Wiley Encyclopedia of Clinical Trials

1. Abbreviated New Drug Application (ANDA) 2. Absolute Risk Reduction 3. Accelerated Approval 4. Adverse Drug Reaction Reporting 5. Adverse Event Evaluation 6. Adverse Event Report System (AERS) 7. Advisory Committees 8. AIDS Clinical Trials Group (ACTG) 9. Algorithm-Based Designs 10. Aligned Rank Test 11. Allocation Concealment 12. Alpha-Spending Function 13. Analysis of Variance ANOVA 14. Analysis Population 15. Application of New Designs in Phase I Trials 16. ASCOT Trial 17. Assay Sensitivity 18. Assessment Bias 19. Assessment of Health-Related Quality of Life 20. Audit 21. Audit Certificate 22. Audit Report 23. Bayesian Dose-Finding Designs in Healthy Volunteers 24. Benefit/Risk Assessment in Prevention Trials 25. Biased-Coin Randomization 26. Bioequivalence (BE) Testing for Generic Drugs 27. Biological Assay, Overview 28. Block Randomization 29. Bootstrap 30. Cardiac Arrhythmia Suppression Trial (CAST) 31. Categorical Response Models 32. Causal Inference 33. Cell Line 34. Censored Data 35. Center for Devices and Radiological Health (CDRH) 36. Center for Drug Evaluation and Research (CDER) 37. Central Nervous System (CNS) 38. CFR 21 Part 11 39. Change/Percent Change From Baseline 40. Chemistry, Manufacturing and Controls (CMC) 41. Citizen Petition 42. Clinical Data Coordination 43. Clinical Data Management 44. Clinical Hold Decision 45. Clinical Significance 46. Clinical Trial Misconduct 47. Clinical Trials, Early Cancer and Heart Disease 48. Clinical Trials to Support Prescription to Over-the-Counter Switches 49. Cluster Randomization 50. Code of Federal Regulations (CFR)

51. Coherence in Phase I Clinical Trials 52. Cohort vs. Repeated Cross-Sectional Survey Designs 53. Collinearity 54. Combination Therapy 55. Committee for Medicinal Product for Human Use (CHMP) 56. Common Technical Document (CTD) 57. Community-Based Breast and Cervical Cancer Control Research in Asian Immigrant Populations 58. Compliance and Survival Analysis 59. Composite Endpoints in Clinical Trials 60. Computer-Assisted Data Collection 61. Conditional Power in Clinical Trial Monitoring 62. Confidence Interval 63. Confidence Intervals and Regions 64. Confirmatory Trials 65. Confounding 66. CONSORT 67. Contract Research Organization (CRO) 68. Control Groups 69. Cooperative North Scandinavian Enalapril Survival Study (CONSENSUS) 70. Cooperative Studies Program, US Department of Veterans Affairs 71. Coordinating Committee 72. Coordinating Investigator 73. Coronary Drug Project 74. Correlation 75. Cost-Effectiveness Analysis 76. Covariates 77. Cox Proportional Hazard Model 78. Cronbach's Alpha 79. Crossover Design 80. Crossover Trials 81. Data Mining 82. Data Monitoring Committee 83. Data Safety and Monitoring Board (DSMB) 84. Data Standards 85. Dermatology Trials 86. Designs with Randomization Following Initial Study Treatment 87. Diagnostic Studies 88. Discriminant Analysis 89. Disease Trials for Dental Drug Products 90. Disease Trials in Reproductive Diseases 91. Disease Trials on Pediatric Patients 92. DNA Bank 93. Dose Escalation and Up-and-Down Designs 94. Dose Escalation Guided by Graded Toxicities 95. Dose Finding Studies 96. Dose Ranging Crossover Designs 97. Double-Dummy 98. Drift (For Interim Analysis) 99. Drug Development

100. Drug Packaging 101. Drug Registration and Listing System (DRLS) 102. Drug Supply 103. Eastern Cooperative Oncology Group (ECOG) 104. Eligibility and Exclusion Criteria 105. Emergency Use Investigational New Drug (IND) 106. End of Phase II Meeting 107. End-of-Phase I Meeting 108. Enrichment Design 109. Environmental Assessments (EAs) 110. Equivalence Trials and Equivalence Limits1 Essential Documents 111. 112. Ethical Challenges Posed by Cluster Randomization 113. Ethical Issues in International Research 114. European Medicines Agency (EMEA) 115. European Organization for Research and Treatment of Cancer (EORTC) 116. Factor Analysis: Confirmatory 117. Factorial Designs in Clinical Trials 118. Fast Track 119. FDA Division of Pharmacovigilance and Epidemiology (DPE) 120. FDA Modernization Act (FDAMA) of 1997 121. Federal Food, Drug and Cosmetic Act 122. Federal Register 123. Fileable New Drug Application (NDA) 124. Financial Disclosure 125. Fisher's Exact Test 126. Flexible Designs 127. Food and Drug Administration (FDA) 128. Frailty Models 129. Futility Analysis 130. Generalized Estimating Equations 131. Generalized Linear Models 132. Generic Drug Review Process 133. Gene Therapy 134. Genetic Association Analysis 135. Global Assessment Variables 136. Gold Standard 137. Good Clinical Practice (GCP) 138. Good Laboratory Practice (GLP) 139. Goodness of Fit 140. Group-Randomized Trials 141. Group Sequential Designs 142. Hazard Rate 143. Hazard Ratio 144. Heritability 145. Historical Control 146. Hypothesis 147. Hypothesis Testing

148. Identifying the Most Successful Dose (MSD) in Dose-Finding Studies 149. Imaging Science in Medicine 150. Imputation 151. Incompetent Subjects and Proxy Consent 152. Independent Ethics Committee (IEC) 153. Inference, Design-Based vs. Model-Based 154. Informed Consent Process, Forms, and Assent 155. Institution 156. Institutional and Independent Review Boards 157. Institutional Review Boards (IRB) 158. Integrated Database 159. Intention-to-Treat Analysis 160. Interaction Model 161. Interim Analyses 162. Interim Clinical Trial/Study Report 163. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) 164. International Studies of Infarct Survival (ISIS) 165. Interrater Reliability 166. Interval Censored 167. Intraclass Correlation Coefficient 168. Intrarater Reliability 169. Investigational Device Exemption (IDE) 170. Investigational New Drug Application Process (IND) 171. Investigational Product 172. Investigator 173. Investigator/Institution 174. Investigator's Brochure 175. Kaplan–Meier Plot 176. Kappa 177. Kefauver–Harris Drug Amendments 178. Lan-DeMets Alpha-Spending Function 179. Large Simple Trials 180. Linear Model 181. Logistic Regression 182. Logrank Test 183. Longitudinal Data 184. Manual of Policies and Procedures (MaPPs) 185. Masking 186. Maximum Duration and Information Trials 187. Maximum Tolerable Dose (MTD) 188. Metadata 189. Methods for Conduct of Rigorous Group-Randomization 190. Microarray 191. Minimum Effective Dose (MinED) 192. Ministry of Health, Labour and Welfare (MHLW, Japan) 193. Min Test 194. Missing Data Monitoring 195. 196. Monotherapy

197. Mother to Child Human Immunodeficiency Virus Transmission Trials 198. Multicenter Trial 199. Multinational (Global) Trial 200. Multiple Comparisons 201. Multiple Endpoints 202. Multiple Evaluators 203. Multiple Risk Factor Intervention Trial (MRFIT) 204. Multiple Testing in Clinical Trials 205. Multistage Genetic Association Studies 206. National Cancer Institute (NCI) 207. National Center for Toxicological Research (NCTR) 208. National Cooperative Gallstone Study 209. National Eye Institute (NEI) 210. National Heart, Lung, and Blood Institute (NHLBI) 211. National Human Genome Research Institute (NHGRI) 212. National Institute of Allergy and Infectious Disease (NIAID) 213. National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS) 214. National Institute of Biomedical Imaging and Bioengineering (NIBIB) 215. National Institute of Child Health and Human Development (NICHD) 216. National Institute of Dental and Craniofacial Research (NIDCR) 217. National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) 218. National Institute of Environmental Health Science (NIEHS) 219. National Institute of General Medical Sciences (NIGMS) 220. National Institute of Mental Health (NIMH) 221. National Institute of Neurological Disorders and Stroke (NINDS) 222. National Institute of Nursing Research (NINR) 223. National Institute on Aging (NIA) 224. National Institute on Alcohol Abuse and Alcoholism (NIAAA) 225. National Institute on Deafness and Other Communication Disorders (NIDCD) 226. National Institute on Drug Abuse (NIDA) 227. National Institutes of Health (NIH) 228. National Institutes of Health Stroke Scale (NIHSS) 229. National Library of Medicine (NLM) 230. New Drug Application (NDA) Process 231. N of 1 Randomized Trial 232. Non-Compartmental Analysis 233. Noncompliance 234. Non-Inferiority Trial 235. Nonparametric Methods Non-Randomized Trial 236. 237. Objectives 238. Office of Orphan Products Development (OOPD) 239. Office of Pharmacoepidemiology and Statistical Science (OPaSS) 240. Office of Regulatory Affairs (ORA) 241. One-Sided Versus Two-Sided Tests 242. Open-Labeled Trial 243. Optimal Biological Dose for Molecularly-Targeted Therapies

244. 245. 246. 247. 248. 249. 250. 251. 252. 253. 254. 255. 256. 257. 258. 259. 260. 261. 262. 263. 264. 265. 266. 267. 268. 269. 270. 271. 272. 273. 274. 275. 276. 277. 278. 279. 280. 281. 282. 283. 284. 285. 286. 287. 288. 289. 290. 291. 292. 293.

Optimizing Schedule of Administration in Phase I Clinical Trials Orphan Drug Act (ODA) Orphan Drugs Orphan Products Grant Program Outliers Over dispersion Over-the-Counter (OTC) Drug Product Review Process Over the Counter (OTC) Drugs Overview of Anti-Infective Drug Development Overview of Safety Pharmacology Paired t Test Parallel Track Partially Balanced Designs “Patients” vs. “Subjects” Permutation Tests in Clinical Trials Pharmacoepidemiology, Overview Pharmacovigilance Phase 2/3 Trials Phase I/II Clinical Trials Phase III Trials Phase II Trials Phase I Trials Phase I Trials in Oncology Phase IV Trials Physicians' Health Study (PHS) Placebos Planning A Group-Randomized Trial Poisson Regression Population Pharmacokinetic and Pharmacodynamic Methods Postmenopausal Estrogen/Progestin Intervention Trial (PEPI) Power Transformations Predicting Random Effects in Community Intervention Preference Trials Premarket Approval (PMA) Premarket Notification 510(k) Premature Termination or Suspension Pre-NDA Meeting Prescription Drug User Fee Act (PDUFA) Prescription Drug User Fee Act (PDUFA) II Prescription Drug User Fee Act (PDUFA) IV Prescription Drug User Fee Act III Prevention Trials Primary Efficacy Endpoint Priority Review Prognostic Variables in Clinical Trials Propensity Score Proportional Odds Model Proportions, Inferences, and Comparisons Prostate Cancer Prevention Trial Protocol

294. Protocol Amendment 295. Protocol Deviators 296. Protocol Violators 297. Publication Bias 298. P Value 299. Quality Assessment Of Clinical Trials 300. Quality Assurance 301. Quality Control 302. Quality of Life Query Management: The Route to a Quality Database 303. 304. Question-Based Review (QbR) 305. Randomization-Based Nonparametric Analysis of Covariance 306. Randomization Procedures 307. Randomization Schedule 308. Rank-Based Nonparametric Analysis of Covariance Record Access 309. 310. Refuse to File Letter 311. Registration of Drug Establishment Form 312. Regression 313. Regression Models to Incorporate Patient Heterogeneity 314. Regression to the Mean 315. Regulatory Authorities 316. Regulatory Definitions 317. Relative Risk Modeling 318. Reliability Study 319. Repeatability and Reproducibility 320. Repeated Measurements 321. Repository 322. Response Adaptive Randomization 323. Response Surface Methodology 324. Risk Assessment 325. Robust Two-Stage Model-Guided Designs for Phase I Clinical Studies 326. Run-In Period 327. Safety Information 328. Sample Size Calculation for Comparing Means 329. Sample Size Calculation for Comparing Proportions 330. Sample Size Calculation for Comparing Time-to-Event Data 331. Sample Size Calculation for Comparing Variabilities 332. Sample Size Considerations for Morbidity/Mortality Trials 333. Screening, Models of 334. Screening Trials 335. Secondary Efficacy Endpoints 336. Sensitivity, Specificity and Receiver Operator Characteristic (ROC) Methods 337. Sequential Analysis 338. Serious Adverse Event (SAE) 339. Simple Randomization 340. Software for Genetics/Genomics 341. Sponsor 342. Sponsor-Investigator

343. Spontaneous Reporting System (SRS) 344. Stability Analysis 345. Stability Study Designs 346. Standard Operating Procedures (SOP) 347. Statins 348. Stepped Wedge Design 349. Stopping Boundaries 350. Stratification 351. Stratified Designs Stratified Randomization 352. 353. Subgroup 354. Subgroup Analysis 355. Subinvestigator 356. Superiority Trial Surrogate Endpoints 357. 358. Survival Analysis, Overview 359. Suspension or Termination of IRB Approval 360. The Belmont Report 361. The Carotene and Retinol Efficacy Trial (Caret) 362. The Center for Biologics Evaluation and Research 363. The Cochrane Collaboration 364. The Community Intervention Trial for Smoking Cessation (COMMIT) 365. The FDA and Regulatory Issues 366. Therapeutic Dose Range 367. Therapeutic Equivalence 368. Therapeutic Index 369. TNT Trial 370. Treatment-by-Center Interaction 371. Treatment Interruption 372. Treatment Investigational New Drug (IND) 373. Trial Site 374. True Positives, True Negatives, False Positives, False Negatives 375. UGDP Trial 376. Update in Hyperlipidemia Clinical Trials 377. Using Internet in Community Intervention Studies1 378. Vaccine 379. Vaccine Adverse Event Report System (VAERS) Web Based Data Management System 380. 381. Wei-Lin-Weissfeld Method for Multiple Times to Events 382. Wilcoxon–Mann–Whitney Test 383. Wilcoxon Signed-Rank Test 384. Women's Health Initiative: Statistical Aspects and Selected Early Results 385. Women's Health Initiative Dietary Trial 386. Women's Health Initiative Hormone Therapy Trials 387. World Health Organization (WHO): Global Health Situation

ABBREVIATED NEW DRUG APPLICATION (ANDA)

patent protection for the new medicines they developed to make up for time lost while their products were going through the FDA’s approval process. Brand-name drugs are subject to the same bioequivalence tests as generics upon reformulation.

An Abbreviated New Drug Application (ANDA) contains data that when submitted to the U.S. Food and Drug Administration’s (FDA) Center for Drug Evaluation and Research, Office of Generic Drugs provides for the review and ultimate approval of a generic drug product. Once approved, an applicant may manufacture and market the generic drug product to provide a safe, effective, low-cost alternative to the American public. Generic drug applications are termed ‘‘abbreviated’’ because they are generally not required to include preclinical (animal) and clinical (human) data to establish safety and effectiveness. Instead, generic applicants must scientifically demonstrate that their product is bioequivalent (i.e., performs in the same manner as the innovator drug). One way scientists demonstrate bioequivalence is to measure the time it takes the generic drug to reach the bloodstream in 24 to 36 healthy volunteers. This gives them the rate of absorption, or bioavailability, of the generic drug, which they can then compare with that of the innovator drug. The generic version must deliver the same amount of active ingredients into a patient’s bloodstream in the same amount of time as the innovator drug. Using bioequivalence as the basis for approving generic copies of drug products was established by the ‘‘Drug Price Competition and Patent Term Restoration Act of 1984,’’ also known as the Waxman-Hatch Act. This Act expedites the availability of less costly generic drugs by permitting the FDA to approve applications to market generic versions of brand-name drugs without conducting costly and duplicative clinical trials. At the same time, the brand-name companies can apply for up to 5 additional years longer This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/regulatory/applications/ ANDA.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

ABSOLUTE RISK REDUCTION

studies or in cross-sectional designs. (Some hypothesis testing procedures such as the Fisher exact test are conditional also on the total number of ‘‘successes’’ in the two groups combined. This alternative conditioning is inappropriate for confidence intervals for a difference of proportions; in particular in the event that no successes are observed in either group, this approach fails to produce an interval.) The outcome variable is binary: 1 if the event of interest occurs, 0 if it does not. (We do not consider here the case of an integer-valued outcome variable; typically, this involves the number of episodes of relapse or hospitalization, number of accidents, or similar events occurring within a defined follow-up period. Such an outcome would instead be modeled by the Poisson distribution.) We observe that r1 subjects in group 1 and r2 subjects in group 2 experience the event of interest. Then the proportions having the event in the two groups are given by p1 = r1 /n1 and p2 = r2 /n2 . If responses in different individuals in each group are independent, then the distribution of the number of events in each group is binomial. Several effect size measures are widely used for comparison of two independent proportions:

ROBERT G. NEWCOMBE Cardiff University Cardiff, Wales, United Kingdom

Many response variables in clinical trials are binary: the treatment was successful or unsuccessful; the adverse effect did or did not occur. Binary variables are summarized by proportions, which may be compared between different arms of a study by calculating either an absolute difference of proportions or a relative measure, the relative risk or the odds ratio. In this article we consider several point and interval estimates for the absolute difference between two proportions, for both unpaired and paired study designs. The simplest methods encounter problems when numerators or denominators are small; accordingly, better methods are introduced. Because confidence interval methods for differences of proportions are derived from related methods for the simpler case of the single proportion, which itself can also be of interest in a clinical trial, this case is also considered in some depth. Illustrative examples relating to data from two clinical trials are shown. 1

Difference of proportions p1 – p2 Ratio of proportions (risk ratio or relative risk) p1 /p2 Odds ratio (p1 /(1 – p1 ))/(p2 /(1 – p2 ))

PRELIMINARY ISSUES

In most clinical trials, the unit of data is the individual, and statistical analyses for efficacy and safety outcomes compare responses between the two (or more) treatment groups. When subjects are randomized between these groups, responses of subjects in one group are independent of those in the other group. This leads to unpaired analyses. Crossover and split-unit designs require paired analyses. These have many features in common with the unpaired analyses and will be described in the final section. Thus, we study n1 individuals in group 1 and n2 individuals in group 2. Usually, all analyses are conditional on n1 and n2 . Analyses conditional on n1 and n2 would also be appropriate in other types of prospective

In this article we consider in particular the difference between two proportions, p1 – p2 , as a measure of effect size. This is variously referred to as the absolute risk reduction, risk difference, or success rate difference. Other articles in this work describe the risk ratio or relative risk and the odds ratio. We consider both point and interval estimates, in recognition that ‘‘confidence intervals convey information about magnitude and precision of effect simultaneously, keeping these two aspects of measurement closely linked’’ (1). In the clinical trial context, a difference between two proportions is often referred to as an

Wiley Encyclopedia of Clinical Trials, Copyright © 2007, John Wiley & Sons, Inc.

1

2

ABSOLUTE RISK REDUCTION

absolute risk reduction. However, it should be borne in mind that any term that includes the word ‘‘reduction’’ really presupposes that the direction of the difference will be a reduction in risk—such terminology becomes awkward when the anticipated benefit does not materialize, including the nonsignificant case when the confidence interval for the difference extends beyond the null hypothesis value of zero. The same applies to the relative risk reduction, 1 – p1 /p2 . Whenever results are presented, it is vitally important that the direction of the observed difference should be made unequivocally clear. Moreover, sometimes confusing labels are used, which might be interpreted to mean something other than p1 – p2 ; for example, Hashemi et al. (2) refer to p1 – p2 as attributable risk. It is also vital to distinguish between relative and absolute risk reduction. In clinical trials, as in other prospective and cross-sectional designs already described, each of the three quantities we have discussed may validly be used as a measure of effect size. The risk difference and risk ratio compare two proportions from different perspectives. A halving of risk will have much greater population impact for a common outcome than for an infrequent one. Schechtman (3) recommends that both a relative and an absolute measure should always be reported, with appropriate confidence intervals. The odds ratio is discussed at length by Agresti (4). It is widely regarded as having a special preferred status on account of its role in retrospective case-control studies and in logistic regression and meta-analysis. Nevertheless, it should not be regarded as having gold standard status as a measure of effect size for the 2 × 2 table (3, 5). 2 POINT AND INTERVAL ESTIMATES FOR A SINGLE PROPORTION Before considering the difference between two independent proportions in detail, we first consider some of the issues that arise in relation to the fundamental task of estimating a single proportion. These issues have repercussions for the comparison of proportions because confidence interval methods for p1 – p2 are generally based closely on those

for proportions. The single proportion is also relevant to clinical trials in its own right. For example, in a clinical trial comparing surgical versus conservative management, we would be concerned with estimating the incidence of a particular complication of surgery such as postoperative bleeding, even though there is no question of obtaining a contrasting value in the conservative group or of formally comparing these. The most commonly used estimator for the population proportion π is the familiar empirical estimate, namely, the observed proportion p = r/n. Given n, the random variable R denoting the number of subjects in which the response occurs has the binomial B(n,p) distribution, with Pr[R = r] = {n!/r!(n – r!)} pr qn – r where q = 1 – p. The simple empirical estimator is also the maximum likelihood estimate for the binomial distribution, and it is unbiased— in the usual statistical sense that the expectation of R given n, E[R|n] = π. However, when r = 0, many users of statistical methods are uneasy with the idea that p = 0 is an unbiased estimate. The range of possible values for π is the interval from 0 to 1. Generally, this means the open interval 0 < π < 1, not the closed interval 0 ≤ π ≤ 1, as usually it would already be known that the event sometimes occurs and sometimes does not. As the true value of π cannot then be negative or zero but must be greater than zero, the notion that p = 0 should be regarded as an unbiased estimate of π seems highly counterintuitive. Largely with this issue in mind, alternative estimators known as shrinkage estimators are available. These generally take the form pψ = (r + ψ)/(n + 2ψ) for some ψ > 0. The quantity ψ is known as a pseudo-frequency. Essentially, ψ observations are added to the number of ‘‘successes’’ and also to the number of ‘‘failures.’’ The resulting estimate pψ is intermediate between the empirical estimate p = r/n and 1/2, which is the midpoint and center of symmetry of the support scale from 0 to 1. The degree of shrinkage toward 1/2 is great when n is small and minor for large n. Bayesian analyses of proportions lead naturally to shrinkage estimators, with ψ = 1

ABSOLUTE RISK REDUCTION

and 1/2 corresponding to the most widely used uninformative conjugate priors, the uniform prior B(1,1) and the Jeffreys prior B( 1/2, 1/2). It also is important to report confidence intervals, to express the uncertainty due to sampling variation and finite sample size. The simplest interval p ± z × SE(p), where √ SE(p) = (pq/n), remains the most commonly used. This is usually called the Wald interval. Here, z denotes the relevant quantile of the standard Gaussian distribution. Standard practice is to use intervals that aim to have 95% coverage, with 2.5% noncoverage in each tail, leading to z = 1.9600. Unfortunately, confidence intervals for proportions and their differences do not achieve their nominal coverage properties. This is because the sample space is discrete and bounded. The Wald method for the single proportion has three unfavorable properties (6–9). These can all be traced to the interval’s simple symmetry about the empirical estimate. The achieved coverage is much lower than the nominal value. For some values of π , the achieved coverage probability is close to zero. The noncoverage probabilities in the two tails are very different. The location of the interval is too distal—too far out from the center of symmetry of the scale, 1/2. The noncoverage of the interval is predominantly mesial. The calculated limits often violate the boundaries at 0 and 1. In particular, when r = 0, a degenerate, zero-width interval results. For small non-zero values of r (1, 2, and sometimes 3 for a 95% interval), the calculated lower limit is below zero. The resulting interval is usually truncated at zero, but this is unsatisfactory as the data tells us that 0 is an impossible value for π . Corresponding anomalous behavior at the upper boundary occurs when n – r is 0 or small. Many improved methods for confidence intervals for proportions have been developed. The properties of these methods are evaluated by choosing suitable parameter space points (here, combinations of n and π ), using these to generate large numbers of simulated random samples, and recording how often the resulting confidence interval includes the true value π . The resulting coverage probabilities are then summarized by

3

calculating the mean coverage and minimum coverage across the simulated datasets. Generally, the improved methods obviate the boundary violation problem, and improve coverage and location. The most widely researched options are as follows. A continuity correction may be incorpo√ rated: p ± {z (pq/n) + 1/(2n)}. This certainly improves coverage and obviates zerowidth intervals but increases the incidence of boundary overflow. The Wilson score method (10) uses the theoretical value π , not the empirical estimate p, in the formula for the standard error of p. Lower and upper limits are obtained as the two solutions of the equation p = π ± √ z × SE(π ) = π ± z × (π (1 – π )/n), which reduces to a quadratic in π . The two roots are given in closed form as √ {2p + z2 ± z (z2 + 4rq)}/{2(n + z2 )}. It is easily demonstrated (7) that the resulting interval is symmetrical on the logit scale—the other natural scale for proportions—by considering the product of the two roots for π , and likewise for 1 – π . The resulting interval is boundary respecting and has appropriate mean coverage. In contrast to the Wald interval, location is rather too mesial. The midpoint of the score interval, on the ordinary additive scale, is a shrinkage estimator with ψ = ( 1/2)z2 , which is 1.92 for the default 95% interval. With this (and also Bayesian intervals) in mind, Agresti and Coull (8) proposed a pseudo-frequency method, which adds ψ = 2 to the numbers of successes (r) and failures (n – r) before using the ordinary Wald formula. This is also a great improvement over the Wald method, and is computationally and conceptually very simple. It reduces but does not eliminate the boundary violation problem. A variety of alternatives can be formulated, with different choices for ψ, and also using something other than n + 2ψ as the denominator of the variance. Alternatively, the Bayesian approach described elsewhere in this work may be used. The resulting intervals are best

4

ABSOLUTE RISK REDUCTION

referred to as credible intervals, in recognition that the interpretation is slightly different from that of frequentist confidence intervals such as those previously described. Bayesian inference starts with a prior distribution for the parameter of interest, in this instance the proportion π . This is then combined with the likelihood function comprising the evidence from the sample to form a posterior distribution that represents beliefs about the parameter after the data have been obtained. When a conjugate prior is chosen from the beta distribution family, the posterior distribution takes a relatively simple form: it is also a beta distribution. If substantial information about π exists, an informative prior may be chosen to encapsulate this information. More often, an uninformative prior is used. The simplest is the uniform prior B(1,1), which assumes that all possible values of π between 0 and 1 start off equally likely. An alternative uninformative prior with some advantages is the Jeffreys prior B( 1/2, 1/2). Both are diffuse priors, which spread the probability thinly across the whole range of possible values from 0 to 1. The resulting posterior distribution may be displayed graphically, or may be summarized by salient summary statistics such as the posterior mean and median and selected centiles. The 2 1/2 and 97 1/2 centiles of the posterior distribution delimit the tail-based 95% credible interval. Alternatively, a highest posterior density interval may be reported. The tail-based interval is considered preferable because it produces equivalent results when a transformed scale (e.g., logit) is used (11). These Bayesian intervals perform well in a frequentist sense (12). Hence, it is now appropriate to regard them as confidence interval methods in their own right, with theoretical justification in the Bayesian paradigm but empirical validation from a frequentist standpoint. They may thus be termed beta intervals. They are readily calculated using software for the incomplete beta function, which is included in statistical packages and also spreadsheet software such as Microsoft Excel. As such, they should now be regarded as computationally of ‘‘closed form,’’ though less transparent than Wald methods.

Many statisticians consider that a coverage level should represent minimum, not average, coverage. The Clopper-Pearson ‘‘exact’’ or tail-based method (13) achieves this, at the cost of being excessively conservative; intervals are unnecessarily wide. There is a trade-off between coverage and width; it is always possible to increase coverage by widening intervals, and the aim is to attain good coverage without excessive width. A variant on the ‘‘exact’’ method involving a mid-P accumulation of tail probabilities (14, 15) aligns mean coverage closely with the nominal 1 – α. Both methods have appropriate location. The Clopper-Pearson interval, but not the mid-P one, is readily programmed as a beta interval, of similar form to Bayes intervals. A variety of shortened intervals have also been developed that maintain minimum coverage but substantially shrink interval length (16, 17). Shortened intervals are much more complex, both computationally and conceptually. They also have the disadvantage that what is optimized is the interval, not the lower and upper limits separately; consequently, they are unsuitable when interest centers on one of the limits rather than the other. Numerical examples illustrating these calculations are based on some results from a very small randomized phase II clinical trial performed by the Eastern Cooperative Oncology Group (18). Table 1 shows the results for two outcomes, treatment success defined as shrinkage of the tumor by 50% or more, and life-threatening treatment toxicity, for the two treatment groups A and B. Table 2 shows 95% confidence intervals for both outcomes for treatment A. These examples show how Wald and derived intervals often produce inappropriate limits (see asterisks) in boundary and near-boundary cases. 3 AN UNPAIRED DIFFERENCE OF PROPORTIONS We return to the unpaired difference case. As described elsewhere in this work, hypothesis testing for the comparison of two proportions takes a quite different form according to whether the objective of the trial is to

ABSOLUTE RISK REDUCTION

5

Table 1. Some Results from a Very Small Randomized Phase II Clinical Trial Performed by the Eastern Cooperative Oncology Group Treatment A

Treatment B

14 0 2

11 0 1

Number of patients Number with successful outcome: tumor shrinkage by ≥ 50% Number with life-threatening treatment toxicity Source: Parzen et al. J Comput Graph Stat. 2002: 11; 420–436.

Table 2. 95% Confidence Intervals for Proportions of Patients with Successful Outcome and With Life-Threatening Toxicity on Treatment A in the Eastern Cooperative Oncology Group Trial Outcome

Successful tumor shrinkage

Life-threatening toxicity

0 0 to 0* < 0* to 0.0357 0 to 0.2153 0.1111 < 0* to 0.2563 0 to 0.2180 0 to 0.1616 0 to 0.2316 0 to 0.1926

0.1429 < 0* to 0.3262 < 0* to 0.3619 0.0401 to 0.3994 0.2222 0.0302 to 0.4143 0.0433 to 0.4046 0.0309 to 0.3849 0.0178 to 0.4281 0.0247 to 0.3974

Empirical estimate Wald interval Wald interval with continuity correction Wilson score interval Agresti-Coull shrinkage estimate Agresti-Coull interval Bayes interval, B(1,1) prior Bayes interval, B( 1/2, 1/2) prior Clopper-Pearson ‘‘exact’’ interval Mid-P interval

Note: Asterisks denote boundary violations. Source: Parzen et al. J Comput Graph Stat. 2002: 11; 420–436.

ascertain difference or equivalence. When we report the contrast between two proportions with an appropriately constructed confidence interval, this issue is taken into account only when we come to interpret the calculated point and interval estimates. In this respect, in comparison with hypothesis testing, the confidence interval approach leads to much simpler, more flexible patterns of inference. The quantity of interest is the difference between two binomial proportions, π 1 and π 2 . The empirical estimate is p1 – p2 = r1 /n1 – r2 /n2 . It is well known that, when comparing means, there is a direct correspondence between hypothesis tests and confidence intervals. Specifically, the null hypothesis is rejected at the conventional two-tailed α = 0.05 level if and only if the 100(1 – α) = 95% confidence interval for the difference excludes the null hypothesis value of zero. A similar property applies also to the comparison of proportions—usually, but not invariably. This is because there are several options for constructing a confidence interval for the difference of proportions, which have

different characteristics and do not all correspond directly to purpose-built hypothesis tests. The Wald interval is calculated as p1 – √ p2 ± z (p1 q1 /n1 + p2 q2 /n2 ). It has poor mean and minimum coverage and fails to produce an interval when both p1 and p2 are 0 or 1. Overshoot can occur when one proportion is close to 1 and the other is close to 0, but this situation is expected to occur infrequently in practice. Use of a continuity correction improves mean coverage, but minimum coverage remains low. Several better methods have been developed, some of which are based on specific mathematical models. Any model for the comparison of two proportions necessarily involves both the parameter of interest, δ = π 1 −π 2 , and an additional nuisance parameter γ . The model may be parametrized in terms of δ and π 1 + π 2 , or δ and (π 1 + π 2 )/2, or δ and π 1 . We will define the nuisance parameter as γ = (π 1 + π 2 )/2. Some of the better methods substitute the profile estimate γ δ , which is the maximum likelihood estimate of γ conditional on a

6

ABSOLUTE RISK REDUCTION

hypothesized value of δ. These include scoretype asymptotic intervals developed by Mee (19) and Miettinen and Nurminen (20). Newcombe (21) developed tail-based exact and mid-P intervals involving substitution of the profile estimate. All these intervals are boundary respecting. The ‘‘exact’’ method aligns the minimum coverage quite well with the nominal 1 – α; the others align mean coverage well with 1 – α, at the expense of fairly complex iterative calculation. Bayesian intervals for p1 – p2 and other comparative measures may be constructed (2, 11), but they are computationally much more complex than in the single proportion case, requiring use of numerical integration or computer-intensive methodology such as Markov chain Monte Carlo (MCMC) methods. It may be more appropriate to incorporate a prior for p1 – p2 itself rather than independent priors for p1 and p2 (22). The Bayesian formulation is readily adapted to incorporate functional constraints such as δ ≥ 0 (22). Walters (23) and Agresti and Min (11) have shown that Bayes intervals for p1 – p2 with uninformative beta priors have favorable frequentist properties. Two computationally simpler, effective approaches have been developed. Newcombe (21) also formulated square-and-add intervals for differences of proportions. The concept is a very simple one. Assuming independence, the variance of a difference between two quantities is the sum of their variances. In other words, standard errors ‘‘square and add’’—they combine in the same way that differences in x and in y coordinates combine to give the Euclidean distance along the diagonal, as in Pythagoras’ theorem. This is precisely how the Wald interval for p1 – p2 is constructed. The same principle may be applied starting with other, better intervals for p1 and p2 separately. The Wilson score interval is a natural choice as it already involves square roots, though squaring and adding would work equally effectively starting with, for instance, tail-based (24) or Bayes intervals. It is easily demonstrated that the square-and-add process preserves the property of respecting boundaries. Thus, the square-and-add interval is obtained as follows. Let (li , ui ) denote the

score interval for pi , for i = 1,2. Then the square-and-add limits are p1 − p2 − p1 − p2 +

√ √

{(p1 − l1 )2 + (u2 − p2 )2 }, {(u1 − p1 )2 + (p2 − l2 )2 }.

This easily computed interval aligns mean coverage closely with the nominal 1 – α. A continuity correction is readily incorporated, resulting in more conservative coverage. Both intervals tend to be more mesially positioned than the γ δ -based intervals discussed previously. The square-and-add approach may be applied a second time to obtain a confidence interval for a difference between differences of proportions (25); this is the linear scale analogue of assessing an interaction effect in logistic regression. Another simple approach that is a great improvement over the Wald method is the pseudo-frequency method (26, 27). A pseudofrequency ψ is added to each of the four cells of the 2 × 2 table, resulting in the shrinkage estimator (r1 + ψ)/(n1 + 2ψ) – (r2 + ψ)/(n2 + 2ψ). The Wald formula then produces the limits √ pψ1 − pψ2 ± z {pψ1 (1 − pψ1 )/(n1 + 2ψ) + pψ2 (1 − pψ2 )/(n2 + 2ψ)}, where pψi = (ri + ψ)/(ni + 2ψ) i = 1, 2. Agresti and Caffo (27) evaluated the effect of choosing different values of ψ, and they reported that adding 1 to each cell is optimal here. So here, just as for the single proportion case, in total four pseudo-observations are added. This approach also aligns mean coverage effectively with 1 – α. Interval location is rather too mesial, very similar to that of the square-and-add method. Zero-width intervals cannot occur. Boundary violation is not ruled out but is expected to be infrequent. Table 3 shows 95% confidence intervals calculated by these methods, comparing treatments A and B in the ECOG trial (18).

ABSOLUTE RISK REDUCTION

7

Table 3. 95% Confidence Intervals for Differences in Proportions of Patients with Successful Outcome and with Life-Threatening Toxicity between Treatments A and B in the Eastern Cooperative Oncology Group Trial Outcome

Successful tumor shrinkage

Life-threatening toxicity

0 0* to 0* −0.2588 to 0.2153 −0.2667 to 0.2223 −0.2849 to 0.2316 −0.2384 to 0.1926 −0.2198 to 0.1685 −0.1768 to 0.1361 −0.2588 to 0.2153 −0.0144 −0.2016 to 0.1728

0.0519 −0.1980 to 0.3019 −0.2619 to 0.3312 −0.2693 to 0.3374 −0.2721 to 0.3514 −0.2539 to 0.3352 −0.2432 to 0.2986 −0.2288 to 0.3008 −0.2524 to 0.3192 0.0337 −0.2403 to 0.3076

Empirical estimate Wald interval Mee interval Miettinen-Nurminen interval Tail-based ‘‘exact’’ interval Tail-based mid-P interval Bayes interval, B(1,1) priors for p1 and p2 Bayes interval, B( 1/2, 1/2) priors for p1 and p2 Square-and-add Wilson interval Agresti-Caffo shrinkage estimate Agresti-Caffo interval

Note: Asterisks denote boundary violations. Source: Parzen et al. J Comput Graph Stat. 2002: 11; 420–436.

4

NUMBER NEEDED TO TREAT

In the clinical trial setting, it has become common practice to report the number needed to treat (NNT), defined as the reciprocal of the absolute risk difference: NNT = 1/(p1 – p2 ) (28, 29). This measure has considerable intuitive appeal, simply because we are used to assimilating proportions expressed in the form of ‘‘1 in n,’’ such as a 1 in 7 risk of lifethreatening toxicity for treatment A in Table 1. The same principle applies to differences of proportions. These tend to be small decimal numbers, often with a leading zero after the decimal point, which risk being misinterpreted by the less numerate. Thus if p1 = 0.35 and p2 = 0.24, we could equivalently report p1 – p2 = 0.11, or as an absolute difference of 11% or an NNT of 9. The latter may well be an effective way to summarize the information when a clinician discusses a possible treatment with a patient. As always, we need to pay careful attention to the direction of the difference. By default, NNT is read as ‘‘number needed to treat for (one person to) benefit,’’ or NNTB. If the intervention of interest proves to be worse than the control regime, we report the number needed to harm (NNTH). A confidence interval for the NNT may be derived from any good confidence interval method for p1 – p2 by inverting the two limits. For example, Bender (30) suggests an

interval obtained by inverting square-andadd limits (21). But it is when we turn attention to confidence intervals that the drawback of the NNT approach becomes apparent. Consider first the case of a statistically significant difference, with p1 – p2 = +0.25, and 95% confidence interval from +0.10 to +0.40. Then an NNT of 4 is reported, with 95% confidence interval from 2.5 to 10. This has two notable features. The lower limit for p1 – p2 gives rise to the upper limit for the NNT and vice versa. Furthermore, the interval is very skewed, and the point estimate is far from the midpoint. Neither of these is a serious contraindication to use of the NNT. But often the difference is not statistically significant— and, arguably, reporting confidence intervals is even more important in this case than when the difference is significant. Consider, for example, p1 – p2 = +0.10, with 95% confidence interval from –0.05 to +0.25. Here, the estimated NNT is 1/0.10 = +10. Inverting the lower and upper confidence limits for p1 − p2 gives −20 and +4. This time, the two limits do not change places apparently. But there are two problems. The point estimate, +10, is not intermediate between −20 and +4. Moreover, the interval from −20 to +4 does not comprise the values of the NNT that are compatible with the data, but rather the ones that are not compatible with it. In fact, the confidence region for the NNT in this case consists of two intervals that extend to infinity, one from + 4 to +

8

ABSOLUTE RISK REDUCTION

∞ in the direction of benefit, the other from –20 to –∞ in the direction of harm. It could be a challenge to clinicians and researchers at large to comprehend this singularity that arises when a confidence interval spanning 0 is inverted (31). Accordingly, it seems preferable to report absolute risk reductions in percentage rather than reciprocal form. The most appropriate uses of the NNT are in giving simple bottomline figures to patients (in which situation, usually only the point estimate would be given), and in labeling a secondary axis on a graph. 5

A PAIRED DIFFERENCE OF PROPORTIONS

Crossover and split-unit trial designs lead to paired analyses. Regimes that aim to produce a cure are generally not suitable for evaluation in these designs, because in the event that a treatment is effective, there would be a carryover effect into the next treatment period. For this reason, these designs tend to be used for evaluation of regimes that seek to control symptomatology, and thus most often give rise to continuous outcome measures. Examples of paired analyses of binary data in clinical trials include comparisons of different antinauseant regimes administered in randomized order during different cycles of chemotherapy, comparisons of treatments for headache pain, and split-unit studies in ophthalmology and dermatology. Results can be reported in either risk difference or NNT form, though the latter appears not to be frequently used in this context. Other examples in settings other than clinical trials include

longitudinal comparison of oral carriage of an organism before and after third molar extraction, and twin studies. Let a, b, c, and d denote the four cells of the paired contingency table. Here, b and c are the discordant cells, and interest centers on the difference of marginals: p1 − p2 = (a + b)/n − (a + c)/n = (b − c)/n. Hypothesis testing is most commonly performed using the McNemar approach (32), using either an asymptotic test statistic expressed as z or chi-square, or an aggregated tail probability. In both situations, inference is conditional on the total number of discordant pairs, b + c. Newcombe (33) reviewed confidence interval methods for the paired difference case. Many of these are closely analogous to unpaired methods. The Wald interval performs poorly. So does a conditional approach, based on an interval for the simple proportion b/(b + c). Exact and tail-based profile methods perform well; although, as before, these are computationally complex. A closedform square-and-add approach, modified to take account of the nonindependence, also aligns mean coverage with 1 – α, provided that a novel form of continuity correction is incorporated. Tango (34) developed a score interval, which is boundary respecting and was subsequently shown to perform excellently (35). Several further modifications were suggested by Tang, Tang, and Chan (36). Agresti and Min (11) proposed pseudo-frequency methods involving adding ψ = 0.5 to each cell

Table 4. Crossover Trial of Home Versus Hospital Physiotherapy: Treating Physiotherapist’s Assessment of Whether the Patient Benefited from Either Type of Treatment Number of patients benefiting from: Treatment sequence Home versus hospital Hospital versus home Difference Half

Difference (first minus second treatment)

Both

First

Second

Neither

Estimate

95% confidence interval

11 9

6 4

1 5

3 1

+ 0.2381 −0.0526 + 0.2907 + 0.1454

−0.0127 to + 0.4534 −0.3372 to + 0.2434 −0.0973 to + 0.6475 −0.0486 to + 0.3238

Source: R. G. Newcombe, Estimating the difference between differences: measurement of additive scale interaction for proportions. Stat Med. 2001; 20: 2885–2893. Reproduced with permission.

ABSOLUTE RISK REDUCTION

and demonstrated good agreement of mean coverage with 1 – α. However, overshoot can occasionally occur. The above methods are appropriate for a paired difference of proportions. But for crossover and simultaneous split-unit studies, a slightly different approach is preferable. Thus, in a crossover study, if the numbers of subjects who get the two treatment sequences AB and BA are not identical, the simple difference of marginals contains a contribution from period differences. A more appropriate analysis is based on the analysis of differences of paired differences described by Newcombe (25). The example in Table 4 relates to a crossover trial of home versus hospital physiotherapy for chronic multiple sclerosis (37). Twenty-one patients were randomized to receive home physiotherapy followed by hospital physiotherapy, and 19 to receive these treatments in the reverse order. Following Hills and Armitage (38) and Koch (39), the treatment effect is estimated as half the difference between the within-subjects period differences in the two treatment order groups. The resulting estimate, + 0.1454 and 95% confidence interval, –0.0486 to + 0.3238, are very similar but not identical to those obtained by direct application of the modified square-and-add approach (33), + 0.1500 and –0.0488 to + 0.3339. REFERENCES 1. K. Rothman, Modern Epidemiology. Boston: Little, Brown, 1986. 2. L. Hashemi, B. Nandram, and R. Goldberg, Bayesian analysis for a single 2 × 2 table. Stat Med. 1997; 16: 1311–1328. 3. E. Schechtman, Odds ratio, relative risk, absolute risk reduction, and the number needed to treat—which of these should we use? Value Health. 2002; 5: 431–436. 4. A. Agresti, Categorical Data Analysis, 2nd ed. Hoboken, NJ: Wiley, 2002. 5. R. G. Newcombe, A deficiency of the odds ratio as a measure of effect size. Stat Med. 2006; 25: 4235–4240. 6. S. E. Vollset, Confidence intervals for a binomial proportion. Stat Med. 1993; 12: 809–824. 7. R. G. Newcombe, Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med. 1998; 17: 857–872.

9

8. A. Agresti and B. A. Coull, Approximate is better than ‘‘exact’’ for interval estimation of binomial proportions. Am Stat. 1998; 52: 119–126. 9. L. D. Brown, T. T. Cai, and A. DasGupta, Interval estimation for a proportion. Stat Sci. 2001; 16: 101–133. 10. E. B. Wilson, Probable inference, the law of succession, and statistical inference. J Am Stat Assoc. 1927; 22: 209–212. 11. A. Agresti and Y. Min, Frequentist performance of Bayesian confidence intervals for comparing proportions in 2 × 2 contingency tables. Biometrics. 2005; 61: 515–523. 12. B. P. Carlin and T. A. Louis, Bayes and Empirical Bayes Methods for Data Analysis. London: Chapman & Hall, 1996. 13. C. J. Clopper and E. S. Pearson, The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika. 1934; 26: 404–413. 14. H. O. Lancaster, The combination of probabilities arising from data in discrete distributions. Biometrika. 1949; 36: 370–382. 15. G. Berry and P. Armitage, Mid-P confidence intervals: a brief review. Statistician. 1995; 44: 417–423. 16. H. Blaker, Confidence curves and improved exact confidence intervals for discrete distributions. Can J Stat. 2000; 28: 783–798. 17. J. Reiczigel, Confidence intervals for the binomial parameter: some new considerations. Stat Med. 2003: 22; 611–621. 18. M. Parzen, S. Lipsitz, J. Ibrahim, and N. Klar, An estimate of the odds ratio that always exists. J Comput Graph Stat. 2002: 11; 420–436. 19. R. W. Mee, Confidence bounds for the difference between two probabilities. Biometrics. 1984; 40: 1175–1176. 20. O. S. Miettinen and M. Nurminen, Comparative analysis of two rates. Stat Med. 1985; 4: 213–226. 21. R. G. Newcombe, Interval estimation for the difference between independent proportions: comparison of eleven methods. Stat Med. 1998; 17: 873–890. 22. R. G. Newcombe, Bayesian estimation of false negative rate in a clinical trial of sentinel node biopsy. Stat Med. 2007; 26: 3429–3442. 23. D. E. Walters, On the reliability of Bayesian confidence limits for a difference of two proportions. Biom. J. 1986; 28: 337–346. 24. T. Fagan, Exact 95% confidence intervals for differences in binomial proportions. Comput

10

ABSOLUTE RISK REDUCTION

Biol Med. 1999; 29: 83–87. 25. R. G. Newcombe, Estimating the difference between differences: measurement of additive scale interaction for proportions. Stat Med. 2001; 20: 2885–2893. 26. W. W. Hauck and S. Anderson, A comparison of large-sample confidence interval methods for the difference of two binomial probabilities. Am Stat. 1986; 40: 318–322. 27. A. Agresti and B. Caffo, Simple and effective confidence intervals for proportions and differences of proportions result from adding 2 successes and 2 failures. Am Stat. 2000; 54: 280–288. 28. A. Laupacis, D. L. Sackett, and R. S. Roberts, An assessment of clinically useful measures of the consequences of treatment. N Engl J Med. 1988; 318: 1728–1733. 29. D. G. Altman, Confidence intervals for the number needed to treat. BMJ. 1998; 317: 1309–1312. 30. R. Bender, Calculating confidence intervals for the number needed to treat. Control Clin Trials. 2001; 22: 102–110. 31. R. G. Newcombe, Confidence intervals for the number needed to treat—absolute risk reduction is less likely to be misunderstood. BMJ. 1999; 318: 1765. 32. Q. McNemar, Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika. 1947; 12: 153–157. 33. R. G. Newcombe, Improved confidence intervals for the difference between binomial proportions based on paired data. Stat Med. 1998; 17: 2635–2650. 34. T. Tango, Equivalence test and CI for the difference in proportions for the paired-sample design. Stat Med. 1998; 17: 891–908. 35. R. G. Newcombe, Confidence intervals for the mean of a variable taking the values 0, 1 and 2. Stat Med. 2003; 22: 2737–2750. 36. M. L. Tang, N. S. Tang, and I. S. F. Chan, Confidence interval construction for proportion difference in small sample paired studies. Stat Med. 2005; 24: 3565–3579. 37. C. M. Wiles, R. G. Newcombe, K. J. Fuller, S. Shaw, J. Furnival-Doran, et al., Controlled

randomised crossover trial of physiotherapy on mobility in chronic multiple sclerosis. J Neurol Neurosurg Psych. 2001; 70: 174–179. 38. M. Hills and P. Armitage, The two-period cross-over clinical trial. Br J Clin Pharmacol. 1979; 8: 7–20. 39. G. G. Koch, The use of non-parametric methods in the statistical analysis of the twoperiod change-over design. Biometrics. 1972; 28: 577–584.

FURTHER READING Microsoft Excel spreadsheets that implement chosen methods for the single proportion, unpaired and paired difference, and interaction cases can be downloaded from the author’s website: http://www.cardiff.ac.uk/medicine/ epidemiology statistics/research/statistics/ newcombe The availability of procedures to calculate confidence intervals for differences of proportions is quite patchy in commercial software. StatXact (Cytel Statistical Software) includes confidence intervals for differences and ratios of proportions and odds ratios. These are ‘‘exact’’ intervals, designed to guarantee minimum coverage 1 – α. The resulting intervals are likely to be relatively wide compared with methods that seek to align the mean coverage approximately with 1 – α.

CROSS-REFERENCES Estimation Confidence interval Categorical variables Comparing proportions Sample size for comparing proportions (superiority and noninferiority) Relative risk Odds ratio Bayesian approach Equivalence analysis Noninferiority analysis

ACCELERATED APPROVAL

In 1987, four major initiatives were drafted to speed drug delivery, cut cost, and make drugs available sooner. Two of these initiatives, ‘‘Treatment IND’’ and ‘‘Parallel Track,’’ focused on expanding access of potentially life-saving drugs to specific patient populations prior to approval of the drug. In contrast, the ‘‘Subpart E’’ and ‘‘Accelerated Approval’’ programs focused on decreasing the amount of clinical and regulatory review time needed to approve a life-saving drug. Subpart E provides a regulatory framework to grant approval to certain drugs after an extended Phase II trial. Accelerated approval allows for a drug to be approved with restrictions on distribution or with the use of unvalidated surrogate endpoints or measures of indirect clinical benefits to determine efficacy (1). The terminology and regulatory implications for various FDA programs can be confusing and are often mislabeled in literature. In understanding the terms, it is worth mentioning that ‘‘Subpart E,’’ which is also known as ‘‘Expedited Development,’’ refers to the sub-part of title 21 in the Code of Federal Regulations (CFR). Likewise, ‘‘Accelerated Approval’’ is sometimes referred to as ‘‘Subpart H,’’ as that is the subpart of 21 CFR that created the regulations allowing for this process by the FDA. Since the 1990s, more initiatives such as Fast Track designation, Priority Review, and Rolling Review have been implemented to facilitate the path of life-saving drugs to market, through separate initiatives.

LOUIS A. CABANILLA M.S. CHRISTOPHER P. MILNE D.V.M, M.P.H, J.D Tufts University Center for the Study of Drug Development, Boston, Massachusetts

The development and approval of a new drug is a complex and time-consuming procedure, which takes many years to accomplish. By the time a drug receives FDA approval, extensive lab and clinical work must be performed to ensure that the drug is both safe and effective, and months or years spent by the FDA reviewing the drug’s application. Although this level of quality control is a great benefit to people who use the drug, the extensive amount of time spent on development and regulatory submission review represents a barrier for a person with a serious or life threatening disease for which no treatment exists, or for a person for whom available treatments have failed. In such cases, a speedy development and regulatory review are of the utmost importance. A patient with no or few therapeutic options is often willing to accept a higher risk to benefit ratio including the use of a treatment whose efficacy is predicated on indirect measures of the expected clinical benefit. 1 ACCELERATED DEVELOPMENT VERSUS EXPANDED ACCESS IN THE U.S.A. In the United States, an emphasis has been placed on making potentially life saving drugs available as soon as possible. In some instances, this process involves making experimental drugs available to patients who are not enrolled in clinical trials, but more typically it involves programs designed to decrease the time to market for these important drugs. This movement took shape in the middle to late 1980s when AIDS drugs were being developed but were not made available to the wider patient population quickly enough, which caused outrage among those who risked dying while waiting for the FDA to approve these drugs for marketing.

2 SORTING THE TERMINOLOGY—WHICH FDA INITIATIVES DO WHAT? • Treatment IND. ‘‘TIND’’ provides for

early access of promising drugs to patients with serious or life-threatening illnesses who have no treatment options, or for patients who have failed available therapies. Drug companies are allowed to recoup the cost of producing the drug for these patients who are not involved in the clinical trials.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

ACCELERATED APPROVAL • Parallel Track. This program is a more

• Priority Review. This review allows the

extensive version of the TIND program aimed at people who do not qualify for participation in clinical trials. The program was a response to the AIDS epidemic, and it is generally intended for HIV/AIDS treatments or related problems. Drug companies collect basic safety data from these patients and are allowed to recoup the cost of producing the drug.

FDA to allocate more resources on the review of a ‘‘priority’’ drug (i.e., one that represents a significant advance over those currently on the market). • Rolling Review.. Under rolling review, companies can submit sections or ‘‘Reviewable Units (RUs)’’ of a NDA or BLA as they complete them, to be reviewed by the FDA. Although an RU may be complete, marketing rights are not granted until all RUs are submitted and approved (1,2).

• Expedited Development (Subpart E).

This provision allows for extensive interaction and negotiation with the FDA to move a drug to the market as quickly as possible. This process includes the possibility of expanding a Phase II trial and using that data for a new drug application (NDA)/biologic licensing application (BLA) submission, and then focusing on postmarketing safety surveillance.

3 ACCELERATED APPROVAL REGULATIONS: 21 C.F.R. 314.500, 314.520, 601.40 The FDA has several programs to expedite the development and approval of lifesaving drugs. Because some similarities are observed among the programs that pertain to the eligibility requirements and regulatory language, they are sometimes confused with each other. Although some overlap does occur, they generally affect different segments of the development and application review timeline (see Fig. 1).

• Accelerated Approval (Subpart H). This

provision allows drugs to gain approval with distribution restrictions, or more commonly, based on unvalidated surrogate endpoints or an indirect clinical benefit. It allows lifesaving drugs to become available while the company completes a long-term Phase IV study. • Fast Track. This program originated as

4 STAGES OF DRUG DEVELOPMENT AND FDA INITIATIVES

a provision of the second Prescription Drug User Fee Act (PDUFA II), and it allows the FDA to facilitate the development of and to expedite the review for drugs intended to treat serious or lifethreatening illness that have the potential to address unmet medical needs.

IND Pre-IND

End of Phase I

End of Phase II

End of Phase III

The intended effect of FDA expedited development programs is to speed development and approval; however, the respective programs focus on disparate aspects of the development and regulatory process. For example,

NDA Submission

NDA Approval Marketing

Priority Rating Rolling Review

Accelerated Approval Sub-Part E Fast Track

Figure 1. Stages of drug development and FDA initiatives. The progressive stages of drug development, and where the initiatives begin/end.

ACCELERATED APPROVAL

3

Accelerated Approval (1992–2005) and... 50% 45% 40% Percentage

35% 30% 25% 20% 15% 10% Figure 2. Accelerated Approval and . . . ’’ The overlap between accelerated approval and various other initiatives.

5% 0%

accelerated approval is often confused with fast track designation and priority review; however, they are quite different. The central mechanism of the accelerated approval program is ‘‘conditional’’ approval based on the use of unvalidated surrogate endpoints or indirect clinical endpoints as evidence of efficacy, or restricted distribution. Fast track designation provides the opportunity for intensive scientific interaction with the FDA and acts as a threshold criterion for rolling review. Priority review is an administrative prioritization scheme implemented by the FDA to give precedence to applications for drugs that represent an improvement to the currently marketed products for a particular disease or condition (also applies to eligible diagnostics and preventatives). Moreover, it should be recognized that the same drug could be a part of accelerated approval, fast track, and other programs at the same time (see Fig. 2). Many accelerated approvals have also benefited from other programs aimed at decreasing the time and resources required to bring crucial drugs to market. In addition, many accelerated approvals are also orphan drugs. Orphan drugs are treatments for rare diseases and conditions, and the FDA designation provides certain economic and regulatory incentives for a drug company to develop them.

Fast Track

Subpart E

Rolling NDA Orphan Drug

5 ACCELERATED APPROVAL REGULATIONS: 21 CFR 314.500, 314.520, 601.40 Accelerated approval regulations were promulgated December 11, 1992. The law stipulates the following (3): FDA may grant marketing approval for a new drug product on the basis of adequate and well-controlled clinical trials establishing that the drug product has an effect on a surrogate endpoint that is reasonably likely, based on epidemiologic, therapeutic, pathophysiologic, or other evidence, to predict clinical benefit or on the basis of an effect on a clinical endpoint other than survival or irreversible morbidity.

Conditions for approval: • Possible FDA restrictions or on distri-

bution and use by facility physician, a mandated qualifying test, or procedural administration requirement • Promotional materials must be submit-

ted to, and approved by the FDA • Streamlined withdrawal mechanisms if • Anticipated benefit is not confirmed

in Phase IV trials • Sponsor fails to exercise due dili-

gence in performing Phase IV trials • Restrictions on use prove insuffi-

cient to ensure safe usage • Violations on restrictions on use

and distribution

4

ACCELERATED APPROVAL • Promotional materials are false or

misleading • Other evidence that the drug is not safe or effective (3) An FDA advisory committee can recommend a drug for accelerated approval based on the set criteria for qualification, which includes the soundness of evidence for the surrogate markers. Following an advisory committee recommendation, the FDA can then review the application and approve the drug for marketing. Drugs must be intended for patients with serious or life-threatening illnesses. Moreover, the data used for the approval must show effectiveness of an unvalidated surrogate endpoint that is ‘‘reasonably likely to predict a clinical effect.’’ In contrast, a validated surrogate endpoint would proceed though the normal approval process. If a company seeks accelerated approval based on restricted distribution, then it must have clear distribution restriction practices and provider/user education programs for the drugs to gain approval. An NDA submission with unvalidated surrogate endpoints must still stand up to the scrutiny of the NDA review process, and it can be rejected for any of the reasons a traditional NDA could be rejected such as safety, efficacy, or concern about product quality. Beyond the issues of approval, all promotional materials for an accelerated approval drug must be submitted to the FDA for approval, and they must be periodically reviewed. This review is another method of ensuring the appropriate understanding and availability of these drugs for both doctors and patients. The initial approval process for a drug by the FDA pertains to a particular or limited set of indications for a particular Sub-population of patients. However, it is common that a drug can later be found to have a benefit for multiple populations and multiple indications. For a drug that is already approved for use, the expansion for a subsequent indication requires a less comprehensive supplemental NDA (sNDA) or supplemental BLA (sBLA). A supplement can be eligible for accelerated approval status, whether the first indication was a traditional approval or not. Over the past 5 years more sNDAs have been granted accelerated approval. In the 2000s,

23 sNDA accelerated approvals were granted by the FDA, whereas in the 1990s (1993–1999) only five sNDAs were approved (4). Most accelerated approvals have been granted to sponsors of small molecule drugs (i.e., chemicals), but a significant minority, which has increased over time, have been granted to large molecule drugs (i.e., biologics). This latter group is generally composed of monoclonal antibodies, although many more designations have been granted for other biologics such as vaccines or recombinant proteins. Nearly 31% of all accelerated approvals have been for biologics, most for oncology, although several accelerated approvals have been granted for rare diseases such as multiple sclerosis and Fabry’s disease (5). Increasingly, accelerated approvals have been given to speed the development and availability of preventative vaccines, such as the influenza vaccine for the 2006–2007 flu season (6) (Fig. 3). 6 ACCELERATED APPROVAL WITH SURROGATE ENDPOINTS The major advantage of the accelerated approval designation is that it can decrease the complexity of late stage clinical trials using an unvalidated surrogate endpoint, which may decrease the number of patients who must be enrolled, decrease the amount of data being collected, and most of all decrease the time required to conduct the necessary studies (7). For example, in 1992 ddI (also called Videx or didanosine) was approved based on CD4 levels rather than the survival rate for HIV infected patients, which greatly reduced the Phase III trial time. However, this endpoint has become validated; as a result, new drugs that use CD4 levels as a clinical measurement now proceed through the normal NDA process. In terms of speed to market, drugs that are designed to increase the survival rate or slow disease progression can take years to properly test, which is generally the case for illnesses such as cancer, HIV, or multiple sclerosis. In such cases, accelerated approval is beneficial in making a drug available to patients while long-term data are collected.

ACCELERATED APPROVAL

The ability to use easily observable and quantifiable data as evidence of clinical effectiveness allows a drug to reach the market much more quickly than using traditional measurement such as overall survival rate (7). Surrogate endpoints are a subset of biological markers, which are believed to have a correlation for endpoints in a clinical trial (8). Although it is not guaranteed that an actual relationship exists, surrogate endpoints are ‘‘based on scientific evidence for clinical benefit based on epidemiologic, therapeutic, pathophysiologic or other scientific evidence’’ (8). A clinical trial testing the efficacy of a statin, for example, might use a reduction in cholesterol as a surrogate for a decrease in heart disease. Elevated cholesterol levels are linked to heart disease, so this correlation is likely. The benefit of using cholesterol levels is that heart disease often takes decades to develop and would require a time-consuming and expensive trial to test. With this in mind, the FDA might approve the statin for marketing contingent on the sponsor company’s agreement to complete postmarketing commitment(s) to test for a clinical effect (Phase III b or Phase IV studies). Phase IIIb and Phase IV studies are distinct from each other in that the former begins after the application submission, but before approval, and it typically continues into the postmarketing period; a Phase IV study is a postmarketing study.

6.1 What Is a Surrogate Endpoint? A surrogate endpoint is a laboratory or physical sign that is used in therapeutic trials as a substitute for a clinically meaningful endpoint (or biomarker) that is a direct measure of how a patient feels, functions, or survives and that is expected to predict the effect of the therapy [8]. 6.2 What Is a Biomarker? A biomarker is a characteristic that is measured and evaluated objectively as an indicator of normal biologic or pathogenic processes or pharmacological responses to a therapeutic intervention. Surrogate endpoints are a subset of biomarkers [8]. 7 ACCELERATED APPROVAL WITH RESTRICTED DISTRIBUTION Although accelerated approval drugs are generally approved on the basis of surrogate endpoints, however, section 21 C.F.R. Section 314.520 allows for approval of a drug based on restrictions on distribution. These restrictions can refer to certain facilities or physicians who are allowed to handle and prescribe the drug, or to certain requirements and tests or medical procedures, which must be performed prior to use of the drug. The drug thalidomide has a restricted accelerated approval for the treatment of Erythema

Product Type of Accelerated Approvals: 1992–2005 80% 70%

Percentage

60% 50% 40% 30% 20% 10% 0%

5

Small Molecule

Biologics Type of Drug

Figure 3. Product type of accelerated approvals. Comparison between small molecule and biologic drugs approved by FDA through the Subpart H.

6

ACCELERATED APPROVAL

Nodosum Leprosum because the drug is associated with a high risk of birth defects for infants whose mothers are exposed to the drug. To mollify this risk, doctors must complete an educational program on the risks and safe usage of thalidomide before they are certified to prescribe the drug. Few accelerated approvals are based on this restriction, although they have become more common over time. The number of restricted distribution approvals averaged one per year between 2000 and 2005. However, in the first half of 2006 there were four restricted approvals, all of which were supplemental approvals (Fig. 4) (4). 8 PHASE IV STUDIES/POSTMARKETING SURVEILLANCE Although a drug that is given accelerated approval might reach the market, the company that sponsors the drug is required to complete the necessary research to determine efficacy with a Phase IV study. These studies are intended to increase the quality of the application dossier to the scientific standards of drugs approved through the traditional NDA/BLA process. Sec. 314.510 (9) stipulates: Approval under this section will be subject to the requirement that the applicant study the drug further, to verify and describe its clinical benefit, where there is uncertainty as to

the relation of the surrogate endpoint to clinical benefit, or of the observed clinical benefit to ultimate outcome. Postmarketing studies would usually be studies already underway. When required to be conducted, such studies must also be adequate and well controlled. The applicant shall carry out any such studies with due diligence.

Some ambiguity exists in this section of the law as no actual time frame was given, until recent years, for companies to complete their Phase IV trials and submit their findings to the FDA. Rather, it was expected that companies would use ‘‘due diligence’’ in completing their research. This nebulous definition has lead to some contention about the level of compliance by the drug industry. This question will be considered in more depth in a subsequent section. 9 BENEFIT ANALYSIS FOR ACCELERATED APPROVALS VERSUS OTHER ILLNESSES Drugs that qualify for accelerated approval are typically intended to treat serious conditions and diseases such as pulnanary Arterial hypertension, HIV, malignancies, Fabry’s disease, and Crohn’s disease. Given that the average median total development time for all FDA-approved drugs during the 1990s and 2000s was just over 7 years (87 months), drugs under the accelerated approval program have spent considerably less time in

Approval Type: 1992–2005 Number of Approvals

90 80 70 60 50

sNDAs/sBLAs NDA/BLA

40 30 20 10 0

Surrogate

Restricted Basis

Figure 4. Accelerated Approval information through 06/06. Approval type: A comparison between Subpart H approvals with restricted distribution versus surrogate basis.

ACCELERATED APPROVAL

the journey from bench to bedside in such critical therapeutic areas as HIV/AIDS and cancer (Fig. 5). To qualify for accelerated approval, a drug must offer a meaningful therapeutic benefit over available therapy, such as: • Greater efficacy • A more favorable safety profile • Improved patient response over existing

therapies (9) Given the severity of the illnesses and the potential for a ‘‘meaningful therapeutic benefit,’’ the FDA is generally willing to accept a higher risk-to-benefit ratio for a potential therapy. A drug with a comparatively high risk of side effects or with less certainty of proof of efficacy can be approved. For example, a drug with a traditional approval for the treatment of insomnia would have an acceptable overall safety and efficacy profile. Conversely, an accelerated approval drug for multiple sclerosis could gain approval with a likely clinical effect, based on surrogate endpoints. This change in approval occurred because patient advocacy groups and public opinion have made it clear that patients in such a situation are willing to bear additional risks. For patients with no other options, a new therapy often represents the only possibility for increasing the length or quality of their life. Likewise, a drug designated as a firstline treatment for a malignancy would have

7

a more rigorous approval process, given the currently available treatments, than a drug designated as a third-line treatment for patients who have failed all conventional treatment options. Within any patient population, a certain percentage will fail to respond to first- or second-line treatments or they may become resistant to therapy. This finding is particularly true with malignancies that have high relapse rates and infections such as HIV, in which the virus gains immunity to treatments over time. In situations such as these, creating the incentive for drug companies to develop second-, third-, and even fourth-line treatments with new mechanisms of action is critical.

10 PROBLEMS, SOLUTIONS, AND ECONOMIC INCENTIVES Accelerated approval is intended to increase the speed to market for important drugs. Although this timeline is a benefit to patients, it is also a benefit to the company that sponsors the drug. The use of surrogate endpoints can greatly decrease the length and complexity of a clinical trial. This use allows a drug company to begin recouping sunk costs earlier and provides returns on sales for a longer period. Without accelerated approval, it is possible that many of these drugs would have been viewed as too risky or expensive to develop in terms of return on investment for

Median Development and Approval Times for Accelerated Approval Drugs: 1992–2005 140

Time (months)

120 100 80 60 40 20

Figure 5. Median development and approval times for accelerated approval drugs. Development and regulatory times by indication for Subpart H.

0 HIV/AIDS

Cancer

Cardiovascular Related Diseases

Other

Median Approval Time Median Development Time

8

ACCELERATED APPROVAL Table 1. 2004 Sales figures for drugs first approved under accelerated approval: sales figures for various accelerated approval drugs in billions of dollars (US) Viracept Biaxin Cipro Eloxatin Taxotere Levaquin Remicade Celebrex Erbitux Temodar Thalomid Casodex Norvir Arimidex Epivir Gleevec Betaseron Kaletra Camptosar Viread

nelfinavir mesylate clarithromycin ciprofloxacin hcl oxaliplatin docetaxel levofloxacin infliximab celecoxib cetuximab temozolomide thalidomide bicalutamide ritonavir anastrozole lamivudine imatinib mesylate interferon beta-1b lopinavir/ritonavir irinotecan hcl tenofovir disoproxil fumarate

drug companies. Even after approval, sponsor companies have some degree of uncertainty. It is possible that some practitioners, patients, and third-party payers may consider such approvals to be conditional and avoid their use until no alternative is available. Moreover, insurance companies may refuse to reimburse experimental drugs, thus the financial burden for the treatment may also place the drugs out of reach of patients. However, these drugs can be very profitable. In 2004, 20 drugs had accelerated approval among the top 200 best-selling drugs, with combined sales of $12.54 billion (Table 1) (10). A prerequisite for accelerated approval designation is that the drug company will complete a Phase IV trial with ‘‘due diligence’’ following the marketing of the drug. Once approved, evidence from a Phase IV trial that does not support the efficacy of the drug may lead to the drug being withdrawn. The original text has proven problematic in that ‘‘due diligence’’ is ambiguous, and it is understood differently by the various stakeholders. Moreover, despite the wording of the provisions for drug withdrawal, it is difficult to do so once market rights have

$0.23 $0.27 $0.27 $0.78 $0.94 $1.70 $1.80 $2.70 $0.24 $0.24 $0.27 $0.28 $0.30 $0.30 $0.31 $0.34 $0.38 $0.45 $0.48 $0.53

been granted. A congressional staff inquiry found that as of March 2005, 50% of outstanding accelerated approval postmarketing studies had not begun, whereas most of the other 50% of studies were underway. Overall, a significant proportion (26%) of trials is taking longer than expected to complete (11). The incentive for a company to perform a Phase IV trial is minimal given that the sponsor company has already received marketing approval, and the completion of the trial is expensive and time consuming. As of 2006, no drug had been withdrawn for failure to complete a Phase IV trial. However, proposed regulations and guidance within the FDA could change this (12). The biggest change to the original procedure for accelerated approval came in 2003 when the FDA reinterpreted the laws regarding the number of accelerated approvals granted per oncology indication. Whereas the old practice allowed only one accelerated approval per indication, the new interpretation viewed a medical need as ‘‘unmet’’ until the sponsoring company completes a Phase IV trial (13). This change provides an incentive for drug companies to complete Phase IV studies quickly to avoid market competition

ACCELERATED APPROVAL

from other drugs. More recently, the FDA has released a guidance that suggests accelerated approvals could be based on interim analysis, which will continue until a final Phase IV report is completed, rather than on separate Phase IV trials (14). This guidance would potentially decrease the cost of a Phase IV study as well as alleviate the issue of patient enrollment, which is often cited as a cause for delays by drug companies. The idea of granting ‘‘conditional’’ approval contingent on Phase IV completion for ‘‘full’’ approval has also been mentioned (15). Increasing Phase IV follow-through is important to the proper functioning of the accelerated approval program. Although the failure to complete a Phase IV trial might hurt the patient population by leaving some uncertainty as to a drug’s safety or efficacy, it also hurts the drug companies in the long run. In 2005, an advisory committee refused to grant the drug Zarnestra, (Johnson & Johnson, New Brunswick, NJ) accelerated approval and cited the FDA’s failure to enforce Phase IV obligations as a reason (16). This decision was an example of the impact that the lack of confidence in FDA enforcement, and sponsor compliance could have on the future performance of the program. A final issue, which the FDA must address, is that of public perception and understanding of exactly what accelerated approval is and is not. There is often the misconception about the conditions under which accelerated approval is granted, that it is better than a full approval, or that it is the same as a fast track approval. In the interest of transparency, that the public must understand that an accelerated approval is less certain than a traditional approval and that less than a full quantum of evidence was used as the basis for efficacy. However, an accelerated approval should not be viewed as inherently more risky, when in fact it has been demonstrated to be more beneficial for certain patients to take the drug than not based on consideration of all the risks and benefits currently known. 11

FUTURE DIRECTIONS

Will there be changes to the accelerated approval program? On one hand, some

9

patient groups believe that accelerated approval is not delivering on its promise for bringing more therapeutic options to patients with diseases for which few or no adequate treatments are available. They question the arduousness of the process, the increasing trepidation of the FDA regarding potential ill effects, as well as political influences being brought to bear on a program that was borne out of an initiative inspired by patients, medical practitioners, and FDA in response to a public health emergency. On the other hand, there is increasing discomfort among some consumer watchdog groups as well as patient groups concerning the diligence with which the confirmatory post-marketing studies are being pursued by biopharmaceutical firms. Addressing this issue was a key focus of the public debates surrounding the reauthorization of the PDUFA in 2007. Will the concept of accelerated approval go global? Research on biomarkers is becoming a global endeavor and the need to get lifesaving drugs to patients more expeditiously is a worldwide concern. It would seem likely that the regulatory agencies in countries that are major centers of biopharmaceutical research and development such as Japan and the European Union would follow the United State’s lead as they did in setting up their orphan drug programs 10 and 20 years after the United States, respectively (with piggy-back programs emerging in other industrialized countries such as Australia and Singapore). However, the EU already has a program that is similar to accelerated approval, which is termed marketing authorization ‘‘under exceptional circumstances’’ for products that do not have, and are likely to never have full safety and efficacy data (17). The EU also recently enacted ‘‘conditional approval’’ that allows for approval based on a less complete approval package than is normally required (18). It is possible that some symbiosis may occur between the EU and U.S. program. The FDA may become more comfortable with the concept of a conditional approval (i.e., annual status review and restrictive marketing until confirmatory studies are complete), whereas the EU may become more expansive in employing conditional approval based on the increasing availability of surrogate biomarkers as

10

ACCELERATED APPROVAL

adequate indicators of safety and efficacy in ‘‘exceptional circumstances.’’ The Japanese regulatory body has also adopted a conditional approval process, which includes provisions for the use of restricted access, and surrogate endpoints. However, they do not have a program that is specific to accelerated approvals. This program, which encompasses many initiatives, has been an effective means of decreasing development and regulatory time (19). Overall, the success of the accelerated approval program to date in bringing some 100 drug products to market months or years faster than would have happened through the traditional process, together with advances in the science and technology of the clinical evaluation of biopharmaceuticals and the more powerful voice of patient advocacy in determining regulatory agency policy, ought to ensure that accelerated approval will expand its reach both therapeutically and geographically.

REFERENCES 1. S. S. Shulman and J. S. Brown. The Food and Drug Administration’s early access and fast-track approval initiatives: how have they worked. Food Drug Law J. 1995; 50: 503–531. 2. Guidance for Industry: Fast Track Drug Development Programs – Designation, Development, and Application Review. January 2006, Procedural, Revision 2. U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER) & Center for Biologics Evaluation and Research (CBER). 3. New Drug, Antibiotic, and Biological Drug Product Regulations; Accelerated Approval. Final Rule. 57 Federal Register 58942 (December 11, 1992). Department of Health and Human Services, Food and Drug Administration. 4. Accelerated Approvals under 21 CFR 314 Subpart H (drugs) & 21 CFR 601 Subpart E (biologics). U.S. Food and Drug Administration. CDER website. Available at http://www.fda. gov/cder/rdmt/accappr.htm. Last accessed 09/15/ 2006. 5. Unpublished Data, Marketed Database, September 2006. Tufts Center for the Study of Drug Development. Boston, MA: Tufts University.

6. FDA News: FDA Approves Additional Vaccine for Upcoming Influenza Season. October 5, 2006. U.S. Food and Drug Administration. [Press Release] Available at: http://www.fda. gov/hhs/topics/NEWS/2006/NEW01478.html. 7. R. L. Schilsky, Hurry up and wait: is accelerated approval of new cancer drugs in the best interests of cancer patients? J. Clin. Oncol. 2003; 21: 3718–3720. 8. A. G. Chakravarty, Surrogate markers – their role in the regulatory decision process. FDA Workshop, American Statistical Association, Toronto, Canada, 2004. 9. H. Subpart, Accelerated Approval of New Drugs for Serious or Life-Threatening Illnesses. 21 CFR 314.50 (Revised April 1, 2004). 10. Top 200 Drugs for 2004 by U.S. Sales. Available at: http://www.drugs.com/top200 2004. html. 11. Conspiracy of Silence: How the FDA Allows Drug Companies to Abuse the Accelerated Approval Process. Responses by the Food and Drug Administration and the Securities and Exchange Commission to Correspondence from Rep. Edward J. Markey (D-MA). Congressional staff summary, June 1, 2005. 12. A. Kaspar, Accelerated approval should be ‘‘conditional’’ panel says. Pink Sheet Daily, June 17, 2005. 13. FDA revising accelerated approval standard in Oncology; biomarkers sought. The Pink Sheet June 9, 2003;65: 23. 14. Susman E. Accelerated approval seen as triumph and roadblock for cancer drugs. J. Natl. Cancer Inst. 2004; 96: 1495–1496. 15. Fox JL. Conflicting signals on US accelerated approvals. Nature Biotechnology, published online, 31 August 2005. Available at: http://www. nature.com/news/2005/050829/full/nbt09051027.html. 16. Iressa casts shadow over Zarnestra advisory committee review. The Pink Sheet May 16, 2005;67: 18. 17. Guideline on procedures for the granting of a marketing authorization under exceptional circumstances, pursuant to article 14 (8) of regulation (EC) No 726/2004. London, 15 December 2005. Doc. Ref. EMEA/357981/2005. European Medicines Agency (EMEA). 18. Guideline on the scientific application and the practical arrangements necessary to implement commission regulation (EC) No 507/2006 on the conditional marketing authorization for medicinal products for human use

ACCELERATED APPROVAL falling within the scope of regulation (EC) No 726/2004. London, 5 December 2006. Doc. Ref. EMEA/509951/2006. European Medicines Agency (EMEA). 19. Japan regulators increase use of ‘‘conditional approvals’’ on drugs. Pacific Bridge Medical: Asian Medical Newsletter 2006; 6: 1–2.

FURTHER READING Federal Register Final Rules, Title 21, Chapter 1, Subchapter D- Drugs for Human Use, Subpart H- Accelerated Approval of New Drugs for Serious or Life Threatening Illnesses. §314 and 600. Guidance for industry: Providing Clinical Evidence of Effectiveness for Human Drug and Biological Products, U.S Department of Health and Human Services Food and Drug Administration, May 6, 1998. Guidance for Industry: Fast Track Drug Development Programs- Designation, Development, and Application Review, U.S Department of Health and Human Services Food and Drug Administration, January 2006.

11

ADVERSE DRUG REACTION REPORTING The sponsor should expedite the reporting to all concerned investigator(s)/institutions(s), to the Institutional Review Board(s) (IRB)/Independent Ethics Committee(s) (IEC), where required, and to the regulatory authority(ies) of all adverse drug reactions (ADRs) that are both serious and unexpected. Such expedited reports should comply with the applicable regulatory requirement(s) and with the ICH Guideline for Clinical Safety Data Management: Definitions and Standards for Expedited Reporting. The sponsor should submit to the regulatory authority(ies) all safety updates and periodic reports, as required by applicable regulatory requirement(s).

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

ADVERSE EVENT EVALUATION

at the time of the data collection, it may be extremely difficult to assess the causality of a single, specific event, and it is only through the review of aggregated data, or after a series of apparently unrelated events have occurred, that a relationship becomes apparent. For example, falls are not uncommon in an elderly population, and it may seem unremarkable that an older study participant has a fall resulting in minor injuries during participation in a study, and that this does not really constitute an adverse event. However, if several subjects in the study seem to be having falls, at a rate higher than would be expected, additional investigation may demonstrate that the investigational product causes dizziness and that the risk of falling is a true safety issue. Although it is often easiest to think of safety assessment in terms of adverse events occurring during a study in which the intervention is a pharmaceutical product, adverse events may also occur in studies in which there is a different type of intervention. For example, emotional distress or nightmares may result from a study in which a new technique for counseling is used in the treatment of survivors of childhood trauma.

LINDSAY A. MCNAIR, M.D., Vertex Pharmaceuticals, Cambridge, MA Boston University School of Public Health Department of Epidemiology Boston, Massachusetts

1

INTRODUCTION

Although much of the focus in later stage clinical trials is often on study design and the evaluation of the efficacy of an intervention or pharmaceutical product, the appropriate collection and assessment of safety data is a critical part of any clinical study. In early stage trials, the primary objective of the study may be the assessment of safety (or the identification of a maximum tolerated dose). For the assessment of safety data, several key factors must be collected for any adverse event. This article will provide information for the identification of adverse events and the data to be collected, to provide the necessary tools for safety assessment in clinical studies. 2

IDENTIFYING AN ADVERSE EVENT 3 INFORMATION COLLECTED FOR ADVERSE EVENTS

An adverse event is defined as any untoward medical occurrence in a patient or clinical investigation subject administered a pharmaceutical product and that does necessarily not have to have a causal relationship with the treatment. It may include worsening of a preexisting disease or condition that occurs during the study period that is outside the normally expected variations (1). Adverse events may also include the abnormal or clinically significant results of laboratory tests or medical procedures (see Adverse Event Definitions). It is important to note that the definition of an adverse event does NOT state or imply any relationship to the intervention or investigational product. Therefore, any medical event regardless of whether the investigator thinks it may have been related to the study intervention must be collected as an adverse event. The rationale for this is that

The first piece of information collected is the adverse event name. The exact term used by the investigator to name the event (e.g., hypertension or diarrhea) is called the verbatim term. Some medical events can be named in many different ways, but they are all referring to the same clinical finding or diagnosis (e.g., hypertension, high blood pressure, elevated blood pressure, increased BP, etc.). For the analysis of adverse events, the verbatim terms will be interpreted through a coding dictionary. The coding dictionary will match each reported verbatim as closely as possible to the appropriate medical term, which will then be grouped with other like terms that all describe the same clinical event. The dictionary includes a hierarchy of terms, so that the more specific, lower level terms can be then grouped into disorders of specific

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

ADVERSE EVENT EVALUATION

mechanisms, diagnoses, and body systems (see Coding Dictionaries).

4

RELATIONSHIP TO THE INTERVENTION

Assessing relationship to the intervention or the investigational product is often one of the more difficult aspects of safety data collection and interpretation. Clinical study subjects often have comorbidities, physical findings related to the disease under study, side effects from other concomitant medications, or even normal variations in health status that complicate the decision about whether a particular adverse event is related to the intervention being studied. Even in early phase studies of healthy volunteers, there are reports of adverse events of alanine aminotransferase elevations in subjects receiving placebo (2), which may be from the change in daily habits during study participation, or to fluctuations in normal laboratory values when watched more closely than is standard clinical practice. In determining the relationship, there are several questions the investigator must consider. What was the timing of the adverse event (start date/time, and stop date/time)? What was the relationship to the administration of the intervention? Depending on the type of investigational product and the pharmacokinetics, this may be related to the actual dosing time, dosing route (for example, an injection site reaction), a maximal plasma concentration, or a cumulative exposure over time. If the intervention or investigational product is dose reduced or withdrawn, does the event improve? If the dose is increased or the intervention is restarted, does the event recur? Are there other comorbid conditions or other medications that may have caused the event? Has this event or this type of event been seen before with this intervention? Is there a reasonable biologic mechanism through which this event could have been caused? After determining the likelihood that the adverse event is related to the intervention, the investigator must choose one of several options that reflects the clinical assessment. The options usually include no relationship,

an unlikely relationship, a possible relationship, a probable relationship, and a definite relationship. The specific wording of the options may vary (‘‘unlikely’’ is sometimes called ‘‘remote’’). There are rarely more options for the relationship categories, but there are often fewer; sometimes the options are collapsed into four, three, or even two categories (related/not related). The CDISC standard, which will be standard for all data used to support regulatory approvals by 2010 (3), includes four options (not related/unlikely related/possibly related/related). 5 ASSESSING SEVERITY The investigator must assess the severity, sometimes called the intensity, of the adverse event. For the interpretation of the data collected, it is important that investigators are using the same guidelines to determine severity, so that an event considered moderate in intensity by one investigator is reasonably similar to the same type of event considered moderate by a different investigator. For this reason, there are many standard guidelines for the assessment of the severity of adverse events. Perhaps the best known of the standard guidelines is the National Cancer Institute’s Common Toxicity Criteria (NCI CTC) (4). This guideline, now in Version 3.0 and more extensive with each version, includes hundreds of possible adverse events with brief descriptions for what would be considered appropriate for each grade of severity, from grade 1 through grade 4. In general, grade 1 events are considered mild, grade 2 events are moderate, grade 3 events are severe, and grade 4 events are life-threatening. Any event that results in death is considered grade 5. For example, nausea is grade 1 if it includes loss of appetite but no real change in eating habits, grade 2 if there is decreased oral intake but no weight loss or malnutrition and IV fluids are needed for less than 24 hours, and grade 3 if oral intake is so low that parenteral nutrition or hydration is required for more than 24 hours. Results of laboratory tests are graded as well, usually by the proportional change from the reference range for the laboratory (i.e., 0.5 times

ADVERSE EVENT EVALUATION

lower limit of normal range) rather than by absolute numbers. The NCI CTC is well known, extensive, and commonly used, but it is not appropriate for all clinical studies. The NCI CTC is designed for use in clinical studies in oncology, and in other populations (particularly healthy volunteers), the grading of events may be considered too lenient, with significant clinical signs and symptoms before an event is considered to be severe. In the design of a new clinical study, it is necessary to review different grading guidelines to find the one best suited to the study and the study population—or, in some cases, to develop a new grading guideline, particularly if there are safety events that are expected or that are of particular interest. In general, for events for which there is not specific grading guidance, the usual recommendation is to consider the impact on the activities of daily life for the study participant, with a mild event having little or no impact, a moderate having some impact, and a severe event having substantial impact. 6

DETERMINING EXPECTEDNESS

The determination of when a specific event is considered to be ‘‘Expected’’ for a specific investigational product is a formal process. In general, an event is considered expected if events of the same nature and severity are described in the product label (or in the case of investigational products, the Investigator’s Brochure). The formal determination of expectedness is important, as it is a significant factor in the requirements for the reporting of adverse events to regulatory authorities. In general, serious adverse events that are considered expected do not have to be reported in the same expedited manner (within 7–15 days, depending on other factors) to investigators and regulatory agencies. Therefore, the persons who are responsible for maintaining the Investigator’s Brochure usually make the decision about what is considered ‘‘Expected.’’ In addition to considering the documented safety profile for the intervention, the investigator may use an informal interpretation of expectedness also taking into account his or her own experience. The expectedness should

3

be considered as part of the determination of relationship, but the investigator will not be asked to specify the expectedness of an event.

7

ACTIONS TAKEN

In addition to collecting the event name, start and stop dates (and sometimes times), the relationship, and the severity, it is also important to know what actions were taken in response to a specific event. Did the dose of the investigational drug have to be decreased, or did dosing have to be interrupted (with possible impact on efficacy)? Did the administration of the intervention have to be stopped completely? Did the subject discontinue from the study? Did they require addition medications to treat the event, or hospitalization? Although in general, the more dramatic actions (stopping administration, etc.) may parallel the event severity, this may not always be the case. Adverse events that do not have significant clinical safety impact may still impact the tolerability of the investigational product, especially if it will be used over a long period of time. Because of the potential effects of tolerability on treatment adherence, this information must be analyzed as part of the overall safety profile.

REFERENCES 1. European Medicines Agency, ICH E2A. Clinical Safety Data Management, Definitions and Guidelines for Expedited Reporting. (March 1995). Available: http://www.fda.gov/cder/ guidance/iche2a.pdf, accessed January 2008. 2. Rosenzweig P, Miget N, Broheir S. Transaminase elevation on placebo in Phase 1 Clinical Trials: Prevalence and significance. J. Clin. Pharmacol. 1999; 48:19–23. 3. Clinical Data Interchange Standards Consortium; mission statement and strategic plan summary. Available: http://www.cdisc.org/ about/index.html, accessed January 2008. 4. National Cancer Institute Common Toxicity Criteria, Version 3.0. Available: http://ctep. cancer.gov/reporting/ctc v30.html, accessed January 2008.

4

ADVERSE EVENT EVALUATION

FURTHER READING Guidance for Industry: Adverse Event Reporting—Improving Human Subject Protection. (Issued in draft, April 2007). Available: http://www.fda.gov/cber/gdlns/advreport.pdf, accessed January 2008. Guidance for Industry: Premarketing Risk Assessment. (Issued March 2005). Available: http:// www.fda.gov/cder/guidance/6357fnl.pdf, accessed January 2008.

CROSS-REFERENCES Coding dictionaries Regulatory definitions Safety assessments Adverse event reporting system

ADVERSE EVENT REPORT SYSTEM (AERS)

labeling information, sending out a ‘‘Dear Health Care Professional’’ letter, or reevaluating an approval decision.

The Adverse Event Reporting System (AERS) is a computerized information database designed to support the U.S. Food and Drug Administration’s (FDA) postmarketing safety surveillance program for all approved drug and therapeutic biologic products. The ultimate goal of the AERS is to improve the public health by providing the best available tools for storing and analyzing safety reports. The FDA receives adverse drug reaction reports from manufacturers as required by regulation. Health-care professionals and consumers send reports voluntarily through the MedWatch program. These reports become part of a database. The structure of this database is in compliance with the International Conference on Harmonisation (ICH) international safety reporting guidelines (ICH E2B). The guidelines describe the content and format for the electronic submission of reports from manufacturers. The FDA codes all reported adverse events using the standardized international terminology of MedDRA (the Medical Dictionary for Regulatory Activities). Among the AERS system features are the on-screen review of reports, searching tools, and various output reports. The FDA staff use AERS reports in conducting postmarketing drug surveillance and compliance activities and in responding to outside requests for information. The AERS reports are evaluated by clinical reviewers in the Center for Drug Evaluation and Research (CDER) and the Center for Biologics Evaluation and Research (CBER) to detect safety signals and to monitor drug safety. They form the basis for further epidemiologic studies when appropriate. As a result, the FDA may take regulatory actions to improve product safety and protect the public health such as updating a product’s This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/aers/default.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

ADVISORY COMMITTEES Advisory committees provide the U.S. Food and Drug Administration (FDA) with independent advice from outside experts on issues related to human and veterinary drugs, biological products, medical devices, and food. In general, advisory committees include a Chair, several members, plus a consumer, industry, and sometimes a patient representative. Additional experts with special knowledge may be added for individual meetings as needed. Although the committees provide advice to the FDA, final decisions are made by the Agency. Nominations for scientific members, consumer, industry, and patient representatives originate from professional societies, industry, consumer and patient advocacy groups, the individuals themselves, or other interested persons. Candidates are asked to provide detailed information regarding financial holdings, employment, research grants and contracts, and other potential conflicts of interest that may preclude membership. Persons nominated as scientific members must be technically qualified experts in their field (e.g., clinical medicine, engineering, biological and physical sciences, biostatistics, and food sciences) and have experience interpreting complex data. Candidates must be able to analyze detailed scientific data and understand its public health significance.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/oc/advisory/vacancies/ acvacfaq.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

AIDS CLINICAL TRIALS GROUP (ACTG)

recognizing that a CD4 cell is infected and destroying it. After a burst of viral replication that occurs immediately after people are initially infected, viral loads drop to a lower ‘‘set point.’’ The effects on the immune system are gradual, so individuals often are unaware they are infected for years. If left untreated, the disruption of the immune system by HIV infection is catastrophic. This leaves people vulnerable to a long list of opportunistic infections (OIs), wasting, neurologic damage, and cancers that people with an intact immune system do not normally have. When this state is reached, people are considered to have AIDS. HIV was initially identified in 1981 when a small but growing number of individuals in cities in the United States and Europe were diagnosed with unusual and rapidly fatal infections and cancers. Once the syndrome was described and when a blood test for antibody to it became available in 1983, it became apparent that this infection already existed worldwide at pandemic proportions. In 2005, it was estimated that 33.4 to 46.0 million people were living with HIV worldwide, 3.4–6.2 million became newly infected, and 2.4–3.3 million people had already died from HIV/AIDS (1).

JANET W. ANDERSEN ACTG Statistics and Data Management Center, Harvard School of Public Health, Boston, Massachusetts,

This article provides a brief overview of HIV and AIDS, a short history of the founding of the ACTG, an overview of the structure of the ACTG, and sections that outline the major areas covered by the agenda of the ACTG, including clinical trial design issues that developed in the context of the studies and science that was developed to answer this agenda. The ACTG website (www.aactg.org) provides brief overviews of Group studies open to enrollment, information about the ACTG and clinical trials in general, and links to a vast array of sites with information about HIV and AIDS. 1

A BRIEF PRIMER ON HIV/AIDS

Acquired immunodeficiency syndrome (AIDS) is a collapse of the immune system caused by infection with the human immunodeficiency virus (HIV). HIV is a retrovirus; a retrovirus is a virus whose genetic material is RNA that is read into DNA (or reverse transcribed to DNA) when the virus enters the target cell. This DNA incorporates itself into the DNA in the host cell’s nucleus, and then it is replicated to produce new retrovirus particles. HIV is passed from person to person primarily by direct blood transfer from transfusions or needles, by unprotected sexual activity, or from a mother to her child at birth or through breast-feeding. HIV’s target cell in humans is the CD4 or helper T-cell. CD4 cells are the part of the blood’s lymphocytes or white blood cells that are responsible for organizing and arousing (activating) the other cells of the immune system to produce antibodies against an invader or to attack a foreign cell directly. HIV infection of the CD4 cells results in their destruction, either directly by the virus or from the immune system

2

ACTG OVERVIEW

2.1 History In 1986, the National Institutes of Allergy and Infectious Diseases (NIAID) established the Division of AIDS (DAIDS). In that same year, NIAID established the AIDS Treatment Evaluation Units, and, in 1987, they established the AIDS Clinical Studies Groups to conduct studies of treatment for AIDS and HIV. In 1988, the AIDS Clinical Trials Group (ACTG) was formed by the merger of these two groups. Two original units had a primary focus in the treatment of children with HIV. This component of the ACTG agenda grew as the Network grew. In 1997, NIAID funded the Pediatric AIDS Clinical Trials Group (PACTG) that was constituted from the sites in the ACTG that specialized in studies in pediatric HIV and prevention of mother-tochild transmission (MTCT). At that time, the

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

AIDS CLINICAL TRIALS GROUP (ACTG)

ACTG was renamed the Adult AIDS Clinical Trials Group (AACTG) to reflect the change in its focus (2). In 2006, the NIAID and DAIDS initiated a wide-ranging reorganization of the AIDS Networks in the United States including the vaccine and prevention networks as well as those like the AACTG that focused on treatment of the disease. In the grant period between 1999 and 2006, the AACTG had increased its agenda to include international membership engaged in both common and country-specific clinical research. In the reorganization of the networks, the AACTG defined and expanded this international effort, restructured its scientific committees, and integrated the Oral HIV/AIDS Research Alliance (OHARA) that studies oral manifestations and complications of HIV. To reflect this revised and refocused agenda, the AACTG reassumed the name ‘‘AIDS Clinical Trials Group’’ and the acronym, ACTG. This overview will refer to the Group as the ACTG throughout.

•

•

2.2 ACTG Structure The infrastructure required for the success of a network of the size of the ACTG is massive, interlocking, and complex. The general structure of the ACTG has been very stable over time, consisting of the following: • The Group Chair, Vice-Chair, and Exec-

utive Committee. The ACTG Executive Committee (AEC) is constituted of Group leadership and representatives from the Scientific Committees, community, sites, Operations Center, Statistical and Data Management Center (SDMC), and DAIDS. Subcommittees of the AEC include one that evaluates the performance of the Group and its sites, and the Scientific Agenda Steering Committee that coordinates and oversees the scientific agenda of the Group. The Principal Investigators of each site are an essential component of the leadership of the Group. • The Operations Center that is sited at Social and Scientific Systems (Silver Springs, MD). This center provides administrative support and coordination for

•

•

leadership and scientific activities, protocol development and implementation, site establishment, regulatory activities, and contract management. The SDMC that is constituted of the Statistical and Data Analysis Center (SDAC) sited at the Harvard School of Public Health (Boston, MA) and the Data Management Center (DMC) sited at the Frontier Science Technology and Research Foundation, Inc. (Amherst, NY). SDAC members provide leadership and expertise in protocol design and implementation, and they prepare both interim and final analyses of studies. SDAC members also conduct methodological research essential to HIV studies. DMC members design the study forms, monitor data and specimen submission and quality, conduct training for sites, and work closely with sites on data issues. The Network Laboratory Committee that is constituted of laboratories studying virology (quantification of HIV and other viruses; testing of in vitro and genetic drug resistance), immunology (measurement of cytokines, lymphocyte proliferation, and cellular immune responses), and pharmacology (measurement of parameters of drug distribution and elimination and drug bioavailability). The laboratories also develop innovative assays and methodology in this changing field. The Network Community Advisory Board and the community at large bring community perspectives and input to the Group’s agenda as a whole and to issues related to specific research areas and protocols. Community representatives are members of ACTG leadership groups as well as of all protocol teams. The sites, site investigators and staff, and subjects who are the backbone of the research.

3 ACTG SCIENTIFIC ACTIVITIES The scientific agenda of the ACTG is carried out through its scientific committees. It is

AIDS CLINICAL TRIALS GROUP (ACTG)

impossible to detail the activity of the ACTG since its inception in this brief overview. A few studies will be mentioned to illustrate the nature and breadth of the integrated research conducted by the Group. References are provided where results are given but not for every study number cited. A search engine such as PubMed (3) can be used to identify ACTG publications by topic and/or study number. Information on current studies is on the ACTG website. 4 DEVELOPMENT OF POTENT ANTIRETROVIRAL THERAPY (ART) Treatment of HIV ultimately depends on preventing components of the retroviral lifecycle. Thus far, no evidence suggests the eradication of progressive HIV disease. However, the development of drugs and combination ART has resulted in prolonged survival and profound improvements in quality of life when drugs are taken as prescribed. The numbers and types of drugs available have mirrored the increasing understanding of the structure and functions of the virus and its components. Although active in early drug development, an essential role of a large, multicenter network like the ACTG is the ability to complete large comparative (i.e., Phase III and IV) studies quickly to make the results about optimal treatments available to the clinical community. Once HIV was identified as a retrovirus, nucleosides such as zidovudine (ZDV), didanosine (ddI), lamivudine (3TC), and zalcitabine (ddC) were developed that interfere with the reverse transcription of the viral RNA into DNA. The class of drugs is called nucleoside reverse transcriptase inhibitor (NRTI). The ACTG conducted pivotal studies that led to the approval of these drugs as monotherapy and also identified the emergence of drug resistance that limits long-term efficacy. To illustrate the scope of the epidemic, ACTG 019 that tested ZDV versus placebo enrolled 3222 subjects in 2 years. ACTG 175 (4) was a randomized, doubleblind, placebo-controlled study of ZDV, ddI, ZDV + ddI, or ZDV + ddC, which demonstrated the superiority of combination NRTIs over monotherapy. Numerous studies investigated other NRTIs and combinations.

3

A second class of drugs, the nonnucleosides or NNRTIs such as nevirapine (NVP) and efavirenz (EFV), was then developed. ACTG 241 (5) was the first study of a 2-class 3-drug regimen (ZDV, ddI, and NVP) that showed the superiority of multiclass therapy. ACTG 364 (6) extensively evaluated EFV + NVP + 2 NRTIs, which showed this arm to be superior to three-drug regimens in subjects with prior NRTI experience. For HIV to replicate, a long protein must be cleaved by an enzyme (a protease) encoded by its genes. The protease inhibitors (PIs) were the next class of effective drugs to be developed. ACTG 320 (7) was a turning point in HIV research. 3TC or 3TC plus the PI indinavir (IDV) were added to ZDV in ZDV-experienced subjects. A data safety monitoring board (DSMB) halted the study early because of the superiority of the IDV arm in preventing AIDS-defining events (OIs, cancer, etc.) or death. Data and experience from this trial, which followed both clinical events and HIV viral loads with a newly developed assay, also provided convincing evidence that quantitative measurement of the HIV virus was a highly relevant and reliable early surrogate endpoint. Subsequent studies investigated increasingly active combination therapy. For example, ACTG 384 was a partially blinded, randomized, 2 × 3 factorial design in which the first factor was two different NRTI backbones, and the second factor was the use of nelfinavir (NFV, a PI) or EFV or both. This study specified second-line regimens to be used if the initial failed; primary endpoints included both the results of the initial regimen and comparisons of consecutive treatments (8). This study demonstrated the superiority of one of the six arms (ZDV/3TC + EFV), that ZDV/3TC in combinations was more effective and safer than ddI + d4T, and that a four-drug regimen was not better than the three-drug regimen (9,10). These findings led to changes in U.S. ART guidelines. Some Phase III studies evaluate the equivalence or noninferiority of new drugs or drug combinations compared with standard regimens. A5095 investigated three PI-sparing regimens hoping to preserve the class for those who failed first-line therapy. It compared a control arm of 3 NRTIs,

4

AIDS CLINICAL TRIALS GROUP (ACTG)

including abacavir (ABC), with two arms that contained NRTIs + EFV. The primary aims were to demonstrate: the noninferiority of ABC/3TC/ZDV to 3TC/ZDV/EFV, the superiority of ABC/3TC/ZDV/EFV to 3TC/ZDV/EFV, and the superiority of ABC/3TC/ZDV/EFV to ABC/3TC/ZDV. At a prespecified review and based on the statistical design that included guidelines for early evaluation of the noninferiority endpoint (11), the DSMB recommended discontinuation of the three-NRTI arm because of inferior results and suggested continuation of the EFV-containing arms (12). This study illustrates numerous issues involved in the design and management of equivalence and noninferiority studies, and it is an example of efficient study design. As the number of drugs in all classes and the surety of HIV control in a high proportion of ART-na¨ıve subjects increased, tablets that combine highly active drugs were formulated, including a ‘‘one pill once-a-day’’ regimen. A5175 is a three-arm, noninferiority study of a once-daily PI-containing and once-daily and twice-daily PI-sparing regimen, which are both primarily targeted for resource limited settings (refrigeration not required) including sites in Africa, India, South America, and Thailand, although sites in the United States are also participating. A5202 is a four-arm equivalence study of once-daily regimens. Although the arms form a factorial design (PI vs. NNRTI as one factor, and the NRTI backbone as the other), the sample size is fully powered for between-arm equivalence comparisons to guard against a between-factor interaction that could limit the interpretability of the results. Studies conducted internationally must consider numerous other factors, such as differences in host and viral genomics; socioeconomic issues, including communication, transportation and laboratory capabilities; malnutrition; the social stigma of HIV infection; and co-infections such as tuberculosis (TB) and malaria. 4.0.1 Side Effects. No regimen is without adverse effects. Some side effects of ART, such as nausea, diarrhea, reduced blood counts, and impaired liver and kidney function, can be managed with dose and

drug changes as well as supportive care. Others are more complex, including ARTrelated neuropathy and metabolic complications, such as disfiguring lipoatrophy and fat redistribution, mitochondrial toxicity, and extremely high cholesterol levels. The first challenges for the ACTG were developing definitions of these new conditions and understanding their etiologies (13). In fact, some of the first investigations involved establishing whether the metabolic complications were a direct effect of ART or a result of dysregulation from the longer-term survival with HIV that ART was making possible. The ACTG has conducted numerous studies to investigate the underlying cause of these conditions and also to prevent and treat them. The use of substudies, which are intensive investigations on a subset of subjects in a large study, is very powerful. For example, A5097s investigated the neuropsychological effects of EFV in A5095 (14), and early effects were found such as ‘‘bad dreams’’ during the first week of therapy. 4.0.2 Viral Load Data, Complex Endpoints. Both the development of sensitive assays to measure the HIV viral load and the advent of fully suppressive ART resulted in the need for innovative methods for study design and analysis. All tests for viral loads as well as many other markers have a lower limit of reliable quantification; a result below this limit does not mean virus has been eliminated. SDAC members have been at the forefront of development for methods for such ‘‘left censored’’ data (15), combined correlated endpoints (16), and the use of surrogate endpoints in clinical trials (17). 4.0.3 Drug Resistance. HIV replication is highly error prone, which results in a high rate of viral mutation. A change in just one base in HIV’s RNA is sufficient to confer resistance not only to a specific drug but also to other drugs from the same class. ACTG researchers have provided breakthroughs in understanding, identifying, and quantifying drug resistance, including extensive modeling to uncover complex patterns of mutations that affect viral control and the use of predicted drug susceptibility to guide regimen choice. Salvage studies such as A5241

AIDS CLINICAL TRIALS GROUP (ACTG)

and treatment options after virologic failure included in front-line studies like A5095 base treatment assignment on an individual’s virus’ drug resistance profile. Additionally, the possibility of resistance developing in the mother’s virus from brief treatment with ART during pregnancy to prevent MTCT and implications for the future treatment of the mother are considered in several studies (A5207, A5208). 4.0.4 ALLRT. As survival with HIV has increased, long-term follow-up of subjects with standardized data and sample collection beyond participation in an individual study became vital. The ACTG mounted the ALLRT (AIDS Longitudinally Linked Randomized Trials or A5001) in 2000 with an accrual target of at least 4500 enrollees. Subjects in targeted, large, randomized ART studies are followed on the ALLRT during and far beyond participation in their initial study. Data available from any concurrently enrolled ACTG study are merged with data collected specifically for the ALLRT, which results in a complete longitudinal database. Numerous analyses that involve long-term response, immunologic recovery, quality of life, complications of HIV and therapy, co-infection, and epidemiologic design have resulted or are underway. 4.1 Drug Development, Pharmacology (PK), and Pharmacogenomics A large component of the ACTG’s activities is the initial evaluation of new therapies for HIV, from the earliest days of ddI, to intensive investigations of novel modalities such as entry inhibitors that target one specific molecule on the surface of the CD4 cell (A5211). Some of the earliest stages of drug development involve establishing the pharmacology of a drug: How fast is it absorbed and eliminated? Even brief monotherapy quickly results in viral drug resistance as noted earlier, so studies of individual drugs are performed in HIV-uninfected volunteers. Subjects with HIV are taking at least three antiretrovirals in their ART. Thus, any PK study in HIV-infected subjects is ultimately a study of drug–drug interactions, either comparing the PK of the target drug between

5

monotherapy in HIV uninfected volunteers and when added to ART in HIV-infected subjects, or comparing the PK of one or more of the ART drugs before and after receipt of the new agent. These studies are essential because one drug can cause another either to be ineffective or to reach toxic levels at normal dosing. Subjects with HIV also receive numerous medications to prevent and treat AIDSdefining illnesses and side effects of ART. Thus, the ACTG also conducts essential studies of interactions between antiretrovirals and other medications. For example, simvastatin is often prescribed to combat ARTrelated dislipidemia. A5047 (18) provided crucial evidence of a life-threatening increase in simvastatin levels when given with RTV plus saquinivir, which is a PI, whereas A5108 (19) established that it was safe when given with EFV. As well as population-based analyses, determination of the genetic basis of drug metabolism as well as other biologic systems is essential both to individualize treatment and also to explain and ultimately predict outcome. In 2002, the ACTG launched a unique study, which is called A5128 (20), to collect informed consent and samples specifically for human genetic data to be linked to clinical data in a highly confidential manner. An example of the power of both A5128 and the interrelationship of ACTG studies is a merge of data from A5128, A5095, and A5097s that determined that a specific allelic variant in a liver enzyme gene that is more common in African Americans is associated with much greater EFV exposure (21). The ACTG conducts some classic doseescalating Phase I studies, but it has performed numerous studies that test the efficacy of new drugs or, perhaps more importantly, identify inefficacious or problematic drugs and combinations. ACTG 290 and 298 confirmed laboratory findings that ZDV and d4T are antagonistic, which led to ART guidelines that they not be used together. ACTG 307 and A5025 demonstrated that hydroxyurea was inefficacious. Additionally, the latter also showed that the risk of pancreatitis was increased when ddI and d4T were given together, which led to the recommendation not to combine them. Phase II

6

AIDS CLINICAL TRIALS GROUP (ACTG)

evaluations of strategies for multidrug class experienced subjects lead to studies of ‘‘class sparing’’ front-line regimens, and research on studies such as ACTG 398 demonstrated the importance to drug resistance of ‘‘minority’’ variants in the HIV genome. 4.2 Immunology, Pathogenesis, and Translational Research Ultimately, central issues in HIV infection are the failure of the individual’s immunity to mount an effective defense when the person first became infected and the inability of the immune system to clear the virus over time. The ACTG’s research in this area includes investigations in immunity and its regulation, immune-based therapies to enhance anti-HIV immunity after infection, immune restoration with ART, and special issues such as reservoirs where virus can lie latent and unique challenges and opportunities in specific populations. 4.2.1 HIV-Specific Immunity. It was not clear whether the inability of the immune system to recognize and destroy the HIV virus was caused solely by the progressive loss of both na¨ıve (unprimed) and memory (lymphocytes that remain after an immune response) antigen-reactive subsets of the CD4 cells, or whether wider intrinsic or HIVrelated defects occurred in cellular immunity. Clinical and basic research informs each other and both were critical in understanding the interaction of complex biological systems involved in the immune disregulation in this disease. Studies in the pre-ART era (e.g., ACTG 137 and 148) found some immunological effect but no clinical benefit of vaccination with proteins derived from the virus’ outer coating, which is a strategy effective in other diseases. Once CD4 counts were being restored with ART, the logical step was to investigate strategies to boost HIV-specific immunity as an adjunct to the initiation of ART and in synergy with successful ART. This strategy required the ACTG to pursue innovative basic research in the underlying biological mechanisms of cellular immunity; the nature of the defects, which includes both those caused by the virus and those lingering despite viral suppression and the level

of immunocompetency required for any given immune-based strategy to be successful. This research also required an intensive effort in the development of specialized laboratories, standardization and validation of research assays, and development of a central specimen repository to ensure proper storage and that samples could be retrieved long after study closure. 4.2.2 Immune Restoration. Once reliable suppressive ART had been defined, it became important to study whether the immunodeficiency caused by HIV is reversible and what factors are associated with the degree and pace of reconstitution. ACTG studies 315 and 375 followed subjects over 6 years of successful ART, finding that most failed to restore normal CD4 counts and function (22). Intensive studies of clinical (A5014, A5015), immunologic (DACS 094, DACS 202), and human genetic (NWCS 233) parameters could not identify strong predictors of who would and would not achieve immunologic reconstitution among subjects who had sustained virologic control on ART. Studies such as ACTG 889 found that recovery of immunity to pathogens other than HIV is also incomplete, which indicated not only that patients might still be susceptible to OIs despite a CD4 increase, but also that the effectiveness of anti-HIV immune therapy might need to be investigated in populations with a relatively intact immune system. Although consistent predictors of immunologic reconstitution have not yet been identified, the Group has produced extensive cutting-edge research in immune activation and modulation, CD4 cell subsets, pathogenspecific immunity, thymic activity, and HIV-1 pathogenesis. 4.2.3 Immune-Based Therapy. Because HIV control alone is insufficient to restore a normal immune system, the ACTG has investigated immune-based treatments as adjuncts to ART including immunomodulators (e.g., growth factors, cyclosporin A, IL-2, and IL12) and therapeutic vaccination with more general polyvalent peptide vaccines (A5176) and also those based on viral vectors (A5197). Future studies might include infusions of autologous cells primed ex vivo

AIDS CLINICAL TRIALS GROUP (ACTG)

with autologous HIV RNA. The ‘‘analytical treatment interruption’’ (ATI) is a unique and powerful tool for evaluating immunomodulation of HIV. After establishment of sustained viral suppression and with careful criteria for CD4 levels, all ARVs are discontinued. HIV viral rebound is closely monitored and ART is restarted if rebound is rapid or the CD4 count drops. Successful immunomodulation provides a substantial period of time with minimal viral replication and with sustained CD4 counts. ATI provides rapid readout to test vaccines and other immune-based modalities. 4.2.4 Latent Reservoirs, Special Populations. Understanding HIV has required innovative research into underlying differences between populations. An intensive study (A5015) included measurement of thymic volume as well as serial quantification of numerous T-cell and serum markers of immune function. Subjects were 13–30 years of age or at least 45 years old. Not only did the study identify age-associated differences in immune restoration with ART, but also numerous new findings were made in the biology of immune restoration. Genomic research is covered elsewhere, but it is an important component of understanding subpopulations. PK studies in women consider interactions between ARVs and contraceptives, and other studies evaluate other aspects of gender differences (e.g., do women and men with the same body mass index have different PK?). HIV can lie latent in resting cells, and several intensive studies consider viral reservoirs other than the blood such as lymph nodes, cerebrospinal fluid, mucosal tissue, and the genital tracts of men and women to understand how to eradicate all HIV viruses to prevent resurgence. 4.3 OIs and other AIDS-Related Complications As noted above, the immune system collapse caused by prolonged infection with HIV makes individuals susceptible to a long list of diseases that are very rare in people with an intact immune system and that often have an unusually rapid course with AIDS. The Centers for Disease Control, the World

7

Health Organization, and other groups have developed a list of AIDS-defining conditions (23). These conditions include the following: fungal infections (candidiasis, Pneumocystis carinii pneumonia, and cryptococcus); bacterial infections [Mycobacterium avium complex (MAC), and TB]; viral infections (cytomegalovirus); wasting; neurological complications [progressive multifocal leukoencephalopathy (PML), dementia, peripheral neuropathy (PN)]; and cancers [Kaposi’s Sarcoma (KS), lymphomas, cervical cancer].

4.3.1 OIs. Before effective ART was developed, numerous trials in the ACTG were performed to study prevention and treatment of these life-threatening diseases and conditions. An example of the prevalence of these unusual conditions in AIDS is ACTG study 196 that tested treatment for MAC, which enrolled over 1200 subjects in 9 months from late 1993 to early 1994. The role of the ACTG in drug development is evident in sequences like ACTG 196 that demonstrated the superiority of clarithromycin over standard therapy (24) followed by ACTG 223 that demonstrated the superiority of clarithromycin plus two other drugs (25). Similar sequences of studies in other OIs lead to enhanced quality of life and prolonged survival for subjects waiting successful drugs for HIV. The development of highly effective ART in the late 1990s produced a paradigm shift. Several OI studies were halted or completely refocused when it became apparent that immune reconstitution with ART dramatically reduced OI incidence. For example, a planned interim analysis of ACTG 362, which is a large three-arm study of prevention of MAC, found only two events when many were anticipated. The study was continued for extended follow-up to focus on cardiovascular, metabolic, and neurologic changes, and the rates of AIDS-defining events in these subjects who once had a very low CD4 count (<50), and has produced numerous important findings. More recent studies examine the timing of ART, for example, whether immediate or deferred ART is beneficial in HIVinfected people with very low CD4 counts who present with a life-threatening OI (A5164).

8

AIDS CLINICAL TRIALS GROUP (ACTG)

4.3.2 Cancer. Uncontrolled infection with HIV causes a 100-fold increase in cancers like KS and non-Hodgkin’s lymphoma. KS is a rare tumor of the skin and connective tissue previously observed in elderly men of Mediterranean descent. Between 1987 and 1998, the ACTG enrolled subjects in 15 KS studies, generally evaluating conventional cancer chemotherapy in unconventional low doses in these patients who were debilitated from wasting, OIs, and a failed immune system. Oncology groups were also conducting trials in AIDS-related cancers. The National Cancer Institute funded the AIDS Malignancy Consortium (26) in 1995 to consolidate this effort. Since then, the ACTG has studied cancers involving human papillomavirus (HPV), including cervical cancer and, more recently, has studies of vaccines to prevent HPV infection in HIV-infected individuals. 4.3.3 Neurology. The neurological effects of HIV are complicated, which include effects of the virus itself (dementia, PN), OIs such as PML and toxoplasmosis, and drugrelated toxicity. As with other complications, early studies examined the use of conventional treatment in severely debilitated and immune-compromised patients. It was also important to ascertain whether conditions such as dementia and PN were similar to that observed in other settings (e.g., Alzheimer’s and diabetes, respectively). Nearly all studies have both treatment and developmental components. Diagnosis, confirmation of response, and research in basic biology in this setting are complicated and often invasive, which require specialized scans and samples of cerebrospinal fluid (CSF), nerves, muscles, and the brain. The need for special expertise in all these areas led to the founding of the Neurologic AIDS Research Consortium (NARC) (27) in 1993. The ACTG and NARC work closely in joint studies that combine therapeutics with extensive scientific investigations. For example, despite being a negative study with respect to treatment, ACTG 243 definitively established that measurement of JC polyomavirus in the CSF can be used for diagnosis of PML, making brain biopsy unnecessary (28). ACTG 700, which was a substudy of 301 that showed a drug active in Alzheimer’s did not have the same activity in HIV cognitive

impairment, extensively evaluated the use of brain magnetic resonance spectroscopy in biopsy-free diagnosis and in research in complex brain metabolism and neuronal activity (29).

4.3.4 Co-Infections. With prolonged survival with ART, co-infection with Hepatitis C (HCV) and B (HBV) viruses has emerged as a significant problem in people with HIV. Evidence suggests that hepatitis progression to liver failure or cancer is faster with HIV, and ART is more difficult to manage in those with poor liver function. The primary treatment for HCV in monoinfection (just HCV) is various forms of Interferon, which is a drug that affects the immune system. A5071 (30) demonstrated the superiority of the pegylated form of Interferon (PEG) in HCV/HIV coinfection, but it also showed that the durability of viral suppression is far lower. A5178 tests weight-adjusted dosing to improve viral control and the use of PEG in those without viral suppression. Biopsy, which is invasive and expensive, is the ‘‘gold standard’’ for assessing liver damage (fibrosis). Several ancillary studies associated with the Group’s hepatitis protocols examine new markers for fibrosis and statistically model panels of standard and these new clinical parameters to develop alternatives to biopsy. International ACTG studies are evaluating HBV in Africa where the disease is more common than in the United States. Coinfections with other pathogens such as TB and malaria occur at high rates in some countries. The standard regimen for TB is generally made up of four drugs. Subjects who initiate treatment for TB and HIV together are starting seven to eight drugs they may never have taken before, each with side effects and with the complexity of managing all the dosing. A5221 is an international study that evaluates immediate versus delayed ART start in subjects who initiate treatment for TB to test whether phased start is superior to starting all the drugs together. This setting is an example of the difficulty of managing HIV in areas with high rates of other endemic serious diseases.

AIDS CLINICAL TRIALS GROUP (ACTG)

5

EXPERT SYSTEMS AND INFRASTRUCTURE

The infrastructure required for the success of a group of this size is massive and interlocking. Systems had to be developed de novo. For example, the DMC developed the Laboratory Data Management System, which is a system that tracks a sample from subject to freezer to repository to assay lab and transfers the sample’s data from the assay lab to the database. Expert systems have been developed by the SDMC to enable web-based transfer of viral genomic data and provide rapid sequence alignment and quality assurance. Both of these systems are used by other networks. The operations center and DMC have moved the ACTG to a ‘‘paperless’’ system, in which essential documents are downloaded from secure websites. Large contracts and regulatory groups ensure seamless study conduct. Training of investigators and site personnel is a constant focus, possibly all the more important as international sites without prior clinical trial experience join the ACTG.

7.

8.

9.

10.

REFERENCES 1. Joint United Nations Programme on HIV/ AIDS. AIDS epidemic update, 2005. (data. unaids.org/pub/GlobalReport/2006/2006 GR ch02 en.pdf) 2. aidshistory.nih.gov/timeline/index.html. 3. PubMed. www.ncbi.nlm.nih.gov/entrez/query. fcgi?DB=pubmed. 4. Hammer SM, Katzenstein D, Hughes MD, Gundacker H, Schooley RT, Haubrich RH, Henry WK, Lederman ML, Phair JP, Niu M, Hirsch MS, Merigan TC. A trial comparing nucleoside monotherapy with combination therapyn in HIV-infected adults with CD4 cell counts from 200 to 500 per cubic millimeter. N. Engl. J. Med. 1996; 335: 1081–1090. 5. D’Aquila RT, Hughes MD, Johnson VA, Fischl MA, Sommadossi JP, Liou SH, Timpone J, Myers M, Basgoz N, Niu M, Hirsch MS. Nevirapine, zidovudine and didanosine compared with zidovudine and didanosine in patients with HIV-1 infection. Ann. Intern. Med. 1996; 124: 1019–1030. 6. Albrecht MA, Bosch RJ, Hammer SM, Liou SH, Kessler H, Para MF, Eron J, Valdez H, Dehlinger M, Katzenstein DA. Nelfinavir,

11.

12.

13.

9

efavirenz or both after the failure of nucleoside treatment of HIV infection. N. Engl. J. Med. 2001: 345: 398–407. Hammer SM, Squires KE, Hughes MD, Grimes JM, Demeter LM, Currier JS, Eron JJ, Feinberg JE, Balfour HH, Deyton LR, Chodakewitz JA, Fischl MA. A controlled trial of two nucleoside analogues plus indinavir in persons with human immunodeficiency virus infection and CD4 cell counts of 200 per cubic millimeter or less. N. Engl. J. Med. 1997; 337: 725–733. Smeaton LM, DeGruttola V, Robbins GK, Shafer RW. ACTG (AIDS Clinical Trials Group) 384: a strategy trial comparing consecutive treatments for HIV-1. Control. Clin Trials 2001; 22: 142–159. Robbins GK, De Gruttola V, Shafer RW, Smeaton LM, Snyder SW, Pettinelli C, Dub´e MP, Fischl MA, Pollard RB, Delapenha R, Gedeon L, van der Horst C, Murphy RL, Becker MI, D’Aquila RT, Vella S, Merigan TC, Hirsch MS; AIDS Clinical Trials Group 384 Team. Comparison of sequential three-drug regimens as initial therapy for HIV-1 infection. N. Engl. J. Med. 2003: 349: 2293–2303. Shafer RW, Smeaton LM, Robbins GK, De Gruttola V, Snyder SW, D’Aquila RT, Johnson VA, Morse GD, Nokta MA, Martinez AI, Gripshover BM, Kaul P, Haubrich R, Swingle M, McCarty SD, Vella S, Hirsch MS, Merigan TC; AIDS Clinical Trials Group 384 Team. Comparison of four-drug regimens and pairs of sequential three-drug regimens as initial therapy for HIV-1 infection. N. Engl. J. Med. 2003: 349: 2304–2315. Ribaudo HR, Kuritzkes DR, Schackman BR, Acosta EP, Shikuma CM, Gulick RM. Design issues in initial HIV-treatment trials: focus on ACTG A5095. Antivir. Ther. 2006; 11: 751–760. Gulick RM, Ribaudo HJ, Shikuma CM, Lustgarten S, Squires KE, Meyer WA 3rd, Acosta EP, Schackman BR, Pilcher CD, Murphy RL, Maher WE, Witt MD, Reichman RC, Snyder S, Klingman KL, Kuritzkes DR. Triple-nucleoside regimens versus efavirenzcontaining regimens for the initial treatment of HIV infection. N. Engl. J. Med. 2004: 350: 1850–1861. Wohl DA, McComsey G, Tebas P, Brown TT, Glesby MJ, Reeds D, Shikuma C, Mulligan K, Dube M, Wininger D, Huang J, Revuelta M, Currier J, Swindells S, Fichtenbaum C, Basar M, Tungsiripat M, Meyer W, Weihe J, Wanke C. Current concepts in the diagnosis and management of metabolic complications

10

AIDS CLINICAL TRIALS GROUP (ACTG)

of HIV infection and its therapy. Clin. Infect. Dis. 2006 43: 645–653. 14. Clifford DB, Evans S, Yang Y, Acosta EP, Goodkin K, Tashima K, Simpson D, Dorfman D, Ribaudo H, Gulick RM. Impact of efavirenz on neuropsychological performance and symptoms in HIV-infected individuals. Ann Intern Med 2005; 143: 714–721. 15. Hughes M. Analysis and design issues for studies using censored biomarker measurements, with an example of viral load measurements in HIV clinical trials. Stat Med. 2000; 19: 3171–3191. 16. DiRienzo AG, De Gruttola V. Design and analysis of clinical trials with a bivariate failure time endpoint, with application to AIDS Clinical Trials Group Study A5142. Control Clin Trials 2003: 24:122-34. 17. Hughes MD. Evaluating surrogate endpoints. Control Clin. Trials 2002; 23: 703–707. 18. Fichtenbaum CJ, Gerber JG, Rosenkranz SL, Segal Y, Aberg JA, Blaschke T, Alston B, Fang F, Kosel B, Aweeka F. Pharmacokinetic interactions between protease inhibitors and statins in HIV seronegative volunteers: ACTG Study A5047. AIDS 2002; 16: 567–577. 19. Gerber JG, Rosenkranz SL, Fichtenbaum CJ, Vega JM, Yang A, Alston BL, Brobst SW, Segal Y, Aberg JA. AIDS Clinical Trials Group A5108 Team. Effects of efavirenz on the pharmacokinetics of simvastatin, atorvastatin, and pravastatin; results of AIDS Clinical Trials Group 5108 Study. J. Acquir. Immune Defic. Syndr. 2005; 39: 307–312. 20. Haas DW, Wilkinson GR, Kuritzkes DR, Richman DD, Nicotera J, Mahon LF, Sutcliffe C, Siminski S, Andersen J, Coughlin K, Clayton EW, Haines J, Marshak A, Saag M, Lawrence J Gustavson J, Bennett J, Christensen R, Matula MA, Wood AJJ. A multi-investigator/institutional DNA bank for AIDS-related human genetic studies: AACTG protocol A5128. HIV Clin. Trials 2003; 4: 287–300 21. Haas DW, Ribaudo HJ, Kim RB, Tierney C, Wilkinson GR, Gulick RM, Clifford DB, Hulgan T, Marzolini C, Acosta EP. Pharmacogenetics of efavirenz and central nervous system side effects: an Adult AIDS Clinical Trials Group study. AIDS 2004; 18: 2391–2400. 22. Smith K, Aga E, Bosch R, Valdez H, Connick El, Landay A, Kuritzkes D, Gross B, Francis I, McCune JM, Kessler H, Lederman M.

Long-term changes in circulating CD4T lymphocytes in virologically suppressed patients after 6 years of highly active antiretroviral therapy. AIDS 2004; 18: 1953–1956. 23. 1993 Revised classification system for HIV infection and expanded surveillance case definition for AIDS among adolescents and adults. www.who.int/hiv/strategic/en/cdc.1993 hivaids def.pdf. 24. Benson CA, Williams PL, Cohn DL, Becker S, Hojczyk P, Nevin T, Korvick JA, Heifets L, Child CC, Lederman MM, Reichman RC, Powderly WG, Notario GF, Wynne BA, Hafner R. Clarithromycin or rifabutin alone or in combination for primary prophylaxis of Mycobacterium avium complex disease in patients with AIDS: A randomized, doubleblind, placebo-controlled trial. The AIDS Clinical Trials Group 196/Terry Beirn Community Programs for Clinical Research on AIDS 009 Protocol Team. J. Infec. Dis. 2000; 181: 1289–1297. 25. Benson CA, Williams PL, Currier JS, Holland F, Mahon LF, MacGregor RR, Inderlied CB, Flexner C, Neidig J, Chaisson R, Notario GF, Hafner R: AIDS Clinical Trials Group 223 Protocol team. A prospective, randomized trial examining the efficacy and safety of clarithromycin in combination with ethambutol, rifabutin or both for the treatment of disseminated Mycobacterium avium complex disease in persons with acquired immunodeficiency syndrome. Clin. Infec. Dis. 2003; 37:1234-43. 26. www.niaid.nih.gov/daids/pdatguide/amc. htm. 27. www.neuro.wustl.edu/narc. 28. Yiannoutsos CT, Major EO, Curfman B, Jensen PN, Gravell M, Hou J, Clifford DB, Hall CD. Relation of JC virus DNA in the cerebrospinal fluid to survival in acquired immunodeficiency syndrome patients with biopsy-proven progressive multifocal leukoencephalopathy. Ann. Neurol. 1999; 45: 816–821. 29. Schifitto G, Navia B, Yiannoutsos C, Marra C, Chang L, Ernst T, Jarvik J, Miller E, Singer E, Ellis R, Kolson D, Simpson D, Nath A, Berger J, Shriver S, Millar L, Colquhoun D, Lenkinski R, Gonzalez R, Lipton S. Memantine and HIV-associated cognitive impairment: a neuropsychological and proton magnetic resonance spectroscopy study. AIDS 2007; 21: 1877–1886.

AIDS CLINICAL TRIALS GROUP (ACTG) 30. Chung R, Andersen J, Volberding P, Robbins G, Liu T, Sherman K, Peters M, Koziel M, Bhan A, Alston-Smith B, Colquhoun D, Nevin T, Harb G, van der Horst C. Peg-interferon alfa-2a plus ribavirin versus interferon alfa-2a plus ribavirin for chronic hepatitis c in HIVcoinfected persons. N. Engl. J. Med. 2004; 351: 451–459.

11

ALGORITHM-BASED DESIGNS

of DLTs is open such that any AE from the CTC catalog of grade 3 and higher related to treatment is considered a DLT (4).

WEILI HE Clinical Biostatistics Merck Research Labs Merck & Co., Inc. Rahway, New Jersey

1.1 Algorithm-based A + B Designs The standard methods for conducting phase I cancer trials are the traditional algorithmbased A + B designs that are unaccompanied by close inspection of the statistical properties (2, 5). Our focus here is on the traditional algorithm-based designs; alternative designs, such as continual reassessment method (CRM) (6), decision theoretic approaches (7, 8), and escalation with overdose control (9), are discussed elsewhere in this book. With algorithm-based designs, the trials begin with the selection of a starting dose low enough to avoid severe toxicity but also high enough for a chance of activity and potential efficacy in humans (4). The starting dose is generally chosen based on animal studies, usually one-tenth of the lethal dose in mice or based on the information from previous trials. The dose escalation usually follows a modified Fibonacci scheme with dose escalations of 100%, 65%, 52%, 40%, 33%, 33%, 33%, and so on (4). Large increments occur at early doses, and smaller ones at higher levels. The traditional 3 + 3 design is a special case of an A + B design. With this design, three patients are assigned to the first dose level. If there is no DLT, the trial proceeds to the next dose level with a cohort of three other patients. If at least two out of the three patients experience the DLT, then the previous dose level is considered as the MTD; otherwise, if only one patient experiences the DLT, then three additional patients are enrolled at the same dose level. If at least one of the three additional patients experience the DLT, then the previous dose is considered as the MTD; otherwise, the dose will be escalated. This design scheme is considered a 3 + 3 design without dose de-escalation. The 3 + 3 design with dose de-escalation is essentially similar, but permits more patients to be treated at a lower dose. When excessive DLT incidences occur at the current dose level, another three patients would be treated at the previous dose level if there were only

JUN LIU MRI Unit, Department of Psychiatry Columbia University New York State Psychiatric Institute New York, New York

1

PHASE I DOSE-FINDING STUDIES

The primary goal of a phase I cancer clinical trial is to determine the maximum tolerated dose (MTD) to be used for subsequent phase II and III trials evaluating efficacy (1, 2). The MTD is determined as the highest dose level of a potential therapeutic agent at which patients experience an acceptable level of dose-limiting toxicity (DLT) (3). The acceptable level of DLT is usually referred to as a tolerable or target toxicity level (TTL) (4). It is generally specified at the design stage of a trial by investigators, and is dependent on the potential therapeutic benefit of the drug, which may include but is not limited to the considerations of disease, disease stage, type of drug being tested, and patient population (4). Hence, the determination of the MTD level chiefly depends on the TTL that the investigators assign to the study drug. As with TTL, DLT is specifically defined in a study protocol at the design stage before the start of the study. In the United States, the common toxicity criteria (CTC) of the U.S. National Cancer Institute (NCI) is used (4). There is a large list of adverse events (AEs) subdivided into organ/symptom categories that can be related to the anticancer treatment. Each AE can be categorized into five classes: grade 0, no toxicity; grade 1, mild toxicity; grade 2, moderate toxicity; grade 3, serious/severe toxicity; and grade 4, very serious or life threatening. Usually, a toxicity of grade 3 or 4 is considered the DLT. That identifies a subset of toxicities from the CTC list, and the limits of grading define the DLTs for the investigational drug. Sometimes the list

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

ALGORITHM-BASED DESIGNS

three patients treated at that level previously. Hence, the dose may de-escalate until reaching a dose level at which six patients are already treated, and at most one DLT is observed out of six patients. The MTD is defined as the highest dose level (≥1st dose level) in which six treated patients have at most one patient experiencing the DLT, or among three patients, none have experienced DLT, and the immediate higher dose level having at least two patients experiencing the DLT out of a cohort of three or six patients. If escalation occurs at the last observed dose level during the study, then the MTD is at or above the last dose level. If the trial stops at the first dose, then the MTD is below the first dose level. In either of the above cases, the prescribed dose levels for the trial may need to be altered to determine the MTD. 1.2 Modification of A + B Designs and Derivation of Key Statistical Properties Lin and Shih (10) and Shih and Lin (11) discussed statistical properties of the traditional and modified algorithm-based designs in a general setting: A + B designs with or without dose de-escalation, which include the popular 3 + 3 design as a special case. They provided exact formulas to derive the key statistical properties of these traditional and modified A + B designs, which include (1) the probability of a dose being chosen as the MTD; (2) the expected number of patients at each dose level; (3) the attained TTL (the expected DLT at the MTD found through the algorithm-based design); (4) the expected number of toxicity at each dose level; and (5) the expected overall toxicity in a trial. They realized that, although the traditional A + B design is quite a general family, it still does not include some important variants of the algorithm-based designs that are often encountered in many phase I cancer clinical trials. They referred to experience by Smith et al. (12): ‘‘[P]hase I trials should consider alternative designs that increase a patient’s chance to receive a therapeutic dose of an agent and also provide a better dose estimate for phase II trials.’’ They introduced three modifications of the traditional A + B design. In the traditional A + B design, the previous lower dose level is always declared as

the MTD. The current dose has no chance at all to be declared as the MTD, which is conservative and is a waste of the already scarce information for phase I cancer trails. The first modification they made is to allow the investigator to declare a current dose level as the MTD (M1 A + B design). Because the starting dose in the traditional A + B design is always the lowest dose level in the study protocol, which may be conservative, the second modification they made, in addition to M1, is to start somewhere in the middle of the prespecified dose levels with escalation or de-escalation in the sequence of treatment (M2 A + B design). This M2 design has the advantage of reaching the MTD faster; when the starting dose is too toxic, the dose can de-escalated to avoid stopping the trial or halting it for protocol amendment. The third modification they made is to have a twostage process. In the first stage, only one or two patients are treated per dose level in an increasing dose scheme until the first sign of toxicity incidence (either minor or DLT) is observed, at which time the second stage of a standard A + B design will begin. Lin and Shih derived the exact formulae for the key statistical properties described above for the traditional A + B design and also for M1 to M3 A + B designs. (A computer program to calculate these statistical quantities is provided on the authors’ Website at http://www2.umdnj.edu/∼linyo.) For clinicians who use the algorithm-based A + B design or M1 to M3 A + B designs, Lin and Shih provide them with powerful tools to gain an understanding of the design properties before the start of a trial. 1.3 Advantages and Drawbacks of Algorithm-based Designs There are two schools of thought in designing phase I cancer clinical trials. One treats the MTD as being observed from the data, and the other treats the MTD as a parameter to be estimated from a model. The former approach of treating the MTD as being identifiable from the data can be seen in the algorithmbased A + B designs. These designs have been continuously and widely used, even though many alternative model-based designs have been proposed and advocated in recent years,

ALGORITHM-BASED DESIGNS

such as the continual reassessment method (CRM) (6), random walk rules (13), decision theoretic approaches (14, 15), and escalation with overdose control (16). The chief reason for the broad use of algorithm-based designs is its simplicity in application. Model-based designs (6, 13–16) have to overcome many operational and logistic issues practitioners, such as real-time data collection and process, real-time estimation of model parameters by statisticians, prompt communication of the dose level to practitioners for the next cohort of patients, and logistics for drug supplies to these patients. These operational issues become increasingly more difficult to manage when a phase I study is conducted in multiple study centers. The traditional A + B design is not without its drawbacks. As already mentioned, the chief complaint is that the recommended dose observed from the data from an algorithmbased design has no interpretation as an estimate of the dose that yields a prespecified rate of target toxicity (6). Hence, it has no intrinsic property that makes the study stop at any particular percentile of toxicity or even on average (5). In addition, there is no mechanism in the trial to estimate the sampling error in the identified MTD dose; thus, the recommended dose for phase II will always be an imprecise estimate of the optimal dose (17). Other criticisms of traditional algorithm-based designs include that if the initial starting dose is far below the true MTD, the length of the trial may be long with many patients—especially the patients who entered early into the trial—being treated at a suboptimal dose (5, 12, 18). Smith et al. (12) recommended that the planning stage for future phase I trials should consider alternative designs that increase a patient’s chance to receive a therapeutic dose of a drug and provide a better dose estimate for phase II studies. 2

ACCELERATED DESIGNS

Because of the perceived drawbacks of the traditional algorithm-based A + B designs, a few variants have been proposed and evaluated (11, 18–22). Storer (21) was probably the first to examine the characteristics of

3

the traditional algorithm-based designs. He proposed several variants of the standard designs, including a two-stage procedure. Operating characteristics of the standard design along with stopping rules have been derived (23). In the first stage, one patient is recruited per dose level, and dose escalation continues until the first DLT is observed. After that, accrual to the second stage begins following the standard design. The MTD is established based on the scheme of the standard design. Simon et al. (22) proposed a family of accelerated titration designs similar to Storer’s. Design 1 was a conventional design (similar to the commonly used modified Fibonacci method) using cohorts of three to six patients, with 40% dose-step increments and no intrapatient dose escalation. Designs 2 through 4 included only one patient per cohort until one patient experienced doselimiting toxic effects or two patients experienced grade 2 toxic effects (during their first course of treatment for designs 2 and 3 or during any course of treatment for design 4). Designs 3 and 4 used 100% dose steps during this initial accelerated phase. After the initial accelerated phase, designs 2 through 4 resorted to standard cohorts of three to six patients, with 40% dose-step increments. Designs 2 through 4 used intrapatient dose escalation if the worst toxicity is grade 0 to 1 in the previous course for the patients. Using simulations, Simon et al. (22) showed that accelerated titration designs appear to effectively reduce the number of patients who are undertreated, speed up the completion of phase I trials, and provide a substantial increase in the information obtained. They cautioned that the use of an accelerated titration design requires careful definition of the level of toxicity that is considered dose limiting and of the level considered sufficiently low that intrapatient dose escalation is acceptable. In most phase I clinical trials, however, intrapatient dose escalation may not be allowed when the toxicity profile of the compound being studied is not well known. Otherwise, it may be difficult to tease out the toxicity response at one dose level from the response at another dose level. Dancey et al. (24) also provided a thorough evaluation of the performance of the family of accelerated designs by Simon et al.

4

ALGORITHM-BASED DESIGNS

and conducted a literature search to assess the usefulness of the accelerated designs in the evaluation of novel oncology therapeutics. They reached similar conclusions about the advantages of accelerated designs as Simon’s evaluation. Based on the results of their review of published phase I studies that used the Simon group’s accelerated titration designs, they observed that the use of minimum one-patient cohorts and larger doseescalation steps may be advantageous under the following circumstances: (1) the agent is of a chemical class that has been widely studied; (2) the agent is predicted to have minimal interpatient variability in pharmacokinetics; and (3) the agent’s anticipated toxicity is unlikely to be severe or irreversible and is amenable to close monitoring and supportive interventions. Conversely, there are situations where an accelerated titration design may not provide the optimal balance between safety and efficiency, as either larger numbers of patients/dose cohort and/or smaller dose increments would be preferable. Agents associated with steep dose-response curves for toxicity, severe irreversible toxicity, unexplained mortality in animal toxicology studies, or large variability in doses or plasma drug levels eliciting effects may require alternative designs to balance safety and efficiency optimally. They found that, despite the advantages, the accelerated designs are not widely used, probably due to the conservativeness of investigators. There may be scenarios under the accelerated designs as proposed by Storer and Simon et al. in which the MTD could be declared at a dose level at which only one patient had been studied if the design is without dose de-escalation. This is not desirable because more than one patient should be evaluated at the MTD dose level. It is also possible that the MTD could not be identified in a trial because no DLT was encountered with the acceleration of one patient through the entire prescribed dose levels. This case is also less than optimal because one would want to strive to identify the MTD within the prescribed dose levels in a trial; restarting the trial with altered dose levels prolongs the study duration and requires additional patient and financial resources.

2.1 A Modification of the Accelerated Designs Liu et al. (25) proposed a modified accelerated design in which an MTD dose will need to be tested in a cohort of A + B patients. Further, if no DLT occurs throughout the prescribed doses in the acceleration phase during the trial, then the last dose level will be studied with a cohort of at least A or A + B patients; this way, more patients are evaluated at the last dose level, thus increasing the chance of identifying an MTD dose for the trial. This modified accelerated 1 + A + B design closely approximates that of the M3 A + B design proposed by Shih and Lin (11) except that this design modifies the standard A + B portion of the study to ensure that A + B patients will be studied at the MTD level and that a cohort of at least A or A + B patients will be studied at the last dose level n if no DLT occurrence at the previous n − 1 doses. Liu et al. (25) used the same notations as in Lin and Shih (10). Let A, B, C, D, and E be integers. Notation C/A indicates that there are C toxicity incidences out of A patients and >C/A means more than C toxicity incidences out of A patients. Similarly, the notation
ALGORITHM-BASED DESIGNS

5

Enter one patient No Escalate to dose l + 1*

Exhibit DLT? Yes Switch to A + B Design

Dose Level l
> D/A

≥C/A + ≤D/A Add B patients

≤E/(A + B)

>E/(A + B)

Escalate to Dose Level l + 1* Dose level de-escalate to l − 1**

≤ E/(A + B)


Dose level l −1 is MTD

Add B patients

≤E/(A + B)

= 0/1

Add A-1 Patients

≤ D/A

>E/(A + B)

>D/A

Dose level l − 1

Dose de-escalate to the lower dose level**

* At the last dose level n

Enroll A patients

>D/A

≤D/A

Add B patients To dose level n − 1

>E/(A + B)

≤E/(A + B) No MTD can be identified

** if l − 1 = 0

No MTD can be identified

as compared with the standard A + B design in two case studies. They concluded that the modified accelerated design and the standard design select a dose level as the MTD with

Figure 1. Diagram of modified accelerated design.

similar probabilities. However, the modified accelerated design requires fewer expected numbers of patients because of its ability to accelerate dose escalation through low-dose

6

ALGORITHM-BASED DESIGNS

levels and to switch to the standard design when DLT is encountered. Under certain dose toxicity profiles such as a dose-toxicity response curve that is gradually increasing, the savings in patient resources and time may be tremendous, translating into much shorter studies. For trials with high starting-dose toxicity or with only a few dose levels, the modified accelerated design may not save the number of patients because it rushes through the initial doses to the higher doses and may require dose de-escalation a few more levels down than the standard design. Their modified accelerated design can also be extended easily to allow the flexibility to start a trial at a dose level that may be several levels above the studies prescribed minimum dose. This flexibility gives researchers increased freedom to expedite a trial as needed while having the option to move downward to the lower doses when necessary. 3 MODEL-BASED APPROACH IN THE ESTIMATION OF MTD In cancer clinical trials, a single universal TTL does not appear to exist based on the literature (5, 12, 21, 26). Depending on the type and severity of a disease, a TTL of approximately 33% is suitable for many cancer trials, and a TTL of 50% may also be reasonable when the toxicities are viewed to be less serious (or more acceptable) relative to the severity of the disease being treated. Among the variety of possible TTLs, it would be difficult to use the algorithm-based approach to derive the MTD of certain TTLs as MTD observed from the data from an algorithmbased design has no interpretation as an estimate of the dose that yields a prespecified rate of target toxicity (6). 3.1 Evaluation of the Traditional Algorithm-based 3 + 3 Design The common perception by most clinicians is that the traditional algorithm-based 3 + 3 designs produce a 33% toxicity rate (5). This is not true based on discussions by Lin and Shih (10) and Kang and Ahn’s simulation study (27, 28). Kang and Ahn showed that the expected toxicity rate at the MTD is between

19% and 22% if the dose-toxicity relationship is assumed to be logistic or hyperbolic tangent functions. He et al. (29) also conducted a simulation study to investigate the properties of the algorithm-based 3 + 3 design. Instead of assuming a particular functional form of the dose-toxicity relationship, they only allowed the dose-toxicity relationship to be monotonically nondecreasing. They found that the expected toxicity levels are in the range of 19% to 24% rather than the anticipated 33%, depending on the number of prescribed doses in a study. They observed that the estimate of the expected toxicity level and its associated standard deviation decrease with the increasing number of dose levels prescribed for a study. This is most likely for the following reasons. For the algorithm-based 3 + 3 designs, the MTD is designed as the highest dose level (≥1st dose level) in which either six patients are treated with at most one patient experiencing the DLT or three patients are treated with no patient experiencing the DLT and the immediate higher dose level having at least two patients experiencing the DLT out of a cohort of three or six patients. The expected toxicity level at the MTD that is identified under this design scenario is more likely associated with DLTs between 0% (0/3) to 16.7% (1/6) because at most zero out of three patients or one out of six patients experiences DLT at the MTD level. With increasing number of prescribed dose levels in a study, the accuracy and precision of identifying a MTD that is associated with an expected toxicity level approaching this limit increase. Because it has been shown that the traditional 3 + 3 algorithm-based designs cannot provide accurate estimates of the MTD when TTLs are set high (27–29), He et al. (29) proposed a model-based approach in the estimation of the MTD following an algorithm-based design. 3.2 Model-based Approach in the Estimation of MTD He et al. (29) assumed a one-parameter family model to depict the dose-toxicity relationship and denoted it by ψ(d1 ,a). Let the dose range be D = {d1 , . . ., dk }, where k is the number of dose levels in the study; let S =

ALGORITHM-BASED DESIGNS

{s1 , . . ., sm } be the dose level in the dose escalation or de-escalation steps during the trial, where m is the total number of steps of dose escalation and/or de-escalation before the study stops; and let Y = {y1 , . . ., ym } be the number of patients experiencing DLT events at the corresponding S = {s1 , . . ., sm } steps, and the elements in Y may take on values (0, 1, 2, 3). Under traditional 3 + 3 designs, at each step i three patients are studied at dose level si , where dose level si may or may not be equal to dose level sj at step j (i = j). They derived the likelihood function as follows: L(a) =

m


Pr(yl |sl ) ∝

l=1

m
{ψ(sl , a)}yl l=1

{1 − ψ(sl , a)}

(3−yl )

.

(1)

These derivations can be easily extended to general traditional A + B algorithm-based designs. Let v = {v1 , . . . , vm } be the number of patients studied at the corresponding S = {s1 , . . . , sm } steps, and the elements in v may take on values (A, B) for the traditional A + B designs. Based on the likelihood function in equation 1, the likelihood function for a traditional A + B design can be written as L(a) ∝

m
{ψ(sl , a)}yl {1 − ψ(sl , a)}vl −yl ) .

(2)

l=1

Because the likelihood function in equation 2 is derived based on the data from each step of dose escalation or de-escalation, this likelihood function can also apply to the modified accelerated 1 + A + B designs mentioned in the previous section. In the case of modified accelerated designs, v = {v1 , . . . , vm } may take on values (1, A, B). Further, for any algorithm-based designs, be they an A + B design or an accelerated design or an M1−M3 A + B design (7), the likelihood function can be written as equation 2, with v = {v1 , . . . , vm } taking on possible integer values in (1 to A) or (1 to B). He et al. (29) employed a Bayesian approach for the model inference. They conducted simulations to investigate the properties of their model-based approach in the estimation of the MTD as compared with the MTD identified from a traditional

7

algorithm-based 3 + 3 design. The simulation results demonstrated that their modelbased approach produced much less biased estimates of the MTDs compared with the estimates obtained from the traditional 3 + 3 designs. Although the traditional 3 + 3 designs produce estimates of the MTD that are generally associated with expected toxicity levels of 19% to 24% regardless of any prespecified target toxicity levels, their model-based approach enables a substantially less biased estimate of the MTD based on the data as compared with the traditional 3 + 3 designs. Their method is applicable in any trial that was conducted using an algorithm-based design, when the intended target toxicity level was much higher than 20%, potentially salvaging the trial and probably the whole development program by bringing forward to later phase trials a therapeutic dose for confirmatory efficacy studies. 4 EXPLORING ALGORITHM-BASED DESIGNS WITH PRESPECIFIED TARGETED TOXICITY LEVELS 4.1 Rationale for Exploring the Properties of Algorithm-based Designs As mentioned in the previous section, the chief reason for the broad use of algorithmbased designs is due to their simplicity in application, even though many alternative model-based designs have been proposed and advocated in recent years. Correspondingly, the chief complaint about these designs is that the recommended MTD dose observed from the data from an algorithm-based design has no interpretation as an estimate of the dose that yields a prespecified rate of target toxicity. The properties of the traditional algorithm-based 3 + 3 designs have been studied extensively by many (27–29). Simulation studies have shown that the traditional algorithm-based 3 + 3 designs are associated with expected toxicity levels in the range of 19% to 24% or even less, depending on the actual algorithm and number of prescribed doses in a trial. Other algorithmbased designs have not been studied, and it is not certain what design properties, especially the expected toxicity level, an identified MTD

8

ALGORITHM-BASED DESIGNS

may be associated with at the conclusion of an algorithm-based trial. In the following section, we study the expected toxicity levels of a few A + B and 1 + A + B designs with varying design parameters by simulations.

4.2 Simulation Study Design and Results Suppose that k dose levels from the dose range D = {d1 , . . . , dk } are chosen for a study. Let θ i be the probability of DLT that corresponds to the dose level di (i = 1, . . . , k). We assume that θ i satisfies θ i = ψ(di ), where ψ( ) is any monotonic function with the constraint θ 1 ≤ θ 2 ≤ . . . θ k . Denote ri = P(MTD = di ) as the probability that dose level di is declared as the MTD. Based on the exact formulas of Lin and Shih (10) and exact formulas of Liu et al. (25), ri can be derived for traditional A + B designs with and without dose de-escalation and the modified accelerated 1 + A + B designs. Once ri is obtained for each dose level di , we can derive the expected toxicity level (ETL) at the MTD as

and the corresponding standard deviation as  k−1 k−1   θi2 ri ri − (ETL)2 . SD =  i=1

The ETL is different from the target toxicity level (TTL) in that TTL is an acceptable toxicity level at the MTD prespecified by investigators at the onset of a trial, whereas ETL is the expected toxicity level at the MTD achieved after the conclusion of a trial. For dose levels d1 , . . . , dk , instead of calculating the ETL for one set of fixed θ 1 , . . . , θ k , we generated N = 4000 sets of the corresponding probabilities of the DLT as θ 1 , . . . , θ k from the uniform distribution (0,1) with θ 1 ≤ θ 2 ≤ . . . ≤ θ k through simulation. Then for each of the N sets of the θ 1 , . . . , θ k , ETL is obtained by the exact computation using the formulas by Lin and Shih (10) and Liu et al. (25), respectively, for A + B and 1 + A + B designs. For our simulations, k = 10 is used. The results of the simulations are then pooled together to evaluate the parameter estimates with the following summary statistics:

ETL = P(toxicity at MTD | dose 1 ≤ MTD < does k) =

k−1  i=1

θi ri

k−1 

i=1

ˆ = ETL

N 

 ˆ ETLl /N,

l=1

ri , ˆ l represents the expected toxicity where ETL

i=1

Table 1. Estimated Expected Toxicity Levels Based On Algorithm-Based Designs: Commonly Used Algorithm-Based Designs Design Parameters

Designs

A

B

C

D

E

A+B

1+A+B

3 3 3 3 3 3 3 3 3 3 3 3 3 3

3 3 3 3 3 6 6 6 6 6 6 6 6 6

1 1 1 2 2 1 1 1 2 2 2 2 2 2

1 1 2 2 2 1 1 1 2 2 2 2 3 3

1 2 2 2 3 2 3 4 2 3 4 5 4 5

20% 27% 30% 33% 42% 21% 26% 30% 25% 31% 38% 45% 39% 48%

21% 29% 32% 32% 42% 22% 27% 32% 25% 32% 38% 45% 39% 47%

ALGORITHM-BASED DESIGNS

level at the lth (l = 1, . . . , N) simulation. The ˆ l is defined as standard deviation of ETL N ˆ = SD(ETL)

ˆ

ˆ 2 − ETL) . N−1

l=1 (ETLl

Using the same definitions for A, B, C, D, E, C/A, >C/A, and
9

toxicity levels that are higher than 19% to 24%. This is extremely desirable because algorithm-based designs are still widely used due to their operational simplicity for implementation. Now that the statistical property in terms of the expected toxicity levels has been studied for these designs, practitioners could choose the one algorithm-based design that fits their prespecified target toxicity level. Table 2 provides the estimated ETLs for less commonly used A + B and 1 + A + B designs. Although the variability for each estimated ETL was not reported, they generally vary in the range of 3% to 6%. In general, the precision of the ETL estimates increases with increasing number of doses included in a trial (29).

Table 2. Estimated Expected Toxicity Levels Based On Algorithm-Based Designs: Less Commonly Used Algorithm-Based Designs Design Parameters

Designs

A

B

C

D

E

A+B

1+A+B

3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 5 5 5 5 5 5 5 5 6 6 6 6 6

2 4 4 4 5 5 3 4 5 5 5 5 6 6 6 3 4 4 4 5 5 5 6 3 3 6 6 6

1 1 1 1 1 1 2 2 1 1 2 2 1 1 2 2 1 2 2 1 2 2 1 2 3 1 2 3

1 1 1 1 1 1 2 2 2 2 2 2 1 2 2 2 2 2 2 1 2 2 2 3 3 3 3 3

1 1 2 3 3 4 2 4 3 4 3 4 5 5 5 3 3 3 4 4 4 5 5 3 4 5 6 6

22% 18% 25% 30% 28% 31% 35% 38% 28% 35% 29% 35% 26% 37% 38% 30% 27% 28% 32% 30% 30% 33% 32% 28% 36% 33% 38% 38%

24% 18% 26% 32% 30% 33% 35% 39% 30% 37% 29% 36% 29% 39% 38% 31% 30% 28% 34% 32% 31% 35% 34% 30% 36% 35% 39% 38%

10

ALGORITHM-BASED DESIGNS

REFERENCES 1. N. L. Geller, Design of phase I and II clinical trials in cancer: a statistician’s view. Cancer Invest. 1984; 2: 483–491. 2. W. Rosenberger and L. Haines, Competing designs for phase I clinical trials: a review. Stat Med. 2002; 21: 2757–2770. 3. S. Green, J. Benedetti, and J. Crowley, Clinical Trials in Oncology. Boca Raton, FL: Chapman & Hall/CRC, 2003. 4. L. Edler, Overview of phase I trials. In: J. Crowley, ed. Handbook of Statistics in Clinical Oncology. New York: Marcel Dekker, 2001, pp. 1–34. 5. B. Storer and D. DeMets, Current phase I/II designs: are they adequate? J Clin Res Drug Dev. 1986; 1: 121–130. 6. J. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: a practical design for phase I clinical studies in cancer. Biometrics. 1990; 46: 33–48. 7. J. Whitehead and H. Brunier, Bayesian decision procedures for dose determining experiments. Stat Med. 1995; 14: 885–893. 8. J. Whitehead, Bayesian decision procedures with application to dose-finding studies. Int J Pharm Med. 1997; 11: 201–208. 9. J. Babb, A. Rogatko, and S. Zacks, Cancer phase I clinical trials: efficient dose escalation with overdose control. Stat Med. 1998; 17: 1103–1120. 10. Y. Lin and W. J. Shih, Statistical properties of the traditional algorithm-based designs for phase I cancer clinical trials. Biostatistics. 2001; 2: 203–215. 11. W. J. Shih and Y. Lin, Traditional and modified algorithm-based designs for phase I cancer clinical trials. In: S. Chevret (ed.), Statistical Methods for Dose-Finding Experiments. New York: Wiley, 2006, pp. 61–90. 12. T. Smith, J. Lee, H. Kantarjian, S. Legha, and M. Rober, Design and results of phase I cancer clinical trials: three-year experience at M.D. Anderson Cancer Center. J Clin Oncol. 1996; 14: 287–295. 13. S. Durham, N. Flournoy, and W. Rosenberger, A random walk rule for phase I clinical trials. Biometrics. 1997; 53: 745–760. 14. J. Whitehead and H. Brunier, Bayesian decision procedures for dose determining experiments. Stat Med. 1995; 14: 885–893. 15. J. Whitehead, Bayesian decision procedures with application to dose-finding studies. Int J Pharm Med. 1997; 11: 201–208.

16. J. Babb, A. Rogatko, and S. Zacks, Cancer phase I clinical trials: efficient dose escalation with overdose control. Stat Med. 1998; 17: 1103–1120. 17. M. Ratain, R. Mick, R. Schilsky, and M. Siegler, Statistical and ethical issues in the design and conduct of phase I and II clinical trials of new anticancer agents. J Natl Cancer Inst. 1993; 85: 1637–1643. 18. E. Korn, D. Midthune, T. Chen, L. Rubinstein, M. Christian, and R. Simon, A comparison of two phase I trial designs. Stat Med. 1994; 13: 1799–1806. 19. C. Ahn, An evaluation of phase I cancer clinical trial designs. Stat Med. 1998; 17: 1537–1549. 20. W. Rosenberger and L. Haines, Competing designs for phase I clinical trials: a review. Stat Med. 2002; 21: 2757–2770. 21. B. Storer, Design and analysis of phase I clinical trials. Biometrics. 1989; 45: 925–937. 22. R. Simon, B. Freidlin, L. Rubinsteir, S. Arbuck, J. Collins, and M. Christian, Accelerated titration designs for phase I clinical trials in oncology. J Natl Cancer Inst. 1997; 89: 1138–1147. 23. E. Reiner, X. Paoletti, and J. O’Quigley, Operating characteristics of the standard phase I clinical trial design. Comput Stat Data Anal. 1999; 30: 303–315. 24. J. Dancey, B. Freidlin, and L. Rubinstein, Accelerated titration designs. In: S. Chevret (ed.), Statistical Methods for Dose-Finding Experiments. New York: Wiley, 2006, pp. 91–113. 25. J. Liu, W. He, and H. Quan, Statistical properties of a modified accelerated designs for phase I cancer clinical trials. Commun Stat Theory Methods. 2007. In press. 26. M. Stylianou, M. Proschan, and N. Flournoy, Estimating the probability of toxicity at the target dose following an up-and-down design. Stat Med. 2003; 22: 535–543. 27. S. Kang and C. Ahn, The expected toxicity rate at the maximum tolerated dose in the standard phase I cancer clinical trial design. Drug Inf J. 2001; 35: 1189–1199. 28. S. Kang and C. Ahn, An investigation of the traditional algorithm-based designs for phase I cancer clinical trials. Drug Inf J. 2002; 36: 865–873. 29. W. He, J. Liu, B. Binkowitz, and H. Quan, A model-based approach in the estimation of the maximum tolerated dose in phase I cancer clinical trials. Stat Med. 2006; 25: 2027–2042.

ALGORITHM-BASED DESIGNS

CROSS-REFERENCES Continual Reassessment Method Maximum Tolerable Dose Phase I Trials

11

ALIGNED RANK TEST

been extended to multi-way designs, and has also been considered for time-to-event data (3). The goal of the procedure is to preserve information regarding interblock variability through the application of interblock rankings while removing the additive blockspecific effects from the data. This permits a more efficient test of the experimental effect under examination. To this end, all observations are centered (aligned) according to their intrablock mean, median, or another symmetric and translation-invariant measure of location. The residuals from centering are then ranked without respect to block or experimental group, and are labeled according to group. After labeling, the groupwise rank sums are compared using a procedure appropriate for the hypotheses. For example, the Wilcoxon rank-sum statistic may be computed for two-sample comparisons, and the Kruskal-Wallis statistic or F statistic (4) may be considered for comparisons of more than two experimental groups. Like their traditional nonaligned rank test counterparts, these aligned procedures provide an exact test of the null hypothesis. The significance of the test may be computed by hand for small samples where derivation of the permutation distribution of the rank labels is convenient; larger samples or multiple blocking factors require the use of computerized software or the chi-square approximation from Sen (5), which was developed further in Puri and Sen (6) and supported by Tardif (7).

REBECCA B. McNEIL Biostatistics Unit Mayo Clinic Jacksonville, Florida,

ROBERT F. WOOLSON Department of Biostatistics, Bioinformatics and Epidemiology Medical University of South Carolina Charleston, South Carolina

When a comparison of means or medians between two or more experimental groups is desired, and differences between the scale parameters (variance) are not of concern, a location shift hypothesis is under consideration. Analysis of variance (ANOVA) methods are the most common hypothesistesting approach in this experimental framework. However, the validity of the parametric approach is dependent on the distribution of the data. Rank tests are a popular alternative method of analysis. The application of these tests generally involves the ranking of all observations without regard to experimental group; a function of the ranks is then compared between the groups. But it is often necessary to block or stratify the data on one or more covariates for the purpose of reducing experimental error. In this situation, it is routine to consider the application of methods such as the Friedman generalization of the Wilcoxon test (1), which ranks observations on an intrablock basis to develop a comparison of experimental groups. Another approach with comparatively higher efficiency in these multi-way designs is that of aligned rank testing, which compares experimental groups based on the ranks of residuals obtained after the alignment of intrablock observations via removal of block effects. 1

2

ASSUMPTIONS

We restrict our further discussion to the twoway layout. Consider a randomized block design with n blocks, p experimental treatment groups, and a total of N measurements. Assume that the block and experimental effects are additive and linear, and that the experimental effect is a fixed effect. Then the endpoint under consideration within the i-th block and j-th experimental group, Yij , may be described as

BACKGROUND

Aligned rank tests were introduced by Hodges and Lehmann in 1962 (2) within the motivating context of the two-way layout. The concept of ranking after alignment according to one or more factors has since

Yij = µ + αi + βj + εij

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

ALIGNED RANK TEST

where µ is the overall mean response, α i is the block effect, β j is the experimental (treatment) effect, and εij denotes residual error. We assume that the vectors εi , which contain the residuals specific to block i (i = 1, . . . , n), constitute independent random variables with a continuous joint cumulative distribution function. In addition, it is assumed that the rank scoring sequence selected is a function F of the sequence m/(N + 1), m = 1, . . . , N, where F is defined according to the Chernoff-Savage criteria. This allows for most of the commonly chosen scoring regimens. 3

EFFICIENCY, SIZE, AND POWER

Akritas (8) shows that when a two-way model is considered, the aligned rank test of the hypothesis of no main or interaction effects due to the experimental variable outperforms the rank transform ANOVA and the F test in efficiency for logistic and double exponential data. The F test exceeds the aligned rank test and the rank transform ANOVA in efficiency when the data have a Gaussian distribution (8). Specifically, the aligned rank test has minimum efficiency of 3/π = 0.955 in comparison with the rank transform F test in normal data (9). When compared with the equivalent test based on interblock rankings, the aligned rank test has efficiency of 3/2 for two experimental groups, and the efficiency declines monotonically to equivalence between methods as the sample size grows. It should be noted that these efficiencies are correct regardless of whether the sample size increases as a result of increasing the number of experimental groups, or as a result of increasing the groupwise sample size while maintaining a limited number of experimental groups (9). The size (type I error rate) of the aligned rank test has not been comprehensively described. O’Gorman (10) presents the results of simulations of empirical size for 4 or 10 blocks and 3 or 6 treatments in a randomized complete block design. The distributions considered were the uniform, normal, and double exponential. The aligned rank test (large sample, with referral to the chisquare distribution) was consistently conservative, with estimated size ranging from 1.4%

to 4.8%, depending on whether the alignment was with respect to the mean or the median. A smaller number of blocks resulted in increased conservatism. An aligned test with referral to the F distribution, but with denominator degrees of freedom modified for improved size, ranged between 3.3% and 5.2% for the same data. The power of the mean- and median-aligned rank tests was slightly inferior to that of the denominatoradjusted test when small sample sizes were considered. In larger samples where at least 15 blocks were present, the aligned rank tests had power comparable to or greater than that of the F test and Friedman’s test, particularly in the simulations of data from double exponential distributions. Thus, the F test is preferred for normal or uniform distributions, and the aligned rank tests are preferred for larger sizes and skewed or long-tailed distributions (10). However, selection of an analysis method should take into account both the comparative size and power of the candidate methods. 4 EXAMPLE We illustrate the use of this method using data collected in a clinical trial of high-dose vitamin D supplementation in pregnant and lactating mothers. Table 1 presents selected data from the trial. The measurements represent the serum level of 25(OH)D, a form of vitamin D measured through a blood test. The data are drawn from three racial blocks and two dosage groups. We align blocks using the block mean. After computation of the block mean, the observations are aligned and the residuals ranked (from least to greatest) and labeled with respect to dose group, as shown in Table 2. To compare dose groups, we now compute the sum of ranks specific to a single group. For group A, the ranks total W = 57. This sum is referred to the permutation distribution of the rank sums, obtained by varying the group labelings. Specifically, within block 1, the two assignments to group A are permutable among the three observations to yield rank totals for group A of 10, 11, and 19. Within block 2, the two assignments to group

ALIGNED RANK TEST

3

Table 1. Selected 25(OH)D Measurements and Intrablock Means from a Clinical Trial with Two Dose Groups and Three Blocks Block 1 2 3

Dose A

Dose B

42.0, 41.4 36.3, 32.9 39.5, 46.1

14.6 15.9, 18.6 25.0, 25.6, 31.3

Mean 32.7 25.9 33.5

Source: Data kindly provided by Dr. Carol Wagner, Medical University of South Carolina.

Table 2. Calculation, Ranking, and Labeling of Residuals After Alignment by Subtraction of the Intrablock Mean 25(OH)D Block mean Residual Ranking Dose Block

42.0 32.7 9.3 10 A 1

41.4 32.7 8.7 9 A 1

14.6 32.7 –18.1 1 B 1

36.3 25.9 10.4 11 A 2

32.9 25.9 7.0 8 A 2

A can be varied among the four observations to give six sums (some not unique), and 10 (5choose-2) may be obtained from the labelings in block 3. Thus, it can be shown that the total possible value of the rank sum is 57, obtainable only through one sum: that of the largest intrablock values possible through label permutation. The significance of the test is then 1/(3 × 6 × 10) = 0.0056. This exact test of the null hypothesis of no difference between groups indicates that there is a significant difference between the mean 25(OH)D levels of the two dose groups. 5

LARGE-SAMPLE APPROXIMATION

For randomized complete block designs, in which each of n blocks contains p experimental units assigned to p treatments, a large-sample chi-square approximation is available. This may be obtained through referral of the statistic

(p − 1)n2 S=

p  2
R.j − R.. j=1

p  n
2
Rij − Ri. i=1 j=1

2 to the χp−1 distribution.

15.9 25.9 –10.0 2 B 2

6

18.6 25.9 –7.3 5 B 2

39.5 33.5 6.0 7 A 3

46.1 33.5 12.6 12 A 3

25.0 33.5 –8.5 3 B 3

25.6 33.5 –7.9 4 B 3

31.3 33.5 –2.2 6 B 3

RELATED METHODS

The aligned rank test relies on analysis of residuals, after adjustment for block effects, to complete a test of the experimental effect. However, residual analysis is not restricted to aligned tests. It is common to examine residuals after fitting a chosen model so as to assess the lack of fit. Often, this examination is visual, using graphical techniques, but Quade (11) has formalized some residual analysis methods with demonstrated equivalence to preexisting nonparametric tests. The weighted rank tests from Quade permit comparisons between regression models, with weighting matrices applied to residuals to provide emphasis on specific aspects of the models.

REFERENCES 1. M. H. Hollander and D. A. Wolfe, Nonparametric Statistical Methods 2nd ed. New York: Wiley, 1999. 2. J. L. Hodges and E. L. Lehmann, Rank methods for combination of independent experiments in analysis of variance. Ann Math Stat 1962; 33: 482–497. 3. M. D. Schluchter, An aligned rank test for censored data from randomized block designs. Biometrika 1985; 72: 609–618.

4

ALIGNED RANK TEST 4. R. F. Fawcett and K. C. Salter, A Monte Carlo study of the F test and three tests based on ranks of treatment effects in randomized block designs. Commun Stat Simul Comput 1984; 13: 213–225. 5. P. K. Sen, On a class of aligned rank order tests in two-way layouts. Ann Math Stat 1968; 39: 1115–1124. 6. M. L. Puri and P. K. Sen, Nonparametric Methods in Multivariate Analysis New York: Wiley, 1971. 7. S. Tardif, On the almost sure convergence of the permutation distribution for aligned rank test statistics in randomized block designs. Ann Stat 1981; 9: 190–193. 8. M. G. Akritas, The rank transform method in some two-factor designs. J Am Stat Assoc 1990; 85: 73–78. 9. K. L. Mehra and J. Sarangi, Asymptotic efficiency of certain rank tests for comparative experiments. Ann Math Stat 1967; 38: 90–107.

10. T. W. O’Gorman, A comparison of the F-test, Friedman’s test, and several aligned rank tests for the analysis of randomized complete blocks. J Agric Biol Environ Stat 2001; 6: 367–378. 11. D. Quade, Regression analysis based on the signs of the residuals. J Am Stat Assoc 1979; 74: 411–417.

CROSS-REFERENCES Analysis of variance (ANOVA) Balanced design Blocking Rank tests

ALLOCATION CONCEALMENT

and informed consent obtained, in ignorance of the upcoming assignment (5).

KENNETH F. SCHULZ Family Health International Research Triangle Park North Carolina,

1 INDICATIONS OF THE IMPORTANCE OF ALLOCATION CONCEALMENT Trialists recently coined the term ‘‘allocation concealment’’ to describe, as well as highlight the importance of, the process (2, 3). Proper allocation concealment appears to prevent bias. Four empirical investigations have found that trials that used inadequate or unclear allocation concealment, compared with those that used adequate concealment, yielded up to 40% larger estimates of effect, on average (3, 6–8). The poorly executed trials tended to exaggerate treatment effects. One other investigation did not find an overall exaggeration, but suggested that poor quality, including allocation concealment, may distort the trial results in either direction (9). In any case, an updated meta-analysis of the relevant studies provides empirical evidence that inadequate allocation concealment allows bias to seep into trials (10). The bias could fluctuate in either direction. The worst concealed trials yielded greater heterogeneity in results (i.e., the results vacillated extensively above and below the estimates from better trials) (3). Indeed, having a randomized sequence should make little difference without adequate allocation concealment. Even random, unpredictable assignment sequences can be subverted (3, 11, 12). For example, if investigators implemented a truly randomized sequence by merely posting the sequence on a bulletin board, everyone involved with the trial can obtain the upcoming assignments. Similarly, the allocation sequence could be implemented through placing method indicator cards in regular (translucent) envelopes. This inadequate allocation concealment process could be deciphered by simply holding the envelopes to a bright light. With both these inadequate approaches, awareness of

Randomization eliminates bias in the allocation of participants to treatment (intervention) groups. Generating an unpredictable randomized allocation sequence represents the first vital step in randomization (1). Implementing that sequence while concealing it, at least until assignment occurs, a process termed allocation concealment, represents the second step (2, 3). Without adequate allocation concealment, randomization crumples in a trial. Misunderstandings thrive concerning allocation concealment. Some researchers attempting to discuss it digress into flipping coins or computer random number generators. Allocation concealment, however, is distinct from the techniques that generate the sequence. Allocation concealment techniques implement the sequence (3). Still other people confuse allocation concealment with blinding (2–4). However, unmistakable theoretical and practical differences separate the terms. Allocation concealment primarily prevents selection bias and protects a sequence before and until assignment. By contrast, blinding prevents ascertainment bias and protects the sequence after assignment. Furthermore, allocation concealment can, and must, always be successfully implemented in a randomized control trial (RCT); but in many trials, blinding cannot be successfully implemented. Proper allocation concealment secures strict implementation of a sequence without foreknowledge of treatment assignments. It safeguards the upcoming assignments from those who admit participants to a trial. The judgment to accept or reject a participant must be made,

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

ALLOCATION CONCEALMENT

the next assignment could lead to the exclusion of certain patients based on their prognosis. Alternatively, awareness of the next assignment could lead to directing some participants preferentially to groups, which can easily be accomplished by delaying a participant’s entry into the trial until the next ‘‘preferred’’ allocation appears. Bias could easily be introduced, despite an adequate randomized sequence (11). Understandably, however, investigators seldom document the sensitive details of subverting allocation schemes. Nevertheless, when investigators responded anonymously, many did relate instances of deciphering (11). The descriptions varied from holding translucent envelopes to bright lights, to holding opaque envelopes to ‘‘hot lights’’ in radiology, to rifling files in the dead of night (11). Most deciphering stemmed from inadequate allocation concealment schemes. The value of randomization paradoxically produces its most vexing implementation problems. Randomization antagonizes humans by frustrating their clinical inclinations (11, 13, 14). Thus, many involved with trials will be tempted to undermine randomization, if afforded the opportunity to decipher assignments. To minimize the impact of this human tendency, trialists must devote meticulous attention to concealing allocation schemes. Proper randomization hinges on adequate allocation concealment. 2 ADEQUATE METHODS OF ALLOCATION CONCEALMENT The following approaches to allocation concealment are generally considered as adequate (2, 3, 15): sequentially numbered, opaque, sealed envelopes (SNOSE); pharmacy-controlled; numbered or coded containers; central randomization (e.g., by telephone to a trials office); or other methods whose description contained elements convincing of concealment (e.g., a secure computer-assisted method). In assessing allocation concealment from published reports, readers will be fortunate to find even these minimal standards reasonably met (see Block 1). Investigators should, however, embrace even more rigorous approaches to assure that

selection and confounding biases have been averted. Methods based on envelopes are more vulnerable to exploitation through human ingenuity and resourcefulness than most other approaches. Therefore, some trialists consider envelopes a less than ideal method of concealment (16). However, they can be adequate and they are eminently practical, even in developing world settings. With an envelope method, investigators must diligently develop and monitor the allocation process to preserve concealment. In addition to using SNOSE, investigators must ensure that the envelopes are opened sequentially, and only after the participant’s name and other details are written on the appropriate envelope (17). Also using pressure-sensitive or carbon paper inside the envelope transfers the information written on the outside of the envelope to the assigned allocation card inside the envelope, which promotes randomization adherence and also creates a valuable audit trail. Cardboard or aluminum foil placed inside the envelope further inhibits detection of assignments via ‘‘hot lights’’ in radiology or the like. Pharmacy-controlled methods of allocation concealment tend to be adequate, but also are subject to allocation concealment and sequence generation problems. Although reports in which the assignment was made by the pharmacy have generally been classified as having ‘‘adequate’’ allocation concealment (2, 3, 15), comformity with proper randomization by pharmacists is unknown. Their detailed methods should have been reported. The author is aware of instances in which pharmacists have violated assignment schedules (11). For example, one pharmacy depleted the stock of one of the two drugs being compared in a randomized trial. They proceeded to allocate the other drug to all newly enrolled participants to avoid slowing recruitment. Another pharmacy allocated (what they inaccurately called ‘‘randomization’’) by alternate assignment. Investigators should not assume that pharmacists grasp RCT methodology. For pharmacy-controlled allocation, investigators should develop the allocation concealment procedures in concert with the pharmacy, instruct in the details if necessary, and monitor the process.

ALLOCATION CONCEALMENT

3

Table 1. Important Elements in Adequate Allocation Concealment Schemes General Allocation Concealment Scheme

Details on Adequate Approaches for Implementing Allocation Concealment

Envelopes

Should be sequentially numbered, opaque, sealed envelopes (SNOSE). Must ensure that the envelopes are opened sequentially only after participant details are written on the envelope. Pressure-sensitive or carbon paper inside the envelope transfers that information to the assignment card inside the envelope (also creates an audit trail). Cardboard or aluminum foil inside the envelope renders the contents of the envelope impermeable to intense light.

Containers

Must ensure that the containers are sequentially numbered or coded to prevent deciphering. Must ensure that containers are tamper-proof, equal in weight, identical in appearance, and equivalent in sound upon shaking.

Pharmacy-controlled

Investigators should develop, or at least validate in concert with the pharmacy, a proper randomization scheme, including allocation concealment. Should instruct the pharmacy in the details of proper allocation concealment, if necessary, and monitor the process.

Central randomization

They must determine the mechanism for contact (e.g., telephone, fax, or e-mail), the rigorous procedures to ensure enrollment before randomization, and the comprehensive training for those individuals staffing the central randomization office.

Others such as automated electronic assignment systems

Investigator may use other systems for allocation concealment. Any such system must safeguard allocations until enrollment is assured and assignment confirmed. A computer-based system must be impenetrable for it to be adequate. Simple computer-based systems that merely store assignments or naively protect assignments may prove as transparent as affixing an allocation list to a bulletin board.

Proper use of sequentially numbered containers prevents foreknowledge of treatment assignment, but only if investigators take proper precautions. They must assure that all of the containers are tamper-proof, equal in weight, identical in appearance, and equivalent in sound upon shaking. It is also encouraged that some audit trail be established, such as writing participant identifiers on the empty containers. Although central randomization persists as

an

excellent

allocation

concealment

approach, indeed, perhaps the ‘‘gold standard,’’ investigators must establish effective trial procedures and monitor adherence in

their execution. At a minimum, they determine the mechanism for contact (e.g., telephone, fax, or e-mail), the rigorous procedures to ensure enrollment before randomization, and the comprehensive training for those individuals staffing the central randomization office. Other methods may suffice for adequate allocation concealment. For example, an investigator may use a secure computerassisted method to safeguard assignments until enrollment is assured and confirmed. Undoubtedly, automated electronic assignment systems will become more common (18, 19). However, simple computer-based systems that merely store assignments or naively protect assignments may prove as transparent as affixing an allocation list to

4

ALLOCATION CONCEALMENT

a bulletin board. A computer-based system must be impenetrable for it to be adequate. Finally, in their trial report, investigators must fully describe the details of their allocation concealment approach (11, 20). Many reports, however, neglect to provide even the bare essentials (21). Only 4% of trials in a recent review of 2000 trials in schizophrenia described a method of concealment (22). Another paper found that over 20% of trials in head injury described an allocation concealment method, but the reports were more likely to describe an inadequate method rather than an adequate method (23), which hampers attempts by readers to assess RCTs. Fortunately, the situation is improving with more medical journals adopting CONSORT reporting standards for RCTs (20, 24, 25). With transparent reporting, more investigators will have to design and conduct methodologically sound trials. 3 BLOCK 1: EXAMPLES OF DESCRIPTIONS OF ALLOCATION CONCEALMENT

. . . concealed in sequentially numbered, sealed, opaque envelopes, and kept by the hospital pharmacist of the two centres (26). Treatments were centrally assigned on telephone verification of the correctness of inclusion criteria . . . (27). Glenfield Hospital Pharmacy Department did the randomisation, distributed the study agents, and held the trial codes, which was disclosed after the study(28). The various placebo and treatment blocks were then issued with a medication number and assigned to consecutive patients in a sequential order. Two copies of the randomisation list were prepared: one was used by the packaging department, . . . supplied in blister packs containing 20 capsules for morning and evening administration over 10 days. These blister packs were supplied in labeled boxes–ie, one box for each patient and each dose (29).

of randomization from reports of controlled trials published in obstetrics and gynecology journals. JAMA 1994; 272(2): 125–128. 3. K. F. Schulz, I. Chalmers, R. J. Hayes, and D. G. Altman, Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995; 273(5): 408–412. 4. K. F. Schulz, I. Chalmers, and D. G. Altman, The landscape and lexicon of blinding in randomized trials. Ann. Intern. Med. 2002; 136(3): 254–259. 5. T. C. Chalmers, H. Levin, H. S. Sacks, D. Reitman, J. Berrier, and R. Nagalingam, Meta-analysis of clinical trials as a scientific discipline. I: control of bias and comparison with large co-operative trials. Stat. Med. 1987; 6(3): 315–328. 6. D. Moher et al., Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? Lancet 1998; 352(9128): 609–613. 7. L. Kjaergard, J. Villumsen, and C. Gluud, Quality of randomised clinical trials affects estimates of intervention. Abstracts for Workshops and Scientific Sessions, 7th International Cochrane Colloquium. Rome, Italy, 1999. ¨ 8. P. Juni, D. G. Altman, and M. Egger, Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ 2001; 323(7303): 42–46. 9. E. M. Balk, P. A. Bonis, H. Moskowitz, C. H. Schmid, J. P. Ioannidis, C. Wang, and J. Lau, Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA 2002; 287(22): 2973–2982. ¨ 10. P. Juni and M. Egger, Allocation concealment in clinical trials. JAMA 2002; 288(19): 2407–2408; author reply 2408–2409. 11. K. F. Schulz, Subverting randomization in controlled trials. JAMA 1995; 274(18): 1456–1458. 12. S. Pocock, Statistical aspects of clinical trial design. Statistician 1982; 31: 1–18.

REFERENCES 1. K. F. Schulz and D. A. Grimes, Generating allocation sequences in randomised trials: chance, not choice. Lancet 2002; 359: 515–519. 2. K. F. Schulz, I. Chalmers, D. A. Grimes, and D. G. Altman, Assessing the quality

13. K. F. Schulz, Unbiased research and the human spirit: the challenges of randomized controlled trials. CMAJ 1995; 153(6): 783–786. 14. K. F. Schulz, Randomised trials, human nature, and reporting guidelines. Lancet 1996; 348(9027): 596–598.

ALLOCATION CONCEALMENT 15. D. G. Altman and C. J. Dor’e, Randomisation and baseline comparisons in clinical trials. Lancet 1990; 335(8682): 149–153. 16. C. L. Meinert, Clinical Trials: Design, Conduct, and Analysis. New York: Oxford University Press, 1986. 17. C. Bulpitt, Randomised Controlled Clinical Trials. The Hague, The Netherlands: Martinus Nijhoff, 1983. 18. K. Dorman, G. R. Saade, H. Smith, and K. J. Moise Jr., Use of the World Wide Web in research: randomization in a multicenter clinical trial of treatment for twin-twin transfusion syndrome. Obstet. Gynecol. 2000; 96(4): 636–639. 19. U. Haag, Technologies for automating randomized treatment assignment in clinical trials. Drug Inform. J. 1998; 118: 7–11. 20. D. Moher, K. F. Schulz, and D. Altman, The CONSORT statement: revised recommendations for improving the quality of reports or parallel-group trials. Lancet 2001; 357: 1191–1194. 21. K. F. Schulz and D. A. Grimes, Allocation concealment in randomised trials: defending against deciphering. Lancet 2002; 359(9306): 614–618. 22. B. Thornley and C. Adams, Content and quality of 2000 controlled trials in schizophrenia over 50 years. BMJ 1998; 317(7167): 1181–1184. 23. K. Dickinson, F. Bunn, R. Wentz, P. Edwards, and I. Roberts, Size and quality of randomised controlled trials in head injury: review of published studies. BMJ 2000; 320(7245): 1308–1311.

5

24. D. G. Altman et al., The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann. Intern. Med. 2001; 134(8): 663–694. 25. D. Moher, A. Jones, and L. Lepage, Use of the CONSORT statement and quality of reports of randomized trials: a comparative beforeand-after evaluation. JAMA 2001; 285(15): 1992–1995. 26. T. Smilde, S. van Wissen, H. Wollersheim, M. Trip, J. Kastelein, and A. Stalenhoef, Effect of aggressive versus conventional lipid lowering on atherosclerosis progression in familial hypercholesterolaemia (ASAP): a prospective, randomised, double-blind trial. Lancet 2001; 357: 577–581. 27. Collaborative Group of the Primary Prevention Project (PPP), Low-dose aspirin and vitamin E in people at cardiovascular risk: a randomised trial in general practice. Collaborative Group of the Primary Prevention Project. Lancet 2001; 357(9250): 89–95. 28. C. E. Brightling et al., Sputum eosinophilia and short-term response to prednisolone in chronic obstructive pulmonary disease: a randomised controlled trial. Lancet 2000; 356(9240): 1480–1485. 29. I. McKeith, T. Del Ser, P. Spano et al., Efficacy of rivastigmine in dementia with Lewy bodies: a randomised, double-blind, placebocontrolled international study. Lancet 2000; 356(9247): 2031–2036.

ALPHA-SPENDING FUNCTION

or asymmetric boundaries. In general, a test statistic Z(j), j = 1,2,3, . . . J is computed at each successive interim analysis. In the large sample case and under the null hypothesis, the Z(j)s are standard N(0,1). At each analysis, the test statistic is compared with a critical value Zc (j). The trial would continue as long as the test statistic does not exceed the critical value. That is, continue the trial as long as

DAVID L. DEMETS University of Wisconsin-Madison Madison, Wisconsin

K. K. GORDON LAN Johnson & Johnson Raritan, New Jersey

The randomized control clinical trial (RCT) is the standard method for the definitive evaluation of the benefits and risks of drugs, biologics, devices, procedures, diagnostic tests, and any intervention strategy. Good statistical principles are critical in the design and analysis of these RCTs (1,2). RCTs also depend on interim analysis of accumulating data to monitor for early evidence of benefit, harm, or futility. This interim analysis principle was established early in the history of RCTs (3) and was implemented in early trials such as the Coronary Drug Project (4,5). Evaluation of the interim analysis may require the advice of an independent data monitoring committee (DMC) (6,7), including certain trials under regulatory review (8,9). However, although ethically and scientifically compelling, interim repeated analysis of accumulating data has the statistical consequence of increased false positive claims unless special steps are taken. The issue of sequential analysis has a long tradition (10,11) and has received special attention for clinical trials (12,13). In particular, increasing the frequency of interim analysis can substantially increase the Type I error if the same criteria are used for each interim analysis (13). This increase was demonstrated in the Coronary Drug Project, which used sequential analysis for monitoring several treatment arms compared with a placebo (4). Most of the classical sequential methods assumed continuous analysis of accumulating data, a practice not realistic for most RCTs. Rather than continuous monitoring, most clinical trials review accumulating data periodically after additional data has been collected. Assume that the procedure is a one-sided test but that the process can be easily generalized to introduce two one-sided symmetric

Z(j) < Zc (j) for j = 1, 2, 3, . . . , J − 1 Otherwise, the trial might be considered for termination. We would fail to reject the null hypothesis if Z(j) < Zc (j) for all j (j = 1,2,...,J). We would reject the null hypothesis if at any interim analysis, Z(j) ≥ Zc (j)

for j = 1, 2, . . . , J

Peto et al. (14) recommended using a very conservative critical value for each interim analysis, say a standardized value of Zc (j) = 3.0 for all j (j = 1,2, . . . ,J), such the impact on the overall Type I error would be minimal. In 1977, Pocock (15) published a paper based on the earlier work of Armitage and colleagues (13) that formally introduced the idea of a group sequential approach. This modification developed a more conservative critical value than the na¨ıve (e.g., 1.96 for one-sided Type I error of 0.025) to be used at each analysis such that the overall Type I error was controlled. For example, if a total of 5 interim analyses were to be conducted with an intended Type I error of 0.025, then Zc (j) = 2.413 would be used at each interim analysis (j = 1,2,. .,5). Note that this final critical value is much larger than the standard critical value. In 1979, O’Brien and Fleming (16) introduced an alternative group sequential boundary for evaluating interim analysis. In this approach, the critical values change with each interim analysis, starting with a very conservative (i.e., large) value and shrinking to a final value close to the nominal critical value at the scheduled completion. The exact form for each critical value is Zc (j) = √ ZOBF (J) (J/j). In this case, for the same 5 interim analyses and an overall Type I error of 0.025, the ZOBF (5) value is 2.04, which

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

ALPHA-SPENDING FUNCTION

√ makes the 5 critical values 2.05 (5/j) for j = 1,2, . . . ,5 or (4.56, 3.23, 2.63, 2.28, and 2.04). Both of these latter models assume an equal increment in information between analyses and that the number of interim analyses J is fixed in advance. These three group sequential boundaries have been widely used, and examples are shown in Fig. 1. In fact, the OBF group sequential method was used in the Betablocker Heart Attack Trial (BHAT) (17), which terminated early because of an overwhelming treatment benefit for mortality. In 1987, Wang and Tsiatis generalized the idea of Pocock and O’Brien-Fleming and introduced a family of group sequential boundaries. For given α, J and a shape parameter φ, a constant C will be chosen so that Z(j) ≥ C (J/j)φ for some j = 1,2, . . . , J is equal to α. The choice of φ = 0.5 yields the OBF boundary, and φ = 0 yields the Pocock boundary. 1 ALPHA SPENDING FUNCTION MOTIVATION The BHAT trial was an important factor in the motivation for the alpha spending function approach to group sequential monitoring. BHAT was a cardiovascular trial that

evaluated a betablocker class drug to reduce mortality following a heart attack (17). An independent DMC reviewed the data periodically, using the OBF group sequential boundaries as a guide. A beneficial mortality trend emerged early in the trial and continued to enlarge with subsequent evaluations. At the sixth of a planned seven interim analyses, the logrank test statistic crossed the OBF boundary. After careful examination of all aspects, the DMC recommended that the trial be terminated, approximately 1 year earlier than planned. However, although the OBF sequential boundaries were used, the assumptions of these models were not strictly met. The increment in the number of deaths between DMC meetings was not equal. Furthermore, additional interim analyses were contemplated although not done. This experience suggested the need for more flexible sequential methods for evaluating interim results. Neither the number nor the timing of interim analyses can be guaranteed in advance. A DMC may need to add additional interim analyses as trends that suggest benefit or harm emerge. As described by Ellenberg, Fleming, and DeMets (7), many

Group Sequential Boundaries 4.0 Reject H0 3.0

Standardized Statistic

2.0 1.0

Accept H0

0 −1.0 −2.0 −3.0

O’Brien-Fleming

−4.0

Truncated O’Brien-Fleming .25

.50 .75 Information Fraction

1.0

Figure 1. Upper boundary values corresponding to the α 1 (t*) spending function for α − 0.05 at information fraction t* = 0.25, 0.50, 0.75, and 1.0 and for a truncated version at a critical value of 3.0

ALPHA-SPENDING FUNCTION

3

Table 1. Comparison of boundaries using spending functions with Pocock (P) and O’Brien-Fleming (OBF) methods (α = 0.05, t* = 0.2, 0.4, 0.6, 0.8, and 1.0 t*

α 1 (t*)

OBF

α 2 (t*)

P

0.2 0.4 0.6 0.8 1.0

4.90 3.35 2.68 2.29 2.03

4.56 3.23 2.63 2.28 2.04

2.44 2.43 2.41 2.40 2.36

2.41 2.41 2.41 2.41 2.41

factors must be considered before recommendations for early termination are made, and an additional interim analysis may be necessary to confirm or more fully evaluate these issues. Thus, the need for a flexible group sequential method seemed compelling.

2

THE ALPHA SPENDING FUNCTION

The initial (or classical) group sequential boundaries are formed by choosing boundary values such that the sum of the probability of exceeding those critical values during the interim analyses is exactly the specified alpha level set in the trial design, assuming the null hypothesis of no intervention effect. That is, the total available alpha is allocated or ‘‘spent’’ over the prespecified times of interim analyses. The alpha spending function proposed by Lan and DeMets (18) allocated the alpha level over the interim analyses by a continuous function, α(t), where

t is the information fraction, 0 ≤ t ≤ 1. Here t could be the fraction of target patients recruited (n/N) or the fraction of targeted deaths observed (d/D) at the time of the interim analysis. In general, if the total information for the trial design is I, then at the j-th analysis, the information fraction tj = Ij /I. The total expected information I should have been determined by the trial design if properly done. The function α(t) is defined such that α(0) = 0 and α(1) = α. Boundary values Zc (j), which correspond to the α-spending function α(t), can be determined successfully so that P0 {Z(1) ≥ Zc (1), or Z(2) ≥ Zc (2), or . . . , or Z(j) ≥ Zc (j)} = α(tj )

(1)

where {Z(1), . . . , Z(j),} represent the test statistics from the interim analyses 1, . . . , . The specification of α(t) will create a boundary of critical values for interim test statistics, and we can specify functions that approximate Spending Functions

.05

Alpha

.04

a2(t*)

.03 )

t* a 3(

.02

a1(t*) .01 Figure 2. Comparison of spending functions α 1 (t*), α 2 (t*), and α 3 (t*) at information fractions t* = 0.2, 0.4, 0.6, 0.8, and 1.0

0

.2

.4 .6 Information Fraction

.8

1

4

ALPHA-SPENDING FUNCTION

O’Brien–Fleming or Pocock boundaries as follows: √ α1 (t) = 2 − 2 − (Zα/2 ´ / t) O’Brien-Fleming Type α2 (t) = α1n(1 + (e − 1)t) Pocock Type where  denotes the standard normal cumulative distribution function. The shape of the alpha spending function is shown in Fig. 2 for both of these boundaries. Two other families of spending functions proposed (19,20) are α(θ , t) = αtθ for θ >0 α(γ , t) = α[(1 − e−γ t )/(1 − e−γ )],

for γ # 0

The increment α(tj ) – α(tj −1) represents the additional amount of alpha or Type I error probability that can be used at the jth analysis. In general, to solve for the boundary values Zc (j), we need to obtain the multivariate distribution of Z(1), Z(2), . . . , Z(J). In the cases to be discussed, the distribution is asymptotically multivariate normal with covariance structure  = (σ jk ) where σjk = cov(Z(j), Z(k)) √ √ = (tj /tk ) = (iJ /iK ) for j ≤ k where ij and ik are the amount of information available at the j-th and k-th data monitoring, respectively. Note that at the jth data monitoring, ij and ik are observable and σ jk is known even if I (total information) is unknown. However, if I is not known during interim analysis, we must estimate I by Iˆ and tj by tj = Ij /Iˆ so that we can estimate α(tj ) by (tˆ). If these increments have an independent distributional structure, which is often the case, then derivation of the values of the Zc (j) from the chosen form of α(t) is relatively straightforward using Equation (1) and the methods of Armitage et al. (21,22). If the sequentially computed statistics do not have an independent increment structure, then the derivation of the Zc (j) involves a more complicated numerical integration and sometimes is estimated by simulation. However, as discussed later, for the most frequently used test statistics, the independent increment structure holds.

This formulation of the alpha spending function provides two key flexible features. Neither the timing nor the total number of interim analyses have to be fixed in advance. The critical boundary value at the j-th analysis only depends on the information fraction tj and the previous j-1 information fractions, t1 , t2 , . . . , tj−1 , and the specific spending function being used. However, once an alpha spending function has been chosen before the initiation of the trial, that spending function must be used for the duration of the trial. A DMC can change the frequency of the interim analyses as trends emerge without appreciably affecting the overall α level (23,24). Thus, it is difficult to abuse the flexibility of this approach. The timing and spacing of interim analyses using the alpha spending function approach have been examined (19, 25–27). For most trials, two early analyses with less than 50% of the information fraction are adequate. An early analysis say at 10% is often useful to make sure that all of the operational and monitoring procedures are in order. In rare cases, such early interim reviews can identify unexpected harm such as in the Cardiac Arrhythmia Suppression Trial (28) that terminated early for increased mortality at 10% of the information fraction using an alpha spending function. A second early analysis at 40 or 50% information fraction can also identify strong convincing trends of benefit as in two trials that evaluated beta blocker drugs in chronic heart failure (29,30). Both trials terminated early at approximately 50% of the information fraction with mortality benefits. Computation of the alpha spending function can be facilitated by available software on the web (www.biostat.wisc.edu/landemets) or by commercial software packages (www .cytel.com/Products/East/default.asp). 3 APPLICATION OF THE ALPHA SPENDING FUNCTION Initial development of group sequential boundaries was for comparison of proportions or means (15,16,26). In these cases, the increments in information are represented by additional groups of subjects and their responses to the intervention. For comparing means or proportions, the information

ALPHA-SPENDING FUNCTION

fraction t can be estimated by the n/N, the observed sample size divided by the expected sample size. However, later work expanded the use to other common statistical procedures. Tsiatis and colleagues (31,32) demonstrated that sequential logrank test statistics and the general class of rank statistics used in censored survival data had the independent increment structure that made the application to group sequential boundary straightforward. Later, Kim and Tsiatis (33) demonstrated that the alpha spending function approach for sequential logrank tests was also appropriate. In this case, the information fraction is approximated by d/D, the number of observed events or deaths divided by the expected or design for number of events or deaths (34). Application of the alpha spending function for logrank tests has been used in several clinical trials (e.g., 28–30). Group sequential procedures including the alpha spending function have also been applied to longitudinal studies using a linear random effects model (35,36). Longitudinal studies have also been evaluated using generalized estimating equations (37). In a typical longitudinal clinical trial, subjects are added over time, and more observations are gathered for each subject during the course of the trial. One statistic commonly used is to evaluate the rate of change by essentially computing the slope of the observations for each subject and then taking a weighted average of these slopes over the subjects in each intervention arm. The sequential test statistics for comparison of slopes using the alpha spending function must take into account their distribution. If the information fraction is defined in terms of the Fisher information (i.e., inverse of the variance for the slopes), then the increments in the test statistic are independent, and the alpha function can be applied directly (38). The total expected information may not be known exactly, but it often can be estimated. Wu and Lan (36) provide other approaches to estimate the information fraction in this setting. Scharfstein and Tsiatis (39) demonstrated that any class of test statistics that satisfies specific likelihood function criteria will have this property and thus can be used directly in this group sequential setting.

5

4 CONFIDENCE INTERVALS AND ESTIMATION Confidence intervals for an unknown parameter θ following early stopping can be computed by using the same ordering of the sample space described by Tsiatis et al. (32) and by using a process developed by Kim and DeMets (25,40) for the alpha spending function procedures. The method can be briefly summarized as follows: A 1–γ lower confidence limit is the smallest value of θ for which an event at least as extreme as the one observed has a probability of at least γ . A similar statement can be made for the upper limit. For example, if the first time the Z-value exists the boundary at tj with the observed Z*(j) ≥ Zc (j), then the upper θ U and lower θ L confidence limits are θ U = sup {θ : Pθ {Z(1) ≥ Zc (1), or · · · , or Z(j − 1) > Zc (j − 1), or Z(j) ≥ Z∗ (j)} ≤ 1 − γ }} and θ L = inf {θ : Pθ {Z(1) ≥ Zc (1), or · · · , or Z(j − 1) ≥ Zc (j − 1), or Z(j) ≥ Z1 (j)} ≥ γ }} Confidence intervals obtained by this process will have coverage closer to 1 – γ than na¨ıve ˆ confidence intervals using θˆ ±Zγ /2 SE(θ). As an alternative to computing confidence intervals after early termination, Jennison and Turnbull (41) have allocated the calculation of repeated confidence intervals. This calculation is achieved by inverting a sequential test to obtain the appropriate coefficient Z∗α/2 in the general form for the confidence ˆ This inversion can be interval, θˆ ± Z∗α/2 SE(θ). achieved when the sequential test is based on an alpha spending function. If we compute the interim analyses at the tj , obtaining corresponding critical values Zc (j), then the repeated confidence intervals are of the form θˆk ± Zc (j)SE(θˆj ) where θˆj is the estimate for the parameter θ at the j-th analysis. Methodology has also been developed to obtain adjusted estimates for the intervention effect (42–47). Clinical trials that terminate early are prone to exaggerate the

6

ALPHA-SPENDING FUNCTION

magnitude of the intervention effect. These methods shrink the observed estimate closer to the null. The size of the adjustments may depend on the specific sequential boundary employed. Conservative boundaries such as that proposed by Peto or O’Brien and Fleming generally require less adjustment, and the na¨ıve point estimate and confidence intervals may be quite adequate. Another issue is the relevance of the estimate to clinical practice. The population sample studied is usually not a representative sample of current and future practice. Subjects were those who passed all of the inclusion and exclusion criteria and volunteered to participate. Early subjects may differ with later subjects as experience is gained with the intervention. Thus, the intervention effect estimate may represent populations like the one studied, the only solid inference, but not as relevant to how the intervention will be used. Thus, complex adjustments may not be as useful. 5

TRIAL DESIGN

If any trial is planning to have interim analyses for monitoring for benefit or harm, then that plan must be taken into account in the design. The reason is that group sequential methods will impact the final critical value, and thus power, depending on which boundary is used. For the alpha spending function approach, the specific alpha function must be chosen in advance. In addition, for planning purposes, the anticipated number of interim analyses must be estimated. This number does not have to be adhered to in the application, but it is necessary for the design. Variance with this number in the application will not practically affect the power of the trial. Thus, the design strategy for the alpha spending function is similar to that strategy described by Pocock for the initial group sequential methods (15). The key factor when the sample size is computed is to take into consideration the critical value at the last analysis when the information fraction is 1.0. One simple approach is to use this new critical value in the standard sample size formula. This estimate will reasonably approximate a more exact approach described below.

To illustrate, consider a trial that is comparing failure rates of successive groups of subjects. Here, H0 :pC − pT = 0 HA :pC − pT = δ # 0 where pC and pT denote the unknown response rates in the control and newtreatment groups, respectively. We would estimate the unknown parameter by pˆ C and pˆ T , the observed event rates in our trial. For a reasonably large sample size, we often use the following test statistics, Z=


pˆ C − pˆ T p(1 ˆ − p)(1/m ˆ C + 1/mT

to compare event rates where pˆ is the combined event rate across treatment groups. For sufficiently large n where n = mC = mT , this statistics has an approximate standard normal distribution with mean and unit variance under the null hypothesis H0 : = 0. In this case, assuming equal sample size (n) per group in each arm,
√ = n(pC − pT / 2p(1 − p)
√ = nδ/ 2p(1 − p) where p = (pC + pT )/2 It follows that n=

2 2 p(1 − p) δ2

To design our studies, we evaluate the previous equation for n, the sample size per treatment per sequential group. Because the plan is to have J groups each of size 2n, the total sample size 2N equals 2nJ. Now, to obtain the sample size in the context of the alpha spending function, we proceed as follows: 1. For planning purposes, estimate the number of planned interim analyses J at equally spaced increments of information (i.e., 2n subjects). It is also possible to specify unequal increments, but equal space is sufficient for design purposes.

ALPHA-SPENDING FUNCTION

2. Obtain the boundary values of the K interim analyses under the null hypothesis H0 to achieve a prespecified overall alpha level, α, for a specific spending function α(t). 3. For the boundary obtained, obtain the value of to achieve a desired power (1–β). 4. Determine the value of n that determines the total sample size 2N = 2nJ. 5. Having computed these design parameters, one may conduct the trial with interim analysis to be done based on the information fraction tj approximated by

6

7

CONCLUSIONS

The alpha spending function approach for group sequential interim analysis has provided the necessary flexibility that allows data monitoring committees to fulfill their task. DMCs can adjust their analysis as data accumulates and trends emerge. As long as the alpha spending function is specified in advance, there is little room for abuse. Many trials sponsored by industry and government have successfully used this approach. Although the decision to terminate any trial early, for benefit or harm, is a very complex decision process, the alpha spending function can be an important factor in that process.

tj = Number of subjects observed/2N at the jth analysis (38). The number of actual interim analyses may not be equal to J, but the alpha level and the power will be affected only slightly (26). As a specific example, consider using an O’Brien–Fleming-type alpha spending function α 1 (t) with a one-sided 0.025 alpha level and 0.90 power at equally spaced increments at t = 0.2, 0.4, 0.6, 0.8 and 1.0. Using previous publications (16) or available computer software, we obtain boundary values 4.56, 3.23, 2.63, 2.28, and 2.04. Using these boundary values and available software, we find that = 1.28 provides the desired power of 0.90. If we specify pC = 0.6, pT = 0.4 (p = 0.5) under the alternative hypothesis, then we can obtain a sample size as follows. For = 1.28, n=

2(1.28)2 (0.5)(0.5) = 20.5, (0.2)2

and we have a total sample size of 2(21)5 = 210 subjects. We can then proceed to conduct interim analysis times at information fraction tj equal to the observed number of subjects divided by 210. Similar formulations can be developed for the comparison of means, repeated measures, and survival analysis (48). However, for most applications, the standard sample size formulas with the new alpha spending function final critical value will be a very good approximation.

REFERENCES 1. L. Friedman, C. Furberg, and D. DeMets, Fundamentals of Clinical Trials. Littleton, MA: John Wright – PSG Inc., 1981. 2. S. J. Pocock, Clinical Trials: A Practical Approach, New York: Wiley, 1983. 3. Heart Special Project Committee. Organization, review and administration of cooperative studies (Greenberg Report): a report from the Heart Special Project Committee to the National Advisory Council, May 1967. Control. Clin. Trials 1988; 9: 137–48. 4. P. L. Canner, Monitoring treatment differences in long–term clinical trials. Biometrics 1977; 33: 603–615. 5. Coronary Drug Project Research Group. Practical aspects of decision making in clinical trials: The Coronary Drug Project as a case study. Control. Clin. Trials 1982; 9: 137–148. 6. D. L. DeMets, L. Friedman, C. D. Furberg, Data Monitoring in Clinical Trials: A Case Studies Approach. 2005. Springer Science + Business Media, New York, NY. 7. S. Ellenberg, T. Fleming and D. DeMets, Data Monitoring Committees in Clinical Trials: A Practical Perspective. West Sussex, UK: John Wiley & Sons, Ltd., 2002. 8. ICH Expert Working Group: International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline. Statistical principles for clinical trials. Stats. Med. 1999; 18: 1905–1942. 9. U.S. Department of Health and Human Services.Food and Drug Administration.

8

ALPHA-SPENDING FUNCTION Docket No. 01D–0489. Guidance for Clinical Trial Sponsors on the Establishment and Operations of Clinical Trial Data Monitoring Committees. Federal Register 66: 58151–58153, 2001. Available: http://www.fda.gov/OHRMS/DOCKETS/98fr/ 112001b.pdf.

10. F. J. Anscombe, Sequential medical trials. Journal of the American Statistical Association 1963; 58: 365–383.

24.

25.

26.

11. I. Bross, Sequential medical plans. Biometrics 1952; 8: 188–205. 12. P. Armitage, Sequential Medical Trials, 2nd ed. New York: John Wiley and Sons, 1975. 13. P. Armitage, C. K. McPherson, and B. C. Rowe, Repeated significance tests on accumulating data. J. Royal Stat. Soc. Series A 1969; 132: 235–244. 14. R. Peto, M. C. Pike, P. Armitage, et al., Design and analysis of randomized clinical trials requiring prolonged observations of each patient. 1. Introduction and design. Br. J. Cancer 1976; 34: 585–612.

27.

28.

29.

15. S. J. Pocock: Group sequential methods in the design and analysis of clinical trials. Biometrika 1977; 64: 191–199. 16. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics 1979; 35: 549–556.

30.

17. Beta–Blocker Heart Attack Trial Research Group. A randomized trial of propranolol in patients with acute myocardial infarction. I Mortality results. J. Amer. Med. Assoc. 1982; 247: 1707–1714. 18. K. K. G. Lan and D. L. DeMets, Discrete sequential boundaries for clinical trials. Biometrika 1983; 70: 659–663. 19. K. Kim and D. L. DeMets. Design and analysis of group sequential tests based on the type I error spending rate function. Biometrika 1987; 74: 149–154. 20. I. K. Hwang, W. J. Shih, and J. S. DeCani, Group sequential designs using a family of type I error probability spending function. Stats. Med. 1990; 9: 1439–45. 21. K. K. G. Lan and D. L. DeMets, Group sequential procedures: Calendar versus information time. Stats. Med. 1989; 8: 1191–1198. 22. D. M. Reboussin, D. L. DeMets, K. M. Kim, and K. K.G. Lan, Computations for group sequential boundaries using the Lan–DeMets spending function method. Control. Clin. Trials 2000; 21: 190–207. 23. M. A. Proschan, D. A. Follman, and M. A. Waclawiw. Effects of assumption violations

31.

32.

33.

34.

35.

36.

on type I error rate in group sequential monitoring. Biometrics 1992; 48: 1131–1143. K. K. G. Lan and D. L. DeMets, Changing frequency of interim analyses in sequential monitoring. Biometrics 1989 Sept; 45(3): 1017–1020. K. Kim and D. L. DeMets, Confidence intervals following group sequential tests in clinical trials. Biometrics 1987; 4: 857–864. K. Kim and D. L. DeMets, Sample size determination for group sequential clinical trials with immediate response. Stats. Med. 1992; 11: 1391–1399. Z. Li and N. L. Geller, On the choice of times for date analysis in group sequential trials. Biometrics 1991; 47: 745–750. Cardiac Arrhythmia Suppression Trial (CAST) Investigators. Preliminary report: Effect of endainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N. Engl. J. Med. 1989; 321: 406–412. MERIT–HF Study Group. Effect of metoprolol CR/XL in chronic heart failure: Metoprolol CR/XL randomised intervention trial in congestive heart failure. Lancet 1999; 353: 2001–2007. M. Packer, A. J. S. Coats, M. B. Fowler, H. A. Katus, H. Krum, P. Mohacsi, J. L. Rouleau, M. Tendera, A. Castaigne, C. Staiger, et al. for the Carvedilol Prospective Randomized Cumulative Survival (COPERNICUS) Study Group. Effect of Carvedilol on survival in severe chronic heart failure. New Engl. J. Med. 2001; 334: 1651–1658. A. A. Tsiatis, Repeated significance testing for a general class of statistics used in censored survival analysis. J. Am. Stat. Assoc. 1982; 77: 855–861. A. A. Tsiatis, G. L. Rosner, and C. R. Mehta, Exact confidence intervals following a group sequential test. Biometrics 1984; 40: 797–803. K. Kim and A. A. Tsiatis, Study duration for clinical trials with survival response and early stopping rule. Biometrics 1990; 46: 81–92. K. K. G. Lan and J. Lachin, Implementation of group sequential logrank tests in a maximum duration trial. Biometrics 1990; 46: 759–770. J. W. Lee and D. L. DeMets, Sequential comparison of changes with repeated measurement data. J. Am. Stat. Assoc. 1991; 86: 757–762. M. C. Wu and K. K. G. Lan, Sequential monitoring for comparison of changes in a response variable in clinical trials. Biometrics 1992; 48: 765–779.

ALPHA-SPENDING FUNCTION 37. S. J. Gange and D. L. DeMets, Sequential monitoring of clinical trials with correlated categorical responses. Biometrika 1996; 83: 157–167. 38. K. K. G. Lan, D. M. Reboussin, D. L. DeMets: Information and information fractions for design and sequential monitoring of clinical trials. Communicat. Stat.– Theory Methods 1994; 23: 403–420. 39. D. O. Scharfstein, A. A. Tsiatis, and J. M. Robins, Semiparametric efficiency and its implication on the design and analysis of group–sequential studies. J. Am. Stat. Assoc. 1997; 92: 1342–1350. 40. K. Kim, Point estimation following group sequential tests. Biometrics 1989; 45: 613–617. 41. C. Jennison and B. W. Turnbull: Interim analyses: The repeated confidence interval approach. J. Royal Stat. Soc., Series B 1989; 51: 305–361. 42. S. S. Emerson and T. R. Fleming, Parameter estimation following group sequential hypothesis testing. Biometrika 1990; 77: 875–892. 43. M. D. Hughes and S. J. Pocock, Stopping rules and estimation problems in clinical trials. Stats. Med. 1981; 7: 1231–1241. 44. Z. Li and D. L. DeMets, On the bias of estimation of a Brownian motion drift following group sequential tests. Stat. Sinica 1999; 9: 923–937. 45. J. C. Pinheiro and D. L. DeMets: Estimating and reducing bias in group sequential designs with Gaussian independent structure. Biometrika 1997; 84: 831–843. 46. D. Siegmund, Estimation following sequential tests. Biometrika 1978; 65: 341–349. 47. J. Whitehead, On the bias of maximum likelihood estimation following a sequential test. Biometrika 1986; 73: 573–581. 48. D. L. DeMets and K. K. G. Lan, The alpha spending function approach to interim data analyses. In: P. Thall (ed.), Recent Advances in Clinical Trial Design and Analysis. Dordrecht, The Netherlands: Kluver Academic Publishers, 1995, pp. 1–27.

FURTHER READING M. N. Chang and P. C. O’Brien, Confidence intervals following group sequential tests. Control. Clin. Trials 1986; 7: 18–26.

9

T. Cook and D. L. DeMets, Statistical Methods in Clinical Trials. Boca Raton, FL: CRC Press/Taylor & Francis Co., 2007. D. L. DeMets, Data monitoring and sequential analysis – An academic perspective. J. Acq. Immune Def. Syn. 1990; 3(Suppl 2):S124–S133. D. L. DeMets, R. Hardy, L. M. Friedman, and K. K. G. Lan, Statistical aspects of early termination in the Beta–Blocker Heart Attack Trial. Control. Clin. Trials 1984; 5: 362–372. D. L. DeMets and K. K. G. Lan, Interim analysis: the alpha spending function approach. Stats. Med. 1994; 13: 1341–1352. D. L. DeMets, Stopping guidelines vs. stopping rules: A practitioner’s point of view. Communicat. Stats.–Theory Methods 1984; 13: 2395–2417. T. R. Fleming and D. L. DeMets, Monitoring of clinical trials: issues and recommendations. Control. Clin. Trials 1993; 14: 183–197. J. L. Haybittle, Repeated assessment of results in clinical trials of cancer treatment. Brit. J. Radiol. 1971; 44: 793–797. K. K. G. Lan, W. F. Rosenberger, and J. M. Lachin: sequential monitoring of survival data with the Wilcoxon statistic. Biometrics 1995; 51: 1175–1183. J. W. Lee, Group sequential testing in clinical trials with multivariate observations: a review. Stats. Med. 1994; 13: 101–111. C. L. Meinert, Clinical Trials: Design, Conduct, and Analysis. New York: Oxford University Press, 1986. S. Piantadosi. Clinical Trials: A Methodologic Perspective, 2nd ed. Hoboken, NJ: John Wiley and Sons, Inc., 2005. S. J. Pocock. Statistical and ethical issues in monitoring clinical trials. Stats. Med. 1993; 12: 1459–1469. S. J. Pocock. When to stop a clinical trial. Br. Med. J. 1992; 305: 235–240. E. Slud and L. J. Wei, Two–sample repeated significance tests based on the modified Wilcoxon statistic. J. Am. Stat. Assoc. 1982; 77: 862–868. A. Wald, Sequential Analysis. New York: John Wiley and Sons, 1947.

ANALYSIS OF VARIANCE ANOVA

The basis for generalizability of a successful clinical trial is strengthened when the coverage of a study is as broad as possible with respect to geographical area, patient demographics, and pre-treatment characteristics as well as other factors that are potentially associated with the response variables. At the same time, heterogeneity among patients becomes more extensive and conflicts with the precision of statistical estimates, which is usually enhanced by homogeneity of subjects. The methodology of the ANOVA is a means to structure the data and their validation by accounting for the sources of variability such that homogeneity is regained in subsets of subjects and heterogeneity is attributed to the relevant factors. The method ANOVA is based on the use of sums of squares of the deviation of the observations from respective means (→ Linear Model). The tradition of arraying sums of squares and resulting F-statistics in an ANOVA table is so firmly entrenched in the analysis of balanced data that the extension of the analysis to unbalanced data is necessary. For unbalanced data, many different sums of squares can be defined and then be used in the numerators of F-statistics providing it tests for a wide variety of hypotheses. In order to provide a practically relevant and useful approach, the ANOVA through the cell means model will be introduced. The concept of the cell means model was introduced by Searle (2, 3), Hocking and Speed (4), and Hocking (5) to resolve some of the confusion associated with ANOVA models with unbalanced data. The simplicity of such a model is readily apparent: No confusion exists on which functions are estimable, what their estimators are, and what hypotheses can be tested. The cell means model is conceptually easier, it is useful for understanding the ANOVA models, and it is, from the sampling point of view, the appropriate model to use. In many applications, the statistical analysis is characterized by the fact that a number of detailed questions need to be answered. Even if an overall test is significant, further

¨ Dr. JORG KAUFMANN

AG Schering SBU Diagnostics & Radiopharmaceuticals Berlin, Germany

1

INTRODUCTION

The development of analysis of variance (ANOVA) methodology has in turn had an influenced on the types of experimental research being carried out in many fields. ANOVA is one of the most commonly used statistical techniques, with applications across the full spectrum of experiments in agriculture, biology, chemistry, toxicology, pharmaceutical research, clinical development, psychology, social science, and engineering. The procedure involves the separation of total observed variation in the data into individual components attributable to various factors as well as those caused by random or chance fluctuation. It allows performing hypotheses tests of significance to determine which factors influence the outcome of the experiment. However, although hypothesis testing is certainly a very useful feature of the ANOVA, it is by no means the only aspect. The methodology was originally developed by Sir Ronald A. Fisher (1), the pioneer and innovator of the use and applications of statistical methods in experimental design who coined the name ‘‘Analysis of Variance – ANOVA.’’ For most biological phenomena, inherent variability exists within the response processes of treated subjects as well as among the conditions under which treatment is received, which results in sampling variability, meaning that results for a subject included in a study will differ to some extent from those of other subjects in the affected population. Thus, the sources of variability must be investigated and must be suitably taken into account when data from comparative studies have to be evaluated correctly. Clinical studies are in particular a fruitful field for the application of this methodology.

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

ANALYSIS OF VARIANCE ANOVA

analyses are, in general, necessary to assess specific differences in the treatments. The cell means model provides within the ANOVA framework the appropriate model for a correct statistical inference and provides such honest statements on statistical significance in a clinical investigation. 2

FACTORS, LEVELS, EFFECTS, AND CELLS

One of the principal uses of statistical models is to explain variation in measurements. This variation may be caused by the variety of factors of influence, and it manifests itself as variation from one experimental unit to another. In well-controlled clinical studies, the sponsor deliberately changes the levels of experimental factors (e.g., treatment) to induce variation in the measured quantities to lead to a better understanding of the relationship between those experimental factors and the response. Those factors are called independent and the measured quantities are called dependent variables. For example, consider a clinical trial in which three different diagnostic imaging modalities are used on both men and women in different centers. Table 1 shows schematically how the resulting data could be arrayed in a tabular fashion. The three elements used for classifications (center, sex, and treatment) identify the source of variation of each datum and are called factors. The individual classes of the classifications are the levels of the factor (e.g., the three different treatments T1, T2, and T3 are the three levels of the factor treatment). Male and female are the two levels of the

Table 1. Factors, Levels and Cells (ijk)

Treatment (k) Center (i)

Sex (j)

1

male female

2

male female

3

male female

T1

T2

cell (2 1 2)

T3

factor sex, and center1, center2, center3 are the three levels of the factor center. A subset of the data present for a ‘‘combination’’ of one level of each factor under investigation is considered a cell of the data. Thus, with the three factors, center (3 levels), sex (2 levels), and treatment (3 levels), 3 × 2 × 3 = 18 cells numbered by triple indexing i j k exist. Repeated measurements in one cell may exist, which they usually do. Unbalanced data occur when the number of repeated observations per cell nijk are different for at least some of the indices (i, j, k). In clinical research, this occurrence is rather the role than the exception. One obvious reason could be missing data in an experiment. Restricted availability of patients for specific factor combination is another often experienced reason. 3 CELL MEANS MODEL A customary practice since the seminar work of R. A. Fisher has been that of writing a model equation as a vehicle for describing ANOVA procedures. The cell means model is now introduced via a simple example in which only two treatments and no further factors are considered. Suppose that yir , i = 1, 2, r = 1, . . . , ni represents a random sample of two normal populations with means µ1 and µ2 and common variance σ 2 . The data point yir denotes the rth observation on the ith population of size ni and its value assumed to follow a Gaussian normal distribution: yir ∼ N(µi , σ 2 ). The fact that the sizes n1 and n2 of the two populations differ indicates that a situation of unbalanced data exists. In linear model form, it is written yir = µi + eir , i = 1, 2; r = 1, . . . , ni

(1)

where the errors eir are identically independent normal N(0, σ 2 ) distributed (i.i.d. variables). Notice, a model consists of more than just a model equation: It is an equation such as Equation (1) plus statements that describe the terms of the equation. In the example above, µi is defined as the population mean of the ith population and is equal to the expectation of yir E( yir ) = µi , r = 1, . . . , ni , for i = 1, 2

(2)

ANALYSIS OF VARIANCE ANOVA

The difference yir − E( yir ) = yir − µi = eir is the deviation of the observed yir value from the expected value E( yir ). This deviation denoted eir is called the error term or residual error term, and from the introduction of the model above, it is a random variable with expectation zero and variance v(eir ) = σ 2 . Notice, in the model above, the variance is assumed to be the same for all eir s. The cell means model can now be summarized as follows:

yir = µi + eir

(3)

E( yir ) = µi E(eir ) = 0 v(eir ) = σ 2 for all i and r

Notice that Equation (3) does not assume explicitly the Gaussian normal distribution. In fact, one can formulate the cell means model more generally by only specifying means and variances. Below, however, it is restricted to the special assumption eir ≈ i.i.d. N(0, σ 2 ). It will be shown that the cell means model can be used to describe any of the models that are classically known as ANOVA models.

4

ONE–WAY CLASSIFICATION

Let us begin with the case of a cell means model in which the populations are identified by a single factor with i levels and ni observations at the ith level for i = 1, . . . , i. 4.1 Example 1 A clinical study was conducted to compare the effectiveness of three different doses of a new drug and placebo for treating patients with high blood pressure. For the study, 40 patients are included. To control for unknown sources of variation, 10 patients each were assigned at random to the four treatment groups. As response, one considered the difference in diastolic blood pressure measurement between baseline (pre-value) and the measurement four weeks after administration of treatment. The response measurements yir , sample means yi , and sample variances s2i are shown in Table 2. The cell means model to analyze the data of the example 1 is then given as yir = µi + eir i = 1, 2, 3, 4; r = 1, . . . , 10

dose 1

dose 2

dose 3

1.6 0.1 −0.3 1.9 −1.3 0.6 −0.6 1.0 −1.1 1.1

1.7 4.7 0.2 3.5 1.7 2.6 2.3 0.2 0.3 0.7

3.6 2.3 2.1 1.6 1.8 3.3 4.3 1.2 1.5 2.4

3.1 3.5 5.9 3.0 5.2 2.6 3.7 5.3 3.4 3.6

where µi defines the population mean at the ith level as E( yir ) = µi and σ 2 is the common variance of the eir .

yi.

0.30

1.79

2.41

3.93

sample means

s2i

1.24

2.31

1.01

1.26

sample variances

σˆ 2 = 1.46

(4)

eir ≈ i.i.d. N(0, σ 2 )

Table 2. Example 1, Data for a Dose Finding Study

Placebo

3

estimated common error variance of the model

4

5

ANALYSIS OF VARIANCE ANOVA

PARAMETER ESTIMATION

The µi s in Equation (4) are usually estimated by the methods of least squares. The estimator for µi (i = 1, . . . , i) is then given by µˆ i =

1
yi =: yi. nir r

(5)

2 σˆ =

1

( yir − µˆ i )2 n.- I r

(6)

i

where n. =




ni

i

The sample means and sample variances for each treatment and the estimated common error variance σ 2 are given in Table 2.

SST =

6 THE R(.) NOTATION – PARTITIONING SUM OF SQUARES


( yi − y.)2

(7)

with 1
yi ni

y2i r

(10)

r

The error sum of squares after fitting the model E( yir ) = µi is SSE(µi ) =



i

i

y. =



i

The ANOVA procedure can be summarized as follows: Given n observations yi s, one defines the total sum of squared deviations from the mean by Total SS =

error sum of squares, respectively. Thus, Total SS = Model SS + Error SS. The Total SS always has the same value for a given set of data because it is nothing other than the sums of squares of all data points relative to the common mean. However, the partitioning into Model SS and Error SS depends on model selection. Generally, the addition of a new factor to a model will increase the Model SS and, correspondingly, reduce the Error SS. When two models are considered, each sum of squares can be expressed as the difference between the sums of squares of the two models. Therefore, the approach related to given sum of squares allows the comparison of two ANOVA models very easily. In the one-way classification in Equation (4), the total sum of squares of each observation is

( yir − µˆ i )2

(11)

r

R(µi ) = SST − SSE(µi ) is denoted the reduction in sum of squares because of fitting the model E( yir ) = µi

i

The ANOVA technique partitions the variation among observations into two parts: the sum of squared deviations from the model to the overall mean
( yˆ i − y.)2 (8) Model SS = i

and the sum of squared deviations from the observed values yi values to the model Error SS = SSE =




( yi − yˆ i )2

(9)

i

These two parts are called the sum of squares because of the model and the residual or

Fitting the simplest of all linear models, the constant mean model E( yir ) = µ, the estimate of E( yir ) would be yˆ ir = µˆ = y and the error sum of squares results as SSE(µ) =



i

( yir − µ) ˆ 2

r

R(µ) = SST − SSE(µ) is denoted the reduction in sum of squares because of fitting the model E( yir ) = µ. The two models E( yir ) = µi and E( yir ) = µ can now be compared in terms of their respective reductions in sum of

ANALYSIS OF VARIANCE ANOVA

5

Table 3a. ANOVA—Partioning the Total Sum of Squares 1-way Classification, Cell Means Model

Source of variation

df

Sum of square

Mean square

Model µ

1

R(µ)

R(µ)

Model µi

I−1

R(µi /µ)

R(µi /µ) I−1

Residual

n. − I

SSE = SST − R(µi )

SST − R(µi ) n. − I

Total

n.

F statistic

R(µi /µ) (I − 1)



SST − R(µi ) n. − I

SST

Table 3b. Partitioning the Total Sum Sqares Adjusted for Mean

Source of variation

df

Sum of square

Mean square

Model µi

I−1

R(µi /µ)

R(µi /µ) I−1

Residual

n. − I

SSE = SST − R(µi )

SST − R(µi ) n. − I

Total a.f.m

n. − 1

SST − R(µ)

squares given by R(µi ) and R(µ). The difference (R(µi ) − R(µ) is the extent to which fitting E( yir ) = µi brings about a greater reduction in sum of squares than does fitting E( yir ) = µ. Obviously, the R(.) notation is a useful mnemonic for comparing different linear models in terms of the extent to which fitting each accounts for a different reduction in the sum of squares. The works of Searle (2, 3), Hocking (5), and Littel et al. (6) are recommended for deeper insight. It is now very easy to partition the total sum of squares SST into terms that develop in the ANOVA. Therefore, the identity SST = R(µ) + (R(µi ) − R(µ)) + (SST − R(µi )) (12) = R(µ) + R(µi /µ) + SSE(µi ) or SST − R(µ) = R(µi /µ) + SSE(µi ) are used with R(µi /µ) = R(µi ) − R(µ). The separation of the Table 3a and Table 3b is

F statistic  R(µi /µ) SST − R(µi ) (I − 1) n. − I

appropriate to Equation (12) the first and last line. Table 3a displays the separation into the components atributable to the model µ in the first line, to the model µi extent µ in the second line, and, in the third line to the error term, the last line to the total sum of squares. Table 3b displays only the separation into the two components attributable to the model µi extent µ and in the second line to the error term. 7

ANOVA–HYPOTHESIS OF EQUAL MEANS

Consider the following inferences about the cell means. This analysis includes the initial null hypothesis of equal means (global hypotheses, all means simultaneous by the same) so-called ANOVA hypothesis contingent with pairwise comparisons, contrasts, and other linear function comprising either hypothesis tests or confidence intervals. In starting off with the model E( yij ) = µI , the global null hypothesis H0 : µ1 = µ2 = . . . = µI

6

ANALYSIS OF VARIANCE ANOVA

Table 3c. ANOVA Example 1

Source of variation

df

Sum of square

Mean square

F statistic

Pr > F

Model µi Residual Total a.f.m.

3 36 39

67.81 52.52 120.33

22.60 1.46

15.5

<0.0001

is of general interest. A suitable F-statistic can be used for testing this hypothesis H0 (see standard references (2–6). The F-statistic testing H 0 and the sums of squares of Equation (12) are tabulated in Table 3a and Table 3b, columns 3–5. The primary goal of the experiment in example 1 was to show that the new drug is effective compared with placebo. At first, one may test the global null-hypotheses H0 : µ1 = µ2 = µ3 = µ4 with E( yir ) = µi . Table 3c shows information for testing the hypothesis Ho . R(µi /µ) = R(µi ) − R(µ) = 67.81 is the difference for the respective reductions in sum of squares for the two models E( yir ) = µi and E( yir ) = µ. From the mean squares R(µi /µ)/(I − 1) = 22.60 and the error term (SST − SSE(µi ))/ (n. − I) = σˆ 2 = 1.46 one obtains the F-statistic F = 15.5 for testing the null-hypothesis H 0 and the probability Pr(F > Fα ) < 0.0001. As this probability is less then the type I error α = 0.05, the hypothesis Ho can be rejected in favor of the alternative µi = µj for at least one pair i and j of the four treatments. Rejection of the null hypothesis H0 indicates that differences exist among treat ments, but it does not show where the differences are located. Investigators’ interest is rarely restricted to this overall test, but rather to comparisons among the doses of the new drug or placebo. As a consequence, multiple comparisons comparing the three classes with placebo are required. 8

MULTIPLE COMPARISONS

In many clinical trials, often more than two drugs or more than two levels of one drug are considered. Having rejected the global

hypothesis of equal treatment means (e.g., when the probability of the F-statistic in Table 3c, last column, is less then 0.05), questions related to picking out drugs that are different from others or determining what dose level is different from the others and placebo have to be attacked. These analyses generally constitute making many (multiple) further comparisons among the treatments in order to detect effects of prime interest to the researcher. The excessive use of multiple significance tests in clinical trials can greatly increase the chance of false-positive findings. A large amount of statistical research has been devoted to multiple comparison procedures and the control of false-positive results caused by multiple testing. Each procedure usually has the objective of controlling the experiment-wise or family-wise error rate. A multiple test controls the experiment-wise or family-wise multiple level α, if the probability to reject at least one of the true null hypotheses does not exceed α, irrespective of how many hypotheses and which of them is in fact true. For ANOVA, several types of multiple comparison procedures exist that adjust the critical value of the test statistic. For example, the Scheffe -, Tukey-, and Dunnetttest, and procedures that adjust the compari son-wise P-values (e.g., the Bonferroni-Holm procedure), and the more general closed-test procedures exist (7, 8). Marcus et al. (9) introduced those so-called closed multiple test procedures, which keeps the family-wise multiple level α under control. The closed-test principle requires a special structure among the set of null hypotheses, and it can be viewed as a general tool for deriving a multiple test. In general, the use of an appropriate multiple-comparison test for inference concerning treatment comparisons is indicated (10):

ANALYSIS OF VARIANCE ANOVA

1. to make an inference concerning a particular comparison that has been selected on the basis of how the data have turned out; 2. to make an inference that requires the simultaneous examination of several treatment comparisons, for example, the minimum effected dose in dose findings studies; and 3. to perform so-called ‘‘data dredging,’’ namely, assembling the data in various ways in the hope that some interesting differences will be observable.

9

At the same time, the more extensive heterogeneity among patients conflicts with the precision of statistical estimates, which is usually enhanced by requiring homogeneity of subjects (12). Two study design strategies are followed: 1. Stratified assignment of treatments to subjects who are matched for similarity on one or more block factors, such as gender, race, age, initial severity, or on strata, such as region or centers. Separately and independently within each stratum, subjects are randomly assigned to treatment groups in a block randomization design (11). When a stratified assignment is used, treatment comparisons are usually based on appropriate within stratum/center differences (13). 2. A popular alternative to the block randomization design is the stratification of patients according to their levels on prognostic factors after complete randomization in the analysis phase. This alternative strategy, termed poststratification, leads exactly to the same kind of statistical analysis, but it has disadvantages compared with prestratification. Pre-stratification guards by design against unlikely but devastating differences between the groups in their distributions on the prognostic factors and in sample sizes within stratum. With pre-stratification, these will be equal—or least close to equal—by design. With post-stratification due to randomization, they will be equal only in a long-term average sense (14).

TWO–WAY CROSSED CLASSIFICATION

Two basic aspects of the design of experiments are the error control and the optimal structuring of the treatment groups. A general way of reducing the effect of uncontrolled variations on the error of treatment comparisons is grouping the experimental units (patients) into sets of units being alike (uniform) as much as possible. All comparisons are then made within and between sets of similar units (11). Randomization schemes (e.g., as complete randomization, randomized blocks) are part of the error control of the experimental design. On the other hand, the structure of treatments, what factor(s) and factor levels are to be observed is called the treatment design (6). The factorial treatment design is one of the most important and widely used treatment structures. Thereby one distinguishes between treatment and classification factors. The factorial design can be used with any randomization scheme. Factorial experiments can be compared with the one factor at a time approach, and they have the advantage of giving greater precision in the estimating of overall factor effects. They enable the exploration of interactions between different factors, which are being and allow an extension of the range of validity of the conclusions by the insertion of additional factors. Factorial designs enhance the basis for any generalizability of the trial conclusion with respect to geographical area, patient demographics, and pre-treatment characteristics as well as other factors that are potentially associated with the response variable.

7

10

BALANCED AND UNBALANCED DATA

When the number of observations in each cell is the same, the data shall be described as balanced data. They typically come from well-designed factorial trials that have been executed as planned. The analysis of balanced data is relatively easy and have been extensively described (15). When the number of observations in the cells is not uniform, the data shall be described as unbalanced data.

8

ANALYSIS OF VARIANCE ANOVA

Table 4. Liver Encym (Log Transform)—2-way Crossed Classification Treatment (T), Liver Impairment (B) Case T B X Case T B X

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

2 2 1 2 1 1 1 1 1 2 1 2 1 2 2 1 2 1 2 1

1 1 2 1 1 2 1 1 1 1 1 2 1 1 2 1 1 1 1 1

0.85 1 1.32 0.95 0.90 1.43 1.30 1.08 1 1.08 1.15 1.60 1.34 0.95 1.36 0.95 0.90 1.28 0.95 0.85

T = 1 contrast media 1 T = 2 contrast media 2

21 22 23 24 25 26 27 28 29 30 31 33 34 35 36 37 38 39 40

1 2 2 2 2 1 2 2 1 2 1 2 1 1 2 2 1 2 1

1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 2 1 1

1.11 1.28 0.90 1 1 0.85 1 0.90 0.90 1 1 1.04 1.49 0.85 1.08 0.90 1.64 1.26 1.08

B = 1 liver impairment no B = 2 liver impairment yes

Case

T

B

X

Case

T

B

X

101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120

2 1 1 1 2 1 1 1 1 2 2 2 2 2 1 2 1 1 1 2

1 1 2 1 1 2 1 1 2 2 2 1 2 2 1 1 2 1 1 1

1.15 1.11 1.15 0.95 0.90 1.52 1.36 1.18 1.15 1.46 1.18 1.04 1.34 1 0.95 0.70 1.20 0.85 1.30 1.15

121 122 123 124 125 127 128 129 130 131 132 133 134 135 136 137 138 139 140

2 1 1 1 1 2 2 2 1 2 2 2 1 2 1 1 1 2 2

1 1 2 2 2 1 1 1 1 1 1 2 1 1 1 1 2 1 1

1 0.95 1.04 1.43 1.53 1.04 1 1 1.26 1 0.90 1.15 0.95 1.11 1.32 0.95 1.40 0.90 1.04

ANALYSIS OF VARIANCE ANOVA

This fact will be illustrated using a numerical example. Suppose interest exists in the comparison of two contrast media, T = 1, 2, regarding a liver enzym and a second factor B, with categories B = 1, 2, describing the initial severity of the liver impairment (B = 1 not impaired, B = 2 impaired). Table 4 shows the data of a clinical trial designed as a result. Let xijk be the kth observation on treatment i for impaired liver at severity j, where i = 1, 2, j = 1, 2, and k = 1, . . . , nij , with nij being the number of observations on treatment i and impairment j. For the sake of generality, treatment is referred to as row–factor A with levels i = 1, . . . , i and liver impairment as column-factor B with levels j = 1, . . . , j. Notice the data of Table 4 are represented as logarithms yijk = log(xijk ) of the liver enzyme data xijk . The cell means model is as follows: yijk = µij + eijk , i = 1, . . . , I j = 1, . . . , J, k = 1, . . . , nij

(13)

A normal (Gaussian) distribution is assumed for the log-transformed measurements

An important difference between balanced and unbalanced data is the definition of a mean over rows or columns, respectively. For example, the row mean of the cell means µij in row i is straightforward µi . = 1/J

eijk ≈ i.i.d. N(0, σ 2 )

It follows as for the one-way classification that the cell means µij are estimated by the mean of the observation in the appropriate cells (i, j) yijk =: yij.

(14)

k

and the residual error variance is estimated as σˆ 2 = SSE/(N − s)

(15)

where




SSE =

i

j

( yijk − µˆ ij )2

k

and where  s = I × J − (number of empty nij is the total sample size. cells). N = i

j

µij

(16)

µˆ ij

(17)

j

and its estimator is ˆ i . = 1/J µ


j

with variance estimate ˆ i .) = 1/J 2 1/nij σˆ 2 v(µ ˆ 1. = 1.22 and µ ˆ 2. = 1.15 is For example 2, µ obtained. A different row mean of the cell means µij in row i, sometimes thought to be an interesting alternative to µi ., is the weighted mean with the weights nij /ni. : µi . =


nij j

ˆ i . = µ







ni.

µij

(18)

estimated by the mean over all observation in row i,

yijk = log xijk

µˆ ij = 1/nij

9


nij ni.

µˆ ij = yi.

(19)

with variance estimate 

ˆ i ) = σˆ 2 /ni. v(µ ˆ 1. = 1.16 and µ ˆ 2. = 1.05 is For example 2, µ obtained. The difference between these two cell mean estimates µi. and µi. consists in the fact that µi. is independent from the number of the data points in the cells, whereas µi. depends on the number of observations in each cell. As the sample sizes of participating centers in a multi-center study are mostly different, a weighted mean over the centers is a better overall estimator for treatment comparisons (13, 14, 16). If it can be ascertained that the population in a stratum or block is representative of a portion of the general population of interest, for example, in post-stratification, then it is natural to use

10

ANALYSIS OF VARIANCE ANOVA

a weighted mean for an overall estimator of a treatment effect. Many clinical trials result in unbalanced two-way factorial designs. Therefore, it is appropriate to define row means or column respectively as weighted means resulting in different sum of squares. If weights other than these two possibilities are wanted (e.g., Cochran-MantelHaenszel weights), one may consider a general weighting of the form
µi . = tij µij (20) j

with  j



ˆ i . = tij = 1 and the estimator µ

j



ˆ i ) = σˆ 2 tij yij. with v(µ

 j

t2ij /ni .

11 INTERACTION BETWEEN ROWS AND COLUMNS Consider an experiment with two factors A (with two levels) and B (with three levels). The levels of A may be thought of as two different hormone treatments and the three levels of B as three different races. Suppose that no uncontrolled variation exists and that observations as of Fig 1a are obtained. These observations can be characterized in various equivalent ways: 1. the difference between the observations corresponding to the two levels of A is the same for all three levels of B; 2. the difference between the observations for any two levels of B is the same for the two levels of A; and mij

3. the effects of the two factors are additive. For levels i and i of factor A and levels j and j of factor B, consider the relation among means given by µij − µij = µi j − µi j

(21)

If this case holds, the difference in the means for level i and i of factor A is the same for levels j and j of factor B and, vice versa, the change from level j to level j of factor B is the same for both levels of factor A. If Equation (21) holds for all pairs of levels of factor A and all pairs of levels of factor B, or when the conditions a), b), or c) are satisfied, one can say factor A does not interact with factor B: No interaction exists between factor A and B. If the relation in Equation (21) or the conditions a), b), or c) are not satisfied, then an interaction exists between factor A and factor B. Many ways exist in which interaction can occur. A particular method is shown in Fig 1b. The presence of an interaction can mislead conclusions about the treatment effects in terms of row effects. It is therefore advisable to assess the presence of interaction before making conclusions, which can be done by testing an interaction hypothesis. The test may indicate that the data are consistent with the absence of interaction but may not prove that no real interaction exists.

mij A1

16

16

14

A2

14

12

12

10

10

8

8

6

6 1

2

3

Factor B

Figure 1a. Two-way classification, no interaction 2 rows, 3 columns.

A2

A1 1

2

3

Factor B

Figure 1b. Two-way classification, interaction 2 rows, 3 columns.

ANALYSIS OF VARIANCE ANOVA

Table 5. Sample Size, Means

T1 T2

12

B1

B2

(27) 1.07 (32) 1.00

(12) 1.36 (7) 1.30

ANALYSIS OF VARIANCE TABLE

ANOVA tables have been successful for balanced data. They are generally available, widely documented, and ubiquitously accepted (15). For unbalanced data, often no unique, unambiguous method for presenting the results exists. Several methods are available, but often not as easily interpretable as methods for balanced data. In the context of hypothesis testing, or of arraying sums of squares in an ANOVA format, a variety of sums of squares are often used. The problem in interpreting the output of computerspecific programs is to identify those sums of squares that are useful and those that are misleading. The information for conducting tests for row effects, column effects, and interactions between rows and columns is summarized in an extended ANOVA table.

Various computational methods exist for generating sums of squares in an ANOVA table (Table 5) since the work of Yates (17). The advantage of using the µij -model notation introduced above is that all the µij are clearly defined. Thus, a hypothesis stated in terms of the µij is easily understood. Speed et al. (18), Searle (2, 3), and Pendleton et al. (16) gave the interpretations of four different types of sums of squares computed (e.g., by the SAS, SPSS, and other systems). To illustrate the essential points, use the model in Equation (13), assuming all nij > 0. For reference, six hypotheses of weighted means are listed in Table 6 that will be related to the different methods (16, 18). A typical method might refer to H1 , H 2 , or H 3 as ‘‘main effect of A,’’ row effect. The hypotheses H 4 and H 5 are counterparts of H 2 and H 3 generally associated with ‘‘main effect B,’’ column effect. The hypothesis of no interaction is H 6 , and it is seen to be common to all methods under the assumption nij < 0. The hypotheses H 1 , H 2 , and H 3 agree in the balanced case (e.g., if nij = n for all i and j but not otherwise). The hypothesis H3 does not depend on the nij . All means have the same weights 1/j and are easy to interpret. As it states, no difference exists in the levels of the factor A when averaged over all levels of factor B [Equation (16)].

Table 6. Cell Means Hypotheses

Hypothesis

Main Effect

Weighted Means   j nij µij /ni. = j nij µij /ni.    j nij µij = i j nij nij µij /n.j

H1 ⎫ ⎬ H2 ⎭ H3 H  4

Factor B

µi = µi    i nij µij = j i nij nij µij /ni.

H5

columns

µ.j = µ.j

H6

Interaction A × B

µij − µij − µij + µij = 0

Factor A rows

11

For all i, i, j, j, i?i, j?j H1 weighted mean equation (18) H2 Chochran Mantel-Haenszel weights equation (20) for t = nij nij /n.j H3 weighted mean equation (16) H4 counterpart of H2 with factor B H5 counterpart of H3 with factor B H6 interaction between factor A and factor B

12

ANALYSIS OF VARIANCE ANOVA

The hypotheses H1 and H 2 represent comparisons of weighted averages [Equations (18) and (20)] with the weights being a function of the cell frequencies. A hypothesis weighted by the cell frequencies might be appropriate if the frequencies reflected population sizes but would not be considered as the standard hypothesis. Table 7 specifies the particular hypothesis tested each type of ANOVA sums of squares. Table 8 shows three different analyses of variance tables for the example 2 ‘‘liver encym’’ computed with PROC GLM SAS. The hypotheses for the interaction term TxB is given by H6 . The test is the same for Type I, II, and III sums of squares. In this example, no interaction exists between treatment and liver impairment, P-value Pr > F = 0.94.

The hypothesis for ‘‘main effect A’’—the treatment comparison—is given by H1 , H 2 , or H 3 , corresponding with the different weighting means (Table 6 and Table 7). The range of the results for the treatment comparison are different (Table 8, source T) 1. for Type I, the P-value, Pr > F = 0.0049, is less then 0.005 (highly significant) 2. for Type II, the P-value, Pr > F = 0.074, is between 0.05 < P < 0.1 (not significant) 3. for Type III, the P-value, Pr > F = 0.142, is greater then 0.1 (not significant). The hypothesis H2 (i.e., Type II) is appropriate for treatment effect in the analysis of this

Table 7. Cell Means Hypotheses Being Tested

Sum of Sqares

Row effect, Factor A Column effect, Factor B Interaction, A×B

Typ I

Typ II

Typ III

H1 H4 H6

H2 H4 H6

H3 H5 H6

Typ I, Type II, and Type III agree when balanced data occur Table 8. Liver encyms -Two way classification with interaction term Treatment (T), impairment (B), interaction (T*B)

Source T B T*B Source T B T*B Source T B T*B

DF

Typ I SS

Mean Square

F-Value

Pr > F

1 1 1

0.20615 1.22720 0.00015

0.20615 1.22720 0.00015

8.41 50.08 0.01

0.0049 0.0001 0.9371

DF

Typ III

Mean Square

F-Value

Pr > F

1 1 1

0.05413 1.19127 0.00015

0.05413 1.19127 0.00015

2.21 48.61 0.01

0.1415 0.0001 0.9371

DF

Typ II

Mean Square

F-Value

Pr > F

1 1 1

0.08038 1.22720 0.00015

0.08038 1.22720 0.00015

3.28 50.08 0.01

0.0742 0.0001 0.9371

ANALYSIS OF VARIANCE ANOVA

example—a two-way design with unbalanced data. No interaction exists between treatment and liver impairment. The different test results for H 1 , H 2 , and H 3 result from the unbalanced data and factor—liver impairment. Any rules for combining centers, blocks, or stratas in the analysis should be set up prospectively in the protocol. Decisions concerning this approach should always be taken blind to treatment. All features of the statistical model to be adopted for the comparison of treatments should be described in advance in the protocol. The hypothesis H2 is appropriate in the analysis of multi-centers trials when treatment differences over all centers (13, 14, 19) are considered. The essential point emphasized here is that the justification of a method should be based on the hypotheses being tested and not on heuristic grounds or computational convenience. In the presence of a significant interaction, the hypotheses of main effects may not be of general interest and more specialized hypotheses might be considered. With regard to missing cells, the hypotheses being tested can be somewhat complex for the various procedures or types of ANOVA tables. Complexities associated with those models are simply aggravated when dealing with models for more than two factors. How many factors exist or how many levels each factor has, the mean of the observations in each filled cell, is an estimator of the population mean for that cell. Any linear hypothesis about cell means of any non empty cell is testable; see the work of Searle (3: 384–415). REFERENCES 1. R. A. Fisher, Statistical Methods for Research Workers. Edinburgh: Oliver & Boyd, 1925. 2. S. R. Searle, Linear Models. New York: John Wiley & Sons, 1971. 3. S. R. Searle, Linear Models for Unbalanced Data. New York: John Wiley & Sons, 1987. 4. R. R. Hocking and F. M. Speed, A full rank analysis of some linear model problems. JASA 1975; 70: 706–712. 5. R. R. Hocking, Methods and Applications of Linear Models-Regression and the Analysis

13

of Variance. New York: John Wiley & Sons, 1996. 6. R. C. Littel, W. W. Stroup, and R. J. Freund, SAS for Linear Models. Cary, NC: SAS Institute Inc., 2002. 7. P. Bauer, Multiple primary treatment comparisons on closed tests. Drug Inform. J. 1993; 27: 643–649. 8. P. Bauer, On the assessment of the performance of multiple test procedures. Biomed. J. 1987; 29(8): 895–906. 9. R. Marcus, E. Peritz, and K. R. Gabriel, On closed testing procedures with special reference to ordered analysis of variance. Biometrica 1976; 63: 655–660 10. C. W. Dunnett and C. H. Goldsmith, When and how to do multiple comparison statistics in the pharmaceutical industry. In: C. R. Buncker and J. Y. Tsay, eds. Statistics and Monograph, vol. 140. New York: Dekker, 1994. 11. D. R. Cox, Planning of Experiments. New York: John Wiley & Sons, 1992. 12. G. G. Koch and W. A. Sollecito, Statistical considerations in the design, analysis, and interpretation of comparative clinical studies. Drug Inform. J. 1984; 18: 131–151. 13. J. Kaufmann and G. G. Koch, Statistical considerations in the design of clinical trials, weighted means and analysis of covariance. Proc. Conference in Honor of Shayle R. Searle, Biometrics Unit, Cornell University, 1996. 14. J. L. Fleiss, The Design and Analysis of Clinical Experiments. New York: John Wiley & Sons, 1985. 15. H. Sahai and M. I. Ageel, The Analysis of Variance – Fixed, Random and Mixed Models. ¨ Boston: Birkhauser, 2000. 16. O. J. Pendleton, M. von Tress, and R. Bremer, Interpretation of the four types of analysis of variance tables in SAS. Commun. Statist.Theor. Meth. 1986; 15: 2785–2808. 17. F. Yates, The analysis of multiple classifications with unequal numbers in the different classes. JASA 1934; 29: 51–56. 18. F. M. Speed, R. R. Hocking, and O. P. Hackney, Methods of analysis of linear models with unbalanced data. JASA 1978; 73: 105–112. 19. S. Senn, Some controversies in planning and analysing multi-centre trials. Stat. Med. 1998; 17: 1753–1765.

• Three patients in the A arm and eight

ANALYSIS POPULATION

patients in the B arm were lost to followup during the 2-year duration of the study. • Four patients were missing a lab measure at baseline that could serve as an important covariate. • At least three patients in the A arm were known to be taking medication B, perhaps in addition to A. • During the trial, five patients moved to another state, and although no lab measures or visit data were missing, lab measures for these patients have now been conducted at different facilities, with unknown quality-control procedures.

ANDREW S. MUGGLIN JOHN E. CONNETT Division of Biostatistics, University of Minnesota, Minneapolis, Minnesota

Most applied statisticians, at some time, have been given the request: ‘‘Please go analyze the data and then report back.’’ But hidden in the instruction, ‘‘ . . . go analyze the data’’ are at least three major issues, each of which has many subissues. Exactly which data should be analyzed? What hidden biases exist in the way that these data were collected? Which analysis technique is the most appropriate for these data? None of these questions has a trivial answer, but appropriate, high-quality analysis and interpretation depend on careful attention to these questions. In this article, we examine specifically the question of which data should be analyzed. In data sets that emanate from clinical trials, this task can be surprisingly challenging. 1

Even in a simple setting such as this, it is not at all clear how to ‘‘go analyze the data.’’ The problems only grow when a trial contains thousands of patients, longer follow-up, and more complicated data collection schemes. One simple and common approach to this setting might be to take all patients who have complete datasets and analyze them as they were actually treated. On the surface, this approach sounds appealing on two fronts: First, the notion of ‘‘complete datasets’’ presents no analytical difficulties (statistical techniques are typically well defined on welldefined datasets). Second, to analyze patients as they were actually treated seems to imply that we are studying the actual scientific consequence of treating patients in a specified way. However, these approaches have several major problems. What does ‘‘actually treated’’ mean when patients either forget or refuse to take their medication precisely as prescribed? What kinds of systematic factors lurk behind a patient’s or caregiver’s decision not to comply completely with a specified treatment regimen? And if only complete datasets are analyzed, what systematic factors influence whether a patient’s dataset is incomplete? For instance, if sicker patients tend to have more missing values or tend not to take their medication faithfully, an analysis that ignores incomplete datasets and handles patients according to their actual

APPRECIATING THE DIFFICULTIES

Consider the following hypothetical example. Suppose we have a trial of 100 patients, randomized equally into two study arms: oral medication A and oral medication B. Suppose that the following plausible scenarios all occur: • Three patients who violated the inclu-

sion/exclusion criteria of the trial were enrolled—one patient because of a clerical mistake, one because of an erroneous lab value, and one because of a clinician’s judgment about the intent of the trial. • Because of clerical error, one patient that should have been assigned to A was actually assigned to B. • Three patients assigned to take A refused to take it and dropped out of the study.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

ANALYSIS POPULATION

treatment regimen can produce biased and misleading conclusions. Another approach might be to take all enrolled patients—regardless of whether they had data missing—and analyze them in the groups to which they were assigned by randomization. In this instance, the analyst immediately faces the problem of how to handle the missing values. Is it best to impute the worst possible value for these? Or observed values from some other patients? Or to impute these values via some mathematical model? And considering assignment by randomization (assuming the trial is randomized), analyzing patients according to how they were randomized results in a bias toward no treatment difference in the study arms. (The term ‘‘bias’’ in clinical trials is not intended to be limited to intentional or fraudulent bias but is meant to include the more common situation of hidden influences that systematically affect an outcome.) If the aim of the study is to show that a new treatment is better than an older one, then it is generally deemed to be a small evil, because the bias only makes it harder to demonstrate the superiority of the new treatment. But if the aim of the study is to show that the new treatment is equivalent or noninferior to an existing one, then analyzing patients by assigned treatment can be dangerous—especially if compliance to treatment is low—because the bias may falsely cause an inferior treatment to seem equivalent to a superior one. 2

DEFINITIONS OF COMMON TERMS

Several terms are commonly used to describe a particular analysis data set and the principles by which it was chosen. We describe some of them here.

patient is how they were randomized. (Exceptions and difficult cases have occurred; for example, a patient is accidentally assigned to the wrong treatment, or a patient somehow gets enrolled but is never randomized.) In nonrandomized studies, this principle is more difficult to implement but is probably most likely to be implemented as the intended course of treatment when a patient was initially enrolled, if that can be determined. Intention-to-treat is often the default analysis philosophy because it is conceptually simple and because it is conservative in the sense that lack of perfect compliance to the prescribed therapy tends to bias the treatment effects toward each other, which makes claims of superiority of one therapy over another more difficult to achieve. However, as mentioned previously, in the setting of equivalence or noninferiority trials, this bias can be an anticonservative bias. 2.2 As-Randomized This term is often used interchangeably with intention-to-treat, but it has a few differences. Obviously, the term ‘‘as-randomized’’ does not apply in a nonrandomized study. More subtle differences include situations where a patient is incorrectly assigned to a treatment. For instance, if randomizations are preprinted in a sequence of envelopes, and someone opens the wrong envelope for a given patient, this patient may be randomized to a treatment other than what was intended. Also, if patients are enrolled and intended to be randomized but for some reason are never randomized, an intention-totreat analysis may place those patients into whichever arm seems to be have the most conservative effect, whereas an as-randomized analysis may omit them. 2.3 As-Treated

2.1 Intention-to-Treat (or Intent-to-Treat, or ITT) This principle dictates that patients be divided into analysis groups based on how the investigator intended to treat them, rather than on how they were actually treated. This principle is easiest to apply in a study where patients are randomized to treatments, because in most cases the intended way to treat a

This approach assigns patients to analysis groups based on how they were actually treated. A patient randomized to therapy A but who receives therapy B will be analyzed as belonging to the therapy B group. This approach attempts to study the more purely scientific effect of the therapies as actually implemented, which avoids the bias that occurs under intention to treat. It has

ANALYSIS POPULATION

other biases, however. If patients who take an incorrect therapy do so for some systematic reason (e.g., the assigned therapy has failed, or sicker patients tend to switch therapies because of side effects), this will bias the interpretation in unpredictable ways. Moreover, it can be difficult in practice to determine how a patient is actually treated. What, for example, does one do with data from a patient who has taken the prescribed medication intermittently? Or who took only half the prescribed dose? Or one who, after taking therapy A for awhile, discontinues it and begins taking therapy B? 2.4 Per Protocol Sometimes called an ‘‘adherers only’’ analysis, this term is used to denote an analysis where patients’ data are included in the analysis dataset only if the patients adhered to the requirements of the protocol. This analysis typically means that patients are randomized correctly to a therapy, have met the inclusion and exclusion criteria for the study, and have followed the treatment regimen correctly. Patients who drop out of a treatment arm or who drop in to a different arm (whether or not they discontinue the assigned treatment) are excluded, at least for the periods of time where they were not properly complying with the protocol. This approach attempts to answer the scientific question of how the treatment behaves under ideal conditions, while avoiding the biases inherent in analyzing patients as they were actually treated (see above). However, some biases still exist, because patients who are excluded from the analysis may be eliminated for some systematic reason (e.g., elderly people had more trouble complying with therapy B than with therapy A and thus more elderly people are excluded from the B arm).

3

finished the entire prescribed course of therapy or some minimum amount of the entire course of therapy (making it more of an astreated or per-protocol analysis). It can help with understanding of the long-term effects of therapy, especially among compliers. It is subject to a very serious bias, however, because it ignores patients who drop out of the study or who do not comply. It is dangerous to assume that noncompliance is a random occurrence. 2.6 Evaluable Population This term is not very informative in how it handles patients (e.g., as randomized?), but the intention is that ‘‘nonanalyzable’’ data are excluded. Data can be classified as ‘‘nonanalyzable’’ for a variety of reasons: They can be missing (e.g., a patient has missed a follow-up visit, a serum assay was lost or not performed), technically inadequate (as occurs in certain echocardiographic or other high-tech measurements), or in some cases important endpoints cannot be adjudicated reliably (e.g., whether a death was specific to a certain cause). In some instances, the term seems to refer to patients who have taken the therapy at some minimum level of adherence or had a minimum level of follow-up (making it essentially a completer analysis). This approach is one of convenience. It is subject to many biases, and data interpretation can be difficult. Authors who use this term should take care to define it, and readers should pay careful attention to the authors’ definition. 3

MAJOR THEMES AND ISSUES

In selecting the analysis dataset, certain issues and questions are generated frequently. In this section, we address several of them.

2.5 Completer Analysis Set

3.1 Should We Analyze Data as Intention-To-Treat or Not?

This term is less common and seems to be used in different ways. Some use it to describe an analysis set that includes only patients who have finished the specified minimum follow-up time. When contrasted in publication with an intention-to-treat analysis, it sometimes represents those patients who

The reflexive answer to this question is often affirmative, but the decision is sometimes not easy. Cases where it is easy are settings in which an investigator is trying to prove that an experimental treatment is superior to a standard treatment or placebo. As mentioned previously, the rationale here is that

4

ANALYSIS POPULATION

biases tend to work against the experimental therapy. If patients cannot comply well with the experimental regimen, then it will tend to work against the finding that the experimental treatment is superior. If a large amount of drop-in or drop-out of the experimental arm occurs (e.g., control subjects getting the experimental therapy, or experimental arm patients who follow the control regimen), then it will have the same effect. This conservative bias is also an incentive to investigators to be vigilant in executing the trial according to plan, because mistakes such as accidentally randomizing or assigning patients to treatments will tend to hurt the goals of the trial. Another compelling reason to analyze patients by intention to treat is that assignment of patients to analysis groups is usually straightforward (although it may be challenging to conduct the analysis, such as in the case of missing data). In some settings, however, this approach may not be the best. In an equivalence trial, where the goal is to acquire evidence to establish that two treatments are nearly the same, poor compliance will bias the trial in favor of the desired finding. In some settings, particularly pilot studies, the question of most interest regards the treatment effect when implemented correctly (‘‘efficacy’’), rather than its effect in actual practice (‘‘effectiveness’’). In other settings, an investigator may opt for an intention-to-treat analysis when studying a therapy’s efficacy or effectiveness but not for its safety. It is good practice to think through these issues and document one’s approach prior to knowing the results for any particular analysis. This practice defends against the criticism that a particular cohort was selected simply because it yields the most favorable interpretation. 3.2 What If Inclusion or Exclusion Criteria Are Violated? In trials of moderate or large size, it is common that a few patients are enrolled that should not have been enrolled. Reasons range from simple mistakes, to lab values that were not known when the patient enrolled, to investigator fraud. Hopefully in a wellexecuted trial, this group of patients will be small enough that results do not depend

on whether these patients are included. But there are no guarantees, and in the end, one analysis must be selected as the official trial result. Should false inclusions be contained in this analysis? One argument to exclude data from such patients is that they do not come from the intended study population. It is difficult to know whether their inclusion in the analysis represents a conservative or anticonservative bias. An argument to include them is that this sort of mistake will inevitably happen in real life, so it is best to analyze data that way. Senn (1, p. 157) and Friedman et al. (2, ch. 16) give several other arguments on each side. Once again, it is important that investigators think through their specific cases prior to knowing the analysis results. In our opinion, it is generally best to analyze according to the principle that ‘‘if enrolled, then analyzed.’’ A variant on this question is, ‘‘If I discover a false inclusion during the trial, should follow-up on that patient be discontinued even though I intend to analyze the data I have thus far collected?’’ This question has almost no good answer. If the answer is ‘‘Yes,’’ then potential late adverse events may be lost. If the answer is ‘‘No,’’ then the investigator may be exposing the patient to an inappropriate risk inherent in the therapy. In our opinion, this situation must be handled on a case-by-case basis, with investigators wrestling with the issues and erring, if necessary, on the side of the patient’s best interests and the most disclosure about the nature of the mistake. 3.3 What Do We Do if Data Are Missing? Missing data are inevitable in a study that involves humans. Data go missing for many reasons: patients drop out or withdraw consent, patients miss scheduled visits, urine samples are lost, records are lost, required tests are not conducted, test results are found to be technically inadequate, memory banks in recording devices become full, and so on. Furthermore, many different data items can be missing, and they are not always in the same patients. A patient who relocates and loses contact with the study center will have all data missing after a certain time point. A patient may have complete follow-up data

ANALYSIS POPULATION

but be missing an important baseline lab measure. A patient cannot make a certain scheduled visit but may have full information from all other visits. If data were always missing because of random causes, then it would be sensible to analyze the data that are actually collected and ignore missing cases. Most statistical software packages do this by default. But what assurance is there that the nonmissing data are representative of the missing data? In particular, if the sickest patients are dropping out or skipping scheduled visits because of their condition, or if caregivers are not prescribing certain tests because it would put a very sick patient under excessive stress, then the nonmissing data values are not representative of the missing ones, and any analyses that ignore missingness will be biased. Much statistical research has focused on the topic of statistical adjustment for missing data (3,4). But the fundamental reality is that missing data are still missing, and any statistical adjustment is only as good as the assumptions employed in the adjustment. It is important to assess and disclose the potential impact of missing data on study conclusions through sensitivity analyses. It is even more important to expend maximal energy to prevent the data from going missing in the first place. 3.4 What Is the Value of Subgroup Analyses or Responder Analyses? At the conclusion of a trial, investigators often wish to know if a therapy was especially effective for particular types of patients. A subgroup analysis is one where specific subclasses of patients are selected and their data are analyzed. Typical subgroups are defined by gender, race, geographical location, or many disease-specific categorizations. Here, the analysis is typically well defined (it is usually the primary objective analysis applied to a subset of patients), but the number of subgroups that could be analyzed is practically limitless. Subgroup analyses are often presented graphically, with confidence intervals for the treatment effect in each subgroup presented together in one graph. It is not surprising that the treatment effect is not the same in all

5

subgroups. Inevitably, some will have larger confidence intervals than others (because of varying subgroup sizes or different subgroup variances), and some can seem to be different from the others in the location of the confidence interval. It is important to remember that if enough subgroups are tested, probably every possible finding will be observed, so results should be interpreted with caution, and researchers should examine other studies to corroborate any apparent findings. A related analysis is a post-hoc responder analysis. Here, patients are identified as being a ‘‘responder’’ to a therapy according to some measure (e.g., still alive, or blood pressure adequately controlled), and the task of the analyst is to determine whether any subgroupings, baseline measures, or early indicators significantly predict who the responders will be. This method can be helpful in generating hypotheses for additional research. But because of the potentially limitless number of covariates that can be analyzed, these results should be viewed with extreme caution. 3.5 How Should Analyses be Handled If There is Poor Compliance to Treatment Regimen? In some studies, such as certain medical device studies, compliance is nearly perfect (e.g., if a pacemaker is implanted, it does what it is programmed to do; noncompliance is rare and might be caused by device malfunction or physician’s programming error but not by patient cooperation). In other studies, compliance is much poorer (e.g., when a patient must remember to take a combination of poorly tolerated medications on different dosing schedules). It is often difficult or impossible to determine the extent of the noncompliance accurately, although this fact should not discourage an investigator from attempting its assessment. This assessment is more for the purpose of defending trial conduct and interpretation than to decide whether to analyze any particular patient’s data. At one extreme, the data from all patients are analyzed regardless of compliance, which is consistent with the intention-to-treat approach. This method usually makes sense whenever the intention-to-treat analysis app-

6

ANALYSIS POPULATION

roach makes sense. At the other extreme, absolutely perfect or at least very good compliance is required to be included in an analysis. This analysis makes sense if the aim is to determine whether a treatment works under ideal circumstances or to identify adverse effects of a treatment when applied exactly as intended. It is usually better to analyze more data than less.

4

RECOMMENDATIONS

Hopefully, it is clear at this point that the resolution to most issues in choosing an analysis dataset is seldom simple and that experts can and do disagree. The following sections represent our opinions on some approaches that we have found useful in determining the makeup of an analysis dataset from a clinical trial. 4.1 Wrestle with the Issues It is not possible to eliminate or control all biases in a clinical investigation. It is usually not possible to estimate their magnitudes, and in some cases, it is impossible even to know the direction of bias. But it is possible to understand many of them and take steps to mitigate them, especially in trial conduct but also in data analysis. It can be hard work, but the result of this effort is a great enhancement in the quality and validity of the results. 4.2 Prespecify Wherever Possible Not all issues can be anticipated, but it should not be an excuse for not trying. Identify and document your planned approach to selecting an analysis cohort in advance.

4.4 Know That What Is Conservative in One Setting May Not Be In Another As previously mentioned, the intention-totreat principle is usually conservative in a trial of superiority, although it is not conservative in a trial of equivalence or noninferiority. At times, what is conservative for an efficacy objective may not be conservative for a safety objective. 4.5 Favor General Applicability Over Pure Science Several legitimate reasons can be cited to study a therapy under ideal conditions and also under more realistic conditions. When in doubt, we have found it helpful to favor the realistic side and to attempt to include rather than exclude data. 4.6 Conduct Sensitivity Analyses Check the robustness of your conclusions to varying definitions of the analysis cohort, as well as missing data assumptions and analysis techniques. Disclose in the final report or manuscript whether the conclusions change under reasonable deviations from your planned approach. 4.7 Account Painstakingly For How the Analysis Dataset Was Selected Account carefully in any final report for every patient that is included in or excluded from each analysis. In a journal article, we have less room to describe this in detail, but it is still good practice to account for patients’ data in summary fashion. The CONSORT statement (5) provides a checklist and example statements to illustrate how journal articles should report on methods in general (including specifically the numbers analyzed) for randomized clinical trials. Many of these principles apply to nonrandomized studies as well.

4.3 Take the Conservative Course

4.8 Illustrative Example

When biases cannot be controlled, seek to align them against the conclusion that you wish to reach, if possible. Then, if you achieve statistical significance regarding your desired conclusion, you have firm evidence.

The Lung Health Study (6) provides an excellent example of several issues presented above. The intent of this randomized trial was to study three treatments for chronic obstructive pulmonary disease, which is a

ANALYSIS POPULATION

major cause of morbidity and mortality that occurs almost exclusively in smokers. The three treatments were 1) smoking intervention plus an inhaled bronchodilator, 2) smoking intervention plus placebo, and 3) no intervention. The primary goal was to determine whether the treatments differentially influence the rate of decline in forced expiratory volume in 1 second (FEV1 ). It was intended that 6000 patients be enrolled and followed at annual visits for 5 years. The manuscript in Reference 6 provides an unusually detailed description of the study methods, which includes paragraphs on design, entry criteria, patient recruitment, spirometry, compliance monitoring, follow-up procedures, quality control, data management, and statistical methods. In a table in the Results section, the investigators report the follow-up rates at each annual visit by treatment group, with a maximum follow-up rate of 95% in the intervention plus bronchodilator group at year 5, and a minimum follow-up rate of 88% that occurred at one time in each of the three arms. In year 5, follow-up rates were at least 94% in all three groups. The generally high follow-up rates and especially the 94–95% follow-up rates in year 5 were in part the result of extra efforts by the investigators to ensure attendance at the fifth annual visit by as many surviving participants as possible. This result is important because the fifth annual visit data had the greatest influence on the computation of rates of change of the outcome variable and gave the most accurate estimate of the participant’s lung function status at the time they exited the study. It is possible to estimate the rate of decline in pulmonary function if a person has at least two annual visits (i.e., baseline and at least one follow-up measurement). The implicit assumption in such an analysis is that if a visit is missed, then the outcome data are missing at random (i.e., the missingness of the measurement is independent of the value of the outcome variable itself). This assumption is unlikely to be true in practice. A person with seriously impaired lung function is more likely to be sick or incapacitated, and therefore more likely to miss a visit, than a similar person with better lung function. In this case, no adjustments for

7

nonrandom missingness were applied in the estimates of rates of decline in lung function. Similarly, data for participants who died during the course of the study were included for all lung function measurements made prior to the time of death, and the data were entered into the analysis of the outcome. The manuscript points out several details that bolster the reader’s understanding of the analysis cohort. Among these are statements such as the following: ‘‘At the end of the study, the whereabouts and vital status of only 21 participants (0.4%) were unknown.’’ ‘‘Of the 5887 participants, 149 died during the 5-year follow-up period.’’ In captions in several figures, statements disclose how missing data values were handled: ‘‘Nonattenders of follow-up visits are counted as smokers’’ and ‘‘Participants not attending the visit were classified as noncompliant.’’ In an assessment of patient compliance to the bronchodilation regimen, the investigators report that 20–30% of patients who attended follow-up visits did not bring in their medication canisters for weighing, and this missingness probably represents a source of bias. The investigators discuss this bias, along with overestimates and underestimates of the bias, and report that by either measure, inhaler compliance did not differ between the intervention plus bronchodilator group and the intervention plus placebo group. This example demonstrates several principles found earlier in this article. First, analysis was by intention to treat, which was conservative in this case. (The manuscript words it as a ‘‘comparison of the randomly assigned groups.’’) Strong efforts were made to prevent losses to follow-up, with a high degree of success. Researchers had an understanding of and presented a disclosure of biases involved in missing data items, as well as a sensitivity analysis to how the missing compliance data were handled. And finally, researchers accounted carefully for patients at various time points and how it impacted each analysis. 4.9 Summary The process of selecting a dataset to represent the official results of a clinical trial is far from trivial. The principles involved are not

8

ANALYSIS POPULATION

universally accepted, and even when there is general agreement, there are often difficulties in specific data items that make implementing those principles difficult or even impossible. Nevertheless, some common terms can be used by researchers to describe their approaches to data selection and analysis, and common situations can develop wherein researchers can learn from each other. If this article can be boiled down to its simplest essence, it is this: Selecting a clinical trial analysis dataset is much more difficult than it seems. Therefore, think hard about the issues and potential biases, take a conservative course when you are unsure, and openly disclose how the final datasets were selected.

The difficulty involved in selecting an analysis dataset is just another of the many challenges— legal, ethical, scientific, practical—involved in medical research on humans. But in carefully adhering to good scientific principles and in paying close attention to biases and limitations—which include the selection of data for analysis—the impact of medical research is substantially enhanced. REFERENCES 1. S. Senn, Statistical Issues in Drug Development. New York: John Wiley & Sons, 1997. 2. L. M. Friedman, C. D. Furberg, D. L. DeMets, Fundamentals of Clinical Trials, 3rd ed. New York: Springer-Verlag, 1998. 3. D. B. Rubin, Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons, 1987. 4. J. L. Schafer, Analysis of Incomplete Multivariate Data. New York: Chapman and Hall, 1997. 5. D. Moher, K. F. Schulz, and D. G. Altman, for the CONSORT Group, The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Lancet 2001; 357: 1191–1194. 6. N. R. Anthonisen, J. E. Connett, J. P. Kiley, M. D. Altose, W. C. Bailey, AS. Buist, W. A. Conway Jr, P. L. Enright, R. E. Kanner, P. O’Hara, G. R. Ownes, P. D. Scanlon, D. P. Tashkin, and R. A. Wise, for the Lung Health Study Research Group, Effects of smoking

intervention and the use of an anticholinergic bronchodilator on the rate of decline of FEV1. JAMA 1994; 272: 1497–1505.

FURTHER READING P. Armitage, Exclusions, losses to follow-up, and withdrawals in clinical trials. In: S. H. Sharpiro and T. A. Louis (eds.), Clinical Trials: Issues and Approaches, New York: Marcel Dekker, 1983, pp. 99–113. S. Piantadosi, Clinical Trials: A Methodologic Perspective. New York: John Wiley & Sons, 1997. The ICH Guidelines are also useful references. See especially E3: International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use, ICH Harmonized Tripartite Guideline, Structure and Content of Clinical Study Reports (E3), 1995. Available in Federal Register on July 17, 1996 (61 FR 37320) and at: http://www.ich.org/LOB/media/MEDIA479.pdf and also at: http://www.fda.gov/cder/guidance/ iche3.pdf.

CROSS-REFERENCES Bias Missing Values Intention-to-Treat Analysis Equivalence Trial

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

development in cancer research where cytotoxic drugs were developed, the optimal dose was considered the maximally tolerated dose (MTD): the highest dose that has an acceptable level of toxicity. But there are other optimality criteria, such as minimally effective dose (MinED): the minimum dose that shows sufficient efficacy.

ELIZABETH GARRETT-MAYER Sidney Kimmel Comprehensive Cancer Center Johns Hopkins University Baltimore, Maryland,

Phase I dose finding studies have become increasingly heterogeneous in regards to the types of agents studied, the types of toxicities encountered, and degree of variability in outcomes across patients. For these reasons and others, when designing a phase I study it is especially important to select a design that can accommodate the specific nature of the investigational agent. However, most current phase I studies use the standard 3 + 3 design, which has been demonstrated to have poor operating characteristics, allows little flexibility, and cannot properly accommodate many of current dose-finding objectives. Novel phase I trials have been introduced over the past 20 years that have superior properties and are much more flexible than the 3 + 3 design, yet they have not been fully embraced in practice. These newer designs are in general not cumbersome and provide a more accurate and precise estimate of the appropriate dose to pursue in phase II and later studies. 1

2 STANDARD DESIGNS AND THEIR SHORTCOMINGS 2.1 The Standard 3 + 3 Design The standard 3 + 3 is the most commonly used and well-known phase I design (1). However, it is also probably the most inappropriate in most cases. It is an algorithmic design where a set of rules are followed for dose escalation or de-escalation based on observed dose limiting toxicities (DLTs); at the end of the trial, a maximally tolerated dose (MTD) is declared without any analysis of the data. The way the algorithm works is as follows. Enter three patients on dose level k: 1. If 0 of 3 patients have a DLT, dose escalate to dose level k + 1. 2. If 2 or 3 of 3 patients have a DLT, de-escalate to dose level k – 1. 3. If 1 of 3 patients has a DLT, add an additional 3 patients at dose level k. (a) If 1 of the 6 patients treated at k have a DLT, escalate to dose level k + 1. (b) If 2 or more of the 6 patients at k have a DLT, de-escalate to dose level k – 1.

OBJECTIVES OF A PHASE I TRIAL

It is important to be reminded of the goals of the phase I trial when selecting a design. Phase I studies are generally dose-finding studies, where the goal of the trial is to identify the optimal dose to take forward for further testing (usually for efficacy testing). For ethical reasons, patients cannot be randomized to doses: one cannot treat patients at high doses until lower ones have been explored. The standard approach, therefore, has been to administer a low dose of an agent to a small number of patients and, if the dose appears to be well-tolerated within those patients, to administer a higher dose to another small number of patients. Historically, due to the roots of phase I trial

When de-escalations occur, an additional 3 patients are usually treated at the dose unless 6 patients have already been treated at that dose. The MTD is then defined as the highest dose at which 0 or 1 of 6 patients has experienced a DLT. Note that there are slight variations to this design, but the way it is described here is one of the more common implementations (1). Many agents currently under evaluation in phase I studies do not have the same toxi-

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

city concerns that cytotoxic agents have had in the past. The standard 3 + 3 phase I design makes several assumptions that are not practical for many current phase I studies. Some of these assumptions are the following: 1. As dose increases, the chance of a response increases (i.e., there is a monotonic increasing association between dose and response). 2. As dose increases, the chance of toxicity increases (i.e., there is a monotonic increasing association between dose and toxicity). 3. The dose that should be carried forward should be the highest dose with acceptable toxicity. 4. Acceptable toxicity is approximately 15% to 30% (regardless of the definition of a toxic event). These assumptions are very limiting and are not practical in many of our current settings. For example, angiogenesis inhibitors or colony-stimulating factors may have very low probability of toxicity, and a dose-finding study may be more concerned with evaluating pharmacodynamic response over an increasing range of doses. As such, we would not want to escalate until we saw toxicity: we would want to escalate until we saw sufficient pharmacodynamic response. We may also assume that there may be a range of doses that provides a pharmacodynamic response, but doses below or above this range may be ineffective. This invalidates the assumption of monotonicity between dose and response. 2.1.1 What Is Wrong with the Standard 3 + 3 Design? There are a number of flaws that can be identified in the standard 3 + 3 design, and a few are highlighted here. One of the most problematic issues with this design is that it does not allow for the desired toxicity rate to be user-specified. For example, in certain cancers with very high mortality rates (e.g., brain cancer), the acceptable toxicity rate (i.e., the proportion of patients who experience a DLT) is higher than in some other more curable and less deadly cancers. Also, acceptable toxicity rates for short-term treatments (e.g., several cycles

of chemotherapy) will likely be higher than for long-term treatments (e.g., a daily dose of tamoxifen for 5 years for breast cancer survivors). The standard 3 + 3 design is prespecified in such a way that these considerations cannot be accommodated. The target toxicity rate for these trials is not well-defined (although it is often assumed that they target a rate between 15% and 30%) because it depends heavily on the spacing between prespecified dose levels and how many dose levels are included. In many cases, the 3 + 3 design will significantly underestimate or overestimate the desired toxicity rate. Another major shortcoming of the standard 3 + 3 is that, when choosing the MTD, it ignores any toxicity information observed at lower doses and has very poor precision. This is a common feature of algorithmic designs, but it is still a serious inefficiency. Newer designs estimate a dose-toxicity curve after all the data have been observed to not only help in choosing the most appropriate MTD, but also to provide an estimate of precision. By using the data that have accrued throughout the trial at all doses, there will be greater precision in the estimate of the MTD. Even if the target toxicity rate is approximately 20% and the goal of the trial is to find the MTD, the 3 + 3 will be a less efficient and more error-prone way of identifying a dose than model-based approaches described later in this chapter. Lastly, the 3 + 3 design will not achieve the objectives of a phase I trial when the primary outcome of interest for dose selection is not the maximally tolerated dose. When identifying the MinED, escalating based on DLTs is illogical. However, the standard 3 + 3 only allows for toxicity-based outcomes. 2.2 The Accelerated Titration Design There have been some improvements made to the standard 3 + 3 approach. Simon et al. (2) introduced the accelerated titration design, which has two major advantages over the standard design: (1) it treats only one or two patients per dose level until a DLT is observed; and (2) at the end of the trial, all of the data are incorporated into a statistical model for determining the appropriate dose to recommend for phase II testing. These

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

designs also allow for intrapatient dose escalation. It is clearly a more ethical and more appealing design to patients (patients are less likely to be treated on ineffective doses and can be escalated if they tolerate the treatment). This design still has the shortcoming that escalation is determined based on the same rules as the 3 + 3. However, this is mitigated to some extent by choosing the MTD based on a fitted statistical model. The accelerated titration design is also limited in that it is only appropriate for dose escalation that is toxicity based: for agents where toxicity is not the major outcome for dose finding, the accelerated titration design should not be used. 3

SOME NOVEL DESIGNS

In the past 20 years, a number of designs have been introduced for increased efficiency in phase I trials, but, unfortunately, most of these designs have not been adopted into mainstream use despite their significant improvements as compared with the standard algorithmic designs. Part of the reason for this is that they are based on statistical models and those without a strong background in statistics may be wary of them. There is a misconception that these novel designs are not as safe as the standard 3 + 3, but the evidence is quite to the contrary: many investigators have shown that, for example, the continual reassessment method is safer than the standard 3 + 3 because it exposes fewer patients to toxic doses and requires a smaller sample size (3–7). Even so, proponents of the 3 + 3 continue to promote to it with the idea that such a simple and standard design is safer than the alternatives and that its historic presence is a testament to its utility. An adaptive design is one in which certain decisions about how to proceed (e.g., what doses to give patients, how many patients to treat, which treatment to apply) are made for future patients in the trial based on results observed on all patients treated earlier in the trial. Adaptive phase I designs are designs in which the dose for a future patient is chosen based on the results from patients already treated on the study. There are several ways

3

that adaptive designs can be used in phase I studies, and just a few are described here. These designs have well-documented advantages over nonadaptive approaches and are becoming more commonly seen in many phase I clinical trials (3–7). 3.1 The Continual Reassessment Method The continual reassessment method (CRM) was first introduced by O’Quigley et al. in 1990 (8) and has undergone many modifications in the past 16 years. It is fundamentally different from the standard 3 + 3 design and most other ‘‘up and down’’ designs because (1) it relies on a mathematical model, (2) it assumes that dose is continuous (i.e., doses are not from a discrete set), and (3) it allows and requires the user to select a target toxicity rate. The original CRM used a Bayesian framework and had relatively few safety considerations. Newer and more commonly used versions have included several safety concerns (e.g., not allowing dose to increase by more than 100%) and a simpler estimation approach (i.e., maximum likelihood estimation). Although there are a number of CRMs currently used in practice, the model of Piantadosi et al. (9) will be the approach described here because of its simplicity. For other modifications, see Goodman et al. (4), Faries et al. (10), and Moller (11). The design requires several user-specified inputs: 1. The target dose-limiting toxicity rate 2. A dose that is expected to have a low level of toxicity (e.g., 5% or 10%) 3. A dose that is expected to have a high level of toxicity (e.g., 90% or 95%) 4. The number of patients per dose level (usually 1, 2, or 3). The number of patients per cohort usually will depend on the anticipated accrual. The mathematical model used is prespecified; with the knowledge of the three items listed above, the dose for the first cohort of patients can be calculated. This is demonstrated in Figure 1A where three assumptions are made: a 5% DLT rate at a dose of 100 mg, a 95% DLT rate at a dose of 900 mg, and a logistic dose-toxicity model. (For more

4

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

detail on the logistic dose-toxicity model, see Piantadosi et al. [9]). The dose-toxicity curve is drawn according to this information, with a horizontal line at the target toxicity rate (25%). The dose corresponding to a 25% DLT rate is 343 mg, and this is the dose for our first cohort of patients. Let us assume that two patients are treated at dose level 1 (dose = 343 mg). The DLT information observed has three possibilities: (1) neither patient has a DLT, (2) one patient has a DLT, or (3) both patients have a DLT. We add this information to our presumed information described in the previous paragraph. In Figures 1B to 1D, the updated

dose-toxicity curves are drawn using this new information, corresponding to the three possible outcomes, with the toxicity outcomes included in the graphs (0 = no DLT, 1 = DLT). If neither patient has a DLT, the dose is escalated to 418 mg (see Figure 1B); if one patient has a DLT, the dose is decreased to 218 mg (see Figure 1C); if both have a DLT, then dose is decreased to 147 mg for the next cohort (see Figure 1D). The approach continues where, after each cohort of two patients is observed, the additional data are incorporated to update the estimated dose-toxicity curve. The CRM is considered ‘‘adaptive’’ because we are using

600

800

1.0 0.8 0.6 0.4

1000

0

200

400

600

800

1000

DOSE (MG)

C. 1/2 DLTs: next dose is 233mg

D. 2/2 DLTs: next dose is 147mg

600

800

1000

0.8 0.6 0.4 0.2

PROBABILITY OF DLT 400

DOSE (MG)

0.0

0.8 0.6 0.4 0.2

200

1.0

DOSE (MG)

0.0 0

0.2

PROBABILITY OF DLT 400

0.0

1.0 0.8 0.6 0.4 0.2

200

1.0

0

PROBABILITY OF DLT

B. 0/2 DLTs: next dose is 418mg

0.0

PROBABILITY OF DLT

A. Starting dose: 343mg

0

200

400

600

800

1000

DOSE (MG)

Figure 1. (A) Estimated dose-toxicity curve based on a priori information about likely low and high toxicity doses. The starting dose is 343 mg. (B) Updated dose-toxicity curve if no dose-limiting toxicities (DLTs) are seen in two patients treated at 343 mg. The updated dose for next cohort would be 418 mg. (C) Updated dose-toxicity curve if one DLT is seen in two patients treated at 343 mg. The updated dose for next cohort would be 233 mg. (D) Updated dose-toxicity curve if two DLTs are seen in two patients treated at 343 mg. The updated dose for next cohort would be 147 mg.

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

the data collected as we proceed to determine the doses for future cohorts. And, unlike the algorithmic designs, when choosing a dose for the next cohort, we use information collected from all previous cohorts and not just the last cohort of patients. There are a variety of approaches that may be used to determine when the CRM trial should stop. Suggested approaches include when the recommended dose differs by no more than 10% from the current dose (9), defining a prespecified number of patients (4), or based on a measure of precision of the curve (12). For a detailed discussion of sample size choice, see Zohar and Chevret (13). Although in theory CRMs are markedly better than the standard 3 + 3 design, they do have several potential limitations. First, they require a statistician to be intimately involved with the daily coordination of the trial. The dose for a new patient depends on the data observed up until the new patient is enrolled and on the statistical model that has been chosen. Determining the next dose is not a trivial endeavor and requires that a statistician be available and familiar with the trial to perform these calculations on short notice. Second, the mathematical model that is assumed may not be flexible enough to accommodate the true or observed dosetoxicity relationship. The approach that we have highlighted here by Piantadosi et al. (9) is more flexible than some others, but in some cases, the chosen model may not be robust. 3.2 Extensions of the CRM for Efficacy Outcomes Although the CRM was developed to be used for dose-toxicity relationships, dose-finding based on dose-efficacy relationships can also be explored. This is most appropriate in the setting where a targeted therapy is being evaluated and, for example, there is a desired level of inhibition of a particular target. A target effectiveness rate could be defined as the dose that achieves inhibition of the target in at least 80% of patients. Pharmacokinetic parameters (e.g., area under the curve) could also be the basis for the design of a CRM study.

5

3.3 Bayesian Adaptive Designs Bayesian adaptive designs comprise a wide range of designs that are all based on the same general principles. For phase I studies, Bayesian adaptive designs have many similarities to the CRM described in the previous sections. They tend to be more complicated mathematically, so many of the details of these designs are beyond the scope of this article. However, the basic elements required for Bayesian decision making are (1) a statistical model, (2) a prior distribution that quantifies the presumed information about the toxicity of doses before the trial is initiated, (3) a set of possible actions to take at each look at the data, and (4) a ‘‘gain’’ function. We will consider each of these in turn.

1. The statistical model. This is a mathematical representation of the relationship between dose and toxicity. In the CRM we have described, we assumed a logistic model, but there are many possible models. 2. A prior distribution. This is similar to the quantification of toxicity a priori that is shown in Figure 1A. In the Bayesian approach, this is more formally described using an actual probability distribution but will take a form similar to that shown in Figure 1A. Often these are taken to be quite conservative by assuming that high doses tend to be very toxic so that the dose escalation is less likely to investigate high doses unless there is relatively strong evidence that they are safe, based on the accumulated information at lower doses. 3. Possible actions. These are the actions that can be taken when new patients are enrolled and include all of the possible doses that could be given to them. 4. Gain function. This is an attribute of the Bayesian approach that is quite different from the CRM. The gain function reflects what is best for the next cohort of patients. This is particularly relevant in phase I trials because of the tradeoff of efficacy and toxicity: a higher dose likely means an increase in both, and

6

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

the gain function allows us to include that tradeoff in our decision making about the next cohort. However, the gain function can take many forms and need not represent the tradeoff between toxicity and efficacy.

mathematical details needed to fully describe the approach, but the general idea is shown in Figure 2. Each curve (i.e., ‘‘contour’’) in Figure 2 represents a set of efficacy–toxicity tradeoffs that would be considered equally desirable. The user must elicit information from the clinical investigator to determine what a target contour would be. Dose is escalated based on the combined outcomes of toxicity and efficacy, and new doses are determined as a function of previously observed results. Another adaptive design recently proposed uses a three category outcome variable to describe both efficacy and toxicity (22). This approach has many nice properties, such as maximizing efficacy with constraints on toxicity instead of allowing them equal importance. This may be more practical in many settings, especially when it is expected that the agent under investigation has very low toxicity.

These designs are described in detail by Gatsonis and Greenhouse (14), Whitehead (15), and Haines et al. (16). Descriptions aimed at clinical investigators are provided by Zhou (17) and Whitehead et al. (18). Zohar et al. (19) have developed publicly available software for conducting Bayesian dose-finding studies. 3.4 Efficacy and Toxicity for Dose Finding Recently developed dose escalation designs have formally accounted for the desire to find doses that have both acceptable toxicity and maximal efficacy. This is in response to the trend in recent years of applying standard designs to find MTDs and subsequently looking at efficacy outcomes to see which, if any, of the doses explored showed efficacy either based on clinical outcomes, such as tumor response, or based on correlative outcomes, such as modulation of a genetic marker known to be in the cancer pathway. A more efficient approach is to simultaneously maximize efficacy while minimizing toxicity. Thall and Cook (20, 21) have proposed an adaptive Bayesian method that chooses the optimal dose based on both efficacy and toxicity. The principles are like those described in the previous section, but with the added complexity of two outcomes. There are many

4 DISCUSSION

0.0

Probability of Toxicity 0.2 0.4 0.6 0.8 1.0

In recent years, there have been quite a few novel designs proposed for phase I trials, many of which are Bayesian and most of which are adaptive. The trend has been to steer away from the old-fashioned algorithmic designs that have been shown repeatedly to be inefficient. Chevret (23) has recently published a book on the dose-finding methods that addresses many of the issues discussed here in more depth and detail. There are a number of approaches that could not be discussed in this chapter due to space considerations. Readers interested

0.0

0.2

0.4 0.6 Probability of Efficacy

0.8

1.0

Figure 2. An example of contours for Thall and Cook’s adaptive Bayesian design using both efficacy and toxicity. The target contour is shown by the thick line. Several other contours are shown with thinner lines.

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

in learning more about novel designs should also consider investigating the curve-free method (24) and the biased coin up-and-down design with isotonic regression (25) that have been introduced in recent years. REFERENCES 1. L. Edler, Overview of phase I trials In: J. Crowley (ed.), Handbook of Statistics in Clinical Oncology. New York: Marcel Dekker, 2001, pp. 1–34. 2. R. M. Simon, B. Freidlin, L. V. Rubinstein, S. Arbuck, J. Collins, and M. Christian, Accelerated titration design for phase I clinical trials in oncology. J Natl Cancer Inst. 1997; 89: 1138–1147. 3. B. Storer, An evaluation of phase I clinical trial designs in the continuous dose-response setting. Stat Med. 2001; 20: 2399–2408. 4. S. N. Goodman, M. L. Zahurak, and S. Piantadosi, Some practical improvements in the continual reassessment method for phase I studies. Stat Med. 1995; 14: 1149–1161. 5. E. Garrett-Mayer, The continual reassessment method for dose-finding: a tutorial. Clin Trials. 2006; 3: 57–71. 6. J. O’Quigley, Another look at two phase I clinical trial designs. Stat Med. 1999; 18: 2683–2690. 7. C. Ahn, An evaluation of phase I cancer clinical trial designs. Stat Med. 1998; 17: 1537–1549. 8. J. O. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: a practical design for phase I clinical trials in cancer. Biometrics. 1990; 46: 33–48. 9. S. Piantadosi, J. D. Fisher, and S. Grossman, Practical Implementation of a modified continual reassessment method for dose-finding trials. Cancer Chemother Pharmacol. 1998; 41: 429–436. 10. D. Faries, Practical modifications of the continual reassessment method for phase I cancer clinical trials. J Biopharm Stat. 1994; 4: 147–164. 11. S. Moller, An extension of the continual reassessment methods using a preliminary up-and-down design in a dose finding study in cancer patients, in order to investigate a greater range of doses. Stat Med. 1995; 14: 911–922. 12. N. Ishizuka and Y. Ohashi, The continual reassessment method and its applications: a

7

Bayesian methodology for phase I cancer clinical trials. Stat Med. 2001; 20: 2661–2681. 13. S. Zohar and S. Chevret, The continual reassessment method: comparison of Bayesian stopping rules for dose-ranging studies. Stat Med. 2001; 20: 2827–2843. 14. C. Gatonis and J. Greenhouse, Bayesian methods for phase I clinical trials. Stat Med. 1992; 11: 1377–1389. 15. J. Whitehead, Using Bayesian decision theory in dose-escalation studies In: S. Chevret (ed.), Statistical Methods in Dose-Finding Experiments. Chicester, UK: Wiley, 2006, pp. 149–171. 16. L. M. Haines, I. Perevozskaya, and W. F. Rosenberger, Bayesian optimal designs for phase I clinical trials. Biometrics. 2003; 59: 591–600. 17. Y. Zhou, Choice of designs and doses for early phase trial. Fundam Clin Pharmacol. 2004; 18: 1–7. 18. J. Whitehead, Y. Zhou, S. Patterson, N. D. Webber, and S. Francis, Easy-to-implement Bayesian methods for dose-escalation studies in healthy volunteers. Biostatistics. 2001; 2: 47–61. 19. S. Zohar, A. Latouche, M. Tacconet, and S. Chevret, Software to compute and conduct sequential Bayesian phase I and II doseranging clinical trials with stopping rules. Comput Methods Program Biomed. 2003; 72: 117–125. 20. P. F. Thall and J. D. Cook, Dose-finding based on efficacy-toxicity trade-offs. Biometrics. 2004; 60: 684–693. 21. P. F. Thall and J. D. Cook, Using both efficacy and toxicity for dose-finding In: S. Chevret (ed.), Statistical Methods in DoseFinding Experiments. Chicester, UK: Wiley, 2006, pp. 275–285. 22. W. Zhang, D. J. Sargent, S. Mandrekar, An adaptive dose-finding design incorporating both toxicity and efficacy. Stat Med. 2006; 25: 9243–9249. 23. S. Chevet, ed. Statistical Methods for DoseFinding Experiments. Chicester, UK: Wiley, 2006. 24. M. Gasparini and J. Eisele, A curve-free method for phase I clinical trials. Biometrics. 2000; 56: 609–615. 25. M. Stylianou and N. Flourny, Dose-finding using the biased coin up-and-down design and istonic regression. Biometrics. 2002; 58: 171–177.

8

APPLICATION OF NOVEL DESIGNS IN PHASE I TRIALS

FURTHER READING

CROSS-REFERENCES

J. O’Quigley, Dose-finding designs using the continual reassessment method In: J. Crowley (ed.), Handbook of Statistics in Clinical Oncology. New York: Marcel Dekker, 2001, pp. 35–72. B. Storer, Choosing a phase I design In: J. Crowley (ed.), Handbook of Statistics in Clinical Oncology. New York: Marcel Dekker, 2001, pp. 73–92.

Continual reassessment method Dose escalation design Phase I trials Bayesian approach

ASCOT TRIAL

trial of a lipid-lowering agent (lipid-lowering arm, ASCOT-LLA, 5). Between 1998 and 2000, patients were recruited to an independent, investigator-inititated, investigatorled, multicenter, prospective, randomizedcontrolled trial. Patients were eligible for ASCOT-BPLA if they were aged 40–79 years at randomization, and had either untreated hypertension with a systolic blood pressure of 160 mm Hg or more, diastolic blood pressure of 100 mm Hg or more, or both, or they were treated for hypertension with systolic blood pressure of 140 mm Hg or more, diastolic blood pressure of 90 mm Hg or more, or both. In addition, the patients had to have at least three other cardiovascular risk factors and they were to be free of any previous cardiac events or current cardiac disease

JESPER MEHLSEN Frederikesberg Hospital—Clinical Physiology & Nuclear Medicine Frederiksberg, Denmark

One of the first randomized, placebocontrolled trials in hypertensive subjects (1) demonstrated that whereas treatment dramatically reduced mortality and the incidence of strokes, it did not prevent myocardial infarction. This paradox was confirmed by metaanalyses of randomized trials—all of which used standard diuretic and/or betablocker therapy—showing treatment effects similar to those predicted by prospective, observational studies on stroke incidence but not on the occurrence of coronary heart disease (CHD; 2). It has been speculated that adverse effects of the older antihypertensive drugs on serum lipids, glucose, and potassium could blunt the cardioprotection conferred by lowering of blood pressure. 1

2.1 Endpoints Primary objectives were as follows: 1) to compare the effects on the combined outcome of non-fatal myocardial infarction (MI) and fatal CHD of a beta-blocker-regimen (atenolol) + if necessary a diuretic (bendroflumethiazide-K) with a calcium channel blocker-based regimen (amlodipine) + if necessary an ACE inhibitor (perindopril); 2) to compare the effect on the combined outcome of non-fatal MI and fatal CHD of a statin (atorvastatin) with that of placebo among hypertensive patients with total cholesterol < 6.5 mmol/L.

OBJECTIVES

The rationale for the ASCOT study (3) was thus to answer the issue of whether a newer combination of antihypertensive agents, a dihydropyridine calcium channel blocker (CCB) and an angiotensin converting enzyme (ACE) inhibitor, would produce greater benefits in terms of reducing CHD events than the standard beta-blocker/diuretic combination. The second main issue of ASCOT was whether lipid lowering with a statin would provide additional beneficial effects in those hypertensive patients with average or below average levels of serum cholesterol. 2

2.2 Drug Treatment The ASCOT-BPLA used forced titration of the study drugs with specific add-on drugs, the first of which was the alfa-blocker doxazosin, gastrointestinal transport system (GITS). The target for antihypertensive treatment was 140/90 mm Hg for nondiabetics and 130/80 mm Hg for patients with diabetes mellitus. The ASCOT-LLA used a standard dose of atorvastatin 10 mg for all patients assigned to active treatment without any specific target level for cholesterol.

STUDY DESIGN

ASCOT involved two treatment comparisons in a factorial design—a prospective, randomized, open, blinded endpoint design comparing two antihypertensive regimens (blood pressure-lowering arm, ASCOT-BPLA, 4) and, in a subsample of those hypertensives studied, a double-blind, placebo-controlled

2.3 Sample Size Assuming an adjusted yearly rate of nonfatal myocardial infarction and fatal CHD events

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

ASCOT TRIAL

of 1.42% and an intention-to-treat effect of 15% reduction in risk, it was estimated that a sample size of 18,000 was required to generate 1150 primary endpoints. This sample size would provide 80% power to detect such an effect. In the ASCOT-LLA, a 30% reduction in cholesterol was expected and assumed to translate into a reduction in nonfatal MI and fatal CHD of 30%. Under these conditions, a sample of 9000 patients would have 90% power to detect such an effect. 2.4 Data Safety Monitoring Plan The Data Safety Monitoring Committee (DSMC) did unblinded interim analysis during the trial and used the symmetric Haybittle–Peto statistical boundary (critical value Z = 3) as a guideline for deciding to recommend early termination of the trial. 3

RESULTS

In the Nordic countries, 686 family practices randomized patients, and in the United Kingdom and Ireland, 32 regional centers to which patients were referred by their family doctors recruited patients. A total of 19,342 patients were randomized, but 85 patients had to be excluded because of irregularities in blood-pressure measurements resulting in 19,257 evaluable patients. Participants were well matched between groups; over 80% were on previous antihypertensive treatment, they were mainly white and male, and had a mean age of 63 years, a mean body mass index (BMI) of almost 29 kg/m2 , a mean total cholesterol of 5.9 mmol/L, and a mean baseline sitting blood pressure of 164/95 mm Hg. The average number of the additional cardiovascular risk factors required for inclusion in the trial was 3.7. From the main study, 10,305 patients were found to be eligible for inclusion in ASCOT-LLA. In 2002, the data safety monitoring board (DSMC) recommended that ASCOT-LLA be stopped on the grounds that atorvastatin had resulted in a highly significant reduction in the primary endpoint of CHD events compared with placebo. All patients in the ASCOT-LLA were offered atorvastatin 10 mg daily to be continued to the end of ASCOTBPLA. In 2004, the DSMB recommended

the ASCOT-BPLA to be stopped as those patients allocated the atenolol-based regimen had significantly higher mortality as well as worse outcomes on several other secondary endpoints than those allocated the amlodipine-based regimen. ASCOT-LLA accumulated 33,041 patient years of follow-up (median, 3.3 years). At the close of follow-up for the lipid-lowering arm, complete information was obtained on 10,186 (98.8%) of the patients originally randomized. By the end of the study, 87% of patients originally assigned atorvastatin were still taking a statin, and 9% of those in the placebo group had been prescribed open-label statins. Comparing atorvastatin treatment to placebo, total and LDL-cholesterol were lowered by 1.0 mmol/L and 1.0 mmol/L (19% and 29%), respectively, and triglycerides were reduced by 14%. Changes in HDL-cholesterol concentrations were minimal and blood pressure control throughout the trial was similar in the two groups. The primary endpoint of nonfatal myocardial infarction and fatal CHD was significantly lower by 36% (hazard ratio 0.64 [95% CI 0.50–0.83], P = 0.0005) in the atorvastatin group than in the placebo group. There were also significant reductions in four of seven secondary endpoints, some of which incorporated the primary endpoint: total cardiovascular events including revascularization procedures (21%); total coronary events (29%); the primary endpoint excluding silent myocardial infarction (38%); and fatal and nonfatal stroke (27%). Effects of statin on the secondary endpoints of heart failure or cardiovascular mortality did not differ significantly from those of placebo. ASCOT-BPLA accumulated 106,153 patient years of follow-up (median, 5.5 years). In October, 2004, the DSMC recommended the trial be stopped on the grounds that compared with those allocated the amlodipinebased regimen, those allocated the atenololbased regimen had significantly higher mortality as well as worse outcomes on several other secondary endpoints. Complete endpoint information was collected at the end of the study for 18,965 patients (99%). On average, blood pressure dropped from a mean of 164.0/94.7 (SD 18.0/10.4) mm Hg to a mean of 136.9/78.3 (16.7/9.8) mm Hg. At the trial close

ASCOT TRIAL

out, 32% of patients with diabetes and 60% of those without had reached both the systolic and the diastolic blood pressure targets. Compared with those allocated the atenolol-based regimen, blood pressure values were lower in those allocated the amlodipine-based regimen and the average difference throughout the trial was 2.7/1.9 mm Hg. By the end of the trial, as intended by design, most patients (78%) were taking at least two antihypertensive agents. The primary endpoint of nonfatal myocardial infarction plus fatal CHD was nonsignificantly lowered by 10% in those allocated the amlodipine-based regimen compared with those allocated the atenolol-based regimen. There were significant reductions in most of the secondary endpoints (except fatal and nonfatal heart failure): nonfatal myocardial infarction (excluding silent myocardial infarction) and fatal CHD (reduced by 13%); total coronary events (13%); total cardiovascular events and procedures (16%); allcause mortality (11%); cardiovascular mortality (24%); and fatal and nonfatal stroke (23%). The difference in all-cause mortality was caused by the significant reduction in cardiovascular mortality, with no apparent difference in noncardiovascular mortality. Of the tertiary endpoints, there were significant reductions associated with the amlodipinebased regimen for unstable angina (32%), peripheral arterial disease (35%), development of diabetes (30%), and development of renal impairment (15%). Twenty-five percent of patients stopped therapy because of an adverse event, with no significant difference between the allocated treatment groups. 4

DISCUSSION & CONCLUSIONS

The lipid-lowering arm of ASCOT (ASCOTLLA) showed that cholesterol lowering with atorvastatin compared with placebo conferred a significant reduction in nonfatal myocardial infarction and in fatal CHD in hypertensive patients at moderate risk of developing cardiovascular events. Observational data have indicated a relatively weak association between serum cholesterol and the risk of stroke (6), but previous randomized trials of statin use have shown significant reductions in stroke events of the

3

same order of magnitude as in ASCOT-LLA (7). There were no significant adverse effects on any of the prespecified secondary or tertiary endpoints in association with the use of atorvastatin. The relative magnitude of the benefits in ASCOT-LLA are notably larger for CHD prevention than are the effects of blood-pressure lowering in randomized, placebo-controlled trials (2), whereas the relative reduction in stroke seems somewhat smaller. However, the results show the benefits of statin treatment are additional to those of good blood-pressure control. The findings support the concept that strategies aimed at reducing cardiovascular disease should depend on global assessment of risk, and that benefits of lipid lowering are present across the whole range of serum cholesterol concentrations. Subsequent economic analysis has indicated that adopting the treatment strategy used in ASCOT-LLA would be costeffective (8). ASCOT-BPLA showed that amlodipine-based treatment was superior to an atenolol-based therapy in hypertensive patients at moderate risk of developing cardiovascular events in terms of reducing the incidence of all types of cardiovascular events and all-cause mortality, and in terms of risk of subsequent new-onset diabetes. The effective blood pressure lowering achieved by the amlodipinebased regimen, particularly in the first year of follow-up, is likely to have contributed to the differential cardiovascular benefits. However, the systolic blood pressure difference observed would, based on previous randomized trials (2) and by observational studies (9), be expected to generate a difference in coronary events and in strokes far below that achieved. A large substudy in ASCOT, the Conduit Artery Function Evaluation (CAFE) Study (10), showed that the two drug regimens had substantially different effects on central aortic pressures and hemodynamics despite a similar impact on brachial blood pressure in those included. The study indicated that differences in central aortic pressures could be a potential mechanism to explain the different clinical outcomes between the two blood pressure treatment arms in ASCOT.

4

ASCOT TRIAL

Another explanation for the difference observed could possibly be found in a prespecified assessment of whether any synergistic effects were apparent between the lipid-lowering and blood-pressure-lowering regimens (11). This analysis revealed that atorvastatin reduced the relative risk of CHD events by 53% (P < 0.0001) among those allocated the amlodipine-based regimen, but nonsignificantly (16%) among those allocated the atenolol-based regimen (P < 0.025 for heterogeneity). A significant excess of new-onset diabetes was observed in those allocated the atenololbased regimen and is compatible with the results of previous studies (12). The effect on short-term cardiovascular outcomes of individuals who became diabetic during the course might not be apparent compared with those who did not develop diabetes, although adverse outcomes associated with type 2 diabetes could reasonably be expected with extended follow-up. The ASCOT-BPLA reaffirm that most hypertensive patients need at least two agents to reach recommended blood pressure targets, and that most can reach current targets if suitable treatment algorithms are followed. Economic analysis has indicated that amlodipine-based therapy would be costeffective when compared with the atenololbased therapy (13). Both arms of the ASCOT trial have impacted on current recommendations for the treatment of hypertension. The European guidelines (14) now recommend the addition of a statin to the antihypertensive treatment in hypertensive patients aged less than 80 years who have an estimated 10 years risk of cardiovascular disease of more than 20% or of cardiovascular death of 5% or more. The British National Institute for Health and Clinical Excellence (15) no longer recommends beta-blockers as a first-line therapy, and the European guidelines express several reservations regarding the use of beta-blockers in hypertension particularly in combination with a thiazide diuretic. REFERENCES 1. VA Cooperative Study Group, Effects of treatment on morbidity in hypertension. II. Results

in patients with diastolic blood pressure averaging 90 through 114mm Hg. JAMA 1970; 213: 1143–1152. 2. Blood Pressure Lowering Treatment Trialists’ Collaboration, Effects of different blood pressure lowering regimens on major cardiovascular events: Results of prospectively designed overviews of randomised trials. Lancet 2003; 362: 1527–1535. 3. Sever PS, Dahl¨of B, Poulter NR, et al. Rationale, design, methods and baseline demography of participants of the Anglo-Scandinavian Cardiac Outcomes Trial. J. Hypertens. 2001; 6: 1139–1147. 4. Dahl¨of B, Sever PS, Poulter NR, et al. Prevention of cardiovascular events with an antihypertensive regimen of amlodipine adding perindopril as required versus atenolol adding bendroflumethiazide as required, in the Anglo-Scandinavian Cardiac Outcomes Trial-Blood Pressure Lowering Arm (ASCOTBPLA): A multicentre randomised controlled trial. Lancet 2005; 366: 895–906. 5. Sever PS, Dahlof B, Poulter NR, et al. Prevention of coronary and stroke events with atorvastatin in hypertensive patients who have average or lower-than-average cholesterol concentrations, in the Anglo-Scandinavian Cardiac Outcomes Trial-Lipid Lowering Arm (ASCOT-LLA): A multicentre randomised controlled trial. Lancet 2003; 361: 1149–1158. 6. Eastern Stroke and Coronary Heart Disease Collaborative Research Group. Blood pressure, cholesterol, and stroke in eastern Asia. Lancet 1998; 352: 1801–1807. 7. Crouse JR III, Byington RP, Furberg CD. HMG-CoA reductase inhibitor therapy and stroke risk reduction: An analysis of clinical trials data. Atherosclerosis 1998; 138: 11–24. 8. Lindgren P, Buxton M, Kahan T, et al. Costeffectiveness of atorvastatin for the prevention of coronary and stroke events: an economic analysis of the Anglo-Scandinavian Cardiac Outcomes Trial–lipid-lowering arm (ASCOTLLA). Eur. J. Cardiovasc. Prev. Rehabil. 2005; 12: 29–36. 9. Lewington S, Clarke R, Qizilbash N, et al. Age-specific relevance of usual blood pressure to vascular mortality: A meta-analysis of individual data for one million adults in 61 prospective studies. Lancet 2002; 360: 1903–1913. 10. Williams B, Lacy PS, Thom SM, et al. Differential Impact of Blood Pressure–Lowering Drugs on Central Aortic Pressure and Clinical Outcomes: Principal Results of the Conduit

ASCOT TRIAL Artery Function Evaluation (CAFE) Study. Circulation 2006; 113: 1213–1225. 11. Sever P, Dahl¨of B, Poulter N, et al. Potential synergy between lipid-lowering and bloodpressure-lowering in the Anglo-Scandinavian Cardiac Outcomes Trial. Eur. Heart J. 2006; 27: 2982–2988. 12. Opie LH, Schall R. Old antihypertensives and new diabetes. J. Hypertens. 2004; 22: 1453–1458. 13. Lindgren P, Buxton M, Kahan K, et al. Economic Evaluation of ASCOT-BPLA: Antihypertensive treatment with an amlodipinebased regimen is cost-effective compared to an atenolol-based regimen. Heart Online, October 2007. 14. The Task Force for the Management of Arterial Hypertension of the European Society of Hypertension (ESH) and of the European Society of Cardiology (ESC). 2007 Guidelines for the Management of Arterial Hypertension. J. Hypertens. 2007; 25: 1105–1187. 15. National Institute of Clinical Excellence. Hypertension: management of hypertension in adults in primary care. 2006; Available from: http://www.nice.org.uk/ nicemedia/pdf/CG034NICEguideline.pdf.

5

ASSAY SENSITIVITY

estimate for the primary endpoint, one can calculate the power at any effect size, yielding the power function. This power function takes a value equal to the type I error rate when the effect size is 0 and a value equal to the desired power when the effect size is
. Because the goal of a superiority trial is to detect a difference, high assay sensitivity will help a trial achieve its objective in the presence of a true treatment difference. In such a trial, the power function can help provide some quantification for assay sensitivity. The higher the power to detect a treatment effect is, the greater the assay sensitivity is. For a superiority trial, several factors could reduce the true treatment effect or increase the variability, leading to a smaller signal-to-noise ratio when comparing an effective treatment with a less effective or an ineffective one. These factors include, but are not limited to:

CHRISTY CHUANG-STEIN Pfizer Global Research and Development Kalamazoo, Michigan

A good source for assay sensitivity is the International Conference on Harmonisation of Technical requirements for Registration of Pharmaceuticals for Human Use (ICH) guidance on Choice of Control Group and Related Issues in Clinical Trials (E10) (1). According to ICH E10, the assay sensitivity of a clinical trial is defined as the trial’s ability to distinguish an effective treatment from a less effective or ineffective one. Since it is always important for a trial to tell treatments of different effects apart, assay sensitivity is an important consideration when designing and conducting a trial. The lack of adequate assay sensitivity has different implications for trials designed to demonstrate superiority and trials designed to demonstrate noninferiority as we will discuss below. 1

1. High measurement or ascertainment errors caused by the use of multiple equipments, multiple technicians, and multiple raters that produce different results. 2. Poor compliance with therapy such as study medication administration, use of concomitant medications, visit schedule, sample collection, and data recording. 3. Large patient dropout rate that makes analysis challenging and renders findings suspicious. 4. The enrollment population is not what the effective treatment will benefit the most. 5. The dose or the administration schedule is not optimal, which results in a suboptimal exposure. 6. A large placebo effect in the enrolled population makes it hard to demonstrate a treatment benefit. 7. Poorly or inconsistently applied diagnostic criteria that result in the inclusion of subjects who might not have the disorder under investigation. 8. Treatment randomization that fails to take into consideration crucial baseline covariates, leading to imbalance

SUPERIORITY TRIAL

The primary objective of a superiority trial is to demonstrate the superiority of a treatment over a concurrent comparator. The comparator could be a placebo or another treatment. The primary endpoint could be related to efficacy, safety, or benefit/risk. When determining the sample size for a superiority study, a trialist needs to consider the type I error (false-positive) rate and the power (one minus the false-negative rate) at a particular effect size. The allowable type I error rate, typically set to be 0.05 for a two-sided test in a confirmatory trial, determines the level at which statistical significance will be claimed. As for statistical power, a prespecified requirement (80% or 90%) at a prespecified effect size (
) gives the probability that we will conclude a treatment effect difference if the treatment effect difference is truly of the amount
. Given a sample size, the type I error rate, and the variability associated with the

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

ASSAY SENSITIVITY

between treatment groups with respect to important covariates. 9. Lack of clarity in the protocol, leaving investigators to interpret the requirements themselves and leading to discrepancies in implementation. When a superiority trial concludes a treatment effect difference, the finding demonstrates that the trial has assay sensitivity. If the trial fails to conclude a treatment effect difference, it could be because of the absence of a true treatment effect difference. It could also be that the study is lacking adequate assay sensitivity because of one or more of the reasons listed above. Since the latter is something that a trialist could control with proper steps, it is important that a trialist ensures adequate assay sensitivity through careful design and excellent execution. By comparison, it is hard to change the innate property of a drug candidate. 2

NONINFERIORITY TRIALS

For convenience, we will focus our discussion on noninferiority trials (2) even though many of the comments are equally applicable to equivalence trials. Here, we assume that a noninferiority decision will be based on a confidence interval constructed for the treatment effect difference. Furthermore, we assume that a positive value for the difference signals a beneficial treatment effect (for the new treatment). Noninferiority will be concluded if the lower confidence limit exceeds a predetermined quantity. When noninferiority in efficacy is being investigated, the hope is to confirm the efficacy of the new treatment over a placebo based on the proven efficacy of the active control (over a concurrent placebo) observed in previous trials. The idea is that if the efficacy of the new treatment is ‘‘reasonably’’ close to that of the active control and the latter is believed to have a better efficacy than a placebo, then one could conclude the efficacy of the new treatment over a placebo if a placebo had been included in the trial. To make the above work, it is critical that the active control successfully and consistently demonstrated a benefit over a concurrent placebo in past trials. In other words,

historical evidence of sensitivity to drug effects is available showing that similarly designed trials regularly distinguished the active control from a placebo. This condition, although sounds simple, is not always true. In fact, we know this condition often fails for disorders such as depression. For depression, a non-negligible chance exists that an effective treatment fails to demonstrate efficacy over a placebo comparator in a given trial. Even when historical evidence for assay sensitivity exists, the effect over the concurrent placebo is not necessarily similar from trial to trial. When this happens, no assurance is given that similarly designed noninferiority trials involving the active control will have similar assay sensitivity. All trials should be planned carefully. A trialist should consider all things that could possibly go wrong and make previsions for them at the design stage. On the other hand, the appropriateness of trial conduct could only be fully evaluated after the trial is completed. To extrapolate the efficacy of the active control observed in previous trials to the current one, the current trial should be similar to those from which historical evidence has been drawn. For example, we should have similar inclusion/exclusion criteria, similar allowance for concomitant medications, similar measurement process, and similar follow-up and data collection/analysis methods. Since historical evidence for assay sensitivity only applies to similarly designed trials, the current trial should be conducted with the same high quality as the previous ones. 3 IMPACT OF SLOPPY TRIAL EXECUTION Careful conduct of a clinical trial according to the protocol has a major impact on the credibility of the results (3). Sloppy trial conduct affects both noninferiority (4,5) and superiority trials, albeit differently. Since the primary objective of a noninferiority trial is to demonstrate the absence of a difference, the lack of assay sensitivity could reduce the treatment effect difference and increase the chance of erroneously concluding noninferiority. On the other hand, the variability could be increased as a result of increased sources

ASSAY SENSITIVITY

of variability. The variability contributes to a greater confidence interval width, which in turn could reduce our chance to conclude noninferiority correctly. The above discussion is further complicated in the case of a binary endpoint (6). Use the case of an antibiotic trial where the success rate of the new treatment and the positive control is expected to be around 70%. If the observed rates are close to 50%, this would result in a higher than expected variability when constructing the confidence interval for the difference in the success rates. Several factors could play against each other in regard to the noninferiority decision. Clearly, the observed treatment effect is a major factor. However, the magnitude of the change in the point estimate for treatment effect needs to be evaluated against a likely increase in the variability for their combined effect on the final conclusion. Koch and Rohmel (7) conducted a simulation study to evaluate the impact of sloppy study conduct on noninferiority studies. They showed that sloppy study conduct could affect the rate of false-positive conclusions in noninferiority trials. In their investigations, Koch and Rohmel examined the following four factors: (1) the objective of the trial; (2) the population analyzed and the mechanism for imputation of nonevaluable measurements; (3) the mechanism generating nonevaluable measurements; and (4) the selected measure for treatment effect. They encouraged researchers to perform simulation studies routinely to investigate the effect of these often counteracting factors in practical situations. By comparison, sloppy study conduct in a superiority trial generally leads to a decrease in assay sensitivity, and therefore, it reduces our chance to conclude superiority even if the treatment of interest is better than the comparator. 4

DESIGNS INCLUDING THREE ARMS

A design using three arms is used increasingly to help gauge assay sensitivity in a trial (8). This applies to trials designed from the superiority or noninferiority perspective. The three arms discussed here are placebo, an active control, and a new treatment.

3

In a superiority trial, the inclusion of an active control helps decide whether a study has adequate assay sensitivity. If the study can conclude a significant treatment effect between the active control (known to have an effect) and the placebo, one can conclude that the study has adequate assay sensitivity. If the new treatment fails to demonstrate a significant effect over the placebo in the same study, one could conclude that the new treatment, as applied to the patients in the study, is not likely to possess the kind of treatment effect used to size the study at the design stage. The three-arm design has gained popularity for proof of concept or dose-ranging studies also. Since most product candidates entering into clinical testing will fail, many drug development programs are designed so that the sponsors could terminate a program early if the candidate is not likely to meet the required commercial profile. To this end, the inclusion of an active control in an early trial (proof of concept or dose-ranging) helps answer the question of whether the observed effect of the new treatment is reasonably close to the truth based on the observed response to the active control. Similarly, for situations where product differentiation is important, including an active control allows the sponsor to compare the new treatment head-to-head with the active control after efficacy of the new treatment is confirmed. In some cases, a sponsor might cease further development of a new treatment because it does not meet the commercial requirements relative to the active control even if the new treatment demonstrates efficacy over a placebo. A classic noninferiority trial designed from the efficacy perspective does not include a placebo. The basis for this is typically because of ethical consideration. For example, in trials involving patients with life-threatening conditions for which treatments are available, it will be unethical to treat patients with a placebo. For confirmatory trials that do contain a placebo and an active control, the primary analysis is typically to compare the new treatment with the placebo (9). A secondary analysis is to compare the new treatment with the active control for product differentiation.

4

ASSAY SENSITIVITY

In some situations, the primary objective of the trial is to compare the new treatment with a placebo on safety to rule out a safety signal. The design is basically a noninferiority design. The hope is to conclude that the primary safety endpoint associated with the new treatment is within a certain limit of that associated with the placebo. This is the case with the ‘‘thorough QT/QTc’’ study discussed in ICH E14 (10). ICH E14 states that the ‘‘thorough QT/QTc’’ study is intended to determine whether a drug candidate under development has a threshold pharmacologic effect on cardiac repolarization as detected by QT/QTc prolongation. Such a study, which is typically conducted early in a clinical development program, is intended to provide maximum guidance for the collection of ECG data in later trials. For example, a negative ‘‘thorough QT/QTc’’ study will often allow the collection of on-therapy ECGs in accordance with the current practices in each therapeutic area. On the other hand, a positive ‘‘thorough QT/QTc study’’ will almost always lead to an expanded ECG safety evaluation during later stages of drug development. Because of the critical role of this study, the confidence in the study’s findings can be greatly enhanced by the use of a concurrent positive control (pharmacological or nonpharmacological) to establish assay sensitivity. Detecting the positive control’s effect in the study will establish the ability of the study to detect an effect similar to that of the positive control. A common choice of the positive control is a drug or an intervention that has an effect on the mean QT/QTc interval of about 5 ms. Positive controls play a similar role in the assessment of mutagenicity studies. According to Hauschke et al. (11), the classification of an experiment as being negative or positive should be based also on the magnitude of the responses in the positive control. In such studies, the determination of the maximum safe dose is often done by incorporating a biologically meaningful threshold value, which is expressed as a fraction of the difference between positive and vehicle control. Therefore, the inclusion of a positive control not only determines the quality of the experiment qualitatively, but it is also part of the summary results involving a new compound in a quantitative manner.

In summary, assay sensitivity is a critical factor to the success of a trial, whether the trial is designed from the efficacy or the safety perspective. REFERENCES 1. International Conference on Harmonisation E10. (2000). Choice of control group and related issues in clinical trials. Step 5. Available: http://www.ich.org/cache/compo/276254-1.html. 2. Hung HMJ, Wang SJ, O’Neill R. (2006). Noninferiority trials. Wiley Encyclopedia of Clinical Trials. 3. International Conference on Harmonisation E9. (1998). Statistical principles for clinical trials. Step 5. Available: http://www.ich.org/ cache/compo/276-254-1.html. 4. Jones B, Jarvis P, Lewis JA, Ebbutt AF. Trials to assess equivalence: the importance of rigorous methods. Br. Med. J. 1996; 313: 36–39. 5. Sheng D, Kim MY. The effect of noncompliance on intent-to-treat analysis of equivalence trials. Stat Med. 2006; 25: 1183–1199. 6. Chuang-Stein C. Clinical equivalence—a clarification. Drug Information J. 1999; 33: 1189–1194. 7. Koch A, Rohmel J. The impact of sloppy study conduct on non-inferiority studies. Drug Information J. 2002; 36: 3–6. 8. Chuang C, Sanders C, Snapinn S. An industry survey on current practice in the design and analysis of active control studies. J. Biopharm. Stat. 2004; 14(2): 349–358. 9. Koch A, Rohmel J. Hypothesis testing in the ‘‘gold standard’’ design for proving the efficacy of an experimental treatment relative to placebo and a reference. J. Biopharm. Stat. 2004; 14(2): 315–325. 10. International Conference on Harmonisation E14. (2005). The clinical evaluation of QT/QTc interval prolongation and proarrhythmic potential for non-antiarrhythmic drugs. Step 4 document. Available: http://www.ich.org/ cache/compo/276-254-1.html. 11. Hauschke D, Slacik-Erben R, Hensen S, Kaufmann R. Biostatistical assessment of mutagenicity studies by including the positive control. Biometrical J. 2005; 47: 82–87.

ASSESSMENT BIAS

when they assessed the effect openly but not when they assessed the effect blindly in the same patients (4). Some outcomes can only be meaningfully evaluated by the patients (e.g., pain and well-being). Unfortunately, blinding patients effectively can be very difficult, which is why active placebos are sometimes used. The idea behind an active placebo is that patients should experience side effects of a similar nature as when they receive the active drug, although it contains so little of a drug that it can hardly cause any therapeutic effect. As lack of blinding can lead to substantial bias, it is important in blinded trials to test whether the blinding has been compromised. Unfortunately, this test is rarely done (Asbjørn Hr´objartsson, unpublished observations), and, in many cases, double-blinding is little more than window dressing. Some outcome assessments are not made until the analysis stage of the trial (see below). Blinding should, therefore, also be used during data analysis, and it should ideally be preserved until two versions of the manuscript— written under different assumptions which of the treatments is experimental and which is control—have been approved by all the authors (5).

PETER C. GØTZSCHE Nordic Cochrane Centre Rigshospitalet, København Ø, Denmark

1

INTRODUCTION

Assessment bias in a clinical trial occurs if bias in the assessment of the outcome exists. It is also called ascertainment bias, diagnostic bias, or detection bias (1). A major cause of assessment bias is lack of blinding. Other problems relate to differential identification of harmless or false-positive cases of disease, bias in assessment of disease-specific mortality, the use of composite outcomes, competing risks, timing of the assessments, and bias in assessment of harms. 2

LACK OF BLINDING

One of the most important and most obvious causes of assessment bias is lack of blinding. In empirical studies, lack of blinding has been shown to exaggerate the estimated effect by 14%, on average, measured as odds ratio (2). These studies have dealt with a variety of outcomes, some of which are objective and would not be expected to be influenced by lack of blinding (e.g., total mortality). When patient reported outcomes are assessed, lack of blinding can lead to far greater bias than the empirical average. An example of a highly subjective outcome is the duration of an episode of the common cold. A cold does not stop suddenly, and awareness of the treatment received could therefore bias the evaluation. In a placebo-controlled trial of Vitamin C, the duration seemed to be shorter when an active drug was given, but many participants had guessed they received the vitamin because of its taste (3). When the analysis was restricted to those who could not guess what they had received, the duration was not shorter in the active group. Assessments by physicians are also vulnerable to bias. In a trial in multiple sclerosis, neurologists found an effect of the treatment

3 HARMLESS OR FALSE-POSITIVE CASES OF DISEASE Assessment bias can occur if increased diagnostic activity leads to increased diagnosis of true, but harmless cases of disease. Many stomach ulcers are silent (i.e., they come and go and give no symptoms). Such cases could be detected more frequently in patients who receive a drug that causes unspecific discomfort in the stomach. Similarly, if a drug causes diarrhea, it could lead to more digital, rectal examinations, and therefore also to the detection of more cases of prostatic cancer, most of which would be harmless, because many people die with prostatic cancer but rather few die from prostatic cancer. Assessment bias can also be caused by differential detection of false-positive cases

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

ASSESSMENT BIAS

of disease. Considerable observer variation often exists with common diagnostic tests. For gastroscopy, for example, a kappa value of 0.54 has been reported for the interobserver variation in the diagnosis of duodenal ulcers (6), which usually means that rather high rates of both false-positive findings and false-negative findings occur. If treatment with a drug leads to more gastroscopies because ulcers are suspected, one would therefore expect to find more (false) ulcers in patients receiving that drug. A drug that causes unspecific, non-ulcer discomfort in the stomach could therefore falsely be described as an ulcer-inducing drug. The risk of bias can be reduced by limiting the analysis to serious cases that would almost always become known (e.g., cases of severely bleeding ulcers requiring hospital admission or leading to death). 4

DISEASE-SPECIFIC MORTALITY

Disease-specific mortality is very often used as the main outcome in trials without any discussion of how reliable it is, even in trials of severely ill patients in which it can be difficult to ascribe particular causes for the deaths with acceptable error. Disease-specific mortality can be highly misleading if a treatment has adverse effects that increases mortality from other causes. It is only to be expected that aggressive treatments can have such effects. Complications to cancer treatment, for example, cause mortality that is often ascribed to other causes, although these deaths should have been added to the cancer deaths. A study found that deaths from other causes than cancer were 37% higher than expected and that most of this excess occurred shortly after diagnosis, suggesting that many of the deaths were attritutable to treatment (7). The use of blinded endpoint committees can reduce the magnitude of misclassification bias, but cannot be expected to remove it. Radiotherapy for breast cancer, for example, continues to cause cardiovascular deaths even 20 years after treatment (8), and it is not possible to distinguish these deaths from cardiovascular deaths from other causes. Furthermore, to work in an unbiased

way, death certificates and other important documents must have been completed and patients and documents selected for review without awareness of status, and it should not be possible to break the masking during any of these processes, including review of causes of death. This result seems difficult to obtain, in particular because those who prepare excerpts of the data should be kept blind to the research hypothesis (1). Fungal infections in cancer patients with neutropenia after chemotherapy or bonemarrow transplantation is another example of bias in severely ill patients. Not only is it difficult to establish with certainty that a patient has a fungal infection and what was the cause of death, but evidence also exists that some of drugs (azole antifungal agents) may increase the incidence of bacteriaemias (9). In the largest placebocontrolled trial of fluconazole, more deaths were reported on drug than on placebo (55 vs 46 deaths), but the authors also reported that fewer deaths were ascribed to acute systemic fungal infections (1 vs 10 patients, P = 0.01) (10). However, if this subgroup result is to be believed, it would mean that fluconazole increased mortality from other causes (54 vs 36 patients, P = 0.04). Bias related to classification of deaths can also occur within the same disease. After publication of positive results from a trial in patients with myocardial infarction (11), researchers at the U.S. Food and Drug Administration found that the causeof-death classification was ‘‘hopelessly unreliable’’ (12). Cardiac deaths were classified into three groups: sudden deaths, myocardial infarction, or other cardiac event. The errors in assigning cause of death nearly all favored the conclusion that sulfinpyrazone decreased sudden death, the major finding of the trial. 5 COMPOSITE OUTCOMES Composite outcomes are vulnerable to bias when they contain a mix of objective and subjective components. A survey of trials with composite outcomes found that when they included clinician-driven outcomes, such as hospitalization and initiation of new antibiotics, in addition to objective outcomes such

ASSESSMENT BIAS

as death, it was twice as likely that the trial reported a statistically significant effect (13). 6

COMPETING RISKS

Composite outcomes can also lead to bias because of competing risks (14), for example, if an outcome includes death as well as hospital admission. A patient who dies cannot later be admitted to hospital. This bias can also occur in trials with simple outcomes. If one of the outcomes is length of hospital stay, a treatment that increases mortality among the weakest patients who would have had long hospital stays, may spuriously appear to be beneficial. 7

given in the protocol, the conversion of these data into publishable bits of information can be difficult and often involves subjective judgments. Particularly vulnerable to assessment bias is exclusion of reported effects because they are not felt to be important, or not felt to be related to the treatment. Trials that have been published more than once illustrate how subjective and biased assessment of harms can be. Both number of adverse effects and number of patients affected can vary from report to report, although no additional inclusion of patients or follow-up have occurred, and these re-interpretations or reclassifications sometimes change an insignificant difference into a significant difference in favor of the new treatment (16).

TIMING OF OUTCOMES

Timing of outcomes can have profound effects on the estimated result, and the selection of time points for reporting of the results is often not made until the analysis stage of the trials, when possible treatment codes have been broken. A trial report of the anti-arthritic drug, celecoxib, gave the impression that it was better tolerated than its comparators, but the published data referred to 6 months of follow-up, and not to 12 and 15 months, as planned, when little difference existed; in addition, the definition of the outcome had changed, compared with what was stated in the trial protocol (15). Trials conducted in intensive care units are vulnerable to this type of bias. For example, the main outcome in such trials can be total mortality during the stay in the unit, but if the surviving patients die later, during their subsequent stay at the referring department, little may be gained by a proven mortality reduction while the patients were sedated. A more relevant outcome would be the fraction of patients who leave the hospital alive. 8

3

ASSESSMENT OF HARMS

Bias in assessment of harms is common. Even when elaborate, pretested forms have been used for registration of harms during a trial, and guidelines for their reporting have been

REFERENCES 1. A. R. Feinstein, Clinical Epidemiology. Philadelphia: Saunders, 1985. ¨ 2. P. Juni, D. G. Altman, and M. Egger, Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ 2001; 323: 42–46. 3. T. R. Karlowski, T. C. Chalmers, L. D. Frenkel, A. Z. Kapikian, T. L. Lewis, and J. M. Lynch, Ascorbic acid for the common cold: a prophylactic and therapeutic trial. JAMA 1975; 231: 1038–1042. 4. J. H. Noseworthy et al., The impact of blinding on the results of a randomized, placebocontrolled multiple sclerosis clinical trial. Neurology 1994; 44: 16–20. 5. P. C. Gøtzsche, Blinding during data analysis and writing of manuscripts. Controlled Clin. Trials 1996; 17: 285–290. 6. T. Gjørup et al., The endoscopic diagnosis of duodenal ulcer disease. A randomized clinical trial of bias and interobserver variation. Scand. J. Gastroenterol. 1986; 21: 561–567. 7. B. W. Brown, C. Brauner, and M. C. Minnotte, Noncancer deaths in white adult cancer patients. J. Natl. Cancer Inst. 1993; 85: 979–987. 8. Early Breast Cancer Trialists’ Collaborative Group, Favourable and unfavourable effects on long-term survival of radiotherapy for early breast cancer: an overview of the randomised trials. Lancet 2000; 355: 1757–1770. 9. P. C. Gøtzsche and H. K. Johansen, Routine versus selective antifungal administration for

4

ASSESSMENT BIAS

control of fungal infections in patients with cancer (Cochrane Review). In: The Cochrane Library, Issue 3. Oxford: Update Software, 2003. 10. J. L. Goodman et al., A controlled trial of fluconazole to prevent fungal infections in patients undergoing bone marrow transplantation. N. Engl. J. Med. 1992; 326: 845–851. 11. The Anturane Reinfarction Trial Research Group, Sulfinpyrazone in the prevention of sudden death after myocardial infarction. N. Engl. J. Med. 1980; 302: 250–256. 12. R. Temple and G. W. Pledger, The FDA’s critique of the anturane reinfarction trial. N. Engl. J. Med. 1980; 303: 1488–1492. 13. N. Freemantle, M. Calvert, J. Wood, J. Eastaugh, and C. Griffin, Composite outcomes in randomized trials: greater precision but with greater uncertainty? JAMA 2003; 289: 2554–2559. 14. M. S. Lauer and E. J. Topol, Clinical trials - multiple treatments, multiple end points, and multiple lessons. JAMA 2003; 289: 2575–2577. ¨ 15. P. Juni, A. W. Rutjes, and P. A. Dieppe, Are selective COX 2 inhibitors superior to traditional non steroidal anti-inflammatory drugs? BMJ 2002; 324: 1287–1288. 16. P. C. Gøtzsche, Multiple publication in reports of drug trials. Eur. J. Clin. Pharmacol. 1989; 36: 429–432.

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

(7) Role-Emotional, and (8) Mental Health. These eight domains can be further summarized by two summary scales: Physical Component Summary (PCS) and Mental Component Summary (MCS) scales. This article is intended to provide an overview of assessment of HRQOL in clinical trials. For more specific details on a particular topic mentioned in this article, the readers should consult the cited references. The development of a new HRQOL questionnaire and its translation into various languages are separate topics and are not covered in this article.

C. S. WAYNE WENG Department of Biomedical Engineering Chung Yuan Christian University Chungli, Taiwan

1

INTRODUCTION

Randomized clinical trials are the gold standard for evaluating new therapies. The primary focus of clinical trials has traditionally been evaluation of efficacy and safety. As clinical trials evolved from traditional efficacy and safety assessment of new therapies, clinicians were interested in an overall evaluation of the clinical impact of these new therapies on patient daily functioning and well-being as measured by health-related quality of life (HRQOL). As a result, HRQOL assessments in clinical trials have risen steadily throughout the 1990s and continue into the twentyfirst century. What is HRQOL? Generally, quality of life encompasses four major domains (1): 1. 2. 3. 4.

2

CHOICE OF HRQOL INSTRUMENTS

HRQOL instruments can be classified into two types: generic instruments and diseasespecific instruments. The generic instrument is designed to evaluate general aspects of a person’s HRQOL, which should include physical functioning, mental functioning, and social well-being. A generic instrument can be used to evaluate the HRQOL of a group of people in the general public or a group of patients with a specific disease. As such, data collected with a generic instrument allow comparison of HRQOL among different disease groups or against a general population. As a generic instrument, is designed to cover a broad range of HRQOL issues and it may be less sensitive regarding important issues for a particular disease or condition. Disease-specific instruments focus assessment in a more detailed manner for a particular disease. A more specific instrument allows detection of changes in disease-specific areas that a generic instrument is not sufficiently sensitive to detect. For example, the Health Assessment Questionnaire (HAQ) is developed to measure functional status of patients with rheumatic disease. The HAQ assesses the ability to function in eight areas of daily life: dressing and grooming, arising, eating, walking, hygiene, reach, grip, and activities. Table 1 (3–30) includes a list of generic and disease-specific HRQOL instruments for common diseases or conditions.

Physical status and functional abilities Psychological status and well-being Social interactions Economic or vocational status and factors

World Health Organization (WHO) defines ‘‘health’’ (2) as a ‘‘state of complete physical, mental, and social well-being and not merely the absence of infirmity and disease.’’ HRQOL focuses on parts of quality of life that are related to an individual’s health. The key components of this definition of HRQOL include (1) physical functioning, (2) mental functioning, and (3) social wellbeing, and a well-balanced HRQOL instrument should include these three key components. For example, the Medical Outcomes Study Short Form-36 (SF-36), a widely used HRQOL instrument, includes a profile of eight domains: (1) Physical Functioning, (2) Role-Physical, (3) Bodily Pain, (4) Vitality, (5) General Health, (6) Social Functioning,

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

Table 1. Generic and Disease-Specific HRQOL Instruments for Common Diseases or Conditions Generic HRQOL Instruments

Reference

Short Form-36 (SF-36) Sickness Impact Profile Nottingham Health Profile Duke Health Profile McMaster Health Index Questionnaire Functional Status Questionnaire WHO Quality of Life Assessment

3–5 6 7 8 9 10 11

Disease-Specific HRQOL Instruments: Disease Instrument

Reference

Pain

Brief Pain Inventory McGill Pain Questionnaire Visual Analogue Pain Rating scales

Depression

Beck Depression Inventory Center for Epidemiologic Studies Depression Scale (CES-D) Hamilton Rating Scale for Depression The Hospital Anxiety and Depression Questionnaire Zung Self-Rating Depression Scale WHO Well-Being Questionnaire Health Assessment Questionnaire (HAQ)

Rheumatic Disease: • Rheumatoid Arthritis • Osteoarthritis • Ankylosing Spondylitis • Juvenile Rheumatoid Arthritis Inflammatory Bowel Disease Asthma Airway Disease Seasonal Allergic Rhinitis Parkinson’s Disease

Cancer Both have tumor-specific modules

Inflammatory Bowel Disease Questionnaire (IBDQ) Asthma Quality of Life Questionnaire (AQLQ) St. George’s Respiratory Questionnaire (SGRQ) Rhinoconjunctivitis Quality of Life Questionnaire (RQLQ) Parkinson’s disease Questionnaire −39 item (PDQ-39) Parkinson’s Disease Quality of Life Questionnaire (PDQL) EORTC QLQ-C30 Functional Assessment of Cancer Therapy (FACT)

A comprehensive approach to assessing HRQOL in clinical trials can be achieved

12 13 Various authors; see Reference 14, p. 341 15 16 17 18 19 20 21

22, 23 24 25 26 27 28 29 30

relevant HRQOL components or a ‘‘module’’ approach, which includes a core measure of HRQOL domains supplemented in

using a battery of questionnaires when a single questionnaire does not address all

the same questionnaire by a disease- or treatment-specific set of items. The battery

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

approach combines a generic HRQOL instrument plus a disease-specific questionnaire. For example, in a clinical trial on rheumatoid arthritis (RA), one can include the SF-36 and HAQ to evaluate treatment effect on HRQOL. The SF-36 allows comparison of RA burden on patients’ HRQOL with other diseases as well as the general population. The HAQ, being a disease-specific instrument, measures patients’ ability to perform activities of daily life and is more sensitive to changes in a RA patient’s condition. The module approach has been widely adopted in oncology as different tumors impact patients in different ways. The most popular cancer-specific HRQOL questionnaires, EORTC QLQ-C3O and FACT, both include core instruments that measure physical functioning, mental functioning, and social well-being as well as common cancer symptoms, supplemented with a list of tumor- and treatment-specific modules. In certain diseases, a disease-specific HRQOL instrument is used alone in a trial because the disease’s impact on general HRQOL is so small that a generic HRQOL instrument will not be sensitive enough to detect changes in disease severity. For example, disease burden of allergic rhinitis on generic HRQOL is relatively small compared with the general population. Most published HRQOL studies in allergic rhinitis use Juniper’s Rhinoconjunctivitis Quality of Life Questionnaire (RQLQ), a disease-specific HRQOL questionnaire for allergic rhinitis. 3 ESTABLISHMENT OF CLEAR OBJECTIVES IN HRQOL ASSESSMENTS A clinical trial is usually designed to address one hypothesis or a small number of hypotheses, evaluating a new therapy’s efficacy, safety, or both. When considering whether to include HRQOL assessment in a study, the question of what additional information will be provided by the HRQOL assessment must be asked. As estimated by Moinpour (31), the total cost per patient is $443 to develop an HRQOL study, monitor HRQOL form submission, and analyze HRQOL data. Sloan et al. (32) have revisited the issue of the cost of HRQOL assessment in a number of settings including clinical trials and

3

suggest a wide cost range depending on the comprehensiveness of the assessment, which is not a trivial sum of money to be spent in a study without a clear objective in HRQOL assessment. The objective of HRQOL assessment is usually focused on one of the four possible outcomes: (1) improvement in efficacy leads to improvement in HRQOL, (2) treatment side effects may cause deterioration in HRQOL, (3) the combined effect of (1) and (2) on HRQOL, and (4) similar efficacy with an improved side effect profile leads to improvement in HRQOL. After considering possible HRQOL outcomes, one will come to a decision whether HRQOL assessment should be included in the trial. In many published studies, HRQOL was included in the studies without a clear objective. These studies generated HRQOL data that provided no additional information at the completion of the studies. Goodwin et al. (33) provide an excellent review of HRQOL measurement in randomized clinical trials in breast cancer. They suggest that, given the existing HRQOL database for breast cancer, it is not necessary to measure HRQOL in every trial, at least until ongoing trials are reported. An exception is interventions with a psychosocial focus where HRQOL has to be the primary outcome. 4

METHODS FOR HRQOL ASSESSMENT

The following components should be included in a study protocol with an HRQOL objective: – Rationale for assessing HRQOL objective(s) and for the choice of HRQOL instrument(s): – To help study personnel understand the importance of HRQOL assessment in the study, inclusion of a clear and concise rationale for HRQOL assessment is essential, along with a description of the specific HRQOL instrument(s) chosen. – HRQOL hypotheses: – The study protocol should also specify hypothesized HRQOL outcomes with respect to general and specific domains. It is helpful to identify

4

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

the primary domain and secondary domains for HRQOL analysis in the protocol. – Frequency of HRQOL assessment: – In a clinical trial, the minimum number of HRQOL assessments required is two, at baseline and the end of the study for studies with a fixed treatment duration, and most patients are expected to complete the treatment. One or two additional assessments should be considered between baseline and study endpoint depending on the length of the study so that a patient’s data will still be useful if endpoint data was not collected. More frequent assessments should be considered if the treatment’s impact on HRQOL may change over time. Three or more assessments are necessary to characterize patterns of change for individual patients. In oncology trials, it is common to assess HRQOL on every treatment cycle, as patients’ HRQOL is expected to change over time. However, assessment burden can be minimized if specific time points associated with expected clinical effects are of interest and can be specified by clinicians (e.g., assess HRQOL after the minimum number of cycles of therapy required to observe clinical activity of an agent). Another factor to be considered for the frequency of HRQOL assessment is the recall period for a particular HRQOL instrument. The recall period is the duration a subject is asked to assess his/her responses to an HRQOL questionnaire. The most common recall periods are one week, two weeks, and four weeks. – Administering HRQOL questionnaires: – To objectively evaluate HRQOL, one needs to minimize physician and study nurse influence on patient’s response to HRQOL questions. Therefore, the protocol should indicate that patients are to complete the HRQOL questionnaire in a quiet place in the doctor’s office at the beginning of his/her office

visit, prior to any physical examination and clinical evaluation by the study nurse and physician. – Specify the magnitude of difference in HRQOL domain score that can be detected with the planned sample size: – This factor is especially important when the HRQOL assessment is considered a secondary endpoint. As the sample size is based on the primary endpoint, it may provide only enough power to detect a relatively large difference in HRQOL scores. The question of whether to increase the sample size to cover HRQOL assessment often depends on how many more additional patients are needed and the importance of the HRQOL issue for the trial. To collect HRQOL data when power will be insufficient to detect effects of interest is a waste of clinical resources and the patient’s time. – Specify how HRQOL scores are to be calculated and analyzed in the statistical analysis section: – Calculation of HRQOL domain scores should be stated clearly, including how missing items will be handled. As a result of the nature of oncology studies, especially in late-stage disease, patients will stop treatment at different time points because of disease progression, intolerance to treatment side effects, or death and therefore fail to complete the HRQOL assessment schedule. For example, if data are missing because of deteriorating patient health, the study estimates of effect on HRQOL will be biased in favor of better HRQOL; the term ‘‘informative missing data’’ is the name for this phenomenon and must be handled with care. Fairclough (34) has written a book on various longitudinal methods to analyze this type of HRQOL data. However, analyzing and interpreting HRQOL data in this setting remain a challenge. – Strategies to improve HRQOL data collection:

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

↑ Education at the Investigators’ meeting and during site initiation visit: It is important to have investigators and study coordinators committed to the importance of HRQOL assessment. Without this extra effort, HRQOL assessment is likely to be unsuccessful, simply because collecting HRQOL data is not part of routine clinical trial conduct. – Emphasize the importance of the HRQOL data: ↑ Baseline HRQOL forms should be required in order to register a patient to a trial. Associate grant payment per patient with submission of a patient’s efficacy data. Specifying some portion of the grant payment with the submission of HRQOL form has significantly increased the HRQOL completion rate in the author’s clinical experience. • Establish a prospective reminder system for upcoming HRQOL assessments and a system for routine monitoring of forms at the same time clinical monitoring is being conducted. The following checklist (Table 2) may be helpful when considering inclusion of HRQOL assessment in a clinical trial protocol. 5

HRQOL AS THE PRIMARY ENDPOINT

To use HRQOL as the primary endpoint in a clinical trial, prior information must demonstrate at least comparable efficacy of a study treatment to its control. In this context, to design a study with HRQOL as the primary endpoint, the sample size will have to be large enough to assure adequate power to detect meaningful differences between treatment groups in HRQOL. Another context for a primary HRQOL endpoint is in the setting of treatment palliation. In this case, treatment efficacy is shown by the agent’s ability to palliate disease-related symptoms and overall HRQOL without incurring treatmentrelated toxicities. For example, patient report

5

of pain reduction can document the achievement of palliation [e.g., see Tannock et al. (35) example below]. A HRQOL instrument usually has several domains to assess various aspects of HRQOL. Some HRQOL instruments also provide for an overall or total score. The HRQOL endpoint should specify a particular domain, or the total score, as the primary endpoint of the HRQOL assessment in order to avoid multiplicity issues. If HRQOL is included as a secondary endpoint, it is a good practice to identify a particular domain as the primary focus of the HRQOL assessment. This practice forces specification of the expected outcomes of HRQOL assessments. Some investigators have applied multiplicity adjustments to HRQOL assessments. The approach may be statistically prudent, but it does not provide practical value. The variability of HRQOL domain scores is generally large. With multiple domains being evaluated, only a very large difference between groups will achieve the required statistically significance level. When evaluating HRQOL as a profile of a therapy’s impact on patients, clinical judgment of the magnitude of HRQOL changes should be more important than the statistical significance. However, this ‘‘exploratory’’ analysis perspective should also be tempered with recognition that some significant results may be marginally significant and subject to occurrence by chance. 6

INTERPRETATION OF HRQOL RESULTS

Two approaches have been used to interpret the meaningfulness of observed HRQOL differences between two treatment groups in a clinical trial: distribution-based and anchor-based approaches. The most widely used distribution-based approach is the effect size, among other methods listed in Table 3 (36–45). Based on the effect size, an observed difference is classified into (1) 0.2 = a small difference, (2) 0.5 = a moderate difference, and (3) 0.8 = a large difference. To advocate using the effect size to facilitate the interpretation of HRQOL data, Sloan et al. (46) suggested a 0.5 standard deviation as

6

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

Table 2. Checklist for HRQOL Assessment • • • • •

Rationale to assess HRQOL and the choice of HRQOL instrument(s) Hypothesis in terms of expected HRQOL outcomes Frequency of HRQOL assessment Procedures for administering HRQOL questionnaires Specify the magnitude of difference in HRQOL domain score that can be detected with the planned sample size • Specify in the statistical analysis section how HRQOL scores are to be calculated and analyzed • Strategies to improve HRQOL data collection

a reasonable benchmark for a 0–100 scale to be clinically meaningful. This suggestion is consistent with Cohen’s (47) suggestion of one-half of a standard deviation as indicating a moderate effect and therefore clinically meaningful. The anchor-based approach compares observed differences relative to an external standard. Investigators have used this approach to define the minimum important difference (MID). For example, Juniper and Guyatt (26) suggested that a 0.5 change in RQLQ be the MID (RQLQ score ranges from 1 to 7). Osoba (48) suggested that a 10-point change in EORTC QLQ-C30 questionnaire would be a MID. Both these two MIDs are group average scores. How these MIDs apply to individual patients is still an issue. Another issue in using MID is related to the starting point of patients’ HRQOL scores. Guyatt et al. (49) provides a detailed overview of various strategies to interpret HRQOL results. 7

EXAMPLES

7.1 HRQOL in Asthma To evaluate salmeterol’s effect on quality of life, patients with nocturnal asthma were enrolled into a double-blind, parallel group, placebo-controlled, multicenter study (50). The study rationale was that patients with nocturnal asthma who are clinically stable have been found to have poorer cognitive performance and poorer subjective and objective sleep quality compared with normal, healthy patients. To assess salmeterol’s effect on reducing the impact of nocturnal asthma on patients’ daily functioning and well-being, patients were randomized to

receive salmeterol, 42 µg, or placebo twice daily. Patients were allowed to continue theophylline, inhaled corticosteroids, and ‘‘asneeded’’ albuterol. Treatment duration was 12 weeks, with a 2-week run-in period. The primary study objective was to assess the impact of salmeterol on asthma-specific quality of life using the validated Asthma Quality of Life Questionnaire (24) (AQLQ). Patients were to return to the clinic every 4 weeks. Randomized patients were to complete an AQLQ at day 1; weeks 4, 8, 12; and at the time of withdrawal from the study for any reason. Efficacy (FEV1, PEF, nighttime awakenings, asthma symptoms, and albuterol use) and safety assessments were also conducted at these clinic visits. Scheduling HRQOL assessment prior to efficacy and safety evaluations at office visits minimizes investigator bias and missing HRQOL evaluation forms. The AQLQ is a 32-item, self-administered, asthma-specific instrument that assesses quality of life over a 2-week time interval. Each item is scored using a scale from 1 to 7, with lower scores indicating greater impairment and higher scores indicating less impairment in quality of life. Items are grouped into four domains: (1) activity limitation (assesses the amount of limitation of individualized activities that are important to the patient and are affected by asthma); (2) asthma symptoms (assesses the frequency and degree of discomfort of shortness of breath, chest tightness, wheezing, chest heaviness, cough, difficulty breathing out, fighting for air, heavy breathing, difficulty getting a good night’s sleep); (3) emotional function (assesses the frequency of being afraid of not having medications, concerned about medications, concerned about

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

7

Table 3. Common Methods Used to Measure a Questionnaire’s Responsiveness to Change Method

Formula


Relative change
Effect size

Meantest1 − Meantest2 SDtest1

36 

Square of Effect Sizedimension Effect Sizestandard
 Meantest1 − Meantest2 SDdifference
 Meantest1 − Meantest2 SDstable group 
Meantest1 − Meantest2 SEdifference

Relative efficiency

Standardized response mean

Responsiveness statistic

Paired t statistic SE of measurement

Meantest1 − Meantest2 Meantest1

Reference 




SDtest × Square Root  1 − Reliability Coefficienttest

37, 38 39

40

41, 42

43 44, 45

Reprinted with permission

having asthma, frustrated); and (4) environmental exposure (assesses the frequency of exposure to and avoidance of irritants such as cigarette smoke, dust, and air pollution). Individual domain scores and a global score are calculated. A change of 0.5 (for both global and individual domain scores) is considered the smallest difference that patients perceive as meaningful (51). To achieve 80% power to detect a difference of 0.5 in AQLQ between two treatment arms would only require 80 patients per arm at a significance level of 0.05. However, this study was designed to enroll 300 patients per arm so that it could also provide 80% power to detect differences in efficacy variables (e.g., FEV1, nighttime awakening) between two treatment arms at a significance level of 0.05. A total of 474 patients were randomly assigned to treatment. Mean change from baseline for the AQLQ global and each of the four domain scores was significantly greater (P 0.005) with salmeterol compared with placebo, first observed at week 4 and continuing through week 12. In addition, differences between salmeterol and placebo groups were greater than 0.5 at all visits except at week 4 and week 8 for the environmental exposure domain. At week 12, salmeterol significantly (P < 0.001 compared

with placebo) increased mean change from baseline in FEV1 , morning and evening PEF, percentage of symptom-free days, percentage of nights with no awakenings due to asthma, and the percentage of days and nights with no supplemental albuterol use. This study demonstrated that salmeterol’s effect in improving patients’ asthma symptoms had a more profound effect on improving patients’ daily activity and well-being. 7.2 HRQOL in Seasonal Allergy Rhinitis A randomized, double-blind, placebocontrolled study was conducted to evaluate the effects on efficacy, safety, and quality of life of two approved therapies (fexofenadine HCI 120 mg and loratadine 10 mg) for the treatment of seasonal allergy rhinitis (SAR), (52). Clinical efficacy was based on a patient’s evaluation of SAR symptoms: (1) sneezing; (2) rhinorrhea; (3) itchy noses, palate, or throat; and (4) itchy, watery, or red eyes. The primary efficacy endpoint was the total score for the patient symptom evaluation, defined as the sum of the four individual symptom scores. Each of the symptoms was evaluated on a 5-point scale (0 to 4), with higher scores indicating more severe symptoms. Treatment duration was 2 weeks, with a run-in period of

8

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

3–7 days. After randomization at study day 1, patients were to return the clinic every week. During these visits, patients were to be evaluated for the severity of SAR symptoms and to complete a quality of life questionnaire. Patient-reported quality of life was evaluated using a validated disease-specific questionnaire— Rhinoconjunctivitis Quality of Life Questionnaire (RQLQ) (26). The RQLQ is a 28-item instrument that assesses quality of life over a 1-week time interval. Each item is scored using a scale from 0 (i.e., not troubled) to 6 (i.e., extremely troubled), with lower scores indicating greater impairment and higher scores indicating less impairment in quality of life. Items are grouped into seven domains: (1) sleep, (2) practical problems, (3) nasal symptoms, (4) eye symptoms, (5) non-nose/eye symptoms, (6) activity limitations, and (7) emotional function. Individual domain scores and an overall score are calculated. A change of 0.5 (for both global and individual domain scores) is considered the smallest difference that patients perceive as meaningful (53). The RQLQ assessment was a secondary endpoint. No sample size and power justification was mentioned in the published paper. A total of 688 patients were randomized to receive fexofenadine HCI 120 mg, loratadine 10 mg, or placebo once daily. Mean 24-hour total symptom score (TSS) as evaluated by the patient was significantly reduced by both fexofenadine HCI and loratadine from baseline (P ≤ 0.001) compared with placebo. The difference between fexofenadine HCI and loratadine was not statistically significant. For overall quality of life, a significant improvement from baseline occurred for all three treatment groups (mean improvement was 1.25, 1.00, and 0.93 for fexofenadine HCI, loratadine, and placebo, respectively). The improvement in the fexofenadine HCI group was significantly greater than that in either the loratadine (P ≤ 0.03) or placebo (P ≤ 0.005) groups. However, the magnitude of differences among the treatment groups was less than the minimal important difference of 0.5. The asthma example demonstrates that salmeterol not only significantly improved patients’ asthma-related symptoms, both statistically and clinically, but also relieved their

asthma-induced impairments on daily functioning and well-being. On the other hand, the SAR example demonstrates that fexofenadine HCI and loratadine both were effective in relief of SAR symptoms. The difference between fexofenadine HCI and loratadine in HRQOL was only statistically significant, but not clinically. However, Hays and Wooley (54) have cautioned investigators about the potential for oversimplication when applying a single minimal clinically important difference (MCID). 7.3 Symptoms Relief for Late-Stage Cancers Although the main objective for the treatment of early-stage cancers is to eradicate the cancer cells and prolong survival, it may not be achievable in late-stage cancers. More often, the objective for the treatment of latestage cancers is palliation, mainly through relief of cancer-related symptoms. As the relief of cancer-related symptoms represents clinical benefit to patients, the objective of some clinical trials in late-stage cancer was relief of a specific cancer-related symptom such as pain. To investigate the benefit of mitoxantrone in patients with symptomatic hormone-resistant prostate cancer, hormonerefractory patients with pain were randomized to receive mitoxantrone plus prednisone or prednisone alone (35). The primary endpoint was a palliative response defined as a two-point decrease in pain as assessed by a six-point pain scale completed by patients (or complete loss of pain if initially 1+) without an increase in analgesic medication and maintained for two consecutive evaluations at least 3 weeks apart. Palliative response was observed in 23 of 80 patients (29%; 95% confidence interval; range 19–40%) who received mitoxantrone plus prednisone, and in 10 of 81 patients (12%; 95% confidence interval; range 6–22%) who received prednisone alone (P = 0.01). No difference existed in overall survival. In another study assessing gemcitabine effect on relief of pain (55), 162 patients with advanced symptomatic pancreatic cancer completed a lead-in period to characterize and stabilize pain and were randomized to receive either gemcitabine 1000 mg/m2

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

weekly × 7 followed by 1 week of rest, then weekly × 3 every 4 weeks thereafter, or to fluorouracil (5-FU) 600 mg/m2 once weekly. The primary efficacy measure was clinical benefit response, which was a composite of measurements of pain (analgesic consumption and pain intensity), Karnofsky performance status, and weight. Clinical benefit required a sustained (≥4 weeks) improvement in at least one parameter without worsening in any others. Clinical benefit response was experienced by 23.8% of gemcitabine-treated patients compared with 4.8% of 5-FU-treated patients (P = 0.0022). In addition, the median survival durations were 5.65 and 4.41 months for gemcitabinetreated and 5-FU-treated patients, respectively (P = 0.0025). Regarding the use of composite variables, researchers have urged investigators to report descriptive results for all components so that composite results do not obscure potential negative results for one or more of the components of the composite (56, 57). In a third study example, although symptom assessment was not the primary endpoint, it was the main differentiating factor between the two study arms in study outcomes. As second-line treatment of small cell lung cancer (SCLC), topotecan was compared with cyclophosphamide, doxorubicin, and vincristine (CAV) in 211 patients with SCLC who had relapsed at least 60 days after completion of first-line therapy (58). Response rate and duration of response were the primary efficacy endpoints. Patientreported lung-cancer-related symptoms were also evaluated as secondary endpoints. Similar efficacy in response rate, progression-free survival, and overall survival was observed between topotecan and CAV. The response rate was 26 of 107 patients (24.3%) treated with topotecan and 19 of 104 patients (18.3%) treated with CAV (P = 0.285). Median times to progression were 13.3 weeks (topotecan) and 12.3 weeks (CAV) (P = 0.552). Median survival was 25.0 weeks for topotecan and 24.7 weeks for CAV (P = 0.795). However, the proportion of patients who experienced symptom improvement was greater in the topotecan group than in the CAV group for four of eight lung-cancer-related symptoms evaluated, including dyspnea, anorexia,

9

hoarseness, and fatigue, as well as interference with daily activity (P ≤ 0.043). 8

CONCLUSION

Although HRQOL assessment in clinical trials has increased steadily over the years, a substantial challenge remains in interpretation of HRQOL results and acceptance of its value in clinical research. Both issues will require time for clinicians and regulators to fully accept HRQOL assessments. To help build acceptance, existing HRQOL instruments should be validated in each therapeutic area, rather than develop new instruments. The most urgent need in HRQOL research is to increase HRQOL acceptance by clinicians and regulators so that pharmaceutical companies will continue to include financial support for HRQOL assessments in new and existing drug development programs. 9

ACKNOWLEDGMENT

The author is deeply indebted to Carol M. Moinpour for her numerous suggestions and to Carl Chelle for his editorial assistance. REFERENCES 1. J. A. Cramer and B. Spilker, Quality of Life and Pharmacoeconomics: An Introduction. Philadelphia: Lippincott-Raven, 1998. 2. World Health Organization, The First Ten Years of the World Health Organization. Geneva: World Health Organization, 1958, p. 459. 3. J. E. Ware, Jr. and C. D. Sherbourne, The MOS 36-Item Short-Form Health Survey (SF36). I. Conceptual framework and item selection. Med. Care 1992; 30: 473–483. 4. C. A. McHorney, J. E. Ware, Jr., and A. E. Raczek, The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med. Care 1993; 31: 247–263. 5. C. A. McHorney et al., The MOS 30-Item Short-Form Health Survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med. Care 1994; 32: 40–66.

10

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

6. M. Bergner et al., The Sickness Impact Profile: development and final revision of a health status measure. Med. Care 1981; 19: 787–805.

21.

7. S. M. Hunt, J. McEwen, and S. P. McKenna, Measuring Health Status. London: Croom Helm, 1986. 8. G. R. Parkerson, Jr., W. E. Broadhead, and C. K. Tse, The Duke Health Profile. A 17-item measure of health and dysfunction. Med. Care 1990; 28: 1056–1072.

22.

9. L. W. Chambers, The McMaster Health Index Questionnaire (MHIQ): Mehtodologic Documentation and Report of the Second Generation of Investigations. Hamilton, Ontario, Canada: McMaster University, Department of Clinical Epidemiology and Biostatistics, 1982.

23.

10. A. M. Jette et al., The Functional Status Questionnaire: reliability and validity when used in primary care. J. Gen. Intern. Med. 1986; 1: 143–149. 11. S. Szabo (on behalf of the WHOQOL Group), The World Health Organization Quality of Life (WHOQOL) assessment instrument. In: B. Spilker (ed.), Quality of Life and Pharmacoeconomics in Clinical Trials. Philadelphia: Lippincott-Reven, 1996. 12. C. S. Cleeland, Measurement of pain by subjective report. In: C. R. Chapman and J. D. Loeser (eds.), Issue in Pain Measurement. New York: Raven Press, 1989.

24.

25.

26.

27.

28.

13. R. Melzack, The McGill Pain Questionnaire: major properties and scoring methods. Pain 1975; 1: 277–299. 14. I. McDowell and C. Newell, Measuring Health: A Guide to Rating Scales and Questionnaires, 2nd ed. New York: Oxford University Press, 1996.

29.

15. A. T. Beck et al., An inventory for measuring depression. Arch. Gen. Psychiat. 1961; 4: 561–571.

30.

16. L. S. Radloff, The CES-D scale: a self-report depression scale for research in the general population. Appl. Psychol. Measure. 1977; 1: 385–401.

31.

17. M. Hamilton, Standised assessment and recording of depressive symptoms. Psychiat. Neurol. Neurochir. 1969; 72: 201–205.

32.

18. A. Zigmond and P. Snaith, The Hospital Anxiety and Depression Questionnaire. Acta Scand. Psychiat. 1983; 67: 361–368. 19. W. W. K. Zung, A self-rating depression scale. Arch. Gen. Psychiat. 1965; 12: 63–70. 20. P. Bech et al., The WHO (Ten) Well-Being Index: validation in diabetes. [comment]. [Clinical Trial. Journal Article. Multicenter

33.

Study. Randomized Controlled Trial]. Psychother. Psychosomat. 1996; 65: 183–190. J. F. Fries et al., The dimension of health outcomes: the Health Assessment Questionnaire, disability and pain scales. J. Rheumatol. 1982; 9: 789–793. G. H. Guyatt, A. Mitchell, E. J. Irvine et al., A new measure of health status for clinical trials in inflammatory bowel disease. Gastroenterology 1989; 96: 804–810. E. J. Irvine, B. Feagan et al., Quality of life: a valid and reliable measure of therapeutic efficacy in the treatment of inflammatory bowel disease. Gastroenterology 1994; 106: 287–296. E. F. Juniper, G. H. Guyatt, P. J. Ferrie, and L. E. Griffith, Measuring quality of life in asthma. Am. Rev. Respir. Dis. 1993; 147: 832–838. P. W. Jones, F. H. Quirk, and C. M. Baveystock, The St. Geroge’s Respiratory Questionnaire. Respiratory Med. 1991; 85: 25–31. E. F. Juniper and G. H. Guyatt, Development and testing of a new measure of health status for clinical trials in rhinoconjunctivitis. Clin. Exp. Allergy 1991; 21: 77–83. V. Peto et al., The development and validation of a short measure of functioning and well being for individuals with Parkinson’s disease. Qual. Life Res. 1995; 4(3): 241–248. A. G. E. M. De Boer et al., Quality of life in patients with Parkinson’s disease: development of a questionnaire. J. Neurol. Neurosurg. Psychiat. 1996; 61(1): 70–74. N. K. Aaronson et al., The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J. Natl. Cancer Inst. 1993; 85: 365–376. D. F. Cella et al., The Functional Assessment of Cancer Therapy scale: development and validation of the general measure. J. Clin. Oncol. 1993; 11: 570–579. C. M. Moinpour, Costs of quality-of-life research in Southwest Oncology Group trials. J. Natl. Cancer Inst. Monogr. 1996; 20: 11–16. J. A. Sloan et al. and the Clinical Significance Consensus Meeting Group, The costs of incorporating quality of life assessments into clinical practice and research: what resources are required? Clin. Therapeut. 2003; 25(Suppl D). P. J. Goodwin et al., Health-related quality-oflife measurement in randomized clinical trails in breast cancer—taking stock. J. Natl. Cancer Inst. 2003; 95: 263–281.

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

11

34. D. L. Fairclough, Design and Analysis of Quality of Life Studies in Clinical Trials. Boca Raton, FL: Chapman & Hall/CRC Press, 2002.

47. J. Cohen, Statistical Power Analysis for the Behavioral Sciences, 2nd ed. London: Academic Press, 1988.

35. I. F. Tannock et al., Chemotherapy with mitoxantrone plus prednisone or prednisone alone for symptomatic hormone-resistant prostate cancer: a Canadian randomized trial with palliative end points. J. Clin. Oncol. 1996; 14: 1756–1764.

48. D. Osoba, G. Rodrigues, J. Myles, B. Zee, and J. Pater, Interpreting the significance of changes in health-related quality of life scores. J. Clin. Oncol. 1998; 16: 139–144.

36. R. A. Deyo and R. M. Centor, Assessing the responsiveness of functional scales to clinical change: an analogy to diagnostic test performance. J. Chronic Dis. 1986; 39: 897–906. 37. Kazis et al., 1989. 38. R. A. Deyo, P. Diehr, and D. L. Patrick, Reproducibility and responsiveness of health status measures: statistics and strategies for evaluation. Control Clin. Trials 1991; 12(4 Suppl): 142S–158S. 39. C. Bombardier, J. Raboud, and Auranofin Cooperating Group, A comparison of healthrelated quality-of-life measures for rheumatoid arthritis research. Control Clin. Trials 1991; 12(4 Suppl): 243S–256S. 40. J. N. Katz et al., Comparative measurement sensitivity of short and longer health status instruments. Med. Care 1992; 30: 917–925. 41. G. H. Guyatt, S. Walter, and G. Norman, Measuring change over time: assessing the usefulness of evaluative instruments. J. Chronic Dis. 1987; 40: 171–178. 42. G. H. Guyatt, B. Kirshner, and R. Jaeschke, Measuring health status: what are the necessary measurement properties? J. Clin. Epidemiol. 1992; 45: 1341–1345. 43. M. H. Liang et al., Comparative measurement efficiency and sensitivity of five health status instruments for arthritis research. Arthritis Rheum. 1985; 28: 542–547. 44. K. W. Wyrwich, N. A. Nienaber, W. M. Tierney, and F. D. Wolinsky, Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life. Med. Care 1999; 37: 469–478.

49. G. H. Guyatt et al. and the Clinical Significance Consensus Meeting Group, Methods to explain the clinical significance of health status measures. Mayo Clin. Proc. 2002; 77: 371–383. 50. R. F. Lockey et al., Nocturnal asthma – effect of salmeterol on quality of life and clinical outcomes. Chest 1999; 115: 666–673. 51. E. F. Juniper et al., 1994. 52. P. Van Cauwenberge and E. F. Juniper, The Star Study Investigating Group. Comparison of the efficacy, safety and quality of life provided by fexofenadine hydrochloride 120 mg, loratadine 10 mg and placebo administered once daily for the treatment of seasonal allergic rhinitis. Clin. Exper. Allergy 2000; 30: 891–899. 53. E. F. Juniper et al., 1996. 54. R. D. Hays and J. M. Woolley, The concept of clinically meaningful difference in healthrelated quality-of-life research. How meaningful is it? Pharmacoeconomics 2000; 18: 419–423. 55. H. A. Burris, III et al., Improvements in survival and clinical benefit with gemcitabine as first-line therapy for patients with advanced pancreas cancer: a randomized trial. J. Clin. Oncol. 1997; 15: 2403–2413. 56. N. Freemantle, M. Calvert, J. Wood, J. Eastaugh, and C. Griffin, Composite outcomes randomized trials. Greater precision but with greater uncertainty? JAMA 2003; 289: 2554–2559. 57. M. S. Lauer and E. J. Topol, Clinical trials—Multiple treatment, multiple end points, and multiple lessons. JAMA 2003; 289: 2575–2577. 58. J. von Pawel et al., Topotecan versus cyclophosphamide, doxorubicin, and vincristine for the treatment of recurrent smallcell lung cancer. J. Clin. Oncol. 1999; 17: 658–667.

45. K. W. Wyrwich, W. M. Tierney, and F. D. Wolinsky, Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. J. Clin. Epidemiol. 1999; 52: 861–873.

FURTHER READING

46. J. A. Sloan et al., Randomized comparison of four tools measuring overall quality of life in patients with advanced cancer. J. Clin. Oncol. 1998; 16: 3662–3673.

B. Spilker (ed.), Quality of Life and Pharmacoeconomics in Clinical Trials. Philadelphia: Lippincott-Reven, 1996.

12

ASSESSMENT OF HEALTH-RELATED QUALITY OF LIFE

M. A. G. Sprangers, C. M. Moinpour, T. J. Moynihan, D. L. Patrick, and D. A. Revicki, Clinical Significance Consensus Meeting Group. assessing meaningful change in quality of life over time: a users’ guide for clinicians. Mayo Clin. Proc. 2002; 77: 561–571. J. E. Ware et al., SF-36 Health Survey: Manual and Interpretation Guide. Boston: The Health Institute, New England Medical Center, 1993. World Health Organization, International Classification of Impairments, Disabilities, and Handicaps. Geneva: World Health Organization, 1980.

AUDIT

of legal proceedings or investigations. Where required by applicable law or regulation, the sponsor should provide an audit certificate.

An Audit is a systematic and independent examination of trial-related activities and documents to determine whether the evaluated trial-related activities were conducted, and the data were recorded, analyzed, and reported accurately according to the protocol, sponsor’s Standard Operating Procedures (SOPs), Good Clinical Practice (GCP), and applicable regulatory requirement(s). The purpose of a sponsor’s audit, which is independent of and separate from routine monitoring or quality control functions, should be to evaluate trial conduct and compliance with the protocol, SOPs, GCP, and applicable regulatory requirements. The sponsor should appoint individuals who are independent of the clinical trial/data collection system(s) to conduct audits. The sponsor should ensure that the auditors are qualified by training and by experience to conduct audits properly. An auditor’s qualifications should be documented. The sponsor should ensure that the auditing of clinical trials/systems is conducted in accordance with the sponsor’s written procedures on what to audit, how to audit, the frequency of audits, and the form and content of audit reports. The sponsor’s audit plan and procedures for a trial audit should be guided by the importance of the trial to submissions to regulatory authorities, the number of subjects in the trial, the type and complexity of the trial, the level of risks to the trial subjects, and any identified problem(s). The observations and findings of the auditor(s) should be documented. To preserve the independence and the value of the audit function, the regulatory authority(ies) should not request the audit reports routinely. Regulatory authority(ies) may seek access to an audit report on a case-by-case basis, when evidence of serious GCP noncompliance exists, or in the course This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

AUDIT CERTIFICATE An Audit Certificate is the declaration of confirmation by the auditor that an audit has taken place. Where required by applicable law or regulation, the sponsor should provide an audit certificate.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

AUDIT REPORT An Audit Report is the written evaluation by the sponsor’s auditor of the results of the audit. To preserve the independence and the value of the audit function, the regulatory authority(ies) should not request the audit reports routinely. Regulatory authority(ies) may seek access to an audit report on a case-by-case basis, when evidence of serious Good Clinical Practice (GCP) noncompliance exists, or in the course of legal proceedings or investigations.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS

The majority of phase I studies are conducted in healthy volunteers. Current doseescalation designs for healthy volunteer studies are usually crossover designs in which each subject receives a dose in each of a series of consecutive treatment periods separated by washout periods. Such a design is illustrated in Table 1. Groups of subjects, known as cohorts or panels, some on active doses and others on placebo, are treated simultaneously. The lowest dose, d1 , is normally regarded as the only ‘‘safe’’ dose to be used for the first cohort of subjects. Safety data along with pharmacokinetic/pharmacodynamic (PK /PD) data from each treatment period are collected then summarized in tables, listings, and graphs for a safety review committee to make decisions regarding the doses to be given during the next period. The committee will normally assign doses according to some predefined dose-escalation scheme, but they may alter the scheme by repeating the previous doses or de-escalating one dose level; they may even stop the trial if its safety is in question. This is essentially a PK/PD ‘‘clinical judgment’’– guided design. Response variables are analyzed either by simple analysis of variance (ANOVA) approaches (2) or by repeated measures analysis of variance. The PK/PD models may also be estimated and presented at the end of a trial. Under conventional dose-escalation rules, it is likely that healthy volunteers will be treated at subtherapeutic doses. Consequently, information gathered from a trial is mostly not relevant for identifying the optimal dose for phase II studies. Bayesian decision-theoretic designs, motivated by statistical-model–based designs for cancer trials, have been proposed to enhance the precision of the optimal dose for phase II studies, to increase the overall efficiency of the dose-escalation procedure while maintaining the safety of subjects (3–6).

YINGHUI ZHOU Medical and Pharmaceutical Statistics Research Unit The University of Reading Reading, Berkshire, United Kingdom

Phase Istudies have unique characteristics that distinguish them from other phases of clinical research. This is the phase when new experimental drugs are given to human subjects at the first time: a more explicit name for such trials is ‘‘first-into-man’’ (FIM) studies. Although intensive toxicologic studies have been carried out at preclinical trials, the primary concern for FIM studies is always the safety of the participating subjects (1). Safety can be assessed by the incidence of doselimiting events (DLE), such as moderate or severe adverse events, clinically significant changes in cardiac function, exceeding the maximum exposure limits of pharmacokinetic profiles such as the area under the curve (AUC) or the maximum concentration (Cmax ), and changes in pharmacodynamic parameters such as blood flow or heart rate. The majority of FIM studies are conducted on healthy volunteers for whom dose-limiting events should generally be avoided. Only a small proportion of FIM studies is conducted on late-stage patients in cancer trials. For these patients, standard therapies have failed, and low, safe doses will not achieve therapeutic effects, while high, unsafe doses will cause toxicity. A small risk of a DLE is therefore permitted to gain some therapeutic effect. The primary objective of FIM studies is to find an optimal dose, both safe and potentially efficacious, for later phases of clinical research. This involves a dose-escalation scheme: a fixed range of discrete doses, d1 < . . . < dk for some integer k, is predefined by the investigators to be administered in turn to subjects to assess the safety and tolerability of the compound. For healthy volunteer studies, the optimal dose is the maximum safe dose leading to an acceptable concentration of drug in plasma or to an adequate description of the biological effects.

1 A BAYESIAN DECISION-THEORETIC DESIGN In healthy volunteer studies, several PK/PD measurements are normally monitored and recorded. The methodology described here,

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS Table 1. An example of a crossover design Cohort

Dosing intervals 1

1 2 3 . ..

2

X

3 X

X

5

X

(1)

The logarithm of the j-th active dose received by the i-th subject is denoted by ij . The term si is a random effect relating to the i-th subject. The si and εij are modeled as mutually independent, normally distributed random variables with mean zero and variances τ 2 and σ 2 , respectively. The correlation, ρ, between two responses on the same subject can is equal to τ 2 /(σ 2 + τ 2 ). Placebo administrations are ignored in this model because there will be no drug detected in plasma.

6

7

X X

however, focuses on a single pharmacokinetic variable derived from the curve relating the concentration of the drug in plasma to the time since administration (7). Commonly used summaries such as area under the curve (AUC) or peak drug concentration (Cmax ) are often modeled by the normal distribution after a logarithmic transformation (8). The mean value of y, denoting either log(AUC) or log(Cmax ), is modeled as linear in log dose: E(y) = θ 1 + θ 2 log(dose), where θ 1 and θ 2 are unknown population parameters. A maximum exposure level, L, is defined before the start of the study based on the toxicity profile for the compound observed in the most sensitive animal species. Any concentration of the drug leading to an AUC or Cmax in excess of this level will be unacceptable. As each subject receives more than one dose, the multiple responses of each subject are correlated. Furthermore, different subjects vary considerably in terms of pharmacokinetics. Therefore, both random subject effects and random errors are included in the model. A log–log mixed effects model (3–6) is fitted to yij , the response following the j-th dose to the i-th subject: yij = θ1 + θ2 lij + si + εij

4

X X

X

Bayesian decision theory supplies a general framework for making decisions under uncertainty, which can be applied in many scientific and business fields (9–11). Let θ = (θ 1 , θ 2 )
be the vector of the unknown parameters. Some information or expert opinion about θ , from either preclinical data or experience of similar compounds, will be available before the dose-escalation study begins. This information can be formulated as a probability density function for θ , denoted by h0 (θ ) and known as the ‘‘prior density.’’ Let x denote the data collected in the doseescalation study, and f(x | θ ) be its likelihood function of the data. Then, the posterior density function of θ , h(θ | x), can be derived using Bayes’ theorem: h (θ |x) =


h0 (θ ) f (x|θ ) . h0 (ϕ) f (x|ϕ) dϕ

This posterior represents the opinion about θ formed by combining the prior opinion with the data. Potentially, one could treat θ 1 , θ , τ 2 , and σ 2 all as unknown parameters. However, the more parameters that are modeled, the less accurate the resulting estimates, especially as sample sizes are small in phase I trials. One way to solve the problem is to give some parameters fixed values as if they are known. In particular, the within-subject correlation, ρ, might be set to a value such as 0.6. Hence, there will be three unknown parameters in the model (1) instead of four parameters, namely θ 1 , θ 2 , and σ 2 . Fixing ρ is convenient, and the effect of doing so has been studied by Whitehead et al. (5) by taking the alternative strategy of specifying discrete priors on ρ. In the resulting analyses, ρ is underestimated by the Bayesian procedure, and

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS

the estimate of σ 2 is overestimated. Consequently, it was concluded that fixing ρ was a reasonable policy for practical use. Conjugate priors have been proposed for θ 1 , θ 2 , and ν (3–6), where ν is the withinsubject precision ν = σ −2 . In particular, the conditional distribution of θ given ν, and the marginal distribution of ν can be taken to be   θ |ν ∼ N µ0 , (νQ0 )−1 ; ν ∼ Ga (α0 , β0 ) , (2) where N denotes a normal distribution, Ga a gamma distribution, and the values of µ0 , Q0 , α 0 , and β 0 are chosen to represent prior knowledge. Being a conjugate prior, the posterior distribution shares the same form. Suppose that n subjects have been treated. The i-th subject has received pi periods of treatment, i = 1, . . . , n, and so a total of p1 + · + pn = p observations are available. Let the p-dimensional y denote the vector of responses with elements yij ordered by subject and by period within subject. The (p × 2) design matrix X is made up of rows of the form (1, ij ), ordered in the same way as y. The (p × n) matrix U, the design matrix of the random subject effect, is defined as having a 1 in the i-th column of rows p1 + · + pi1 + 1, . . . , p1 + . . . + pi for i = 1, . . . , n, and zeros elsewhere. The identity matrix is denoted by I. The (p × p) matrix P is defined as P = [I + ρ/(1 – ρ)UU
]−1 . Posterior distributions for θ and ν are   (3) θ |v ∼ N µ, (vQ)−1 ; v ∼ Ga (α, β) , where α = α 0 + p/2, β = β0 + (y
Py + µ0Q0µ0−µ
Qµ)/2, µ = (Q0 + X
PX)−1 (Q0
µ0 + X
Py), and Q = (Q0 + X
PX). Priors reflect not only one’s opinions, but also how strongly they are held. Here, a small value for α 0 represents a weak opinion. Consequently, dose escalation may be quick at the beginning as a few safe observations will soon overcome any prior reservations. A bigger value for α 0 represents a strong prior, and the resulting dose escalation will be conservative as prior concerns will only be removed by a clear demonstration of safety. A safety constraint can be used to help control overdosing (3–6). This requires that no dose be given if the predicted response at

3

this dose is likely to exceeds the safety limit L. Mathematically this can be expressed as   (4) P yij > log L|dij , y ≥ π0 , where π 0 , the tolerance level, can be set at a low value such as 0.05 or 0.20. The dose at which the above probability is equal to π 0 is called the maximum safe dose for the i-th subject following the j-th observation. The maximum safe dose is subject related, and posterior estimates may differ among subjects who have already been observed, being lower for a subject who previously had absorbed more drug than average and higher if the absorption was less. After each treatment period of the dose-escalation procedure, the posterior predictive probability that a future response will lie above the safety limit is updated. The decision of which dose to administer to each subject in each dosing period is made using a predefined criterion. This criterion can be based on safety; for example, one could use the maximum safe dose as the recommended dose. It can also be based on accuracy of estimates of unknown parameters; for example, the optimal choice of doses is that which will maximize the determinant of the posterior variance-covariance matrix of the joint posterior distribution or minimize the posterior variance of some key parameter. 2 AN EXAMPLE OF DOSE ESCALATION IN HEALTHY VOLUNTEER STUDIES An example, described in detail by Whitehead et al. (5), in which the principal pharmacokinetic measure was Cmax will be outlined briefly here. The safety cutoff for this response was taken to be yL = log(200). Seven doses were used according to the schedule: 2.5, 5, 10, 25, 50, 100, 150 µg. The actual trial was conducted according to a conventional design, and the dosing structure and the resulting data are listed in Table 2. From a SAS PROC MIXED analysis, maximum likelihood estimates of the parameters in model (1) are θ1 = 1.167, θ2 = 0.822, σ 2 = 0.053, and τ2 = 0.073.

(5)

4

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS

Note that it follows that the within-subject correlation is estimated as 0.579. As an illustration, the Bayesian method described above will be applied retrospectively to this situation. Conjugate priors must first be expressed. It is assumed prior expert opinion suggested that the Cmax values would be 5.32 and 319.2 pg/mL at the doses 2.5 and 150 µg respectively. This prior indicates that the highest dose 150 µg is not a safe dose because the predicted Cmax exceeds the safety limit of 200 pg/mL. The value for ρ is set as 0.6. This forms the bivariate normal distributions for θ ,  N

θ |v ∼    0.756 0.940 − 0.109 , . 1 −0.109 0.04

The value for α 0 is set as 1, as suggested in Whitehead et al. (5). The value for β 0 can be found via the safety constraint: P(y01 > logL | d01 = 2.5) = 0.05. This implies that dose 2.5 µg will be the maximum safe dose for all new subjects at the first dosing period in the first cohort. Therefore, ν ∼ Ga(1, 0.309). To illustrate Bayesian dose escalation, data are simulated based on the parameters found from the mixed model analysis given by Whitehead et al. (5). Thus, Cmax values are generated from three cohorts of eight healthy volunteers, each treated in four consecutive periods and receiving three active doses and one randomly placed placebo. A simulated dose escalation with doses chosen according to the maximum safe dose criterion is shown in Table 3. The first six subjects received the lowest dose, 2.5 µg. All subjects at the next dosing period received dose 10 µg, in which 5 µg dose was skipped for subjects 1 to 4. Subjects 7 and 8, who were on placebo in the first dosing period, skipped two doses, 2.5 and 5 µg, to receive 10 µg in the second dosing period. If this dosing proposal was presented to a safety committee in a real trial, the committee members might wish to alter this dose recommendation. The Bayesian approach provides a scientific recommendation for dose escalations. However, the final decision on which doses are given should come from a safety committee. The procedure would be able to make use of results

from any dose administered. In Table 3, it is shown that the maximum safe dose for a new subject at the beginning of the second cohort is 25 µg. All subjects in the second cohort received 50 µg at least twice. Subjects 17 to 22 in the first dosing period of the final cohort received 50 µg. However, they all had a high value of Cmax . The Bayesian approach then recommended a lower dose, 25 µg for subjects 17 to 20 (subjects 21 and 22 were on placebo). This shows that the Bayesian approach can react to different situations quickly: when escalation looks safe, dose levels can be skipped; when escalation appears to be unsafe, lower doses are recommended. Two high doses that were administered in the real trial, 100 and 150 µg, were never used in this illustrative run. The posterior distributions for θ at the end of the third cohort are

 N

θ |v ∼    1.376 0.024 − 0.006 , , 0.790 −0.006 0.002

and ν ∼ Ga(37, 2.625). Figure 1 shows the doses administered to each subject and the corresponding responses. Table 4 gives the maximum likelihood estimates from the real data in Table 2 that were used as the true values in the simulation, together with the maximum likelihood estimates from the simulated data in Table 3. Results show that σ 2 and τ 2 were underestimated from the simulated data, with there being no evidence of between-subject variation. Consequently, the estimated correlation from the simulated data is zero, in contrast to the true value of 0.579 used in the simulation. This is a consequence of the small dataset, and illustrates the value of fixing ρ during the escalation process. Different prior settings will result in different dose escalations. For example, if the value for α 0 is changed from 1.0 to 0.1, then the dose-escalation will be more rapid. Table 5 summarizes the recommended doses and simulated responses from another simulation run where α 0 = 0.1. In the second dosing period of the first cohort, subjects 1, 2, and 4 skipped two doses, 5 and 10 µg. Subjects 3, 7, and 8 skipped three doses in that cohort.

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS

5

Table 2. Real data from a healthy volunteer trial in Whitehead et al. (2006) Period Subject

1 Dose

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

2.5 2.5 2.5 10

2 Cmax

Cmax

5 5

14.5

24.1 2.5 2.5 5 5

2.5 2.5 2.5 10 10 10 10 10

3

Dose

29.0 15.0 21.8 23.0

50

54.7

50 50 50 50 50

161.5 101.4 194.1 61.6 81.7

10.4 13.5

10 25

14.6 34.1

25 25 25 10 2.5 100 50

34.9 70.6 40.7 16.6 125.6 79.4

100 100

131.3 232.5

100 50

84.2 113.8

4

Dose

Cmax

10

11.0

5 25 5 5 10

10.0 33.3 10.2 20.6 28.5

5 25 50 25

19.4 71.6 37.1

50 25 5

73.4 53.0 14.4

100

108.1

100 150 100

104.5 217.6 178.9

Dose

Cmax

10 10 50 10

20.4 19.7 74.6 29.7

10 10 50

14.6 27.0 47.7

50 50 50

59.2 76.1 166.6

50 10

87.1 17.8

Source: Whitehead et al. Stat Med. 2006; 25: 433–445.

The starting dose for all subjects in the first dosing period of the second and third cohorts was 50 µg (25 µg and 50 µg were the doses in Table 3). All subjects but one occasion repeatedly received 50 µg during the second cohort.Inthe third cohort, the dose of 100 µg was used quite frequently. The highest dose, 150 µg, was never used. This example shows that different prior settings will affect dose-escalation procedures. Multiple simulations should therefore be conducted to gain a better understanding of the properties of a design. Different scenarios should be tried to ensure that the procedure has good properties, whether the drug is safe, unsafe, or only safe for some lower doses. 3

DISCUSSION

Bayesian methods offer advantages over conventional designs. Unlike conventional

designs, more doses within the predefined dose range, or even outside of the predefined dose range, can be explored without necessarily needing extra dosing time as dose level skipping can be permitted. From the simulation runs in Tables 3 and 5, more doses, such as 40, 60, and 80 µg, could perhaps be included in the dose scheme. Simulations have shown that skipping dose levels does not affect either safety or accuracy; on the contrary, it improves safety or accuracy as a wider range of doses is used (12). Providing a greater choice of doses, while allowing dose skipping, leads to procedures that are more likely to find the target dose and to learn about the dose-response relationship curve efficiently. Ethical concerns can be expressed through cautious priors or safety constraints. Dose escalation will be dominated by prior opinion at the beginning, but it will soon

6

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS Table 3. A simulated dose-escalation Period Subject

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

1

2

3

4

Dose

Cmax

Dose

Cmax

Dose

Cmax

2.5 2.5 2.5 2.5 2.5 2.5

9.9 12.4 10.4 11.1 6.5 7.7

10 10 10 10

19.7 27.0 35.6 35.7

25 25

64.1 76.4

25 25 25 25 25 25

50 50 50 50 50 50

10 10 50 50 50 50

38.9 41.4 42.5 53.4 50.8 48.7

50 50 25 25 25 25

138.2 194.4 173.2 139.9 181.6 117.9

50 50

be influenced more by the accumulating real data. Before a Bayesian design is implemented in a specific phase I trial, intensive simulations should be used to evaluate different prior settings, safety constraints, and dosing criteria under a range of scenarios. Investigators can then choose a satisfactory design based on the simulation properties that they

21.3 24.5 84.4 71.0 87.1 67.2

84.4 82.2 73.6 57.9 76.6 43.1

93.6 97.5

25 25 50 25 50 50

50 50 50 50 25 25

25 50 50 50

33.7 62.9 93.2 52.8 53.7 58.6

60.4 66.5 89.4 75.5 46.5 69.2

47.2 87.7 75.5 79.2

Dose

Cmax

25 25 50 50 50 50

44.8 27.1 58.8 57.9 58.0 58.1

50 50 50 50 50 50

92.7 91.9 79.6 110.6 62.6 76.4

25 50 50 50 50 50

37.4 80.2 79.2 63.9 82.5 65.9

are interested in. For instance, they may look for a design that gives the smallest number of toxicities or the most accurate estimates. Until recently, such prospective assessment of design options was not possible, but now that the necessary computing tools are available, it would appear inexcusable not to explore any proposed design before it is implemented.

Table 4. Maximum likelihood estimates (MLE) and Bayesian modal estimates of simulated data in Table 3 (with standard errors or standard deviations)

Truth for simulations (MLE from real data) Final MLE (exclude the prior information) Bayesian prior modal estimates Bayesian posterior modal estimates

θ1

θ2

σ2

τ2

ρ

d∗f

1.167 (0.158) 1.568 (0.143) 0.759 (0.967) 1.376 (0.156)

0.822 (0.046) 0.741 (0.041) 1.00 (0.192) 0.790 (0.042)

0.053

0.073

0.579

73.42

0.090

0.000

0.000

69.45

0.309

0.463

0.6

2.5

0.071

0.106

0.6

58.18

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS

7

Dose 157 135 113 91 69 47 25 3 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Subject

Cmax ∈ (0, 0.2L);

Key: Cmax 210

Cmax ∈ (0.2L, 0.5L);

Cmax ∈ (0, 0.5L, L).

safty limit

180 150 120 50% safty limit 90 60 20% safty limit

30 3 1

2

3

4

5

6

7

8

9 10 11 12 13 14 15 16 17 18 19 20 21 22 23

24

Subject

Figure 1. An illustration of a simulated dose escalation (using data from Table 3).

Although the methodology described here is presented only for a single pharmacokinetic outcome, the principles are easily generalized for multiple endpoints. Optimal doses can be found according to the safety limits for each of the endpoints, and then the lowest of these doses can be recommended. The Bayesian decision-theoretic approach has also been extended for application to an attention deficit disorder study

(13), where a pharmacodynamic response (heart rate change from baseline), a pharmacokinetic response (AUC), and a binary response (occurrence of any dose-limiting events) are modeled. A one-off simulation run indicates that the Bayesian approach controls unwanted events, dose-limiting events, and AUC levels exceeding the safety limit, while achieving more heart rate changes within the therapeutic range.

8

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS Table 5. A simulated dose-escalation (the value for α 0 is 0.1) Period Subject

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

1

2

Dose

Cmax

2.5 2.5 2.5 2.5 2.5 2.5

8.2 9.9 6.2 8.4 8.0 6.4

50 50 50 50 50 50

50 50 50 50 50 50

88.9 104.4 74.9 98.3 86.2 89.4

58.5 48.3 70.5 45.3 55.3 49.1

3

4

Dose

Cmax

Dose

Cmax

25 25 50 25

53.4 40.4 118.5 61.1

50 50

66.1 78.9

25 25 50 50 50 50

50 50 100 100 50 100

100 50

50 50 50 50 50 50

65.1 37.7 64.4 49.5 64.7 84.0

50 50 50 50 50 100

109.9 115.0 172.8 140.7 73.8 89.7

100 100 100 50

157.6 72.8

Bayesian methodology only provides scientific dose recommendations. These should be treated as additional information for guidance of the safety committee of a trial rather than dictating final doses to be administered. Statisticians and clinicians need to be familiar with this methodology. Once the trial starts, data need to be available quickly and presented unblinded with dose recommendations to the safety committee. They can then make a final decision, based on the formal recommendations, together with all of the additional safety, laboratory, and other data available.

4.

REFERENCES

6.

3.

5.

1. U.S. Food and Drug Administration. 1997. Guidance for Industry: General considerations for the clinical evaluation of drugs. 2. K. Gough, M. Hutchison, O. Keene, B. Byrom, S. Ellis S, et al., Assessment of dose proportionality—Report from the

7.

92.7 74.1 60.9 98.2 46.6 46.2

48.4 56.7 46.6 68.5 60.5 98.2

142.0 115.6 102.5 63.9

Dose

Cmax

50 50 50 50 50 50

76.7 84.2 101.7 83.0 78.2 72.0

50 50 100 50 50 50

57.9 72.6 83.6 60.8 59.2 93.4

50 100 100 100 100 100

92.7 119.8 106.5 101.5 156.5 145.3

Statisticians in the Pharmaceutical Industry/Pharmacokinetic UK Joint Working Party. Drug Inf J. 1995; 29: 1039–1048. S. Patterson, S. Francis, M. Ireson, D. Webber, and J. Whitehead, A novel Bayesian decision procedure for early-phase dose finding studies. J Biopharm Stat. 1999; 9: 583–597. J. Whitehead, Y. Zhou, S. Patterson, D. Webber, and S. Francis, Easy-to-implement Bayesian methods for dose-escalation studies in healthy volunteers. Biostatistics. 2001; 2: 47–61. J. Whitehead, Y. Zhou, A. Mander, S. Ritchie, A. Sabin, and A. Wright, An evaluation of Bayesian designs for dose-escalation studies in healthy volunteers. Stat Med. 2006; 25: 433–445. Y. Zhou, Dose-escalation methods for phase I healthy volunteer studies. In: S. Chevret (ed.), Statistical Methods for DoseFinding Experiments. Chichester, UK: Wiley, 2006, pp. 189–204. S. C. Chow and J. P. Liu, Design and Analysis of Bioavailability and Bioequivalence Studies. Amsterdam: Dekker, 1999.

BAYESIAN DOSE-FINDING DESIGNS IN HEALTHY VOLUNTEERS 8. W. J. Westlake, Bioavailability and bioequivalence of pharmaceutical formulations. In: K. E. Peace (ed.), Biopharmaceutical Statistics for Drug Development. New York: Dekker, 1988, pp. 329–352. 9. D. V. Lindley, Making Decisions. London: Wiley, 1971. 10. J. Q. Smith, Decision Analysis: A Bayesian Approach. London: Chapman & Hall, 1988. 11. J. O. Berger, Statistical Decision Theory and Bayesian Analysis, New York: Springer, 1985. 12. Y. Zhou and M. Lucini, Gaining acceptability for the Bayesian decision-theoretic approach in dose escalation studies. Pharm Stat. 2005; 4: 161–171. 13. L. Hampson, Bayesian Methods to Escalate Doses in a Phase I Clinical Trial. M.Sc. dissertation. School of Applied Statistics, University of Reading, Reading, United Kingdom, 2005.

CROSS-REFERENCES Phase I trials Pharmacodynamic study Pharmacokinetic study Crossover design Analysis of variance (ANOVA) Placebo Bayesian approach

9

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

relatively few will receive a preventive benefit from the therapy. In this setting, the use of B/RAs provides an additional mechanism to ensure that all participants comprehend the full extent of potential benefits and risks, and that they make a well-informed decision about the interchange of benefits and risks they are willing to accept by participating in the trial. The use of B/RA in prevention trials also provides a method to evaluate the global effect of the therapy as a safeguard against subjecting trial participants to an unforeseen harmful net effect of treatment. Once the results of the trial are known and the true levels of benefits and risks of the therapy have been established, the individualized B/RA employed in the trial can become the basis for the development of a B/RA methodology that could be used in the clinical setting to facilitate the decision-making process for individuals and their health care providers who may be considering the use of preventive therapy. The trial results can also be used to develop a population-based B/RA to identify changes in patient loads for the outcomes affected by the preventive therapy that would be anticipated as health care professionals incorporate the use of the preventive therapy into their clinical practice. This information could in turn be used for decision-making regarding the planning for and use of health care resources.

JOSEPH P. COSTANTINO University of Pittsburgh, Pittsburgh, PA, USA

Benefit/risk assessment (B/RA) is a mathematical procedure to estimate the probability of detrimental outcomes, beneficial outcomes and the net-effect anticipated from exposure to a given agent. B/RAs of health-related outcomes are used for public health planning, decision-making regarding health care financing and therapeutic decision-making in clinical practice (12,24). Information obtained from B/RAs based on findings from controlled clinical trials, particularly those with doublemasking of treatment, are most informative for health care planning and decision-making because such information is less likely to be biased than is information obtained from observational studies (20,23,26). Thus, B/RAs in prevention trials are an excellent source of information to use as the basis for the types of health care planning and decision-making mentioned above. However, in a prevention trial a B/RA is primarily performed as a supplement for planning, monitoring and analyzing the trial. It is designed to provide a global assessment of all potential beneficial and harmful effects that may occur as a result of a treatment that is being evaluated as a means to reduce the incidence of some particular disease or condition. The Women’s Health Initiative (WHI), the Breast Cancer Prevention Trial (BCPT) and the Study of Tamoxifen and Raloxifene (STAR) are examples of large-scale, multicenter prevention trials which included B/RA as part of the trial methodology (7,8,22,28,29). Compared with treatment trials, the need for the type of information provided by a B/RA may be greater in prevention trials. This situation exists because prevention trials usually involve healthy persons among whom only a small proportion may develop the disease of primary interest during the course of the trial (5–713,22,29,30). As such, all participants are subjected to the risks of therapy during the course of the trial, but

1 TYPES OF B/RAs PERFORMED IN PREVENTION TRIALS In a prevention trial a B/RA can take one of three forms which can be classified according to the nature of the population that constitutes the basis for the assessment. These include assessments based on the general population, those based on the trial cohort and those based on an individual trial participant. Each of these forms of B/RA is

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

performed for specific purposes, namely to support various aspects of the conduct of the trial. A B/RA based on the general population is often performed pre-trial as part of the justification for initiating the trial. The purpose of this form of assessment is to demonstrate the potential net health benefit to society that could be obtained if the therapy being evaluated in the trial actually exhibits the efficacy that is anticipated. This type of assessment is the most generalized form. It is usually accomplished by estimating effects on a national basis assuming the therapy is administered to all susceptible individuals or to a subset of high-risk individuals and demonstrating that there is a significant net benefit when comparing the number of cases prevented with the estimates for the number of additional cases of detrimental outcomes that may be caused as a side-effect of the therapy. A B/RA based on the trial cohort is performed during the course of trial implementation as part of the safety monitoring effort. It can be accomplished in a regimented fashion as part of the formal plan for the interim monitoring of the trial or as an informal tool used by the data monitoring committee to assess the overall safety of the therapy being evaluated. This type of assessment is not usually necessary during a trial if the anticipated effects from the therapy involve only a few outcomes or if the anticipated beneficial effects substantially outweigh the anticipated detrimental effects. However, in complex situations where the anticipated outcomes affected by the therapy involve multiple diseases or conditions and/or the magnitude of the anticipated net benefit may not be large, a B/RA based on the trial cohort can be a very useful supplement for trial surveillance as a method of monitoring the global effect of all beneficial and detrimental outcomes combined. A notable difference between a B/RA based on the general population and one based on the study cohort is in the nature of the measures that are provided by these two forms of assessment. A risk assessment based on a general population provides a measure of the theoretical net effect of the therapy from estimates of

anticipated beneficial and detrimental outcomes. In contrast, a risk assessment based on the trial cohort determines the observed net effect of therapy based on outcomes actually experienced by the cohort during the course of the trial. A B/RA based on an individual trial participant is similar to that of the populationbased assessment in that it is also a theoretical estimate. In this case the assessment is not made for the general population, but instead for a specific subpopulation of persons who have the same risk factor profile (age, sex, race, medical history, family history, etc.) for the anticipated beneficial and detrimental outcomes as that of a particular individual participating in the trial. Information from this type of assessment is used to facilitate the communication to each potential trial participant of the nature of the benefits and risks that are anticipated for them as a result of taking therapy during trial participation. This type of individualized B/RA is used in prevention trials when the nature of anticipated effects is complex and benefit/risk communication is a more difficult task due to the interplay of multiple beneficial and detrimental outcomes. When it is used in this manner, it becomes an integral part of the process to obtain informed consent for each individual’s participation in the trial. 2 ALTERNATIVE STRUCTURES OF THE BENEFIT/RISK ALGORITHM USED IN PREVENTION TRIALS The core components of a B/RA are the measures of the treatment effect for each of the health outcomes that may be affected by the therapy being assessed. In this instance the treatment effect is defined as the difference between the probability that the outcome will occur among individuals who do not receive the therapy being evaluated (p0 ) and the probability that the outcome will occur among those who do receive the therapy (p1 ). For outcomes beneficially affected by therapy, the treatment effect (p0 − p1 ) will have a positive sign, representing cases prevented by therapy. For outcomes detrimentally affected by therapy, the treatment effect will have a

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

negative sign, representing cases caused by therapy. In its simplest structure, the benefit/risk analysis is summarized by an index of net effect (
) as the summation of treatment effects for all outcomes affected. If there are I number of outcomes affected by therapy, then the basic algorithm for the B/RA is defined as:


1 =

I


(p0,i − p1,i ).

(1)

i=1

When the sign of the index of net effect is positive, the therapy exhibits an overall beneficial health effect. When the sign is negative, the therapy has an overall detrimental effect. When dealing with a B/RA based on the trial cohort, the probabilities of (1) are obtained directly from the observations in the trial. When dealing with assessments based on the general population or the individual trial participant, the probabilities utilized as anticipated values among those who do not receive therapy (p0 ) are usually taken from some type of national database or from prospective studies of large populations that included measurements of the outcomes of interest. The probabilities used in these latter types of assessments as anticipated values among those who receive therapy are determined by multiplying the anticipated probability among those not treated by the relative risk (untreated to treated) anticipated as the treatment effect. For example, if we anticipate that treatment will reduce the incidence of a particular outcome by 35% then the anticipated relative risk would be 0.65 and the value used for p1 would be 0.65 p0 . If we anticipate that treatment will increase the incidence of an outcome by 30%, then the anticipated relative risk would be 1.30 and the value used for p1 would be 1.30 p0 . Estimates of the anticipated treatment effects for each outcome are taken from the literature dealing with pharmacokinetics, animal studies and studies in humans undertaken as preliminary investigations of the therapy as an agent to prevent the disease, or from human studies in which the therapy was being used as an agent for the treatment of disease.

3

In the prevention trial setting it is often advantageous to utilize structures of the benefit/risk algorithm other than that defined in (1). Since a B/RA based on the trial cohort is meant to be performed as part of the effort to monitor safety during the trial, an alternative structure of the benefit/risk algorithm can be used to facilitate this effort. This structure incorporates a standardization of the differences between the probabilities among those receiving and not receiving the therapy being evaluated. In this situation the index of net effect is defined as: I
(p0,i − p1,i )


2 =

i=1



I
s.e. (p0,i − p1,i )

.

(2)

i=1

In this structure, the index of net-effect (
2 ) becomes a standardized value with an N(0,1) distribution. As such, the standardized values are Z-scores. Critical values of this index of net effect in the form of Z and −Z can then be used as cut-points for global monitoring indicating that there is a significant net effect that is beneficial or detrimental, respectively. In addition to that for the standardized score, there are other structures of the algorithm used in the prevention trial setting. Instead of expressing the differences between those treated and not treated in terms of the probabilities of the anticipated outcomes, an alternative structure of the algorithm is that based on differences between treatment groups in terms of the number of cases of the outcomes. The structure of the algorithm based on the difference in terms of the number of cases is defined as:
3 =

I
(n0,i − n1,i ),

(3)

i=1

where n0 is the number of cases occurring among those who do not receive the therapy being evaluated and n1 is the number of cases among those who do receive the therapy. This structure of the algorithm is that which is utilized to perform B/RAs based on the general population. This type of assessment is meant

4

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

to justify the need for a trial by demonstrating the potential health benefit to society. The net effect to society is more effectively communicated to a greater proportion of individuals when it is expressed as the number of cases prevented from (3) than when it is expressed as the probability from (1). This facilitation of risk communication is also the reason that (3) is preferred over (1) for B/RAs based on individual trial participants where the specific goal of the assessment is to enhance the individual’s comprehension of benefits and risks that may be experienced as a result of trial participation. For a population-based assessment, the numbers of cases in (3) are determined by multiplying the anticipated probabilities p0,i and p1,i by the number of persons in the general population, frequently that of the total US, to obtain an estimate of the number of cases that may be prevented or caused by treatment for each outcome on an annual basis. For an individual participant-based assessment, the numbers of cases in (3) are determined by multiplying the anticipated probabilities by a fixed sample size (N) of theoretical individuals who all have a risk factor profile similar to that of the individual being assessed. A fixed period of follow-up time (t) is assumed to obtain the number of cases prevented or caused by treatment in t years among N individuals. In scenarios where the length of follow-up is long and/or the population is of older age, the estimation of n0,i and n1,i should incorporate the competing risk of mortality that would be anticipated. If d is the probability of dying and RR is the relative risk anticipated for the outcome of interest, the adjusted expected number of cases among those not treated can be calculated as:   p0,i [1 − exp{−t(p0,i + di )}], n0,i = N (p0,i + di ) (4) and the adjusted expected number of cases among those treated can be calculated as:   RRp0,i n1,i = N (RRp0,i + di ) ×[1 − exp{−t(RRp0,i + di )}].

(5)

In most prevention trials the outcomes that are potentially affected by the therapy

being evaluated encompass a wide range of severity. A simple adding together of the risks of beneficial and detrimental effects without including a consideration of the relative severity of the outcomes may not be appropriate or desirable. For example, suppose a therapy is anticipated to prevent breast cancer and hip fractures, but may cause an increase in uterine cancer and cataracts. Is it appropriate to equate one case of breast cancer prevented to one case of cataracts caused or equate one case of hip fracture prevented to one case of uterine cancer caused? In situations where it is important to include a consideration of the relative severity of the outcomes affected by the therapy, the equations described above for determining the index of net effect can be modified to incorporate a weighting of the outcomes. If wi is used to represent the weight for each of the I outcomes, then the modification to (3) to incorporate weighting of the outcomes is:
4 =

I


wi (n0,i − n1,i ).

(6)

i=1

Equations (1) and (2) can be modified in a similar fashion by including wi as a multiplier of the quantity of difference in the probabilities. 3 METHODOLOGICAL AND PRACTICAL ISSUES WITH B/RA IN PREVENTION TRIALS There are several issues to be faced when performing a B/RA in a prevention trial. These issues concern the variability of the index of net effect, weighting the outcomes by severity, estimating the index of net effect for individuals with specific profiles of risk factors and communicating the findings of a B/RA to individual participants. Some discussion of each of these issues is presented below. The estimates of p0,i and p1,i used in a B/RA have a variability associated with them in terms of the strength of evidence supporting the treatment effect and in terms of the precision of the treatment effect. If this variability is substantial, then it may be necessary to incorporate consideration of the variability into the B/RA. Freedman et al. (8,25) have described a Bayesian approach

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

to incorporating a measure of variability into the index of net effect when it is measured in the form of weighted, standardized probabilities. They assume a skeptical prior distribution based on the strength of the preliminary evidence used as the anticipated treatment effect for each outcome potentially affected by therapy. Gail et al. (10) have described a method to incorporate a measure of variability into the estimate of the index of net effect measured in the form of a weighted number of cases. Their method involves bootstrapping, based on the 95% confidence intervals of the anticipated relative risk associated with treatment for each outcome, to determine the probability that the net number of cases is greater than zero. The values used for weighting the differences between those treated and not treated can be based on a utility function related to the severity of the outcome, preferences in terms of levels of risk acceptability or other considerations. However, the best choice of a utility function is not always obvious. A measure of mortality such as the case-fatality ratio is one possible utility. If this type of weighting is used, then the choice of the one-year, five-year or ten-year case-fatality ratios would be an issue because the relative weighting of the outcomes could likely be very different depending on which time period for case-fatality is used. Also, weights based on case-fatality would eliminate the consideration of any nonfatal outcome, which would not be preferable if there were several nonfatal outcomes of interest or if a nonfatal outcome has a significant impact on morbidity. Issues also arise with the use of rankings based on the impact on quality of life or preferences regarding the acceptability of risk (1,11). The issues with these utilities arise because the rankings are often subjective in nature, based on the opinions of a relatively small panel of individuals, and it is possible that the rankings of outcomes could differ substantially depending on the population from whom the opinions are ascertained (2,15,16). In light of these issues, attempting to identify a basis for weighting a B/RA is a practical problem that can be difficult to resolve. The preferred choice for any particular trial could differ from one group of individuals to another. As such, if a B/RA is planned as part of

5

trial monitoring, it is essential that the data monitoring committee reviews and reaches a consensus regarding the proposed weighting before it initiates review of the outcome data. To accomplish the individualization desired for B/RAs based on individual trial participants, it is necessary to provide estimates of effect specific to the individual’s full spectrum of risk factors for each of the outcomes expected to be affected by the therapy of interest. A problem likely to be faced when performing individualized assessments is the unavailability of probability estimates specific to the individual’s full set of risk factors. For outcomes with several relevant risk factors to be considered or for outcomes that have not been studied in diverse populations, estimates of the outcome probabilities for a specific category of risk factor profiles may not exist. In some cases, multivariate regression models are available that can be used to predict probabilities of outcomes for specific risk factor profiles from data based on the general population. Examples of such models include those for heart disease, stroke and for breast cancer (4,9,1417–19). However, the models currently available are primarily limited to those for the more common diseases and are not generally applicable to all race and sex populations. Also, relatively few of these models have been well validated. Thus, in practice it is often necessary to use estimates of outcome probabilities for individualized B/RAs that are taken from populations that are more representative of the general population than of the population specific to the risk factor profile of the individual being assessed. When this is the case, the limitations of the methodology need to be recognized and used in this light. Nonetheless, a B/RA that has been individualized to the extent possible is more informative to a trial participant than one based on the general population. Additional discussions of the limitations of individualized B/RAs can be found in presentations concerning individualized B/RAs for the use of tamoxifen to reduce breast cancer risk (3,4,27). Communicating the results of a B/RA to an individual is a skilled task. An effort must be made to provide information in a manner that facilitates the individual’s comprehension (21). Tools are needed to facilitate this

6

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

Table 1. Example of Data Presentation Tool for Communicating the Benefits and Risks of Tamoxifen Therapy The information below provides the number of certain events that would be expected during the next five years among 10 000 untreated women of your age (ageX ), race (raceY ) and five-year breast cancer risk (riskZ ). To help you understand the potential benefits and risks of treatment, these numbers can be compared with the numbers of expected cases that would be prevented or caused by five years of tamoxifen use Severity of event Life-threatening events

Other severe events

Other events

Expected number of cases among 10 000 untreated women

Expected effect among 10 000 women if they all take tamoxifen for five years

Invasive breast cancer Hip fracture

N0,1 cases expected N0,2 cases expected

Potential benefits N1,1 of these cases may be prevented N1,2 of these cases may be prevented

Endometrial cancer Stroke Pulmonary embolism

N0,3 cases expected N0,4 cases expected N0,5 cases expected

Potential risks N1,3 more cases may be caused N1,4 more cases may be caused N1,5 more cases may be caused

In situ breast cancer

N0,6 cases expected

Potential benefit N1,6 of these cases may be prevented

Deep vein thrombosis

N0,7 cases expected

Potential risk N2,7 more cases may be caused

Potential benefits:

Tamoxifen use may reduce the risk of a certain type of wrist fracture called Colles’ fracture by about 39%, and also reduce the risk from fractures of the spine by about 26%. Tamoxifen use may increase the occurrence of cataracts by about 14%.

Type of event

Potential risk:

effort. These tools must be developed before the initiation of the trial and included as part of the protocol approved by the Institutional Review Board. Relatively little work has been done in the area of developing tools for communicating the benefits and risks of participation in a prevention trial. However, some tools have been developed that serve as examples for future development. Tools to enhance the communication of B/RA information to women screened for participation were developed for use in the BCPT (7,22). Since the conclusion of this trial, the tools were refined for use in the STAR trial (28). Table 1 provides an example of the type of tool used in the STAR trial to inform potential participants regarding their individualized B/RA. This tool was developed based on the principles put forth by the participants of the National Cancer Institute’s workshop convened to develop information to assist in counseling women about the benefits and risks of tamoxifen when used to reduce

the risk of breast cancer. This workshop and the specific methodology used for the B/RA are described by Gail et al. (10). There were several key working premises that guided the development of the STAR trial tool displayed in Table 1. The premises were considerations of form and format to facilitate the participant’s comprehension of their individualized B/RA. These working premises were to: (1) avoid the use of probabilities and relative risk as these concepts are not readily understood by the nonstatistician; (2) provide information for each outcome anticipated to be affected by therapy; (3) group the information presented by severity of the outcomes; (4) provide detailed information for the outcomes with more severe consequences and provide an estimate of effects among those not treated so the individual can understand the context in which to place the expected treatment effects; and (5) limit the tool to one page of data presentation to reduce the amount of data overload perceived by the individual.

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS

The precise considerations involved in any prevention trial may differ; however, working premises of this nature designed to enhance comprehension should always be employed when developing tools to communicate B/RA information to potential trial participants. REFERENCES 1. Bennett, K. J. & Torrance, G. W. (1996). Measuring health state preferences and utilities: ratings scale, time trade-offs and standard gamble techniques, in Quality of Life and Pharmacoeconomics in Clinical Trials, 2nd Ed., B. Spilker, ed. Lippincott-Raven, Philadelphia, pp. 253–265. 2. Boyd, N. F., Sutherland, H. J., Heasman, K. Z., TritchlerD. L. & Cummings B. J. (1990). Whose utilities for decision analysis?, Medical Decision Making 1, 58–67. 3. Costantino, J. P. (1999). Evaluating women for breast cancer risk-reduction therapy, in ASCO Fall Education Book. American Society of Clinical Oncology, pp. 208–214. 4. Costantino, J. P., Gail, M. H., Pee, D., Anderson, S., Redmond, C. K. & Benichou, J. (1999). Validation studies for models to project the risk of invasive and total breast cancer incidence, Journal of the National Cancer Institute 91, 1541–1548. 5. Cummings, S. R., Echert, S., Krueger, K. A., Grady, D., Powles, T. J., Cauley, J. A., Norton, L., Nickelsen, T., Bjarnason, N. H., Morrow M., Lippman M. E., Black, D., Glusman, J. E. & Jordan, V. C. (1999). The effect of raloxifene on risk of breast cancer in postmenopausal women: results from the MORE randomized trial, Journal of the American Medical Association 281, 2189–2197. 6. Ettinger, B., Black, D. M., Mitlak B. H., Knickerbocker, R. K., Nickelsen, T., Genant, H. K., Christiansen, C., Delmas, P. D., Zanchetta, J. R., Stakkestad, J., Gluer, C. C., Krueger, K., Cohen, F. J., Eckert, S., Ensrud, K. E., Avioli, L. V., Lips, P. & Cummings, S. R. (1999). Reduction of vertebral fracture risk in postmenopausal women with osteoporosis treated with raloxifene: results from a 3-year randomized clinical trial, Journal of the American Medical Association 282, 637–645. 7. Fisher, B., Costantino, J. P., Wickerham, D. L., Redmond, C. K., Kavanah, M., Cronin, W. M., Vogel, V., Robidoux, A., Dimitrov, N., Atkins, J., Daly, M., Wieand, S., Tan-Chiu, E., Ford, L. & Wolmark, N. (1998). Tamoxifen

7

for prevention of breast cancer: report of the National Surgical Adjuvant Breast and Bowel Project P-1 study, Journal of the National Cancer Institute 90, 1371–1388. 8. Freedman, L., Anderson, G., Kipnis, V., Prentice, R., Wang, C. Y., Rousouw, J., Wittes, J. & DeMets, D. (1996). Approaches to monitoring the results of long-term disease prevention trials: examples from the Women’s Health Initiative, Controlled Clinical Trials 17, 509–525. 9. Gail, M. H., Brinton, L. A., Byar, D. P., Corle, D. K., Green, S. B., Schairer, C. & Mulvihill, J. J. (1989). Projecting individualized probabilities of developing breast cancer for white females who are being examined annually, Journal of the National Cancer Institute 81, 1879–1886. 10. Gail, M. H., Costantino, J. P., Bryant, J., Croyle, R., Freedman, L., Helzlsouer, K. & Vogel V. (1999). Weighing the risks and benefits of tamoxifen for preventing breast cancer, Journal of the National Cancer Institute 91, 1829–1846. 11. Guyatt, G., Feeny, D. & Patrick, D. (1993). Measuring health-related quality of life, Annuals of Internal Medicine 118, 622–629. 12. Haynes, R. B., Sackett, D. L., Gray, J. A. M., Cook, D. J. & Guyatt, G. H. (1996). Transferring evidence from research to practice: 1. The role of clinical care research evidence in clinical decisions, APC Journal Club 125, A14–A15. 13. Hulley, S., Grady, D., Bush, T., Furberg, C., Herrington, D., Riggs, B. & Vittinghoff, E. (1998). Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/Progestin Replacement Study (HER) Research Group, Journal of the American Medical Association 280, 605–613. 14. Liao, Y., McGee, D. L., Cooper, R. S. & Sutkowski, M. B. (1999). How generalizable are coronary risk prediction models? Comparison of Framingham and two other national cohorts, American Heart Journal 137, 837–845. 15. Llewellyn-Thomas, H. A. (1995). Patients’ health care decision making: a framework for descriptive and experimental investigations, Medical Decision Making 15, 101–106. 16. Llewellyn-Thomas, H. A., Naylor, C. D., Cohen, M. N., Baskiniski, A. S., Ferris, L. E. & Williams, J. E. (1992). Studying patients’ preferences in health care decision making,

8

17.

18.

19.

20.

21.

22.

BENEFIT/RISK ASSESSMENT IN PREVENTION TRIALS Canadian Medical Association Journal 147, 859–864. Lloyd-Jones, D. M., Larson, M. G., Beiser, A. & Levy, D. (1999). Lifetime risk of developing coronary heart disease, Lancet 353, 89–92. Manolio, T. A., Kronmal, R. A., Burke, G. L., O’Leary, D. H. & Price, T. R. (1996). Shortterm predictors of incidence stroke in older adults. The Cardiovascular Health Study, Stroke 27, 1479–1486. Menotti, A., Jacobs, D. R., Blackburn, H., Krombout, D., Nissinen, A., Nedeljkovic, S., Buzina, R., Mohacek, I., Seccareccia, F., Giampaoli, S., Dontas, A., Aravanis, C. & Toshima, H. (1996). Twenty-five year prediction of stroke deaths in the seven countries study: the role of blood pressure and its changes, Stroke 27, 381–387. Pocock, S. J. & Elbourne, D. R. (2000). Randomized trials or observational tribulations?, The New England Journal of Medicine 342, 1907–1909. Redelmeier, D. A., Rozin, P. & Kahneman D. (1993). Understanding patients’ decision–cognitive and emotional perspectives, Journal of the American Medical Association 270, 72–76. Redmond, C. K. & Costantino, J. P. (1996). Design and current status of the NSABP Breast Cancer Prevention Trial, Recent Results in Cancer Research 140, 309–317.

23. Sackett, D. L. (1997). Bias in analytical research, Journal of Chronic Diseases 32, 51–63. 24. Simon, G., Wagner, E. & VonKorff, M. (1995). Cost-effectiveness comparisons using ‘‘real world’’ randomized trials: the case of the new

antidepressant drugs, Journal of Clinical Epidemiology 48, 363–373. 25. Spiegelhalter, D. J., Freedman, L. & Parmar, M. K. B. (1994). Bayesian approaches to randomization clinical trials, Journal of the Royal Statistical Society, Series A 157, 357–416. 26. Steineck, G. & Ahlbom, A. (1992). A definition of bias founded on the concept of the study base, Epidemiology 3, 477–482. 27. Taylor, A. L., Adams-Cambell, L. & Wright, J. T. (1999). Risk/benefit assessment of tamoxifen to prevent breast cancer—still a work in progress, Journal of the National Cancer Institute 19, 1792–1973. 28. Wolmark, N., Wickerham, D. L., Costantino, J. P. & Cronin, W. (1999). NSABP Protocol P2: Study of Tamoxifen and Raloxifene (STAR) for the Prevention of Breast Cancer. National Surgical Breast and Bowel Project, Pittsburgh, Pennsylvania. 29. Women’s Health Initiative Study Group (1998). Design of the Women’s Health Initiative clinical trial and observational study, Controlled Clinical Trials 19, 61–109. 30. Writing Group for the PEPI Trial (1995). Effects of estrogen/progestin regimens on heart disease risk factors in postmenopausal women: the Post-menopausal Estrogen/Progestin Intervention (PEPI) Trial, Journal of the American Medical Association 273, 199–208.

BIASED COIN RANDOMIZATION

is b, and the maximum number of consecutive subjects with the same treatment is 2b. This method is most commonly used in clinical trial practice. However, it may be possible to predict or guess future treatments for subjects in the later part of each block with knowledge of the treatment for patients earlier in the block. For example, with two treatment groups and a block size of 4, an initial allocation of two patients to treatment A will mean that the next two subjects will receive treatment B. As such, the treatment allocation is deterministic for patients toward the end of blocks. In blinded trials, the block size is usually not disclosed because of the potential for unblinding and bias. However, even this strategy may not afford total protection from selection bias. If study staff are able to make a good guess at the previous treatment—based on frequent occurrence of known treatmentrelated adverse events or strong efficacy results (sometimes called functional unblinding)— then selection bias may still occur in PBR. The aim of this article is to describe a class of randomization procedures called biased coin randomization procedures, which are designed to eliminate or substantially reduce the problems of the CR and PBR procedures by

MIKE D. SMITH Clinical Statistics Pfizer Global Research & Development New London, Connecticut

1 RANDOMIZATION STRATEGIES FOR OVERALL TREATMENT BALANCE In a ‘‘simple’’ or ‘‘complete’’ randomization (CR) scheme, subjects are allocated to treatment groups based on a fixed probability without any regard to previous allocation of subjects or level of current imbalance. In the case of equal randomization to two groups, this procedure is equivalent to tossing a coin. Because it is based on the inability to predict or guess the treatment for future patients, this procedure is free from selection bias (1) , which is the bias introduced into a trial from awareness of a patient’s treatment allocation. The CR procedure does not guarantee overall treatment balance at the end of the trial, and there is a potential during the trial for a long run of one treatment versus another. For these reasons, the CR procedure is rarely used. Avoiding imbalance at specific time points during the trial is particularly important in the implementation of adaptive designs based on interim analysis of data. An early alternative to CR that addressed the issue of potential treatment imbalance is the nonrandom systematic design (e.g., ABBAABBAA), which is not discussed here as it contains no element of randomization. An alternative randomization procedure that aims to limit imbalance throughout the trial is ‘‘permuted block randomization’’ (PBR) (2). For a fixed number of treatments groups, fixed-sized blocks are generated that contain all possible treatment permutations. The full randomization list is then generated by randomly selecting from this full set of treatment blocks until the required sample size is achieved. Under PBR, the magnitude of treatment imbalance and the maximum number of consecutive subjects receiving the same treatment are both limited. In the case of two treatment groups, with a block size of 2b the maximum treatment imbalance

• minimizing the size any treatment

imbalance in a trial (and within each strata) and reducing the chance of long runs of one treatment group. • removing any deterministic treatment allocation for patients to treatment and so eliminating selection bias. The randomization procedures discussed here, including biased coin randomization and others, are described in detail in Rosenberger and Lachin (3), and summarized in Chow and Chang (4). 2 THE BIASED COIN RANDOMIZATION PROCEDURE Efron (5) first described a biased coin randomization procedure for two treatment groups

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

BIASED COIN RANDOMIZATION

(A and B) where equal treatment allocation over a whole study is required. We can consider this random allocation as tossing a coin. If the current treatment allocation is balanced, then the coin used will be a fair unbiased coin. However, if there is any imbalance in the current treatment allocation, then the coin will have a fixed bias favoring the treatment with the smaller number of subjects. Thus, the bias is independent of the size of treatment imbalance. Let nA and nB be the current number of subjects assigned to treatment A and B, respectively. The probability of the next subject being assigned to treatment A in the trial is   1 − p nA > nB nA = nB Probability (treatment A) = 0.5  p nA < nB , where p represents the bias in the randomization ‘‘coin’’ (1 ≥ p ≥ 0.5). This procedure is labeled BCD(p). Complete randomization is equivalent to using BCD(0.5). A block randomization using a block size of 2 is equivalent to using BCD(1), which is not usually considered as it has deterministic allocation for each alternate subject with the associated potential for selection bias. Efron (5) favored the value BCD(2/3) to provide a clear bias for the coin while maintaining an adequate random element. Pocock (6) suggested that BCD(3/4) is too predictable in the presence of imbalance, BCD(2/3) is appropriate for a relatively small trial; and BCD(3/5) is adequate for a larger trial with at least 100 patients. Operationally, the BCD(p) procedure uses three distinct fixed-randomization schedules, using the probability of being assigned to treatment A. For an investigator-blinded trial or a trial with multiple investigator sites, the three fixed randomization lists would be

administered centrally and not at individual sites. Within a stratified randomization (with one stratification factor), it would be proposed that each stratum be treated as a separate trial. Therefore, an attempt is made to achieve treatment balance for each stratum and thereby over the whole trial. A discussion of a more complex stratification procedure is described later.

3 PROPERTIES The BCD(p) procedure produces a Markov Chain random walk process for the difference DN = nA − nB . Under the assumption that both positive and negative imbalances are equally important to avoid, then the properties of the BCD(p) procedure can be evaluated more simply by considering |DN |. The transition matrix for |DN | is formed by considering the probability of moving from the current state to all possible states for the next patient. If balance is achieved at any stage in the study, the probability of an imbalance of 1 following the next subject is 1. If the study is imbalanced at any stage in the study, the probability of greater imbalance will be 1 − p and the probability of lesser imbalance will be p. The limiting probabilities (N = nA + nB → ∞) of imbalance of size j can be calculated from the transition matrix for |DN | (3). These are shown in Table 1 for even and odd N. The likelihood of perfect balance (for even N) or an imbalance of 1 (for odd N) tends to 1 as r → ∞ (p → 1); however, the deterministic nature of treatment allocation will increase, and so increase the potential for selection bias. An illustration of the limiting probabilities of imbalance for BCD(0.60), BCD(0.75) and BCD(0.90) is given in Figure 1A and B.

Table 1. Probability of imbalance of size j for the BCD(p) in terms of r = p/(1 – p) Even N

Odd N

1 − 1/r (j = 0)

r2 −1 (odd rj+1

r2 −1 (even rj+1

j, j ≥ 2)

j, j ≥ 1)

1.0

1.0

0.8

0.8 Probability

Probability

BIASED COIN RANDOMIZATION

0.6 0.4 0.2

3

0.6 0.4 0.2

0.0

0.0 0

1

2

3

4

5

6

7

8

9

10 11

0

1

2

3

4

Imbalance

5

6

7

8

9

10 11

Imbalance

Figure 1. Probability of imbalance for (A) even and (B) odd total number of subjects using biased coin design with p = 0.60 (blue), 0.75 (red), 0.90 (green).

30 Difference (Dn = nA–nB)

Difference (Dn = nA–nB)

30 20 10 0 −10 −20 −30 0

100 Subject Number

200

20 10 0 −10 −20 −30 0

100 Subject Number

200

Difference (Dn = nA–nB)

30 20 10 0 −10 −20 −30 0

100 Subject Number

200

Figure 2. Difference in number of subjects by additional recruited subjects (A) BCD(0.60), (B) BCD(0.75), and (C) BCD(0.90) with starting imbalance of +20 (red line), 0 (blue line), –20 (green line).

An example of this procedure would be an ongoing clinical trial where a further 200 subjects will be randomized. Let the current level of imbalance in this example be Dn = +20, 0, and −20 shown as the red, blue, and green curves, respectively, in Figure 2A–C. We can

see the difference DN = nA − nB for the additional 200 randomized subjects using the BCD(0.60), BCD(0.75), and BCD(0.90) procedures (as A, B, and C, respectively). The plots indicate that as p increases we achieve faster convergence to Dn = 0, given initial

4

BIASED COIN RANDOMIZATION

imbalance and a lower level of imbalance around Dn = 0. Many investigators use the expected number of correct guesses of future treatment assignment as a surrogate measure of potential selection bias (5). Wei (7) showed that the number of correct guesses of the next treatment assignment is much lower for BCD than for PBR. The CR procedure has the minimum expected number of correct guesses, and so (as expected) it minimizes the potential section bias. The number of correct guess is greater for BCD compared with CR; however, this difference decreases as studies get larger.

4 EXTENSIONS TO THE BIASED COIN RANDOMIZATION 4.1 Unequal Randomization Ratio A simple extension to the BCD procedure is to allow for unequal treatment allocation. If the required randomization ratio (A:B) is k, the probability of assigning the next subjects to treatment A could be Probability (treatment A)  k   k+1 − p nA > knB k = k+1 nA = knB   k + p nA < knB . k+1 This procedure will further modify the bias of the originally biased coin. The initial coin k and this bias is either increased or bias is, k+1 decreased by p in cases of imbalance from the planned randomization ratio, subject to the   k constraint 1 − k+1 > p > 0 (where k > 1). 4.2 More Than Two Treatments The BCD procedure can be easily modified to >2 treatment groups. Under equal randomization (with t treatment groups and a total of N subjects), the expected number of subjects in each treatment group will be N/t. In this procedure, the coin bias is applied to each treatment group and is established by comparing the current individual treatment allocation again the expected level.

We define the probability of assigning the next patient to treatment i(i = 1, . . . , t) as follows: Probability (treatment i)   c(1/t − p) ni > N/t = pi = c(1/t) ni = N/t   c(1/t + p) ni < N/t, where ni is the current number of subjects for treatment i, 1/t > p > 0, and c is a rescaling value such that the probabilities sum to 1( ti=1 pi = 1) 1. As before, larger values of p lead to faster convergence with N/t and subsequently to a lower level of imbalance around N/t. Figure 3(A and B) shows an example of the proportion of subjects assigned to each of four treatment groups for an intended randomization ratio of 1:1:1:1, using values of p equal to 0.05 and 0.15 for 200 subjects. As with before in the two treatment case, a larger value of p will tend the proportion of subjects allocated to each treatment more quickly toward the expected value (here N/t) and maintain a smaller imbalance around the expected value. Over a large number of subjects, the proportion of subjects allocated to each treatment will tend to N/t regardless of the choice of p. However, the choice of p does greatly influence the proportion of subjects allocated to each treatment over a small or moderate number of subjects. The choice of p will also influence the chance of long runs of one treatment group, regardless of the number of subjects in the trial. 5 ADAPTIVE BIASED COIN RANDOMIZATION The coin bias in the original BCD procedure is constant, regardless of the size of the treatment imbalance (DN = nA − nB ) or the current size of the study (N = nA + nB ). For example, the probability of the next subject being assigned to treatment A is the same, regardless of whether DN is 1 or 100, or whether N is 20 or 2000. An extension to the simple BCD(p) procedure is to modify the coin bias according to the current size of the imbalance. We will describe this using

1.0

1.0

0.8

0.8 Proportion

Proportion

BIASED COIN RANDOMIZATION

0.6 0.4

5

0.6 0.4 0.2

0.2

0.0

0.0 0

50

100 150 Subject Number

0

200

50

100

150

200

Subject Number

Figure 3. Proportion of subjects allocated to each of four treatment groups using (A) p = 0.05 and (B) p = 0.15. Table 2. Probability of imbalance of size j for the Big Stick Rule

Even c Odd c

Even N

Odd N

1/c (j = 0 or c) 2/c (even j, c–2 ≥ j ≥ 2) 1/c (j = 0) 2/c (even j, c – 1 ≥ j ≥ 2)

2/c (odd j, c–1 ≥ j ≥ 1) 1/c (j = c) 2/c (odd j, c – 2 ≥ j ≥ 1)

both step and continuous functions for the coin bias. A further extension is to also modify the coin bias based on the current size of the study, and this will be described for continuous coin bias functions.

on whether N and c are odd or even. The

5.1 Step Functions for Coin Bias

ing the deterministic nonrandom treatment

Rosenberger and Lachin (3) describe two modified BCD procedures using a step function for coin bias based on DN. The first is called the Big Stick Rule (8) and uses an unbiased coin for treatment allocation where the current imbalance |DN | < c for some predefined critical value of c. Where the imbalance reaches this critical value, the probability of being assigned to treatment A is 0 (DN = c) or 1 (DN = −c). The imbalance between treatment groups is limited to c, and the maximum consecutive allocation of one treatment group is 2c + 1. There is a chance that treatment allocation is determined in a nonrandomized manner when the maximum imbalance is reached, and the frequency of this depends on the choice of c. A value of c = ∞ gives the BCD(p) design. Using the random walk process, we can calculate the long-term probabilities of imbalance of size j, which depend

allocation is 1/c (when N and c are either both

long-term probability of imbalance if size j for the Big Stick Rule is shown in Table 2. Therefore, the long-term probability of apply-

odd or both even), and it will decrease as c increases. The second modified BCD procedure described by Rosenberger and Lachin (3) using a step function for coin bias is the ‘‘biased coin design with imbalance intolerance’’ (9). This procedure is a mixture of Efron’s original BCD procedure and the Big Stick Rule. The treatment allocation is deterministic where the imbalance exceeds some critical value as in the Big Stick Rule; however, the biased coin is still used for any imbalance below the critical level as in Efron’s

6

BIASED COIN RANDOMIZATION

procedure. This rule is shown here:

and p. Therefore, the treatment assignment in this procedure would be:

Probability (treatment A)   0 Dn = c      1 − p c > Dn > 0 = 1/2 Dn = 0   p 0 > Dn > −c     1 Dn = −c,

  1 − p DN = c Probability (treatment A) = 1/2 |DN | < c   p DN = −c.

where 1 ≥ p ≥ 0.5 (a value of 0.5 gives the big stick design, and a value of 1 gives the big stick design with c = 1). Chen (9) gives the probability of imbalance of size j. In particular, the probability of imbalance of size c (and so the probability of applying the deterministic   c−1  treatment allocac−1 q , where q = 1 − p. tion) is (p−q)p c c p −q pc−1 This is always less than 1/c, so the likelihood of reaching the imbalance boundary is less with this procedure compared with the Big Stick Rule. Similarly even  N), the  (for c−1 (p−q)p , which is probability of balance is pc −qc always greater than 1/c, so the likelihood of achieving balance is greater with this procedure compared with the Big Stick Rule. Both the Big Stick Rule and the biased coin design with imbalance intolerance procedures use nonrandom allocation of treatment to patients when imbalance has reached a predefined limit, which creates hard boundaries of −c and c for Dn . Pocock (6) describes a modified procedure that avoids the deterministic allocation in the Big Stick Rule by replacing the probabilities of 0 and 1 with 1 − p

A similar modification to the biased coin design with imbalance intolerance design might be to replace 0 and 1 with 1 − p2 and p2 (such that 1/2 < p < p2 < 1). This would give greater balance compared with Pocock’s procedure, but at the expense of a more complex strategy now requiring the definition of two levels of coin bias rather than one. In this procedure, the imbalance is now not limited by a hard boundary of c but rather by what might be viewed as a ‘‘soft’’ limit. An illustration of this procedure (with p = 0.60, p2 = 0.90, and a soft limit of 3) is compared with Chen’s design (with p = 0.60 and a hard limit of 3) in Figure 4. 5.2 Continuous Functions for Coin Bias A continuous function can be defined for the treatment allocation coin bias such that it is both (1) directly proportional to the size of the current treatment imbalance and (2) inversely proportional to the current size of the study. This provides the benefit that changes to treatment balance are greatest earlier in the trial and/or where the treatment imbalance is greatest. The use of a continuous function for coin bias means that

Difference (Dn = nA–nB)

5 3

0

−3 −5 0

25

50 Subject Number

75

100

Figure 4. Difference in number of subjects (A-B) by subject number for adaptive step function with hard (red) and soft (blue) limits of 3 using p = 0.60 and p2 = 0.90.

BIASED COIN RANDOMIZATION

the arbitrary and subjective cut-off points in the step functions described previously do not need to be defined. Two examples of a continuous function for coin bias are shown here. As before, for two treatment groups, and shown in terms of the probability of assigning the next subject to treatment A, n2B n2B

+ n2B

(10) and ω + βnB 2ω + β(nA + nB ) for some ω,β ≥ 0(11). Wei (7, 12) describes a general class of functions for coin bias and defines a set of properties that this class of function should meet. The CR, PBR, and BCD procedures are shown to fit within this general framework. For two treatment groups, the probability of assigning the next subject to treatment A would be  nA − nB , p(Dn /N) = p nA + nB where p(x) is a nonincreasing function satisfying p(x) + p(−x) = 1 for −1≥x≥1. This class of functions need not be continuous (e.g., the step functions for coin bias described in section 5.1 would satisfy these conditions). However, Wei (7, 12) shows that if the function p(x) is continuous at x = 0 then selection bias will tend to 0 as the sample size increases. ρ The function p(x) = (1+x)(1−x) ρ +(1−x)ρ with x = Dn /N, yields the following function for the probability of assigning the next subject to treatment A: ρ

ρ nA

nB ρ + nB

(13). This function is a more general form of the rule by Atkinson (10). The function p(x) = (1 − x)/2 with x = Dn /N gives the allocation rule for the probability of being assigned to treatment A as: nB nA + nB

7

(12). In these procedures, as for the previous BCD strategies, the coin bias will favor randomization toward the under represented treatment group. However, by using these continuous functions for coin bias, the coin bias is affected by the size of the imbalance. Additionally, as the denominator in all these functions includes nA and nB , the coin bias will tend to 0 as the study gets larger; thus, these procedures will tend to CR as the sample size increases. Another general class of continuous coin bias functions, given by Baldi Antognini and Giovagnoli (14), is the adjustable biased coin designs (ABCDs). The ABCDs are based on Dn rather than Dn /N. In these functions, the coin bias does not necessarily tend to 0 as the study gets larger. This property can be viewed as either an advantage (14) or a disadvantage (12). One ABCD function highlighted by Baldi Antognini and Giovagnoli (14) is  |D |a n   |Dn |a +1 Dn < 0 Probability(treatment A) = 0.5 Dn = 0   1 D n>0 |Dn |a +1 where a ≥ 0, a function labeled as ABCD(Fa). Smaller values of a will increase the chance of some treatment imbalance but will decrease selection bias (the special case a = 0 would give CR with no selection bias). The function ABCD(Fa) will tend to the big stick design with boundaries of ± 2 as a → ∞. The effect of a can be dramatic even when the treatment balance is not large. For example, the probability of being assigned to treatment A is <0.01 when Dn ≥ 4 and ≥ 5 when a = 4 and 3, respectively. The function ABCD(Fa) is continuous, but the probability of assigning treatment A is equal to 1/2 when Dn is 0 or ± 1, that is, where the current allocation is balanced or as close to being balanced as possible (the latter in the case of an odd number of subjects). Figure 5 shows the probability of assigning treatment A to the next subject based on the current imbalance using ABCD(F0.5), ABCD(F1), and ABCD(F4). Although the ABCD functions are described in this section as continuous functions for coin bias, the noncontinuous step

8

BIASED COIN RANDOMIZATION

functions (section 5.1) could be viewed as a special case of the ABCD as they also represent decreasing probability functions (for treatment A) in Dn and satisfy p(Dn ) + p(−Dn ) = 1. We conclude this section by illustrating a number of the biased coin procedures. Table 3 shows the probability of assigning treatment A for a variety of situations based the current level of treatment imbalance (Dn ) and the current sample size (N). 6

URN MODELS

´ 6.1 Polya Urn Model A different formulation to the BCD in achieving treatment balance is the randomized urn design (UD). A historical review of the UD is given by Rosenberger (15). The P´olya Urn Model (16) consists of an urn containing ω balls of two colors, with each color representing a treatment group. A ball is removed from the urn at random and then replaced, together with β balls of the opposite color. The next patient is allocated the treatment corresponding to the color of the drawn ball. This design may be labeled as UD(ω,β), and the special case of UD(ω,0) is equivalent to simple or complete randomization. This design is sometimes called the Friedman’s or generalized Friedman’s urn (4, 17), although the generalized Friedman’s urn is more general than this procedure (Friedman [18] and Wei [19]) and will be discussed

later. The range of descriptive terms for this design is discussed in Rosenberger [15]. This design also allows the initial number of balls to differ between the two groups: for example, UD(ω1 , ω2 , β). However, when equal allocation to treatment is required throughout the trial, we may assume an equal number of balls for the two treatment groups in the starting urn. The case of an unequal initial composition has relevance where unequal treatment allocation might be required. In this case, the number of balls returned would need to depend not on which ball was drawn but on the comparison of the current urn versus the target randomization ratio. An extension to this design might be to update the urn with a random number of balls of the appropriate color, using some predefined probability-generating function. This is called the Randomized Urn Model (20). The application of the urn model for multiple randomization strata is most simply to treat each strata level as an independent trial with a separate urn (11), as also suggested in the BCD (5). This is described in more detail later. The initial number of balls in the urn will affect the bias in the randomization when treatment imbalance occurs early in the trial. Any imbalance that occurs is more quickly countered with smaller versus larger values of ω (as β has a greater relative effect). As the total number of subjects increases, the effect of ω diminishes, and the procedure tends toward the CR procedure with perfect

Probability Assingning Treatment A

1.0 0.8 0.6 0.4 0.2 0.0 −8 −7 −6 −5 −4 −3 −2 −1 0 1 2 3 4 5 Treatment Imbalance, Dn = Na– Nb

6

7

8

Figure 5. Probability of assigning treatment A by current treatment imbalance difference for ABCD(F0.5) [blue], ABCD(F1) [red] and ABCD(F4) [green].

BIASED COIN RANDOMIZATION

9

Table 3. Probability of assigning treatment A for a variety of situations based on the current level of treatment imbalance (Dn ) and the current sample size (N) Constant coin bias

N

nA , nB

1 1,0 2 1,1 3 2,1 4 3,1 20 15,5 100 55,45 1000 505,495

Dn

1 0 1 2 10 10 10

Step function for coin bias

Continuous function for coin bias

Efron 1971 Chen 1999 Atkinson (5)a (9)b 1982 (10)c

0.4 0.5 0.4 0.4 0.4 0.4 0.4

0.4 0.5 0.4 0.4 0 0 0

0 0.5 0.2 0.1 0.1 0.4 0.49

Wei 1978 Wei 1977 (12)d (11)e

0 0.5 0.33 0.25 0.25 0.45 0.495

0.45 0.5 0.46 0.43 0.33 0.45 0.495

Baldi Antognini & Giovagnoli 2004 (14), ABCD(Fa)f 0.5/0.5/0.5 0.5/0.5/0.5 0.5/0.5/0.5 0.414/0.333/0.059 0.240/0.091/0.001 0.240/0.091/0.001 0.240/0.091/0.001

a BCD(p = 0.6). b BCD

with imbalance intolerance (p = 0.6, c = 5).   A) = n2B / n2A + n2B

c Pr(treatment

d Pr(treatment

A) = nB /(nA = nB ). A) = (ω + β nB )/(2ω + β (nA + nB )), with ω = 10 and β = 2. f Pr(treatment A) as described for ABCD(Fa), with a = 0.5, 1, and 4. e Pr(treatment

balance. This can be seen by the approximate probability of treatment imbalance for a moderate or large study, which is

 

  Pr Dn  > r ≈ 2 1 −  r

3 (nA + nB )



(3). Note that this probability is independent of both ω and β. There is a direct equivalence between the BCD and the P´olya UD, where the UD(ω,β) is equivalent to the BCD using the allocation probability defined  by the continuous  ω+βnB (11). function for coin bias of 2ω+β(n +n ) A B The special case UD(0,1) is equivalent to the BCD with allocation probability proportional to the number ofsubjects  allocated to treatnB ment group A of n +n . A B Wei and Lachin (17) show the following properties of the UD(ω,β) as compared with other randomization procedures: • The chance of imbalance for UD(ω,β) is

far less likely compared with CR.

• The potential for selection bias for UD

(ω,β) is less compared with BCD(p) and PBR, and it tends to the minimum selection bias possible (i.e., under CR) as the study size gets larger. • As the study size gets larger, UD(ω,β) is free from accidental bias (which may be caused by an imbalance of a known or unknown prognostic factor). The extension of this design to more than two treatments is simple (19). The initial urn contains ω distinct balls for each treatment group. A ball is removed from the urn at random and replaced. The next patient is allocated the treatment corresponding to this drawn ball. The urn is updated by adding β balls for all the other treatment groups. The probability of the next subject receiving treatment i(i = 1, . . . , t) is 

ω + β(N − ni ) tω + β(t − 1)N

,

where N is the total number of subjects (N = n1 + . . . + nt ). The probability of imbalance for treatment group i versus perfect balance

10

BIASED COIN RANDOMIZATION

is 

 Pr ni − N/t > r   

 t2 (t + 1)  . ≈ 2 1 −  r N(t − 1)2

Again the long-term probability of imbalance is independent of both ω and β. The UD has a particularly simple application to response-adaptive randomization and is described later. 6.2 Urn Model without Replacement The urn design UD(N/2,0) using sampling from the urn without replacement is equivalent to the random allocation rule (21). In this design, treatment balance at the end of the trial is guaranteed, given that the total number of subjects required for the trial is known and fixed in advance. The major criticism of this procedure is that once all N/2 subjects are allocated for either group then the allocation for the remaining subjects must all be assigned to a single treatment group, which may increase both selection and accidental bias, the latter due to time trends in patient characteristics, for example. Another criticism is that this procedure may still lead to imbalances during the trial or a long series of one treatment group. The former might be an important consideration if the trial uses interim analyses or is terminated early. 6.3 Friedman’s Urn Model Friedman (18) introduced a more general urn model whereby a ball is removed at random from the urn (initially containing ω balls of two colors as before). The ball is then replaced together with α balls of the same color and β balls of the opposite color. Treatment allocation for the next patient is chosen from the color of the selected ball. This design will be labeled as UD(ω,α,β). The model UD(ω,0,β) is the P´olya Urn Model discussed previously, which is usually described using two parameters, UD(ω,β). To achieve treatment balance, we require α, β > 0 and also β > α (the case where β = α is CR). Wei (19) confirms that the UD(ω,α,β)

tends to the CR procedure as the study size gets larger as for the simpler UD(ω,β). The UD(ω,α,β) also has the same properties of asymptotic balance and is free from selection and accidental bias. The UD(ω,α,β) tends to balance more quickly as β/α gets larger (19), so the P´olya design of UD(ω,0,β) will tend to balance more quickly that the generalized Friedman design with α > 0. This model is equivalent to a BCD, with an adaptive probability of the next subject being assigned to treatment A as 

ω + αnA + βnB 2ω + (α + β)(nA + nB )



The extension of the Friedman urn model to more than two treatment groups (t > 2) is described by Wei (19), and it follows the same procedure as for the simpler P´olya urn model. Following the random selection and replacement of a ball from the urn, the urn is then updated by adding β balls for each of the other treatment groups and α balls for the selected treatment group. The probability of the next subject receiving treatment is  i (i = 1, . . . , t) is

ω + αni + β(N − ni ) tω + (α + β(t − 1))N

,

where N is the total number of subjects (N = n1 + . . . + nt ). The probability of imbalance for treatment group i versus perfect balance for large N (where (t + 1) β > α) is 

 Pr ni − N/t > r   

 2 ((t + 1)β − α) t  . ≈ 2 1 −  r N(t − 1)(α + (t − 1)β)

7 TREATMENT BALANCE FOR COVARIATES In this section we discuss amendments to the biased coin designs that attempt to balance the treatment allocation not only over the whole study, but also within levels of important covariates or prognostic factors. In a study with treatment imbalance with respect to prognostic factors, the estimated treatment differences will still be unbiased as long as the prognostic factor is accounted for

BIASED COIN RANDOMIZATION

in the analysis (this is not true if the factor is ignored in the analysis). Therefore, the real benefit of maintaining treatment balance within strata is to reduce the variability of the estimated treatment difference, and so to improve the efficiency of the study (22). The BCD procedures aim to bias the treatment allocation for future subjects based on the treatment allocation of previous subjects. As we have seen, the potential for overall treatment imbalance is greatly reduced in the BCD compared with CR. However, the BCD procedures will not guarantee treatment balance within levels of any important prognostic factor. A common procedure to achieve treatment balance within strata is stratified randomization, and the most common method of this is stratified PBR. This method will help improve balance within strata across the prognostic factors. However, it is possible that the overall study could still have some sizeable imbalance across the strata, particularly where the total study size is small and number of strata is large. In this case, many blocks may be unfilled and potentially imbalanced at any point in the trial. The potential for selection bias would also still exist in this procedure. Another simple procedure to achieve treatment balance within strata is minimization, which attempts to achieve balance within the strata of each main prognostic factor (not in strata across factors as in stratified randomization. For example, we can create balance for both gender and for all study sites, but not at each gender-site level. Minimization has been proposed using: • A deterministic nonrandom allocation

rule (23). • A procedure using a combination of both

nonrandom and random allocation rules, the former for higher imbalance, the latter for no or mild imbalance (2). • A rule maintaining random allocation (10, 24). The latter approaches can be viewed as biased coin randomization procedures, for example, where the coin bias is weighted toward allocation that gives the least treatment imbalance in the marginal covariate

11

factors for the next patient (24), or toward allocation that minimizes the variances of the treatment contrasts under the design containing the current state of the treatment allocation and covariate information and the covariate factors for the next patient (D-optimality [10, 25]). Within the BCD and UD procedures previously described, a common simple solution is simply to treat each strata as a separate experiment. The bias in the coin is then calculated using the current state for the single urn (strata) relevant for the next patient (e.g., the single urn for males, age < 45 years, in study site 10). Again, where the numbers of strata are large and/or the sample size is small, this procedure may not protect against overall treatment imbalance or even imbalance within strata. In this case, Wei (19) suggests balancing the marginal urns, either using some composite score calculated from the imbalance over all relevant urns (e.g., the three urns for males, age < 45 years, and study site 10) or using only the currently most imbalanced marginal urn. This procedure would not aim to achieve balance within cross-factor strata, but it is more likely to achieve balance in the marginal factor levels and across the study. Therefore, the BCD procedure to achieve marginal-factor urn balance is a minimization procedure maintaining the random allocation rule. 8 APPLICATION OF BIASED COIN DESIGNS TO RESPONSE-ADAPTIVE RANDOMIZATION The aim of the various bias coin designs is to use a randomization strategy that tends the treatment allocation toward an intended allocation ratio (most often balanced) while reducing allocation bias and other biases in the trial. These strategies can also be extended to the treatment allocation within important strata (covariate-adaptive randomization). A quite distinct class of techniques are those for response adaptive randomization (RAR); as their name suggests, these techniques allocate subjects to treatment groups based on the known responses of previous subjects, rather than on just the treatment group or covariate values. Broadly, the aim of RAR techniques could be:

12

BIASED COIN RANDOMIZATION • To optimize some key parameter of inter-

est. • To tend the treatment allocation toward the more ‘‘successful’’ treatment groups and away from the inferior groups (with respect to efficacy, safety or toleration, or some combination). Both of these objectives can be viewed as creating a more ethical design for the patients recruited into the clinical trial, for example, by minimizing variance of the treatment contrasts (to increase power or reduce overall sample size) or reducing the number of patients subjected to a clearly inferior treatment. Both the BCD and UD techniques may be used for RAR, whether the aim is to optimize some parameter or to bias allocation toward successful treatment groups. The application is easiest for binary response data as in for the randomized ‘‘play the winner’’ rule (RPW), the most well-known RAR technique (26). In this urn model, the initial composition is ω balls for each treatment group (each treatment identified by different colored balls). The first patient is allocated the treatment according to a ball drawn (with replacement) from the initial urn, and the response is noted. If the subject has a treatment response, β colored balls for that treatment are added into the urn, and α balls for each of the other treatments. If the subject does not have a treatment response, α and β balls are added for the current and each of the other treatments, respectively. The study then continues to recruit subjects, to collect the patient responses, and to update the urn. This design is labeled as the RPW(ω,α,β), where β > α ≥ 0 (CR is a special case where β = α). In the two-group case (treatments A and B) for RPW(ω,0,β), the ratio of balls in the urn and the treatment allocation ratio (A/B) will tend to qB /qA , where qi = 1 − pi is the nonresponse rate for treatment i. For the more general RPW(ω,α,β), the treatment allocation ratio will tend to a function based on α and β. These results show that the BCD, using a continuous function for coin bias based on the responses in each treatment group, could also be used as a strategy for a RAR design with the same objective as for the RPW design.

Many modifications have been made to the initial RPW urn model for RAR. These include strategies for more than two treatment groups, for categorical and continuous outcomes, for delayed responses, for treatment elimination from the randomization, and for targeting response percentiles (3, 4, 15, 27).

REFERENCES 1. D. Blackwell and J. L. Hodges, Design for the control of selection bias. Ann Math Stat. 1957; 28: 449–460. 2. M. Zelen, The randomization and stratification of patients to clinical trials. J Chron Dis. 1974; 27: 365–375. 3. W. F. Rosenberger and J. M. Lachin, Randomization in Clinical Trials Theory and Practice. New York: Wiley, 2002. 4. S. C. Chow and M. Chang, Adaptive Design Methods in Clinical Trials. Boca Raton: Chapman & Hall, 2007. 5. B. Efron, Forcing a sequential experiment to be balanced. Biometrika. 1971; 58: 403–417. 6. S. J. Pocock, Clinical Trials: A Practical Approach. New York: Wiley, 1983. 7. L. J. Wei, A class of treatment assignment rules for sequential experiments. Commun Stat Theory Methods. 1978; A7: 285–295. 8. J. F. Soares and C. F. Wu, Some restricted randomization rules in sequential designs. Commun Stat Theory Methods. 1983; 12: 2017–2034. 9. Y. P. Chen, Biased coin design with imbalance intolerance. Commun Stat Stochastic Models. 1999; 15: 953–975. 10. A. C. Atkinson, Optimum biased coin designs for sequential clinical trials with prognostic factors. Biometrika. 1982; 69: 61–67. 11. L. J. Wei, A class of designs for sequential clinical trials. J Am Stat Assoc. 1977; 72: 382–386. 12. L. J. Wei, The adaptive biased coin design for sequential experiments. Ann Stat. 1978; 6: 92–100. 13. R. L. Smith, Sequential treatment allocation using biased coin designs. J R Stat Soc Ser B Methodol. 1984; 46: 519–543. 14. A. Baldi Antognini and A. Giovagnoli, A new ‘biased coin design’ for the sequential allocation of two treatments. Appl Stat. 2004; 53: 651–664.

BIASED COIN RANDOMIZATION 15. W. F. Rosenberger, Randomized urn models and sequential design. Sequential Analysis. 2002; 21: 1–28. ¨ 16. F. Eggenberger and G. P´olya, Uber die statis¨ tik verketteter vorgange. Zeitschrift fur ¨ angewandte mathematic and mechanik. 1923; 3: 279–289. 17. L. J. Wei and J. M. Lachin, Properties of the urn randomization in clinical trials. Control Clin Trials. 1988; 9: 345–364. 18. B. Friedman, A simple urn model. Commun Appl Math. 1949; 1: 59–70. 19. L. J. Wei, An application of an urn model to the design of sequential controlled clinical trials. J Am Stat Assoc. 1978; 73: 559–563. 20. N. L. Johnson and S. Kotz, Urn Models and Their Applications. New York: Wiley, 1977. 21. J. M. Lachin, Properties of simple randomization in clinical trials. Control Clin Trials. 1988; 9: 312–326. 22. S. Senn, Statistical Issues in Drug Development. Chichester, UK: Wiley, 1997.

13

23. D. R. Taves, Minimization: a new method of assigning patients to treatment and control groups. Clin Pharm Ther. 1974; 15: 443–453. 24. S. J. Pocock and R. Simon, Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. Biometrics. 1975; 31: 103–115. 25. A. C. Atkinson, Optimum biased–coin designs for sequential treatment allocation with covariate information. Stat Med. 1999; 18: 1741–1752. 26. L. J. Wei and S. Durham, The randomized play-the-winner rule in medical trials. J Am Stat Assoc. 1978; 73: 840–843. 27. A. Ivanova and W. F. Rosenberger, A comparison of urn designs for randomized clinical trials of K > 2 treatments. J Biopharm Stat. 2000; 10: 93–107.

BIOEQUIVALENCE (BE) TESTING FOR GENERIC DRUGS To receive approval for an Abbreviated New Drug Application (ANDA), an applicant generally must demonstrate, among other things, that its product has the same active ingredient, dosage form, strength, route of administration, and conditions of use as the listed drug and that the proposed drug product is bioequivalent to the reference listed drug [21 United States Code (U.S.C.) 355(j)(2)(A); 21 Code of Federal Regulations (CFR) 314.94(a)]. Bioequivalent drug products show no significant difference in the rate and extent of absorption of the therapeutic ingredient [21 U.S.C. 355(j)(8); 21 CFR 320.1(e)]. Studies for BE are undertaken in support of ANDA submissions with the goal of demonstrating BE between a proposed generic drug product and its reference listed drug. The regulations governing BE are provided at 21 CFR Part 320.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/6772dft.pdf) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

BIOLOGICAL ASSAY, OVERVIEW

penetrating level of research focusing on the gene-environment interaction in conventional experiments with biological units, and thereby calling for drastically different statistical resolutions for bioassays. We include also a brief synopsis of these recent developments. Traditionally, in a bioassay, a test (new) and a standard preparation are compared by means of reactions that follow their applications to some biological units (or subjects), such as subhuman primates (or human) living tissues or organs; the general objective being to draw interpretable statistical conclusions on the relative potency of the test preparation with respect to the standard one. Usually, when a drug or a stimulus is applied to a subject, it induces a change in some measurable characteristic that is designated as the response variable. In this setup, the dose may have several chemically or therapeutically different ingredients while the response may also be multivariable. Thus the stimulus–response or dose–esponse relationship for the two preparations, both subject to inherent stochastic variability, are to be compared in a sound statistical manner (with adherence to biological standardization) so as to cast light on their relative performance with respect to the set objectives. Naturally, such statistical procedures may depend on the nature of the stimulus and response, as well as on other extraneous experimental (biological or therapeutical) considerations. As may be the case with some competing drugs for the treatment of a common disease or disorder, the two (i.e. test and standard) preparations may not have the same chemical or pharmacological constitution, and hence, statistical modeling may be somewhat different than in common laboratory experimentation. Nevertheless, in many situations, the test preparation may behave (in terms of the response/tolerance distribution) as if it is a dilution or concentration of the standard one. For this reason, often, such bioassays are designated to compare the relative performance of two drugs under the dilution–concentration postulation, and are thereby termed dilution assays.

PRANAB K. SEN Chapel Hill, NC, USA

This article mainly emphasizes the classical aim of biological assay (or bioassay), to estimate relative potency, arising out of a need for biological standardization of drugs and other products for biological usage. There is a basic difference between biological and chemical endpoints or responses: the former exhibits greater (bio)variability and thereby requires in vivo or in vitro biological assays wherein a standard preparation (or a reference material) is often used to have a meaningful interpretation of relative potency. However, the term bioassay, has also been used in a wider sense, to denote an experiment, with biological units, to detect possible adverse effects such as carcinogenicity or mutagenicity; Mutagenicity Study). In the context of environmental impact on biosystems, toxicodynamic and toxicokinetic (TDTK) models as well as physiologically based pharmacokinetic (PBPK) models have been incorporated to expand the domain of bioassays; structure– activity relationship information (SARI) is often used to consolidate the adoption of bioassays in a more general setup; the genesis of dosimetry (or animal studies) lies in this complex. The use of biomarkers in studying environmental toxic effects on biological systems, as well as in carcinogenicity studies, has enhanced the scope of bioassays to a greater interdisciplinary field; we need to appraise, as well, bioassays in this broader sense. Further, recent advances in bioinformatics have added new frontiers to the study of biological systems; bioassay models are gaining more popularity in the developing area of computational biology. Our appraisal of bioassay would remain somewhat incomplete without an assessment of the role of Pharmacogenomics as well as Toxicogenomics in establishing a knowledge base of the chemical effects in biological systems. The developments in genomics during the past eight years, have opened the doors for a far more

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

BIOLOGICAL ASSAY, OVERVIEW

Dilution assays are classified into two broad categories: Direct dilution and indirect dilution assays. In a direct assay, for each preparation, the exact amount of dose needed to produce a specified response is recorded, so that the response is certain while the dose is a nonnegative random variable that defines the tolerance distribution. Statistical modeling of these tolerance distributions enables us to interpret the relative potency in a statistically analyzable manner, often in terms of the parameters associated with the tolerance distributions. By contrast, in an indirect assay, the dose is generally administered at some prefixed (usually nonstochastic) levels, and at each level, the response is observed for subjects included in the study. Thus, the dose is generally nonstochastic and the stochastic response at each level leads to the tolerance distributions that may well depend on the level of the dose as well as the preparation. If the response is a quantitative variable, we have an indirect quantitative assay, while if the response is quantal in nature (i.e. all or nothing), we have a quantal assay. Both of these indirect assays are more commonly addressed in statistical formulations. Within this framework, the nature of the dose–response regression may call for suitable transformations on the dose variable (called the dosage or dose-metameter) and/or the response variable, called the response-metameter. The basic objective of such transformations is to achieve a linear dosage-response regression, which may induce simplifications in statistical modeling and analysis schemes. In view of the original dilution structure, such transformations may lead to different designs for such assays, and the two most popular ones are (i) parallel-line assays and (ii) slope-ratio assays. Within each class, there is also some variation depending on the (assumed) nature of tolerance distributions, and within this setup, the probit (or normit) and logit transformations, based on normal and logistic distributions respectively, are quite popular in statistical modeling and analysis of bioassays. Bliss 2 contains an excellent account of the early developments in this area, while the various editions of Finney 6 capture more up-to-date developments, albeit with a predominantly parametric flavor. We refer to

these basic sources for extensive bibliography of research articles, particularly in the early phase of developments where biological considerations often dominated statistical perspectives. In this framework, it is also possible to include bioassays that may be considered for bioavailability and bioequivalence studies, though basically there are some differences in the two setups: Bioassays for assessing relative potency relate to clinical therapeutic equivalence trials, while in bioequivalence trials, usually, the relative bioavailability of different formulations of a drug are compared. Thus, in bioequivalence studies, the pharmacologic results of administrating essentially a common drug in alternative forms, such as a capsule versus a tablet, or a liquid dose of certain amount, capsules (tablets) or liquid forms of larger dose versus smaller dose with increased frequency of prescription, or even the administration of a drug at different time of the day, such as before breakfast or sometime after a meal, and so on, are to be assessed in a valid statistical manner. In this sense, the active ingredients in the drug in such alternative forms may be essentially the same, and differences in bioavailability reflect the form and manner of administration. We shall see later on that these basic differences in the two setups call for somewhat different statistical formulations and analysis schemes.

1 DIRECT DILUTION ASSAYS As an illustrative example, consider two toxic preparations (say, S and T), such that a preparation is continuously injected into the blood stream of an animal (say, cat) until its heart stops beating. Thus, the response (death) is certain, while the exact amount of the dose (X) required to produce the response is stochastic. Let X S and X T stand for the dose (variable) for the standard and test preparation, and let F S (x) and F T (x), x ≥ 0, be the two tolerance distributions. The fundamental assumption of a direct dilution assay is the following: FT (x) = FS (ρx),

for all x ≥ 0,

(1)

BIOLOGICAL ASSAY, OVERVIEW

where ρ(>0) is termed the relative potency of the test preparation with respect to the standard one. Standard parametric procedures for drawing statistical conclusions on ρ are discussed fully in the classical text of Finney 6, where other references are also cited in detail. If F S (.) is assumed to be a normal distribution function, then ρ is characterized as the ratio of the two means, as well as the ratio of the two standard deviations. Such simultaneous constraints on means and variances vitiate the simplicity of achieving optimality of parametric procedures (in the sense of maximum likelihood estimators and related likelihood ratio tests). On the other hand, if we use the log-dose transformation on the two sets of doses, and the resulting dosage distributions, denoted by FS∗ (.)andFT∗ (, )respectively, are taken as normal, then they have the same variance, while the difference of their means define logρ. Interestingly enough, in the first case, the estimator of ρ is the ratio of the sample arithmetic means, while in the other case, it turns out as the ratio of the sample geometric means. A different estimator emerges when one uses a power-dosage (as is common in slope-ratio assays). Thus, in general, these estimators are not the same, and they depend sensibly on the choice of a dosage. This explains the lack of invariance property of such parametric estimates (as well as associated test statistics) under monotone dosage transformations. From an operational point of view, an experimenter may not have the knowledge of the precise dosage, and hence, it may not be very prudent to assume the normality, lognormality or some other specific form of the tolerance distribution. Therefore, it may be reasonable to expect that an estimator of the relative potency should not depend on the chosen dosage as long as the latter is strictly monotone. For example, if the true tolerance distribution is logistic while we assume it to be (log)normal, the sample estimator may not be unbiased and fully efficient. Even when the two tolerance distributions are taken as normal, the ratio of the sample means is not unbiased for ρ. In this respect, such parametric procedures for the estimation of the relative potency (or allied tests for the fundamental assumption) are

3

not so robust, and any particular choice of a dosage may not remain highly efficient over a class of such chosen tolerance distributions. Nonparametric procedures initiated by Sen 17–19 and followed further by Shorack 26, and Rao and Littell 14, among others, eliminate this arbitrariness of dosage selection and render robustness to a far greater extent. Basically, we may note that ranks are invariant under strictly monotone (not necessarily linear) transformations on the sample observations. As such, a test for the fundamental assumption in (1) based on appropriate rank statistic remains invariant under such transformations. Similarly, if an estimator of the relative potency is based on suitable rank statistics, it remains invariant under such strictly monotone dosage transformations. Both the Wilcoxon–Mann–Whitney twosample rank-sum test and the (Brown–Mood) median test statistics were incorporated by Sen 17 for deriving nonparametric estimators of relative potency, and they also provide distribution-free confidence intervals for the same parameter. If there are m observations X S1 , . . . , X Sm for the standard preparation and n observations X T1 , . . . , X Tn , for the test preparation, we define the differences Yij = XSi − XTj , for i = 1, . . . , m; j = 1, . . . , n.

(2)

We arrange the N(= mn) observations Y ij in ascending order of magnitude, and let Y˜ N bethe median of these N observations. If N is even, we take the average of the two central order statistics. Then Y˜ N isthe Wilcoxon score estimator of logρ, and it is a robust and efficient estimator of logρ. The estimator is invariant under any strictly monotone transformation on the dose. Similarly, the confidence interval for logρ can be obtained in terms of two specified order statistics of the Y ij , and this is a distribution-free and robust procedure. A similar procedure works out for the median procedure; for general rank statistics, generally, an iterative procedure is needed to solve for such robust R-estimators. Rao and Littell 14 incorporated the twosample Kolmogorov–Smirnov test statistics in the formulation of their estimator. For computational convenience, because of

4

BIOLOGICAL ASSAY, OVERVIEW

the invariance property, it is simpler to work with the log-dose dosage, and in that way, the estimators of the log-relative potency correspond to the classical rank estimators in the two-sample location model. These direct dilution assays require the measurement of the exact doses needed to produce the response; this may not be the case if there are some latent effects. For example, the time taken by the toxic preparation to traverse from the point of infusion to the heart multiplied by the infusion rate may account for such a latent effect. In general, the situation may be much more complex. This naturally affects the fundamental assumption in (1), and variations in the modeling and statistical analysis to accommodate such effects have been discussed in 6 and 17 in the parametric and nonparametric cases respectively. 2

INDIRECT DILUTION ASSAYS

As an example, consider two drugs, A and B, each administered at k(≥ 2) prefixed levels (doses) d1 , . . . , dk . Let X Si and Y Ti be the response variable for the standard and test preparation respectively. These drugs may not have the same chemical ingredients, and may not have the same dose levels. It is not necessary to have the same doses for both the preparations, but the modifications are rather straightforward, and hence we assume this congruence. We assume first that both X Si and Y Ti are continuous (and possibly nonnegative) random variables. Suppose further that there exist some dosage xi = ξ (di ), i = 1, . . . , k and response-metameter X* = g(X), Y* = g(Y), for some strictly monotone g(.), such that the two dosage–response regressions may be taken as linear, namely, that ∗ ∗ = αT + βT xi + eTi , XSi YTi

= αS + βS xi + eSi ,

(3)

for i = 1, . . . , k, where for statistical inferential purposes, certain distributional assumptions are needed for the error components eTi and eSi , i = 1, . . . , k. Generally, in the context of log-dose transformations, we have a parallel-line assay, while slope-ratio assays

arise typically for power transformations. Thus, in a parallel-line assay, the two dose– response regression lines are taken to be parallel, and further that the errors eTi and eSi have the same distribution (often taken as normal). In this setup, we have then β S = β T = β (unknown), while α T = α S + β log ρ, where ρ is the relative potency of the test preparation with respect to the standard one. This leads to the basic estimating function log ρ =

{αT − αS } , β

(4)

so that if the natural parameters β, α S and α T are estimated from the acquired bioassay dataset, statistical inference on logρ (and hence ρ) can be drawn in a standard fashion. For normally distributed errors, the whole set of observations pertains to a conventional linear model with a constraint on the two slopes β S , β T , so that the classical maximum likelihood estimators and allied likelihood ratio tests can be incorporated for drawing statistical conclusions on the relative potency or the fundamental assumption of parallelism of the two regression lines. However, the estimator of logρ involves the ratio of two normally distributed statistics, and hence, it may not be unbiased; moreover, generally, the classical Fieller’s theorem [6] is incorporated for constructing a confidence interval for logρ (and hence, ρ), and it is known that this may result in an inexact coverage probability. Because of this difference in setups (with that of the classical linear model), design aspects for such parallel-line assays need a more careful appraisal. For equispaced (log —)doses, a symmetric 2k-point design has optimal information contents, and are more popularly used in practice. We refer to 6 for a detailed study of such bioassay designs in a conventional normally distributed errors model. Two main sources of nonrobustness of such conventional inference procedures are the following: 1. Possible nonlinearity of the two regression lines (they may be parallel but yet curvilinear); 2. Possible nonnormality of the error distributions.

BIOLOGICAL ASSAY, OVERVIEW

On either count, the classical normal theory procedures may perform quite nonrobustly, and their (asymptotic) optimality properties may not hold even for minor departures from either postulation. However, if the two dose–response regressions (linear or not) are not parallel, the fundamental assumption of parallel-line assays is vitiated, and hence, statistical conclusions based on the assumed model may not be very precise. In a slope-ratio assay, the intercepts α S and α T are taken as the same, while the slopes β S and β T need not be the same and their ratio provides the specification of the relative potency ρ. In such slope-ratio assays, generally, a power transformation: dosage = (dose)λ , for some λ > 0 is used, and we have
ρ=

βT βS

1/λ ,

(5)

which is typically a nonlinear function of the two slopes β T and β S , and presumes the knowledge of λ. In such a case, the two error components may not have the same distribution even if they are normal. This results in a heteroscedastic linear model (unless ρ = 1), where the conventional linear estimators or allied tests may no longer possess validity and efficiency properties. Moreover, as ρ λ is a ratio of two slopes, its conventional estimator based on usual estimators of the two slopes is of the ratio-type. For such ratio-type estimators, again the well-known Fieller Theorem (6) is usually adopted to attach a confidence set to ρ or to test a suitable null hypothesis. Such statistical procedures may not have the exact properties for small to moderate sample sizes. Even for large sample sizes, they are usually highly nonrobust for departures from the model-based assumptions (i.e. linearity of regression, the fundamental assumption, and normality of the errors). Again the design aspects for such slope-ratio assays need a careful study, and (6) contains a detailed account of this study. Because of the common intercept, usually a 2k + 1 point design, for some nonnegative integer k is advocated here. The articles on Parallel-line Assay and Slope-ratio Assay should be consulted for further details. The primary emphasis in these articles is on standard parametric

5

methods, and hence we discuss briefly here, the complementary developments of nonparametric and robust procedures for such assays. These were initiated in (21, 22) and also systematically reviewed in (23). First, we consider a nonparametric test for the validity of the fundamental assumption in a parallelline assay. This is essentially a test for the equality of slopes of two regression lines, and as in (21), we consider an aligned test based on the Kendall τ statistic. For each preparation with the set of dosages as independent variate and responses as dependent variable, one can define the Kendall tau statistic in the usual manner. We consider the aligned observations Y Ti − bxi and xi , and denote the corresponding Kendall τ (in the summation but not average form) as K T (b), and for the standard preparation, an aligned Kendall’s τ statistic is defined by K S (b), where we allow b to vary over the entire real line. Let then K ∗ (b) = KT (b) + KS (b), −∞ < b < ∞. (6) Note then that K T (b), K S (b), and hence K*(b) are all nonincreasing in b and have finitely many step-down discontinuities. Equating K*(b) to 0 (20), we obtain the pooled estimator ˆ βofβ. Let us then write L=

ˆ 2 + [KS (β)] ˆ 2} {[KT (β)] , Vn

(7)

where V n is the variance of the Kendall τ statistic under the hypothesis of no regression (and is a known quantity). This statistic has, under the hypothesis of homogeneity of β T and β S , closely central chi-square distribution with one degree of freedom. Lisused as a suitable test statistic for testing the validity of the fundamental assumption of a parallel-line assay where the normality of the error components is not that crucial. In that sense it is a robust test. Moreover, ˆ having obtained the pooled estimator βofβ, under the hypothesis of homogeneity of the slopes, we consider the residuals ˆ i , Yˆ Si = YSi − βx ˆ i, Yˆ Ti = YTi − βx

(8)

for different i, and treating them as two independent samples, as in the case of dilution direct assays, we use the

6

BIOLOGICAL ASSAY, OVERVIEW

Figure 1. Graphical procedure for obtaining a nonparametric confidence interval for the log potency ratio in a parallel-line assay

parameters, and obtain similar robust estimation and testing procedures. It is also possible to use general (aligned) M-statistics for this purpose. In general, such solutions are to be obtained by iterative methods, and hence, for simplicity and computational ease, we prescribe the use of the Kendall tau and twosample rank- sum statistics for the desired statistical inference. Next, we proceed to the case of sloperatio assays, and consider first a nonparametric test for the validity of the fundamental assumption (of a common intercept but possibly different slopes). We define the Kendall tau statistics K T (b) and K S (b) as in the case of the parallel-line assay, and equating them to 0, we obtain the corresponding estimates of β T and β S , which are denoted by βˆT and βˆS respectively. Consider then the residuals Y˜ Ti = YTi − βˆT xi , Y˜ Si = YSi − βˆS xi , ∀i. (9)

Wilcoxon–Mann–Whitney rank-sum test statistic to estimate the difference of the intercepts α T − α S in a robust manner. As in the direct dilution assay, this estimator is the median of the differences of all possible pairs of residuals from the test and standard preparation respectively. A robust, consistent and asymptotically normally distributed estimator of logρ is then obtained by dividing this estimator by the pooled estimaˆ tor β. For drawing a confidence interval for logρ (and hence, ρ), we can then use the Fieller Theorem by an appeal to the asymptotic normality of the estimator, or as in (21), consider a rectangular confidence set for β and α T − α S by computing a coordinate-wise confidence interval for each with coverage probability 1 − γ /2, and as in Figure 1, draw a robust confidence set for logρ with coverage probability 1 − γ. Though this does not have an exact coverage probability, it is quite robust and works out well even for quite nonnormal error distributions. In the above setup, instead of the Kendall τ and the two-sample rank-sum statistics, we may use a general linear rank statistic for regression and a two-sample linear rank statistic for difference of location

We pool all these residuals into a combined set, and use the signed-rank statistic to derive the corresponding rank estimator of the hypothesized common value of the intercept; this estimator, denoted by α,isthe ˜ median of all possible midranges of the set of residuals listed above. Let then Yˆ Ti = ˜ Yˆ Si = Y˜ Si − α, ˜ ∀i,and for each prepaY˜ Ti − α, ration, based on these residuals, we consider the Wilcoxon signed-rank statistic. These are ˆ S respectively. As in the ˆ T and W denoted by W case of parallel-line assays, here we consider a test statistic for testing the validity of the fundamental assumption as L=

ˆ 2} ˆ 2 +W {W T S , Vn

(10)

where V n is the variance of Wilcoxon signedrank statistic under the hypothesis of symmetry of the distribution around 0 (and is a known quantity). When the fundamental assumption holds, the distribution of L isclose to the central chi-square distribution with 1 degree of freedom, and hence a test can be carried out using the percentile point of this chi-square law. This test is quite robust and the underlying normality of the errors may not be that crucial in this context. Note that for the slope-ratio assay, granted the

BIOLOGICAL ASSAY, OVERVIEW

fundamental assumption of a common intercept, a natural plug-in estimator of ρ is given by  ρˆ =

βˆT βˆS

1/λ .

(11)

We may use the Fieller Theorem under an asymptotic setup to construct a confidence interval for ρ. Alternatively, as in the case of a parallel line assay, for a given γ (0 < γ < 1), we may consider a distribution-free confidence interval of coverage probability 1 − γ /2 for each of the two slopes β T and β S , and obtain a confidence interval for ρ λ (and hence ρ). The situation is quite comparable to the Figure for the parallel-line assay, excepting that β T and β S are taken for the two axes. Here also, instead of the Kendall tau statistics and the Wilcoxon signed-rank statistics, general regression rank statistics and (aligned) signed-rank statistics (or even suitable M-statistics) can be used to retain robustness of the procedures without sacrificing much efficiency. However, the solutions are generally to be obtained by iterative procedures, and hence, we prefer to use the simpler procedures considered above. 3

INDIRECT QUANTAL ASSAYS

In this type of (indirect) assays, the response is quantal (i.e. all or nothing) in nature. For each preparation (T or S) and at each level of administered dose, among the subjects, a certain number manifest the response while the others do not; these frequencies are stochastic in nature and their distribution depends on the dose level and the preparation. Thus, for a given dosage x, we denote by F T (x) and F S (x) the probability of the response for the test and standard preparation respectively. It is customary to assume that both F T (x) and F S (x) are monotone increasing in x, and for each α(0 < α < 1), there exits unique solutions of the following FT (ξTα ) = α,

and FS (ξSα ) = α,

(12)

so that ξ Tα and ξ Sα are the α-quantile of the test and standard preparation; they are

7

termed the 100α% effective dosage. In particular, for α = 1/2, they are termed the median effective dosage. Whenever the response relates to death (as is usually the case with animal and toxicologic studies), the ξTα , ξSα are also termed 100α%-lethal dosage. In many studies, generally, low dosages are contemplated so that α is chosen to be small. This is particularly the case with radioimmunoassays, and we shall comment on that later on. Estimation of the ξ Tα and ξ Sα with due attention to their interrelations is the main task in a quantal assay. The concept of parallel-line and slope-ratio assays, as laid down for indirect quantitative assays, is also adoptable in quantal assays, and a detailed account of the parametric theory based on normal, lognormal, logistic, and other notable forms of the distribution F T (x) is available with Finney [6, Chapter 17]. In this context, the probit and logit analyses are particularly notable, and we shall discuss them as well. To set the ideas, we consider a single preparation at k(≥ 2)- specified dosage d1 , . . . ,dk , where d1 < d2 < . . . < dk . Suppose that the dosage di has been administered to ni subjects, out of which ri respond positively while the remaining ni − ri do not, for i = 1, . . . , k. In this setup, the di , ni are nonstochastic, while the ri are random. The probability of a positive response at dosage di , denoted by π (di ), is then expressed as π (di ) = π (θ + βdi ),

i = 1, . . . , k,

(13)

where θ and β are unknown (intercept and regression) parameters, and π (x), −∞ < x < ∞, is a suitable distribution function. In a parametric mold, the functional form of π (.) is assumed to be given, while in nonparametrics, no such specific assumption is made. Note that the joint probability law of r1 , . . . , rk is given by k    ni i=1

ri

π (θ + βdi )ri [1 − π (θ + βdi )]ni −ri , (14)

so that the likelihood function involves only two unknown parameters θ and β. The loglikelihood function or the corresponding estimating equations are not linear in the parameters, and this results in methodological as well as computational complications.

8

BIOLOGICAL ASSAY, OVERVIEW

If π (.) is taken as a logistic distribution, that is, π (x) = {1 + e−x }−1 , then we have from the above discussion 
π (di ) = θ + βdi , i = 1, . . . , k. log [1 − π (di )] (15) This transformation, known as the logit transformation, relates to a linear regression on the dosage, and simplifies related statistical analysis schemes. Thus, at least intuitively, we may consider the sample logits
 ri − ri ) , i = 1, . . . , k, (16) Zi = log ni and attempt to fit a linear regression of the Zi on di . In passing, we may remark that technically ri could be equal to zero or ni (with a positive probability), so that Zi would assume the values − ∞ and +∞ with a positive probability, albeit for large ni , this probability converges to zero very fast. As in practice, the ni may not be all large; to eliminate this impasse, we consider the Anscombe correction to a binomial variable, and in (16), modify the Zi as   (ri + 38 ) Zi = log , i = 1, . . . , k. (17) (ni − ri + 38 ) Though the ri have binomial distributions, the Zi have more complex probability laws, and computation of their exact mean, variance, and so on, is generally highly involved. For large values of the ni , we have the following √

ni (Zi − θ − βdi )

D

→ N(0, {π (di )[1 − π (di )]}−1 ),

(18)

for each i = 1, . . . , k, where the unknown π (di ) can be consistently estimated by the sample proportion pi = ri /ni . Thus, using the classical weighted least squares estimation (WLSE) methodology, we may consider the quadratic norm Q(θ , β) =

k  i=1

and minimize this with respect to θ , β to obtain the WLS estimators. Although the logit transformation brings the relevance of generalized linear models (GLM), the unknown nature of their variance functions makes the WLSE approach more appropriate for the suggested statistical analysis. In any case, the asymptotic flavor should not be overlooked. If π (x) is taken as the standard normal distribution function (x), whose density function is denoted by φ(x), then we may consider the transformation   ri , i = 1, . . . , k, (20) Zi = −1 ni known as the probit or normit transformation. Here also, it would be better to modify Zi as Zi =

−1

(ni + 12 )

,

i = 1, . . . , k. (21)

Note that by assumption, −1 (π (di )) = θ + βdi , i = 1, . . . , k, and this provides the intuitive appeal for a conventional linear regression analysis. However, the likelihood approach based on the product-binomial law encounters computational difficulties and loses its exactness of distribution theory to a greater extent. Here also, we would have complications in the computation of the exact mean, variance, or distribution of the Zi , and hence, as in the logit model, we consider a WLSE approach in an asymptotic setup where the ni are large. By virtue of √ ni (pi − the asymptotic normality of the π (di ))(where again we take pi = (ri + 3/8)/(ni + 1/2)), we obtain that for every i ≥ 1, √

ni [Zi − θ − βdi ]

D

→ N(0,

π (di )[1 − π (di )] ), φ 2 ( −1 (π (di )))

(22)

so that we consider quadratic norm in a WLSE formulation Q(θ , β) =

ni pi (1 − pi ){Zi − θ − βdi }2 , (19)

(ri + 38 )

k  ni φ 2 ( −1 (pi )) i=1

pi (1 − pi )

[Zi − θ − βdi ]2 , (23)

BIOLOGICAL ASSAY, OVERVIEW

and minimizing this with respect to θ , β, we arrive at the desired estimators. For both the logit and probit models, the resulting estimators of θ , β are linear functions of the Zi with coefficients depending on the ni and the pi . Therefore, the asymptotic normality and other properties follow by standard statistical methodology. Moreover the (asymptotic) dispersion matrix of these estimators, in either setup, can be consistently estimated from the observational data sets. Thus, we have the access to incorporate standard asymptotics to draw statistical conclusions based on these estimators. Let us then consider the case of quantal bioassays involving two preparations (S and T), and for each preparation, we have a setup similar to the single preparation case treated above. The related parameters are denoted by θ S , β S and θ T , β T respectively, and for modeling the response distributions, we may consider either the logit or probit model, as has been discussed earlier. If we have a parallel-line assay, as in the case of an indirect assay, we have then βT = βS = β unknown, and θT − θs = β log ρ, (24) so that based on the estimates θˆS , βˆS , θˆT and βˆT ,along with their estimated dispersion matrix, we can incorporate the WLSE to estimate the common slope β and the intercepts θ S and θ T . The rest of the statistical analysis is similar to the case of indirect assays. Moreover, this WLSE methodology is asymptotically equivalent to the classical likelihood-function–based methodology, so it can be regarded, computationally, as a simpler substitute for a comparatively complicated one. For a slope-ratio assay, we have similarly a common intercept while the ratio of the slopes provide the measure of the relative potency, and hence, the WLSE based on the individual preparation estimators can be adopted under this restriction to carryout the statistical analysis as in the case of an indirect assay. Besides the logit or probit method, there are some other quasi-nonparametric methods, of rather an ad hoc nature, and among these, we may mention of the following estimators of the median effective dosage:

9

¨ 1. The Spearman–Karber estimator; 2. The Reed–Muench estimator, and 3. The Dragstedt–Behrens estimator. These procedures are discussed in (7), p. 43. If the tolerance distribution is symmetric, ¨ the Spearman–Karber estimator estimates the median effective dosage closely; otherwise, it may estimate some other characteristic of this distribution. Miller 13 studied the relative (asymptotic) performance of these three estimators, casting light on their bias terms as well. From a practical point of view, none of these estimators appears to be very suitable. Rather, if the π (di ) do not belong to the extreme tails (i.e. are not too small or close to 1), the logit transformation provides a robust and computationally simpler alternative, and is being used more and more in statistical applications. In passing, we may remark that Finney [7, Chapter 10] contains some other techniques that incorporate modifications in the setup of usual quantal assays, such as the numbers ni being unknown and possibly random, multiple (instead of binary) classifications, errors in the doses. In the following chapter, he also introduced the case of doses in mixtures that require a somewhat extended model and more complex statistical designs and analysis schemes. We shall comment on these below. 4 STOCHASTIC APPROXIMATION IN BIOASSAY In the context of a quantal assay, we have the dosage-response model in terms of the tolerance distribution π (d), and the median effective (lethal) dosage, LD50, is defined by the implicit equation π (LD50) = 0.50. In this context, for each preparation (standard or test), corresponding to initial dosage levels d1 , . . . , dk , we have estimates p(d1 ), . . . , p(dk ) of the unknown π (d1 ), . . . , π (dk ). We may set pi = π (di ) + e(di ), i = 1, . . . , k,

(25)

where the errors are (for large ni , the number of subjects treated) closely normally distributed with zero mean and variance n−1 i π (di )[1 − π (di )].Onthe basis of this initial response data, we can choose an appropriate

10

BIOLOGICAL ASSAY, OVERVIEW

do for which the corresponding p(do ) is closest to 1/2. Then, we let d(1) = do + ao [p(do ) − 1/2], for some ao > 0, and recursively we set  1 , d(j+1) = d(j) + aj p(d(j) ) − 2 for some aj > 0; j ≥ 0. (26) The aim of this stochastic approximation procedure, due to Robbins and Monro 15, is to estimate the LD50 without making an explicit assumption on the form of the tolerance distribution π (d). But in this setup, the p(d(j) ) as well as the d(j) are stochastic elements, and for the convergence of this stochastic iteration procedure, naturally, some regularity conditions are needed on the {ai ;i ≥ 0} and π (d) around the LD50. First of all, in order that the iteration scheme terminates with a consistent estimator of the LD50, it is necessary that the ai converge to zero as i increases. More precisely, it is assumed in this context that   an diverges to + ∞, but a2n < +∞. n≥0

n≥0

in many toxicologic studies where a turn occurs at an unknown level. Often this is treated in a general change-point model framework. The main advantage of the stochastic approximation approach over the classical quantal assay approach is that no specific assumption is generally needed on the probability function π (d), so that the derived statistical conclusions remain applicable in a much wider setup. On the other hand, the stochastic iteration scheme generally entails a larger number of subjects on which to administer the study, and often that may run contrary to the practicable experimental setups, especially with respect to cost considerations. In this general context, a significant amount of methodological research work has been carried out during the past 40 years, and an extensive review of the literature on stochastic approximation is made by Ruppert (16) where the relevant bibliography has also been cited. The scope of stochastic approximation schemes is by no means confined to quantal assays; they are also usable for quantitative bioassays, and even to other problems cropping up in far more general setups.

(27) In addition, the continuity and positivity of the density function corresponding to the distribution function π (x) at the population LD50 is also a part of the regularity assumptions. Further assumptions are needed to provide suitable (stochastic) rates of convergence of the estimator of the LD50 and its asymptotic normality and related large sample distributional properties. Once the LD50 values are estimated for each preparation, we may proceed as in the case of a quantal assay, and draw conclusions about the relative potency and other related characteristics. It is not necessary to confine attention specifically to the LD50, and any LD100α, for α ∈ (0, 1) can be treated in a similar fashion. In fact, Kiefer and Wolfowitz (12) considered an extension of the Robbins–Monro stochastic approximation procedure that is aimed to locate the maximum (or minimum) of a dose–response function that is not necessarily (piecewise or segmented) linear but is typically nonmonotone, admitting a unique extremum (maximum or minimum) of experimental importance. Such dose–response regressions arise

5 RADIOIMMUNOASSAY In radioimmunoassays antigens are labeled with radioisotopes, and in immunoradiometric assays antibodies are labeled. For a broad range of antigens, such radioligand assays enable the estimation of potency from very small quantities of materials and usually with high precision. Radioligand assays are based upon records of radiation counts in a fixed time at various doses, so that potency estimation involves the relation between counts of radioactivity and dose, generally both at low levels (8). In many such studies, the regression function of the count of radioactivity on dose has been found to be satisfactorily represented by a logistic curve; however, the lower and upper asymptotes of such a curve are not necessarily equal to zero and one, but are themselves unknown parameters. This difference with the classical logistic distribution is reflected in a somewhat different form of the variance function of radiation counts. Unlike the Poisson process, the variance function may not be

BIOLOGICAL ASSAY, OVERVIEW

equal to the mean level of the radiation counts U(d) (i.e. their expectation at a given dose level d); in many studies, it has been experimentally gathered that the variance function V(d) behaves like [U(d)]λ , where λ(>0) typically lies between 1 and 2. For this reason, the usual Poisson regression model in generalized linear models (GLM) methodology may not be universally applicable in radioimmunoassays. Moreover, such radioligand assays may not be regarded as strictly bioassays, since they may not depend upon responses measured in living organisms or tissues. However, the advent of the use of biologic markers in mutagenesis studies and in molecular genetics, particularly during the past 20 years, has extended the domain of statistical perspectives in radioligand assays to a much wider setup of investigations, and strengthened the structural similarities between radioimmunoassays and the classical bioassays. They involve statistical modeling and analysis schemes of very similar nature, and in this sense, their relevance in a broader setup of bioassays is quite appropriate. 6

DOSIMETRY AND BIOASSAY

As has been noted earlier, a dose–response model exhibits the (mathematical) relationship between an amount of exposure or treatment and the degree of a biological or health effect, generally a measure of an adverse outcome. Bioassay and clinical trials are generally used in such dose–response studies. With the recent advances in pharmacoepidemiology as well as in risk analysis, bioassays have led to another broader domain of statistical appraisal of biological dose–response studies, known as dosimetry (or animal study). Pharmacoepidemiology rests on the basic incorporation of pharmacodynamics (PD) and pharmacokinetics (PK) in the development of the so called structure– activity relationship information (SARI). Though a PD model directly relates to a dose–response model, the PK actions of the exposure or drug needs to be taken into account in the dose–response modeling. This is now done more in terms of SARI where the structure refers to the dose factors

11

and activity refers to the biological reactions that follow the exposure (dose) to a specific species or organism. In a majority of cases, the target population is human, but owing to various ethical and other experimental constraints, human beings may not be usable to the full extent needed for such a dose–response modeling. As such, animal studies are often used to gather good background information, which is intended for incorporation in human studies in bioassay and clinical trials. Dosimetry pertains to this objective. Dosimetry models intend to provide a general description of the uptake and distribution of inhaled (or ingested or absorbed) toxics (or compounds having adverse health effects) on the entire body system. For judgment on human population, such dosimetric models for animal studies need to be extrapolated with a good understanding of the interspecies differences. SARI is a vital component in enhancing such statistical validation of pooling the information from various animal studies and extrapolating to the human population. Most dose–response relationships are studied and through well-controlled animal bioassays with exposure or dose levels generally much higher than typically perceived in human risk analysis. In this respect, dosimetry is directly linked to bioassay, though in dosimetry, the SARI is more intensively pursued to facilitate extrapolation. PDPK aspects not only may vary considerably from subhuman primates to human beings, but also there is much less of control in human exposure to such toxics. Also, metabolism in the human being is generally quite different from that in subhuman primates. An important element in this context is the environmental burden of disease (EBD) factor that exhibits considerable interspecies variation as well as geopolitical variation. Hence, ignoring the SARI part, a conventional dose–response model for a subhuman primate may not be of much help in depicting a similar model for human exposure. For the same reason, conventional statistical extrapolation tools may be of very limited utility in this interspecies extrapolation problems (25). Finally, in many carcinogenicity studies, it has been observed that xenobiotic effects underlie such dose–response

12

BIOLOGICAL ASSAY, OVERVIEW

relations, and this is outlined in a later section. 7

SEMIPARAMETRICS IN BIOASSAYS

The GLM methodology has been incorporated in a broad variety of statistical modeling and analysis schemes pertaining to a wide range of applications, and bioassays are no exceptions. Going back to the direct dilution assays, if we had taken both the distributions, F S and F T , as exponentials with respective means µS and µT , then the two distributions would have constant hazard rates 1/µS and 1/µT respectively, so that the relative potency ρ is equal to the ratio of the two hazard rates. Inspired by this observation, and by the evolution of the Cox (3) proportional hazard model (PHM), research workers have attempted to relate the two survival functions SS (x) = P{X S > x} and ST (x) = P{X T > x} as ST (x) = [SS (x)]ρ , x ≥ 0,

(28)

and interpret ρ as the relative potency of the test preparation with respect to the standard one. Though this representation enables one to import the PHM-based statistical analysis tools for the estimation of the relative potency, for distributions other than the exponential ones, the interpretation of ‘‘dilution assays’’ may no longer be tenable under such a PHM. There is an alternative interpretation in terms of the parallelism of the two log-hazard functions, but that may not fit well with the fundamental assumption in dilution assays. For some related statistical analysis of bioassays based on GLM methodologies, we refer to (24), where indirect bioassays have also been treated in the same manner along with the classical parametrics. 8

NONPARAMETRICS IN BIOASSAYS

The estimators of relative potency and tests for fundamental assumptions in dilution (direct as well as indirect) assays based on rank statistics, considered earlier, spark the first incorporation of nonparametrics in biological assays. However, these may be

characterized more in terms of semiparametrics, in the sense that the assumed linearity of dose–response regressions was essentially parametric in nature, while the unknown form of the underlying tolerance distribution constitutes the nonparametric component. Thus, together they form the so-called semiparametric models. It is possible to incorporate more nonparametrics in bioassays mostly through the nonparametric regression approach. For direct dilution assays, such nonparametric procedures are quite simple in interpretation and actual formulation. We consider the log-dose transformation, so that the dosage for the test and standard preparations have the distributions FT∗ (x) and FS∗ (x), respectively, where FT∗ (x) = FS∗ (x + log ρ), forall x. If we denote the p-quantile of FT∗ and FS∗ byQT (p) and QS (p) respectively, then we have QS (p) − QT (p) = log ρ, ∀ p ∈ (0, 1), (29) so that the well-known Q–Q plot for the two preparations results in a linear regression form, and this provides the statistical information to test for this fundamental assumption as well as to estimate the relative potency. A similar conclusion can also be drawn from a conventional P–P plot. The classical Kolmogorov–Smirnov statistics (in the two-sample case) can be used for drawing statistical conclusions, and we may refer to Rao and Littell (14) for some related work. The situation is a bit more complex with indirect assays. In the classical parametric setup, we work with the expected response at different dosages, assuming of course a linear regression. In a semiparametric approach, this linearity of dosage-response regression is taken as a part of the basic assumption, but the distribution of the errors is allowed to be a member of a wider class, so that robust procedures based on rank or M-statistics are advocated instead of the classical WLSE. In a pure nonparametric setup, the linearity of the dosage-response regression is not taken for granted. Therefore the two dosage-response regression functions may be of quite arbitrary nature, and yet parallel in an interpretable manner. The statistical task is therefore to assess this parallelism without imposing linearity or some other parametric forms. Here

BIOLOGICAL ASSAY, OVERVIEW

also, at a given dosage level, instead of the mean response level, we may consider median or a p-quantile, and based on such robust estimators, we draw statistical conclusions allowing the quantile functions to be of a rather arbitrary nature. Asymptotics play a dominant role in this context, and often this may require a relatively much larger sample size. On the other hand, in terms of robustness and validity, such pure nonparametric procedures have a greater scope than parametric or semiparametric ones. 9 BIOAVAILABILITY AND BIOEQUIVALENCE MODELS As has been explained earlier bioequivalence trials differ from conventional bioassays, as here, generally, the active substances in the drug are the same but the differences in bioavailability reflect the form and manner of administration. Such alternative modes may therefore call for additional restraints in the statistical formulation, and because of anticipated biological equivalence, there is less emphasis on relative potency and more on general equivalence patterns. For such reasons, regulatory requirements for establishing average bioequivalence of two preparations (that are variations of an essentially common drug) relate to a verification of the following: A confidence interval for the relative potency, having the confidence limits ρ L , ρ U , lies between two specified endpoints, say ρ o < 1 < ρ o , with a high coverage probability (or confidence coefficient) γ . Generally, γ is chosen close to 1 (namely, 0.95), and also ρ 0 = (ρ o )−1 is very close to one. These requirements in turn entail a relatively large sample size, and therefore, (group) sequential testing procedures are sometimes advocated (9). For general considerations underlying such bioequivalence trials, we refer to (1,11,29), where other pertinent references are cited. Generally, such statistical formulations are more complex than the ones referred to earlier. As has been mentioned earlier, the term bioassay is used in a more general form, and this is equally true for bioequivalence and bioavailability models. Kinetic measures of

13

bioavailability and pharmacokinetic parameters have been developed to meet the demand for such recent usage. We will illustrate this somewhat differently with pharmacogenomics, which is revolutionizing the field of bioinformatics and experiments with biological units, in general. 10 PHARMACOGENOMICS IN MODERN BIOASSAYS Following Ewens and Grant (5), we take bioinformatics to mean the emerging field of science growing from the application of mathematics, statistics, and information technology, including computers and the theory surrounding them, to study and analysis of very large biological and, in particular, genetic data sets. Having its genesis 50 years ago (28), the field has been fueled by the immense increase in the DNA data generation. Earlier interpretation of bioinformatics with emphasis on computational biology by Waterman (27) also merits serious considerations, while Durbin et al. (4) had a view point geared by computer algorithms along with some heuristic usage of hidden Markov models. At the current stage, gene scientists cannot scramble fast enough to keep up with the genomics, with developments emerging at a furious rate and in astounding detail. Bioinformatics, at least at this stage, as a discipline, does not aim to lay down some fundamental mathematical laws (which might not even exist in such a biological diversity). However, its utility is perceived in the creation of innumerable computer graphics and algorithms that can be used to analyze exceedingly large data sets arising in bioinformatics. In this context, naturally data mining and statistical learning tools (under the terminology Knowledge Discovery and Data Mining (KDDM)) are commonly used (10), though often in a heuristic rather than objective manner. There could be some serious drawbacks of statistical analysis based on such KDDM algorithms alone, and model selection has emerged as a challenging task in bioinformatics. Given the current status of bioinformatics as the information technology (advanced computing) based discipline of analyzing

14

BIOLOGICAL ASSAY, OVERVIEW

exceedingly high dimensional data with special emphasis on genomics, and that genomics looks at the vast network of genes, over time, to determine how they interact, manipulate, and influence biological pathways, networks, as well as physiology, it is quite natural to heed to genetic variation (or polymorphism) in most studies involving biological units. Moreover, because of the drug-response relationship, basic in bioassay, it is natural to appraise the role of pharmacogenomics in this setup. Pharmacology is the science of drugs including materia medica, toxicology, and therapeutics, dealing with the properties and reactions of drugs, especially with relation to their therapeutic values. In the same vein, pharmacodynamics, a branch of pharmacology, deals with reactions between drugs and living structures; pharmacokinetics relates to the study of the bodily absorption, distribution, metabolism, and excretion of drugs. In bioequivalence trials, these tools have already been recognized as fundamental. Pharmacogenetics deals with genetic variation underlying differential response to drugs as well as drug metabolism. The whole complex constitutes the discipline: Pharmacogenomics. In the same way, Toxicogenomics relates to the study of gene-environmental interactions in disease and dysfunction to cast light on how genomes respond to environmental stress or toxics. It is conceived that there are certain genes that are associated with disease phenotype, side effects, and drug efficacy. Also, because of inherent (genetic) variations and an enormously large number of genes as well as a very large pool of diseases and disorders, there is a genuine need of statistical methods to assess the genetic mapping of disease genes. Pharmaco-toxicogenomics is therefore destined to play a fundamental role in biological assays, in the years to come. 11 COMPLEXITIES IN BIOASSAY MODELING AND ANALYSIS There are generally other sources of variations, which may invalidate the use of standard statistical analysis schemes in bioassays to a certain extent. Among these factors, special mention may be made of the following:

1. Censoring of various types, 2. Differentiable / Nondifferentiable measurement errors, 3. Stochastic compliance of dose, 4. Correlated multivariate responses, and 5. Curse of dimensionality in genomics. It is generally assumed that censoring is usually of Type I (truncation of the experiment at a prefixed timepoint), Type II (truncation following a certain prefixed number or proportion of responses), and random, where the censoring time and response time are assumed to be stochastically independent, and moreover, the censoring is assumed to be noninformative, so that the censoring time distribution remains the same for both the preparations. In actual practice, this may not be generally true, and hence, effects of departures from such assumptions on the validity and efficacy of standard statistical procedures are therefore needed to be assessed. Measurement of the actual dose levels in quantal assays, or the response levels in an indirect assay may often be impaired to a certain extent by measurement errors. In statistical analysis, usually such measurement errors are assumed to be either differentiable or nondifferentiable type, and appropriate statistical models and related analysis schemes depend on such assumptions. In radioimmunoassays, dosimetric studies in pharmacokinetics, as well as in other types, not the full amount of a prescribed dose may go into the organ or experimental unit, and the actual consumption of the dose may be (often, highly) stochastic in nature. Therefore, the dose–response regression relation may be subject to nonidentifiability and overdispersion effects. This calls for more modifications of existing models and analysis schemes. Finally, when there are multiple endpoints with possibly binary or polytomous responses, a dimension reduction for the model-based parameters becomes necessary from statistical modeling and inference perspectives. Otherwise, an enormously large sample size may be needed to handle adequately, the full parameter model, and this may run contrary to the practical setup of an assay. The situation is worse when some of the responses are

BIOLOGICAL ASSAY, OVERVIEW

quantitative while the others are quantal or at best polychotomous. These naturally introduce more model complexities and call for more complicated statistical analysis tools.

REFERENCES 1. Anderson, S. & Hauck, W. W. (1990) Considerations of individual bioequivalence, Journal of Pharmacokinetics and Biopharmaceutics 18, 259–273. 2. Bliss, C. I. (1952) The Statistics of Bioassay. Academic Press, New York. 3. Cox, D. R. (1972) Regression models and life tables (with discussion), Journal of the Royal Statistical Society B 34, 187–220. 4. Durbin, R., Eddy, S., Krogh, A. & Mitchison, G. (1998) Biological Sequence Analysis: Probabilistic Models for Proteins and Nucleic Acids. Cambridge University Press, Cambridge, UK. 5. Ewens, W. J. & Grant, G. R. (2001) Statistical Methods in Bioinformatics: An Introduction. Springer-Verlag, New York. 6. Finney, D. J. (1964) Statistical Methods in Biological Assay, 2nd Ed. Griffin, London. 7. Finney, D. J. (1971) Probit Analysis, 3rd ed. University Press, Cambridge. 8. Finney, D. J. (1976) Radioligand assay, Biometrics 32, 721–730. 9. Gould, A. L. (1995) Group sequential extensions of a standard bioequivalence testing procedure, Journal of Pharmacokinetics and Biopharmaceutics 23, 57–86. 10. Hastie, T., Tibshirani, R. & Friedman, J. (2001) The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer-Verlag, New York. 11. Hochberg, Y. (1955) On assessing multiple equivalences with reference to bioequivalence, in Statistical Theory and Applications: Papers in Honor of H. A. David., H. N. Nagaraja, P. K. Sen & D. F. Morrison, eds. Springer-Verlag, New York, pp. 265–278.

15

15. Robbins, H. & Monro, S. (1951) A stochastic approximation method, Annals of Mathematical Statistics 22, 400–407. 16. Ruppert, D. (1991) Stochastic approximation, in Handbook of Sequential Analysis, B. K. Ghosh & P. K. Sen, eds. Marcel Dekker, New York, pp. 503–529. 17. Sen, P. K. (1963) On the estimation of relative potency in dilution (-direct) assays by distribution-free methods, Biometrics 19, 532–552. 18. Sen, P. K. (1964) Tests for the validity of fundamental assumption in dilution (-direct) assays, Biometrics 20, 770–784. 19. Sen, P. K. (1965) Some further applications of nonparametric methods in dilution (-direct) assays, Biometrics 21, 799–810. 20. Sen, P. K. (1968) Estimates of the regression coefficient based on Kendall’s tau, Journal of the American Statistical Association 63, 1379–1389. 21. Sen, P. K. (1971) Robust statistical procedures in problems of linear regression with special reference to quantitative bioassays, I, International Statistical Review 39, 21–38. 22. Sen, P. K. (1972) Robust statistical procedures in problems of linear regression with special reference to quantitative bioassays, II, International Statistical Review 40, 161–172. 23. Sen, P. K. (1984) Nonparametric procedures for some miscellaneous problems, in Handbook of Statistics, Vol. 4: Nonparametric Methods, P. R. Krishnaiah & P. K. Sen, eds. Elsevier, Holland, pp. 699–739. 24. Sen, P. K. (1997) An appraisal of generalized linear models in biostatistical applications, Journal of Applied Statistical Sciences 5, 69–85. 25. Sen, P. K. (2003) Structure-activity relationship information in health related environmental risk assessment, Environmetrics 14, 223–234. 26. Shorack, G. R. (1966) Graphical procedures for using distribution-free methods in the estimation of relative potency in dilution (-direct) assays, Biometrics 22, 610–619.

12. Kiefer, J. & Wolfowitz, J. (1952) Stochastic estimation of the maximum of a regression function, Annals of Mathematical Statistics 23, 462–466.

27. Waterman, M. S. (1995) Introduction to Computational Biology: Maps, Sequences and Genomes. Chapman & Hall, Cambridge.

13. Miller, R. G., Jr. (1973) Nonparametric estimators of the mean tolerance in bioassay, Biometrika 60, 535–542.

28. Watson, J. D. & Crick, F. H. C. (1953) Genetical implications of the structure of deoxyribonucleic acid, Nature 171, 964–967.

14. Rao, P. V. & Littell, R. (1976) An estimator of relative potency, Communication of Statistics Series A 5, 183–189.

29. Westlake, W. J. (1988) Bioavailability and bioequivalence of pharmaceutical formulations, in Biopharmaceutical Statistics for

16

BIOLOGICAL ASSAY, OVERVIEW Drug Developments, K. E. Peace, ed. Marcel Dekker, New York, pp. 329–352.

FURTHER READING Cox, C. (1992) A GLM approach to quantal response models for mixtures, Biometrics 48, 911–928. Moses, L. E. (1965) Confidence limits from rank tests, Technometrics 7, 257–260. Sen, P. K. (2002) Bioinformatics: statistical perspectives and controversies, in Advances in Statistics, Combinatorics and Related Areas, C. Gulati & S. N. Mishra, eds. World Science Press, London, pp. 275–293.

BLOCKED RANDOMIZATION

groups, and similarly there is no control of the balance of allocation to groups over time. Consider the issue of overall imbalance. Let NA and NB represent the numbers of subjects randomized to treatments A and B by simple randomization, respectively. Then the difference NA minus NB is asymptotically normal with a zero mean and variance N = NA + NB . This property can be used to derive the distribution of the difference. The probability that the absolute value of the difference exceeds D is approximately calculated as 2(1–φ(D/n0.5 )) where φ is the cumulative standard normal distribution function. For example, in a trial of 100 subjects, the 95% confidence interval for the difference is calculated as ±19.6, which we round up to 20. Thus, there is a 5% chance of a split that equals or is more extreme than 40 versus 60 or 60 versus 40. Particularly as the sample size increases, the loss of study power that is a consequence of such imbalances is minimal, but other considerations apply. Concerns may include the following: risk of low power at interim analyses, which will have smaller sample sizes than the final; trial credibility to journal or regulatory reviewers; and limited trial supplies. Temporal bias may be caused by too many subjects being assigned to one group early in the study and too few at later time points. If the type of patient enrolled changes over time, then simple randomization may lead to imbalances in the baseline characteristics of the treatment groups. Such changes are entirely feasible and could be caused by investigators who wish to acquire experience of the treatment and trial before they enter the sicker subjects. Similarly, temporal trends in patient outcomes could be caused by the learning curve associated with the trial and treatments. Although theoretically the temporal trends can be accounted for in the analysis, it may not be simple in the typical multicenter trial that starts up centers at different times. For both of the above reasons simple randomization is rarely used.

DAMIAN McENTEGART Head of Statistics, Clinphone Group Ltd, Nottingham, United Kingdom

Randomization techniques can be classified as static, in that allocation is made from a randomization list that is generated before the start of the trial, or they can be dynamic, in that the allocation depends on the current balance of allocated treatments either overall or within particular sub-groups. This article deals with static randomizations from a list composed of one or more blocks. Blocked randomization is the most commonly used randomization technique as judged by reports in medical journals (1) and my own experience (80% of trials on our database of over 1500 randomized trials use blocked randomization). Most theory and examples presented will relate to the case of randomization into two treatment groups with a desired equal allocation. This method is for simplicity of exposition and everything can be generalized to cover multiple treatments and/or unequal randomization ratios. 1

SIMPLE RANDOMIZATION

To understand the rationale for blocked randomization and what it entails, it is first necessary to define simple randomization as a basis for comparison. Simple randomization (alternatively called complete or unconstrained randomization) occurs when each treatment assignment is made at random, and the allocation is completely independent of previous allocations. It has the attractive property of complete unpredictability. One method of constructing a list in a trial of two treatments would be to toss a coin for each allocation. Two disadvantages of simple randomization are that the randomization may yield an imbalance in the overall numbers of subjects assigned to the treatment

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

BLOCKED RANDOMIZATION

2 RESTRICTED RANDOMIZATION THROUGH THE USE OF BLOCKS The alternative to simple randomization is to impose some form of restriction on the randomization. Restricted randomization is defined as any procedure that deviates from simple randomization and controls the randomization to achieve balance between groups in terms of the overall size of the groups and/or their characteristics. The focus of this article is blocked randomization methods, which are used to balance treatment groups overall and, if needed, for time trends and prognostic factors. We will deal with two types of blocked randomization. The first instance is in which the block size or length equals the required sample size (i.e., there is one block for the entire trial). Abel (2) terms this complete balanced randomization. We will devote a section to such blocks. The second is random permuted blocks, in which several blocks make up the randomization list. For convenience, we deal with the second type of scheme first. 2.1 Random Permuted Blocks The permuted blocks procedure involves randomizing subjects to treatment by constructing a randomization list partitioned into blocks. In the simplest case of a constant block size with two treatments and m subjects per treatment in each block, then the block size is 2m. If the list is composed of B blocks, then the total number of entries on the list is 2mB. Allocation should be made sequentially from the list that corresponds to the order that subjects are randomized; otherwise the process is subject to subversion. Thus, the list is enforcing a perfect balance of treatment allocations after every 2m subjects have been recruited. Although this method protects against temporal trends, the constraint of return to periodic balance has the disadvantage that the allocations become increasingly predictable toward the end of each block if the past allocations and block size are known. It may lead to investigator selection bias where the investigator may decide not to enter particular subjects in the trial to avoid the receiving treatments that are not favoured by the investigator (e.g., the

control) or delay their entry until the chances of them receiving a particular treatment (e.g., the experimental treatment) are better than average. We will refer to the topic of selection bias again. Permuted blocks are used when there is a perceived need to protect against time trends in patient characteristics and/or outcomes. Even in the perceived absence of time trends, it is often advantageous to ensure a reasonable degree of overall treatment allocation balance during the trial in case of interim analyses. An additional motivation for permuted blocks may be the need to balance treatment groups within prognostic strata. 2.2 Generation of Blocks Two methods of generating blocks are used: the random allocation rule and the truncated binomial design. Given the block size 2m, the random allocation rule is to generate the randomization probability for each treatment in accordance with the totals of each treatment that still have to be assigned to meet the overall balance requirement. Thus, for example, if we are randomizing the fifth patient in a block of eight, and the allocations in the block to date have been three for treatment A and two for treatment B, then the patient will be randomized to receive treatment A with probability 1/3 and treatment B with probability 2/3. The random allocation rule is equivalent to sampling from the population of all possible randomization orderings with equal probability. If multiple blocks are used, then generally the blocks are generated independently (i.e., sampled with replacement). The alternative method of the truncated binomial design (3) is effected by using simple randomization until m of one of the treatments have been assigned, then assigning the other treatment to the remaining subjects. The chance of a long run of predictable allocations at the end of truncated binomial design blocks has precluded their use in practice, and they will not be referred to again. 2.3 Stratified Randomization Using Permuted Blocks In the absence of fraud or selection bias, there will be no systematic reason for treatment imbalances on prognostic factors using

BLOCKED RANDOMIZATION

simple or blocked randomization but chance imbalances are possible, especially in smaller trials. Kernan et al. (4) investigate the chances of an imbalance for two treatment groups on a binary prognostic factor that is present in 30% of the population. The chance that the two treatment group proportions will arithmetically differ by more than 10% is 43% for a trial of 30 subjects, 38% for a trial of 50 subjects, 27% for a trial of 100 subjects, 9% for a trial of 200 subjects, and 2% for a trial of 400 subjects. These results relate to a single factor; the chance of at least one imbalance on multiple factors is magnified. Chance imbalances on important prognostic variables reduce the precision of the treatment estimate, particularly for smaller trials. But if the prognostic variables are accounted for in the analysis model (poststratification), then the loss in precision for trials of more than 50 subjects per treatment group is slight (5). Of more importance may be the attitude of regulators to chance imbalances. European regulators require sensitivity analyses ‘‘to demonstrate that any observed positive treatment effect is not solely explained by imbalances at baselines in any of the covariates’’ and further note that in the case of a ‘‘very strong baseline imbalance, no adjustment may be sufficiently convincing to restore the results’’ (6). Thus trialists may wish to protect the credibility of their trial by balancing on a few important prognostic factors. A list of other potential reasons for achieving balance is given by McEntegart (5). With a randomization list, balancing takes place within individual strata defined as the crossclassification of prognostic factor levels. For continuous scale variables, this procedure will involve some categorization; methods for deciding on cut-points are available (7). As the number of balancing factors and levels increases, the number of strata increases geometrically. Regulatory guidance suggests that the use of more than two or three balancing factors is rarely necessary (8). If the randomization is to be stratified, then in the absence of temporal trends, a single block (complete balanced randomization) within each stratum would be a possibility in which the numbers of patient to be recruited in each stratum can be fixed in advance. Fixed recruitment, however, is often undesirable

3

because of the implications for the duration of trial recruitment and the extra administrative burden [although interactive voice response (IVR)/web randomization provides a solution where recruitment to each stratum is automatically closed once the stratum limit has been reached (9)]. Alternatively, where stratum sizes are not fixed in advance, permuted blocks will have to be used. Blocks are generated and assigned to each stratum with the effect that a separate randomization list is observed for each stratum. The allocation of blocks to strata is normally done at the time of generating the randomization list. If an IVR/web system is being used to perform the allocation during the trial, then it is not necessary, and allocation of blocks can be performed dynamically as they are needed. This method can be advantageous when center or country is stratification factors. Consider the example of stratification by gender and site. For any particular gender* site stratum, when the first subject with the relevant characteristics enrolls in the study, the system allocates the first free unallocated block of randomization code to that stratum. The current subject is then assigned the treatment associated with the first randomization code in the block, and the block is reserved for future subjects in this stratum. Subsequent subjects who enroll into this stratum are assigned to the next available randomization entry in the block. As a block is completed, then the next available block is reserved if another subject enrolls into that stratum. This allocation saves code that may have some small benefits in terms of checking, maintenance, and system performance. More importantly, it allows flexibility in site recruitment including easily allowing new, unplanned sites to join the study; the system simply allocates blocks of code that follow the same process. When using such a process however, it is important that the randomization (sequence) number is not revealed to the investigator, or else he can deduce the block size from the step change in the randomization numbers when allocation is made from a new block. If it is desired to reveal the randomization number, then a scrambled number that differs from the sequence number on the list should be used. Arguably using dynamic allocation

4

BLOCKED RANDOMIZATION

of blocks makes unscrupulous monitoring by the study team more difficult because it takes more work to deduce the blocks used within each strata. In a stratified design with K strata, then potentially K incomplete (open) blocks are at the end of the trial, each with a maximum imbalance of m and a maximum imbalance for the trial as a whole of Km. If too many strata are relative to the number of subjects and treatments, then the trial is subject to what has been called over stratification (10). Balance may not be achieved for the study as a whole or for the individual strata levels if multiple stratifying factors exist. Hallstrom and Davies (11) provide methodology and formulae to evaluate the risk of an aggregate imbalance combined over strata. For a twotreatment group trial, an approximate 95% confidence interval for the difference between the treatment group allocations can be calculated as ±1.96* (K* (2m + 1)/6)1/2 . Extensions to other numbers of treatment groups and unequal allocation ratios are possible or simulation may be used. Kernan et al. (4) suggest that to avoid the worst effects of incomplete blocks in trials with interim analyses, the number of strata for a two-treatment group trial is limited to the number of subjects divided by four times the block size. This method is to allow for interim analyses. If the number of strata is too large, then it may be necessary to use a dynamic randomization technique such as minimization (12) or dynamic balancing (13). The merits and issues of using such techniques are discussed elsewhere (1,5). If it is desired to balance over a stratification factor at the study level in a multicenter trial, then a central randomization using IVR/web randomization will have to be used. Country is often an appropriate factor in multinational trials (6). Finally, we note that the logistics of organizing medication supplies for stratified randomization in multicenter trials are considerably simpler in IVR/web systems that separate the randomization step from the dispensing step (9). In a traditional trial, supplies are provided to sites in patient numbered packs; normally the patient number will match the number used on the case record form used to

collect the patient data. If the trial is stratified by any factor in addition to site, then typically separate sets of patient numbered packs are reserved for the lists used in each stratum at the site. Unless the numbers to be recruited to each stratum at each site are fixed, which rarely happens as this would slow recruitment, then considerable wastage of patient supplies will occur, as packs are provided that allow for flexibility of strata composition in recruitment. 2.4 Multicenter Trials and Blocking To meet recruitment timelines and to aid generalization of results, most clinical trials are conducted at more than one center. Although a special case of stratification, multicenter trials deserve a separate discussion with regard to blocking because it is the most common use of blocked randomization. European regulatory guidance (6) notes that ‘‘most multicentre trials are stratified by centre (or investigator) for practical reasons or because centre (or investigator) is expected to be confounded with other known or unknown prognostic factors.’’ This regulation reflects the ICH E9 Guidance (8), which states that ‘‘It is advisable to have a separate random scheme for each centre, i.e. to stratify by centre or to allocate several whole blocks to each centre.’’ Most pharmaceutical industry trials are stratified by center, and it seems to be true of all trials. In a survey of trials reported in a 3 month period in 1997 in leading journals, Assmann et al. (14) found that 18 of 25 trials gave sufficient detail stratified by site. One reason for using some form of site stratified, blocked randomization in multicenter trials that are not managed electronically develops from the logistical considerations surrounding supplies and the uncertainty surrounding the recruitment process. One way of organizing the randomization would be to construct a single block of randomization codes equal to the trial sample size and sequentially send several corresponding patient numbered packs to each site—the exact number depending on each site’s agreed recruitment target. If all sites filled their target, then randomization is completely balanced at the study level. But sites

BLOCKED RANDOMIZATION

rarely meet their recruitment target with a consequent high risk of substantial imbalance at the study level. To avoid these problems, randomization for multicenter trials are usually balanced at the site level; this target is most commonly achieved via a blocked randomization, which also guards against temporal trends within each site as well as simplifying the logistics. One way of stratifying by site in a trial with randomization managed electronically is to construct the site stratified blocks dynamically by use of a balancing constraint. This method is a variant of the balanced block randomization scheme first described by Zelen (15). Consider a trial with five treatment groups where it is required to balance at the site level. In the dynamic scheme, subject to the balance constraint, subjects are allocated the treatment that corresponds to the next unused randomization code from a central unstratified randomization list with a block size of five. The balance constraint is that allocation must not cause the treatment imbalance (maximum minus minimum) at site to exceed one. In effect, blocks of size five are being used at the site level with the blocks being dynamically constructed considering the study level balance. A consequence of this method is that the codes on the central list are not always allocated sequentially. The advantage of the scheme is that it allows site balancing without the risk of imbalance at the study level from too many open blocks. It is particularly relevant in studies with many treatments conducted at many sites in which the average recruitment is expected to be less than the number of treatments. If site stratification is employed when it may not otherwise have been (because of the risk of study level imbalance), then a marked supply savings can occur if a system that automatically manages site inventories is also used (16). Assuming subjects enter in a random order, then all randomization sequences within any given center are possible before the trial starts, which is a perfectly valid randomization scheme, albeit that the blocks are not being selected independently and the sample is restricted (for instance, it would not be possible to have all centers commencing the first block with the same treatment). However, the scheme technically violates the

5

ICH E9 Guidance (8), which states that ‘‘The next subject to be randomized into a trial should always receive the treatment corresponding to the next free number in the appropriate randomization list.’’ Wittes (17) has argued that block sizes can be varied by clinic with expected higher recruiting centers having higher block sizes. This option is rarely used because of the problems of predicting the high recruiting clinics. It has, however, been implemented on an ongoing basis in the light of actual experience within an IVR system. In this scheme, the size of block that was dynamically allocated by the system was determined by the calculated recruitment status of the center. 3 SCHEMES USING A SINGLE BLOCK FOR THE WHOLE TRIAL If no stratification occurs, then the simplest way to overcome the problem of unequal allocation to groups is to restrict the randomization list so that the numbers in each group are equal (or in the required proportions). This function is equivalent to using a single block for the whole trial. 3.1 Maximal Procedure Abel (2) described an alternative randomization scheme that constrains the size of the within block imbalance as follows. One specifies a measure of imbalance, I , and a maximum permitted imbalance I0 ; the simplest imbalance measure is the range of treatment allocations. A randomization list of the required length n1 is then generated using simple randomization. Compute the imbalance In for each n ≤ n1 . If, for some n, In > I0 then replace the list. The process is repeated until an acceptable list is found. Berger et al. (18) extended Abel’s concept through the maximal procedure, which also imposes the condition of so called terminal balance on the list. Thus, the maximal procedure retains the desirable characteristics of the randomized permuted block design namely by minimizing imbalance and temporal bias. The permuted block procedure is characterized by its forced returns to perfect balance at the end of each block. For a given block size, the imbalance cannot

6

BLOCKED RANDOMIZATION

exceed half the block size, and this balance is called the maximum tolerated imbalance (MTI). Forced returns to periodic imbalance at the end of every block do not substantially help control temporal bias, yet they do potentially increase selection bias. The maximal procedure considers schedules that satisfy the constraint of never having an imbalance between treatments that exceeds the given MTI and also satisfy the constraint of perfect balance at the end of the sequence. The scheme is therefore less subject to selection bias at the cost of a very small increase in expected imbalance if the trial does not run to completion. Sequences are generated using the random allocation rule until one is found that satisfies the MTI conditions. The technique can be used in stratified trials, but unless the numbers in the strata are fixed, there is a slight increase in the risk of imbalance at the study level. This result occurs because of the slightly higher expected imbalance caused by stopping part way through the maximal block. Nevertheless, it seems a small price to pay, and the technique should be considered in any site stratified trial where the possibility of selection bias is a concern. 3.2 Hadamard Matrix Procedure The maximal scheme has some similarities to Bailey and Nelson’s method (19). This method of generating blocks involves selecting a block at random from an array derived from the permuted columns of a Hadamard matrix (20) with a selection criterion based on the MTI at any point in the block. For instance, with a block size of 12 for a two treatment trial, the example scheme used in the paper has an MTI of 3, and there are 44 possible blocks to choose from. If multiple blocks are selected with replacement, then the resultant scheme is valid in the sense that if terms for treatment and block are included in the analysis model, the model-based estimator of the error variance is also unbiased in that its expectation over the randomization is correct no matter what the order of patient presentation. If multiple blocks are used in an unstratified trial, then we are enforcing periodic returns to balance; thus, the similarity with the maximal procedure ends here. Furthermore, the Hadamard scheme is

less flexible than the maximal scheme, more prone to selection bias, more restrictive (for example for a trial size of 12 and an MTI of 3, there are 792 potential maximal sequences and only 44 in the Hadamard example), and harder to generate for block sizes larger than 12. It does, however, avoid the need for a permutation test in the analysis (see later). 4 USE OF UNEQUAL AND VARIABLE BLOCK SIZES The issue of predictability and selection bias is particularly relevant in site-stratified trials that are not blinded and where the investigator knows the past treatment allocations. One strategy that is sometimes used in an attempt to reduce potential selection bias is to employ a variable block design with some degree of randomness in the block sizes. The block sizes could be chosen at random from a specified subset of block sizes or alternatively be determined as a random ordering of a defined number of each possible block size. The advantage of the latter is that it is possible to fix the exact number of records in randomization list, which is not possible with the former scheme. This strategy is implicitly encouraged in the International Conference on Harmonisation (ICH) E9 Guidance on statistical principles for clinical trials (8) which states that: Care should be taken to choose block lengths that are sufficiently short to limit possible imbalance, but that are long enough to avoid predictability towards the end of the sequence in a block. Investigators and other relevant staff should generally be blind to the block length; the use of two or more block lengths, randomly selected for each block, can achieve the same purpose. (Theoretically, in a double-blind trial predictability does not matter, but the pharmacological effects of drugs may provide the opportunity for intelligent guesswork.)

The considerations that surround such a strategy depend on the assumed type of selection bias. Under the Blackwell-Hodges model (3), the experimenter wishes to bias the study by selecting a patient with a more favorable expected outcome when he guesses that treatment A is the next treatment to

BLOCKED RANDOMIZATION

be allocated. Conversely, the experimenter selects a patient with a poorer prognosis when he guesses that treatment B is the next treatment to be allocated. Clearly the optimal strategy for the investigator is to guess treatment A when it has been allocated least to date, to guess treatment B if it has been allocated least to date, and to guess with equal probability in the case of a tie in prior allocations. This method is called the convergence strategy. Under the Blackwell-Hodges model, the use of random block sizes does not reduce the potential for or degree of selection bias as the investigator’s strategy and expected successes remains the same even if he is unmasked to the block sizes. Rosenberger and Lachin (21) show that a design employing multiple block sizes has an expected bias factor associated with the average block size. So under the Blackwell Hodges model, no advantage exists to the use of random block sizes. Indeed if the MTI condition is not relaxed, then variable blocking will lead to even more scope for prediction than fixed size blocks because of the greater number of blocks and the associated return to balance on the completion of each block (18). But this argument is overstated, as one would not generally know the size of the current block. Furthermore, in my experience the MTI requirement is usually relaxed when variable blocking is used (e.g., practitioners might consider mixed blocks of 4 and 6 rather than fixed blocks of size 4). From the above, one might conclude that there is little advantage in employing variable sized blocks. But it would not be a complete representation of the situation. Arguably with fixed blocks, investigators often can guess the block size from customary practice based on logistical and balance considerations. For example, in a site stratified trial, supplies may be sent to sites in single blocks (22,23). Also, overall study level balance considerations may dictate against the use of larger block sizes; in a site stratified trial of two treatments, the block size is almost always chosen to be four and investigators are aware of this. In this case, the potential for selection bias is a concern as the investigator can determine the tail allocations of the block with complete certainty. Matts and Lachin (24) formally investigate

7

the bias properties in this situation and show that the expected number of treatments that are completely predictable within a block of known length 2m containing m allocations to each of two treatments is (m + 1) – 1 . They investigate the scenario in which the sequence of block sizes is masked but the sizes employed are unmasked. As might be expected, the use of variable block sizes will substantially reduce the potential for prediction with certainty in this scenario. Dupin-Spriet et al. (25) quantify the degree of prediction with certainty in trials with more than two arms and in trials with an unequal allocation ratio. For example in a three-arm trial with balanced blocks of size 6, 20% of treatments assignments are predictable. In a later paper (26), the same authors devise methods to quantify prediction with certainty in the cases of series with two and three unequal sized blocks for schemes that they claim are of a common size (e.g., a fixed number of subjects per center of 8, 12, or 30). For instance in the case of two blocks, the calculations will quantify the probability of identifying a long block when it comes before a short one if it starts with a sequence incompatible with the content of a short block. Both the situations of known block lengths as well as concealed order and concealed block lengths and order are considered. The results are compared against the predictability of the maximal method in the cases where the MTI is known and unknown. For a blocked scheme, if the details of the scheme (block lengths and order) can be assumed to be unknown, then there is actually no reduction in predictability with certainty when using unequal length blocks. The best scheme is always the one with the shortest possible block length as the unknown length order decreases opportunities for deductions about the composition of the blocks. But of course, it is precisely this scheme that is the most likely to be guessed. Furthermore, the scenarios of fixed recruitment per center are relatively uncommon. So although it is interesting to consult this reference when devising a scheme, the use of variable blocking should not be dismissed solely based on this reference.

8

BLOCKED RANDOMIZATION

In summary, the ICH suggestion to use blocks of unequal size is not as straightforward as it might first seem. But it is to be encouraged in open label trials (or blinded trials that are subject to guesswork) where for logistical or other reasons, it is felt necessary to stratify at the site level. It is most relevant when trial designers can be persuaded to use a higher MTI than they would otherwise have considered or where there is no requirement for a fixed number of randomization numbers at each site. In the absence of the latter requirement, block sizes can truly be chosen at random from the specified set of allowable block sizes, which provides less scope for deduction. 4.1 Mixed Randomization The notion of variable blocking has been extended by Schulz and Grimes (27), who recommend their method of mixed randomization for non-double-blind trials. The mixed randomization procedure begins with an uneven block generated with a maximum disparity between the treatments. This procedure is then followed by series of random permuted blocks with randomly varied block sizes interspersed with a simple random sequence. If used in site stratified multicenter trials, the sample size per center would have to be large to allow for the practicalities of the scheme; and more scope for study level imbalances exists compared with conventional site stratification. Nevertheless, this method adds another tool to the armoury of trying to confuse the issue and may be useful. The concept of uneven blocks at the start of the sequence is one I have used on occasion. 5 INFERENCE AND ANALYSIS FOLLOWING BLOCKED RANDOMIZATION The theory underpinning inferences from analyses following the use of blocked randomization is provided by Rosenberger and Lachin (21). On most occasions, a randomization model is used as the basis for inference. Permutation tests based on the randomization model assume that under the null hypothesis, the set of observed patient responses are fixed irrespective of treatment assignment. Then, the observed difference

between the treatment groups depends only on the treatment assigned, and statistical testing can be based around it. Treating the observed patient outcomes as given, the chosen test statistic is computed for all possible permutations of the randomization sequence. The incidence of observing a test statistic that is equal to or is more extreme than the observed test statistic is enumerated, and the probability is calculated by dividing by the total number of possible randomizations. This calculation is assumption free. As enumeration of all possible randomizations may be onerous as the sample size increases, an adequate alternative is to perform a Monte Carlo simulation of 10,000 or 100,000 randomization sequences. Actually, there is no need to perform such analyses following conventional blocked randomization because conventional tests, such as the block-stratified Mantel-Haenszel test, or analysis of variance fitting a factor for the block effect, will provide tests that are asymptotically equivalent to the permutation test (21). To ensure that tests have a true false positive error rate that is close to the nominal rate (e.g., 0.05), the analysis must account for the blocking and any stratification employed. Consider the case of blocking in an unstratified trial. Rosenberger and Lachin (21) show that the correct analysis accounts for the blocks and that the bias of an analysis without inclusion of the blocks depends on the direction of the intrablock correlation coefficient. In addition, an analysis that does not take the blocking into account is likely to be conservative. Omitting the blocks from analyses is common practice because most practitioners doubt the prognostic significance of the blocking factor, and the large number of extra parameters can cause instability and other model-fitting difficulties; this practice is acknowledged by the regulators (28). The situation regarding stratification factors is not treated so liberally, because although an analysis that does not reflect the stratification should again be conservative, there is no excuse for not performing the appropriate test as required under the assumptions of the randomization model. Thus, the ICH E9 Guidance (8) states that ‘‘if one or more factors are used to stratify the

BLOCKED RANDOMIZATION

design, it is appropriate to account for these factors in the analysis,’’ and the European guidance (6) states that, The primary analysis should reflect the restriction on the randomization implied by the stratification. For this reason, stratification variables—regardless of their prognostic value – should usually be included as covariates in the primary analysis.

The European guidance recognizes the problem of low recruiting centers, and if center was used in the randomization but not adjusted for in the analysis then sponsors ‘‘should explain why and demonstrate through well explained and justified sensitivity analyses, simulations, or other methods that the trial conclusions are not substantially affected because of this.’’ But generally the intention of the regulators is clear—you should ‘‘analyse as you randomize.’’ Appropriate theory has not been developed for the other single-block methods described. Thus, to use conventional asymptotic analysis, one must appeal to the homogeneity of the patient population. If one is not prepared to make this assumption, then Kalish and Begg (29) argue that large time trends would need to be observed before there was substantial distortion of the test characteristics; such large trends would be observable and can be accounted for in the model. This issue remains a debate. 6 MISCELLANEOUS TOPICS RELATED TO BLOCKED RANDOMIZATION 6.1 Blocking in Crossover and Group Randomized Designs In certain trials, subjects will be scheduled to receive the treatments in different orders. In these crossover trials, each subject serves as his own control provided that the results from each treatment administration are independent of the preceding allocations; arguably then no need exists to constrain the randomization in any way (i.e., a simple randomization can be used for the sequence of treatment administrations to each patient). But carryover of effects from the previous treatment can never be ruled out, so it is conventional to balance the treatment sequences (e.g., in a

9

two treatment, two period crossover trial, the two possible sequences are usually generated in blocks). In trials with three treatments, the six possible orderings are grouped in a block. In trials with more than three treatments, Williams squares are used as blocks that are balanced Latin square designs constructed so that each treatment is preceded equally often by each other treatment (30,31). Bellamy et al. (32) consider the situation of randomizing subjects in cohorts of a predetermined size to one of two study treatments using blocking factors, the composition of which cannot be determined prior to assembling the cohorts for randomization. This situation develops in trials in which subjects are to be randomized as a group, but not all the subjects attend for the visit. The randomization is effected by alternation into arbitrarily labelled groups within each stratification factor level and then random assignment of these groups to treatment. 6.2 Generation of Blocks: Software and Practical Considerations Most computer statistical packages include pseudo-random number generators, which are so called because the numbers they generate are not the result of a random process but have properties similar to those generated via a random process. These numbers can be used to prepare randomized blocks using the random allocation rule or by generating the block contents and then using the random number generator to assign block position. The SAS procedure PROC PLAN (SAS Institute, Inc., Cary, NC) provides a simple and flexible module for generating randomization lists; Deng and Graz (33) give some useful examples of generating blocked randomizations using this procedure. A variety of freely available software is available for generating blocked randomizations. A recent and flexible package that overcomes the lack of flexibility of some earlier software is described by Saghaei (34). It is important that the trial statistician reviews the generation process used for the randomization list before the trial. Ideally, the trial statistician should not generate the list. My recommended process is that the statistician reviews a ‘‘dummy list’’ that is

10

BLOCKED RANDOMIZATION

created by a validated software package that, based on a set of input parameters, simultaneously generates two lists using different seeds. The dummy list can be used for data set-up, testing, and quality control procedures. Procedures should be in place to ensure secure storage and appropriate limited access to the list after generation. Generally, blocks will be generated independently of one another, that is, they are sampled with replacement, as this method underpins the theory for statistical inference. Generally, there is no need to restrict the number of possible randomization orderings unnecessarily. But marginal advantages for overall treatment balance may be obtained by ensuring a balance of block types within the list for center stratified schemes where incomplete blocks can be expected. No real objection to this practice will occur provided the details are kept away from staff involved in the monitoring of the trial. 6.3 Blocked Randomization also Used for Drug Packaging

kept in separate documents or appendices that are not seen by staff involved with the trial performance or conduct. Good Clinical Practice (36) requires that the protocol contain ‘‘a description of the measures taken to minimize/avoid bias, including randomization and blinding,’’ but it is not necessary to go into specific detail about the type of blocking and block sizes. In contrast after the trial has completed, reports of the trial should contain full details of the randomization procedures. The CONSORT group statement (37) asks for details of the method used to generate the random allocation sequence and details of blocking and stratification. REFERENCES 1. N. W. Scott, G. C. McPherson, C. R. Ramsay, and M. K. Campbell, The method of minimization for allocation to clinical trials: a review. Control. Clin. Trials 2002; 23: 662–674. 2. U. Abel, Modified replacement randomization. Statist. Med. 1987; 6: 127–135.

Trial material packaging is also done using randomized blocks. When the patient number corresponds to the randomization sequence number as it does in a traditional trial, then only one scheme is needed. If, however, the randomization and material dispensing steps are separated, as is often the case with IVR and web trials (9), then separate randomizations will have to be performed for randomization and the packaging list. To avoid unblinding through logical deduction, it is important that after the randomization has identified the appropriate treatment to allocate, that selection from the set of packs available at the site is performed randomly (35). This function is best achieved by use of a double randomized pack list; this block randomized pack list is then scrambled to remove any association between pack number and sequence number (35).

3. D. Blackwell and J. L. Hodges, Design for the control of selection bias. Ann. Math. Stats. 1957; 28: 449–460.

6.4 Documentation and Reporting

8. International Conference on Harmonisation, E-9 Document, Guidance on statistical principles for clinical trials. Federal Register 1998; 63: 49583–49598. Available: http://www.fda.gov/cder/guidance/91698.pdf.

From the preceding discussions, it is clear that the more that can be kept from investigators and monitors, the better it is for study integrity. Thus, before and during the trial conduct, specifics of randomization should be

4. W. N. Kernan, C. M. Viscoli, R. W. Makuch, L. M. Brass, and R. I. Horwitz, Stratified randomization for clinical trials. J. Clin. Epidemiol. 1999; 52: 19–26. 5. D. J. McEntegart, The pursuit of balance using stratified and dynamic randomisation techniques. Drug. Inf. J. 2003; 37: 293–308. Available: http://www.clinphone.com/files/ Stratfied%20Dynamic%20Randomization% 20Techniques.pdf. 6. Committee for Proprietary Medicinal Products. Points to Consider on Adjustment for Baseline Covariates. 2003. CPMP/EWP/283/ 99 Available: http://www.emea.eu.int/pdfs/ human/ewp/286399en.pdf. 7. S. C. Choi, T. P. Germanson, and T. Y. Barnes, A simple measure in defining optimal strata in clinical trials. Control. Clin. Trials 1995; 16: 164–171.

9. B. Byrom, Using IVRS in clinical trial management. Appl. Clin. Trials 2002; 10: 36–42.

BLOCKED RANDOMIZATION http://www.clinphone.com/files/Using IVRS in trial%20management.pdf 10. R. Simon, Patient subsets and variation in therapeutic efficacy. Br. J. Clin. Pharmacol. 1982; 14: 473–482. 11. A. I. Hallstrom and K. Davis, Imbalance in treatment assignments in stratified blocked randomization. Control. Clin. Trials 1988; 9: 375–382. 12. S. J. Pocock and R. Simon, Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. Biometrics 1975; 31: 103–115. 13. S. Heritier, V. Gebski, P. Pillai, Dynamic balancing randomization in controlled clinical trials. Stat. Med. 2005; 24:3729-3741. 14. S. F. Assman, S. J. Pocock, L. E. Enos, L. E. Kasten, et al., Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet 2000; 355: 1064–1069. 15. M. Zelen, The randomization and stratification of patients to clinical trials. J. Chronic. Dis. 1974; 27: 365–375. 16. D. McEntegart and B. O’Gorman, Impact of supply logistics of different randomisation and medication management strategies used within IVRS. Pharmaceut. Engin. 2005; 36–46. Available: http://www. clinphone.com/files/ISPE%20paper1.pdf. 17. J. Wittes, Randomized treatment assignment. P Armitage, T Colton (eds). Encyclopedia of Biostatistics. John Wiley, 1998.:3703-3711 18. V. W. Berger, A. Ivanova, and M. Deloria Knoll, Minimizing predictability while retaining balance through the use of less restrictive randomization procedures. Stat. Med. 2003; 22: 3017–3028. 19. R. A. Bailey and P. R. Nelson, Hadamard randomisation: a valid restriction of random permuted blocks. Biomet. J. 2003; 45: 554–560. 20. A. S. Hedayat and W. D. Wallis, Hadamard matrices and their applications. Ann. Statist. 1978; 6: 1184–1238. 21. W. F. Rosenberger and J. M. Lachin, Randomization in Clinical Trials. New York: John Wiley & Sons, 2002. 22. S. J. Kunselman, T. J. Armstrong, T. B. Britton, and P. E. Forand, Implementing randomisation procedures in the asthma clinical research network. Control. Clin. Trials 2001: 22:181S–195S. 23. D. McEntegart, Implementing randomization procedures in the Asthma Clinical Research Network. Control. Clin. Trials 2002; 23: 424–426.

11

24. J. P. Matts and J. M. Lachin, Properties of permuted-block randomization in clinical trials. Control. Clin. Trials 1988; 9: 327–344. 25. T. Dupin-Spriet, J. Fermanian, and A. Spriet, Quantification of predictability in block randomization. Drug. Inf. J. 2004; 38: 135–141. 26. T. Dupin-Spriet, J. Fermanian, and A. Spriet, Quantification methods were developed for selection bias by predictability of allocations with unequal block randomization. J. Clin. Epidemiol. 2005; 58: 1269–1270. 27. K. F. Schulz and D. A. Grimes, Unequal group sizes in randomised trials: guarding against guessing. Lancet 2002; 359: 966–970. 28. J-M. Grouin, S. Day, Lewis, Adjustment for baseline covariates: an introductory note. J. Stat. Med. 2004; 23: 697–699. 29. L. A. Kalish and C. B. Begg, The impact of treatment allocation procedures on nominal significance levels and bias. Control. Clin. Trials 1987; 8: 121–135. 30. E. J. Williams, Experimental designs balanced for the estimation of residual effects of treatments. Australian J. Sci. Res. 1949; 2A: 149–168. 31. R. G. Newcombe, Sequentially balanced threesquares cross-over designs. Stat. Med. 1996; 15: 2143–2147. 32. S. L. Bellamy, A dynamic block-randomization algorithm for group-randomized clinical trials when the composition of blocking factor is not known in advance. Contemp. Clin. Trials 2005; 26: 469–479. 33. C. Deng and J. Graz, Generating randomisation schedules using SAS programming. SUGI 27, April 14-17 2002 (SAS Users Group) Paper 267-27. Available: http://www2. sas.com/proceedings/sugi27/p267-27.pdf. 34. M. Saghaei, Random allocation software for parallel group randomized trials. BMC Med. Res. Methodol. 2004; 4: 26. Available: http:// www.biomedcentral.com/1471-2288/4/26. 35. M. Lang, R. Wood, and D. McEntegart, Double-randomised Packaging Lists in Trials managed by IVRS. Good Clin. Pract. J. 2005; 10–13. Available: http://www.clinphone.com/ files/GCPJ%20article%20final%20Nov%2020 05.pdf. 36. International Conference on Harmonisation, E-6 Document. Good Clinical Practice. Federal Register 1997; 62: 25691–25709. Available: http://www.fda.gov/ cder/guidance/iche6.htm. 37. CONSORT Group, D. G. Altman, et al., The revised CONSORT statement for reporting

12

BLOCKED RANDOMIZATION randomised trials: explanation and elaboration. Ann. Intern. Med. 2001; 134: 663–694.

FURTHER READING V. W. Berger, Selection Bias and Covariate Imbalances in Clinical Trials. New York: John Wiley & Sons, 2005.

CROSS-REFERENCES Blocking Randomization Randomization Codes Randomization Methods Randomization Procedures Randomization List Simple Randomization Stratified Randomization Interactive Voice Randomization System (IVRS)

to x (the difference is −0.047, to three decimal places). The bootstrap distribution looks normal, with some skewness. This amount of skewness is a cause for concern. This example may be counter to the intuition of many readers, who use normal probability plots to look at data. This bootstrap distribution corresponds to a sampling distribution, not raw data. This distribution is after the central limit theorem has had its one chance to work, so any deviations from normality here may translate into errors in inferences. We may quantify how badly this amount of skewness affects confidence intervals; we defer this to the section on Bootstrap confidence intervals. We first discuss the idea behind the bootstrap, and give some idea of its versatility.

BOOTSTRAP TIM HESTERBERG Insightful Corp., Seattle, Washington

1

INTRODUCTION

We begin with an example of the simplest type of bootstrapping in this section, then discuss the idea behind the bootstrap, implementation by random sampling, using the bootstrap to estimate standard error and bias, the central limit theorem and different types of bootstraps, the accuracy of the bootstrap, confidence intervals, hypothesis tests, planning clinical trials, the number of bootstrap samples needed and ways to reduce this number, and we conclude with references for additional reading. Figure 1 shows a normal quantile plot of Arsenic concentrations from 271 wells in Bangladesh, from http://www.bgs.ac.uk/ arsenic/bangladesh/Data/SpecialStudyData. csv referenced from statlib http://lib.stat.cmu. edu/datasets. The sample mean and standard deviation are x = 124.5 and s = 298, respectively. √The usual formula standard error is s/ n = 18.1, and the√usual 95% confidence interval x ± tα/2,n−1 s/ n is (88.8, 160.2). This interval may be suspect because of the skewness of the data, despite the reasonably large sample size. We may use the bootstrap for inferences for the mean of this dataset. We draw a bootstrap sample, or resample, of size n with replacement from the data, and compute the mean. We repeat this process many times, say 104 or more. The resulting bootstrap means comprise the bootstrap distribution, which we use to estimate aspects of the sampling distribution for X. Figure 2 shows a histogram and normal quantile plot of the bootstrap distribution. The bootstrap standard error is the standard deviation of the bootstrap distribution; in this case the bootstrap standard error is 18.2, which is close to the formula standard error. The mean of the bootstrap means is 124.4, which is close

2

PLUG-IN PRINCIPLE

The idea behind the bootstrap is the plug-in principle (1)—that if a quantity is unknown, then we plug in an estimate for it. This principle is used all the time in statistics. The standard deviation of a sample mean for i.i.d. observations from a √ population with standard deviation σ is σ/ n; when σ is unknown, we plug in an estimate s to obtain √ the usual standard error s/ n. What is different in the bootstrap is that we plug in an estimate for whole population, not just for a numerical summary of the population. Statistical inference depends on the sampling distribution. The sampling distribution depends on the following: 1. the underlying population(s), 2. the sampling procedure, and 3. the statistic, such as X. Conceptually, the sampling distribution is the result of drawing many samples from the population and calculating the statistic for each. The bootstrap principle is to plug in an estimate for the population, then mimic the real-life sampling procedure and statistic calculation. The bootstrap distribu-

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

BOOTSTRAP

1500 1000 0

500

As (ug/L)

2000

2

−2

−1

0

1

2

3

Figure 1. Arsenic concentrations in 271 wells in Bangladesh.

0.025

Quantiles of Standard Normal

180 160 Mean

0.015

100

120

0.010 0.0

80

0.005

Density

0.020

200

Observed Mean

140

−3

80

100

120

140

160

180

200

Mean

−4

−2

0

2

4

Quantiles of standard normal

Figure 2. Bootstrap distribution for arsenic concentrations.

tion depends on 1. an estimate for the population(s), 2. the sampling procedure, and 3. the statistic, such as X. The simplest case is when the original data are an i.i.d. sample from a single population, and we use the empirical distributhe population, where tion Fˆ n to estimate
Fˆ n (u) = (1/n) I(xi ≤ u). This equation gives the ordinary nonparametric bootstrap, which corresponds to drawing samples of size n without replacement from the original data.

2.1 How Useful is the Bootstrap Distribution? A fundamental question is how well the bootstrap distribution approximates the sampling distribution. We discuss this question in greater detail in the section on Accuracy of bootstrap distribution, but note a few key points here. For most common estimators (statistics that are estimates of a population parameter, e.g., X is an estimator for µ, whereas a l statistic is not an estimator), and under fairly general distribution assumptions, center: the center of the bootstrap distribution is not an accurate

BOOTSTRAP

approximation for the center of the sampling distribution. For example, the center of the bootstrap distribution for X is centered at approximately x, the mean of the sample, whereas the sampling distribution is centered at µ. spread: the spread of the bootstrap distribution does reflect the spread of the sampling distribution. bias: the bootstrap bias estimate (see below) does reflect the bias of the sampling distribution. skewness: the skewness of the bootstrap distribution does reflect the skewness of the sampling distribution. The first point bears emphasis. It means that the bootstrap is not used to get better parameter estimates because the bootstrap distributions are centered around statistics θˆ calculated from the data (e.g., x or regression ˆ rather than the unknown populaslope β) tion values (e.g., µ or β). Drawing thousands of boot-strap observations from the original data is not like drawing observations from the underlying population, it does not create new data.

Instead, the bootstrap sampling is useful for quantifying the behavior of a parameter estimate, such as its standard error, bias, or calculating confidence intervals. Exceptions do exist where bootstrap averages are useful for estimation, such as random forests (2). These examples are beyond the scope of this article, except that we give a toy example to illustrate the mechanism. Consider the case of simple linear regression, and suppose that a strong linear relationship exists between y and x. However, instead of using linear regression, one uses a step function the data are split into eight equal-size groups based on x, and the y values in each group are averaged to obtain the altitude for the step. Applying the same procedure to bootstrap samples randomizes the location of the step edges, and averaging across the bootstrap samples smooths the edges of the steps. This method is shown in Fig. 3. A similar effect holds in random forests, using bootstrap averaging of tree models, which fit higher-dimensional data using multivariate analogs of step functions. 2.2 Other Population Estimates Other estimates of the population may be used. For example, if there was reason to assume that the arsenic data followed a gamma distribution, we could estimate parameters for the gamma distribution, then draw

step function bootstrap average

Figure 3. Step function defined by eight equalsize groups, and average across bootstrap samples of step functions.

3

4

BOOTSTRAP

samples from a gamma distribution with those estimated parameters. In other cases, we may believe that the underlying population is continuous; rather than draw from the discrete empirical distribution, we may instead draw samples from a density estimated from the data, say a kernel density estimate. We return to this point in the section entitled ‘‘Bootstrap distributions are too narrow.’’ 2.3 Other Sampling Procedures When the original data were not obtained using an i.i.d. sample, the bootstrap sampling should reflect the actual data collection. For example, in stratified sampling applications, the bootstrap sampling should be stratified. If the original data are dependent, the bootstrap sampling should reflect the dependence; this information may not be straightforward. Some cases exist where the bootstrap sampling should differ from the actual sampling procedure, including: • regression (see ‘‘Examples’’ section) • planning clinical trials (Section 9), (see

‘‘Planning Clinical trials’’ section) • hypothesis testing (see ‘‘Hypothesis test-

ing’’ section), and • small samples (see ‘‘Bootstrap distributions are too narrow’’) 2.4 Other Statistics The bootstrap procedure may be used with a wide variety of statistics mean, median, trimmed mean, regression coefficients, hazard ratio, x-intercept in a regression, and others—using the same procedure. It does not require problem-specific analytical calculations. This factor is a major advantage of the bootstrap. It allows statistical inferences such as confidence intervals to be calculated even for statistics for which there are no easy formulas. It offers hope of reforming statistical practice—away from simple but nonrobust estimators like a sample mean or least-squares regression estimate, in favor of robust alternatives.

3 MONTE CARLO SAMPLING—THE ‘‘SECOND BOOTSTRAP PRINCIPLE’’ The second bootstrap ‘‘principle’’ is that the bootstrap is implemented by random sampling. This aspect is not actually a principle, but an implementation detail. Given that we are drawing i.i.d. samples of size n from the empirical distribution Fˆ n , there are at most nn possible samples. In small samples, we could create all possible bootstrap samples, deterministically. In practice, n is usually too large for that to be feasible, so we use random sampling. Let B be the number of bootstrap samples used (e.g., B = 104 ). The resulting B statistic values represent a random sample of size B with replacement from the theoretical bootstrap distribution that consists of nn values (including ties). In some cases, we can calculate the theoretical bootstrap distribution without simulation. In the arsenic example, parametric bootstrapping from a gamma distribution causes the theoretical bootstrap distribution for the sample mean to be another gamma distribution. In other cases, we can calculate some aspects of the sampling distribution without simulation. In the case of the nonparametric bootstrap when the statistic is the sample mean, the mean and standard deviation of the theoretical bootstrap distribution are x √ and σˆ Fˆ n / n, respectively, where σˆ F2ˆ = n−1 n



(xi − x)2 .

Note that this differs from the usual sample standard deviation in using a divisor of n instead of n − 1. We return to this point in the section ‘‘Bootstrap distribution are too narrow.’’ The use of Monte Carlo sampling adds additional unwanted variability, which may be reduced by increasing the value of B. We discuss how large B should be in the section ‘‘How many bootstrap samples are needed.’’ 4 BIAS AND STANDARD ERROR Let θ = θ (F) be a parameter of a population, such as the mean, or difference in regression

BOOTSTRAP

coefficients between subpopulations. Let θˆ be the corresponding estimate from the data, θˆ ∗ be the estimate from a bootstrap sample, ∗

θˆ = B−1

B 

ˆ EFˆ (θˆ ∗ ) − θ (F)

(2)

The Monte Carlo version in Equation (1) substitutes the sample average of bootstrap statistics for the expected value.

be average of B bootstrap estimates, and s2θˆ ∗ = (B − 1)−1

for population F. Substituting Fˆ for the unknown F in both terms yields the theoretical bootstrap analog

θˆb∗

b=1

5

B  ∗ (θˆb∗ − θˆ )2

5

EXAMPLES

b=1

the sample standard deviation of the bootstrap estimates. Some bootstrap calculations require that θˆ be a functional statistic, which is one that depends on the data only through the empirical distribution, not on n. A mean is a functional statistic, whereas the usual sample standard deviation s with divisor n − 1 is not—repeating each observation twice gives the same empirical distribution but a different s. The bootstrap bias estimate for a functional statistic is ∗

θˆ − θˆ

(1)

Relative risk bootstrap CI t CI

0.0

Proportion in high risk group

0.6 0.4 0.2

An outlier at 10.1 is omitted. Other large observations are indicated below.

0.020

0.030

A major study of the association between blood pressure and cardiovascular disease found that 55 of 3338 men with high blood pressure died of cardiovascular disease during the study period, compared with 21 out of 2676 patients with low blood pressure. The estimated relative risk is θˆ = pˆ 1 /pˆ 2 = 0.0165/0.0078 = 2.12. To bootstrap this, we draw samples of size n1 = 3338 with replacement from the first group, independently draw samples of size n2 = 2676 from the second group, and calculate the relative risk θˆ ∗ . In addition, we record the individual proportions pˆ ∗1 and pˆ ∗2 . The bootstrap distribution for relative risk is shown in the left panel of Fig. 4. It is

Observed Mean

0.0

Density

5.1 Relative Risk

0.010

0.8

Note how this example relates to the plugˆ − in principle. The bias of a statistic is E(θ) θ , which for a functional statistic may be ˆ − θ (F), the expected value expanded as EF (θ) ˆ of θ when sampling from F minus the value

In this section, we consider some examples, with a particular eye to standard error, bias, and normality of the sampling distribution.

1

2

3 4 5 Relative Risk

6

7

0.0

0.005

0.010

Proportion in low risk group Slope = Relative Risk

Figure 4. Bootstrap distribution for relative risk.

0.015

6

BOOTSTRAP

up and down 1.96 residual standard errors from the central point (the original data) to the circled points; the endpoints of the interval are the slopes of the lines from the origin to the circled points. A t interval would not be appropriate in the example because of the bias and skewness. In practice, one would normally do a t interval on a transformed statistic, for example, log of relative risk, or log-odds-ratio log(pˆ 1 (1 − pˆ 2 )/((1 − pˆ 1 )p2 )). Figure 5 shows a normal quantile plot for the bootstrap distribution of the log of relative risk. The distribution is much less skewed than is relative risk, but it is still noticeably skewed. Even with a log transformation, a t interval would only be adequate for work where accuracy is not required. We discuss confidence intervals even more in the section entitled ‘‘Bootstrap confidence intervals.’’ 5.2 Linear Regression The next examples for linear regression are based on a dataset from a large pharmaceutical company. The response variable is a pharmacokinetic parameter of interest, and candidate predictors are weight, sex, age, and dose (3 levels—200, 400, and 800). In all, 300 observations, are provided, one per subject. Our primary interest in this dataset will be to use the bootstrap to investigate the behavior of stepwise regression; however, first we consider some other issues.

1.5 1.0 0.5 0.0

Log Relative Risk

2.0

highly skewed, with a long right tail caused by a divisor relatively close to zero. The standard error, from a sample of 104 observations, is 0.6188. The theoretical bootstrap standard n n error is undefined because some of the n1 1 n2 2 ∗ bootstrap samples have θˆ undefined because the denominator pˆ ∗ is zero; this aspect is not important in practice. The average of the bootstrap replicates is larger than the original relative risk, which indicates bias. The estimated bias is 2.205 − 2.100 = 0.106, which is 0.17 standard errors. Although the bias does not seen large in the figure, this amount of bias can have a huge impact on inferences; a rough calculation suggests that the actual noncoverage of one side of a two-sided 95% confidence interval would be (0.17 + 1.96) = 0.0367 rather than 0.025, or 47% too large. The right panel of Fig. 4 shows the joint bootstrap distribution of pˆ ∗1 and pˆ ∗2 . Each point corresponds to one bootstrap sample, and the relative risk is the slope of a line between the origin and the point. The original data is at the intersection of horizontal and vertical lines. The solid diagonal lines exclude 2.5% of the bootstrap observations on each side; the slopes are the endpoint of a 95% bootstrap percentile confidence interval. The bottom and top dashed diagonal lines are the endpoints of a t interval with standard error obtained using the usual delta method. This interval corresponds to calculating the standard error of residuals above and below the central line (the line with slope θˆ ), going

−4

−2

0

2

Quantiles of Standard Normal

4

Figure 5. Bootstrap distribution for log of relative risk.

BOOTSTRAP

A standard linear regression using main effects gives:

(Intercept) wgt sex age dose

Value

Std. Error

t value

32.0819 0.2394 −7.2192 −0.1507 0.0003

4.2053 0.0372 1.2306 0.0367 0.0018

7.6290 6.4353 −5.8666 −4.1120 0.1695

Pr (>|t|) 0.0000 0.0000 0.0000 0.0000 0.8653

5.2.1 Prediction Intervals and NonNormality. The right panel also hints at the difference between a confidence interval (for mean response given covariates) and a prediction interval (for a new observation). With large n, the regression lines show little variation, but the variation of an individual point above and below the (true) line remains constant regardless of n. Hence as n increases, confidence intervals become narrow but prediction intervals do not. This example is reflected in the standard formulae for confidence intervals:  (3) yˆ ± tα 1/n + (x − x)2 /Sxx and prediction intervals in the simple linear regression case:  yˆ ± tα 1 + 1/n + (x − x)2 /Sxx (4)

60

As n → ∞ the terms inside the square root decrease to zero for a confidence interval but original bootstrap

PK Parameter 30 40

40

10

20

30 10

20

PK Parameter

covariate (age) to the model reduces residual variance. Note that the y values here are the actual data; they are not adjusted for differences between the actual sex and age and the base case. Adjusting the male observations would raise these valves. Adjusting both would make the apparent residual variation in the plot smaller to match the residual variance from the regression.

50

original bootstrap

50

60

The left panel of Fig. 6 contains a scatterplot of clearance versus weight, for the 25 females who received dose = 400, as well as regression lines from 30 bootstrap samples. This graph is useful for giving a rough idea of variability. A bootstrap percentile confidence interval for mean clearance given weight would be the range of the middle 95% of heights of regression lines at a given weight. The right panel shows all 300 observations and predictions for the clearance/weight relationship using (1) all 300 observations, (2) the main-effects model, and (3) predictions for the ‘‘base case’’, for females who received dose = 400. In effect, this graph uses the full dataset to improve predictions for a subset, ‘‘borrowing strength’’. Much less variability is observed than in the left panel, primarily because of the larger sample size and also because the addition of an important

40

50

60

70 Weight

80

90

100

7

40

50

60

70

80

90

100

Weight

Figure 6. Bootstrap regression lines. Left panel: 25 females receiving dose = 400. Right panel: all observations, predictions for females receiving dose = 400.

8

BOOTSTRAP

approach 1 for a prediction interval, the prediction interval approaches yˆ ± zα . Now, suppose that residuals are not normally distributed. Asymptotically and for reasonably large n the confidence intervals are approximately correct, but prediction intervals are not—the interval yˆ ± zα is only correct for normally distributed data. Prediction intervals should approach (ˆy ± Fˆ ε−1 (α/2), yˆ ± Fˆ ε−1 (1 − α/2)) as n → ∞, where Fˆ is the estimated residual distribution. In other words, no central limit theorem exists for prediction intervals. The outcome for a new observation depends primarily on a single random value, not an average across a large sample. Equation (4) should only be used after confirming that the residual distribution is approximately normal. And, in the opinion of this author, Equation (4) should not be taught in introductory statistics to students ill-equipped to understand that it should only be used if residuals are normally distributed. A bootstrap approach that takes into account both the shape of the residual distribution and the variability in regression lines is outlined below in the subsection entited ‘‘Prediction Intervals.’’ 5.2.2 Stepwise Regression. Now, consider the case of stepwise regression. We consider models that range from the intercept-only model to a full second-order model that includes all main effects, all interactions, and quadratic functions of dose, age, and weight. We use forward and backward stepwise regression, with terms added or subtracted to minimize the CP statistic, using the step function of S-PLUS (insightful Corp., Seattle, WA). The resulting coefficients and inferences are:

Value

Std. Error

t value

(Intercept) 12.8035 14.1188 0.9068 wgt 0.6278 0.1689 3.7181 sex 9.2008 7.1634 1.2844 age −0.6583 0.2389 −2.7553 I(ageˆ2) 0.0052 0.0024 2.1670 wgt: sex −0.2077 0.0910 −2.2814

Pr (>|t|) 0.3637 0.0002 0.1980 0.0055 0.0294 0.0218

The sex coefficient is retained even though it has small t value because main effects are included before interactions. We use the bootstrap here to check model stability, obtain standard errors, and check for bias. 6 MODEL STABILITY The stepwise procedure selected a six-term model. We may use the bootstrap to check the stability of the procedure under random sampling (does it consistently select the same model, or is there substantial variation) and to observe which terms are consistently included. Here, we create bootstrap samples by resampling subjects— whole rows of the data—with replacement. We sample whole rows instead of individual values to preserve covariances between variables. In 1000 bootstrap samples, only 95 resulted in the same model as for the original data; on average, 3.2 terms differed between the original model and the bootstrap models. The original model has six terms; the bootstrap models ranged from 4 to 12, with an average of 7.9, which is 1.9 more than for the original data. This result suggests that stepwise regression tends to select more terms for random data than for the corresponding population, which in turn suggests that the original six-term model may also be overfitted. Figure 7 shows the bootstrap distributions for two coefficients: dose and sex. The dose coefficient is usually zero, although it may be positive or negative. This graph suggests that dose is not very important in determining clearance. The sex coefficient is bimodal, with the modes on opposite sides of zero. It turns out that the sex coefficient is usually negative when the weight–sex interaction is included, otherwise it is positive. Overall, the bootstrap suggests that the original model is not very stable. For comparison, repeating the experiment with a more stringent criteria for variable inclusion—a modified Cp statistic with double the penalty—results in a more stable model. The original model has the same six

BOOTSTRAP

Density

0.03

0.04

Observed Mean

0

0.0

0.01

0.02

40 20

Density

60

Observed Mean

9

−0.05

0.0 Coef.dose

0.05

−40

−20

0 Coef.sex

20

40

Figure 7. Bootstrap distribution for dose and sex coefficients in stepwise regression.

terms. In all, 154 bootstrap samples yielded the same model, and on average the number of different terms was 2.15. The average number of terms was 5.93, which is slightly less than for the original data; this number suggests that stepwise regression may now be slightly underfitting. 6.0.2.1 Standard Errors. At the end of the stepwise procedure, the table of coefficients, standard errors, and t values was calculated, ignoring the variable selection process. In particular, the standard errors are calculated under the usual regression assumptions, and assuming that the model were fixed from the outset. Call these nominal standard errors. In each bootstrap sample, we recorded the coefficients and the nominal standard errors. For the main effects the bootstrap standard errors (standard deviation of bootstrap coefficients) and average of the nominal standard errors are:

Intercept wgt sex age dose

boot SE

avg.nominal SE

27.9008 0.5122 9.9715 0.3464 0.0229

14.0734 0.2022 5.4250 0.2137 0.0091

The bootstrap standard errors are much larger than the nominal standard errors.

That information is not surprising— the bootstrap standard errors reflect additional variability because of model selection, such as the bimodal distribution for the sex coefficient. This is not to say that one should use the bootstrap standard errors here. At the end of the stepwise variable selection process, it is appropriate to condition on the model and do inferences accordingly. For example, a confidence interval for the sex coefficient should be conditional on the weight–sex interaction being included in the model. But it does suggest that the nominal standard errors may be optimistic. Indeed they are, even conditional on the model terms, because the residual standard error is biased. 6.0.2.2 Bias. Figure 8 shows bootstrap distributions for R2 (unadjusted) and residual standard deviation. Both show very large bias. The bias is not surprising— optimizing generally gives biased results. Consider ordinary linear regression—unadjusted R2 biased. If it were calculated using the true βs ˆ it would not be biased. instead of estimated βs Optimizing βˆ to minimize residual squared error (and maximize R2 ) makes unadjusted R2 biased. In classic linear regression, with the model selected in advance, we commonly use adjusted R2 to counteract the bias. Similarly, we

10

BOOTSTRAP

Observed Mean

0.8 0.6

Density

0

0.0

0.2

2

0.4

4

Density

6

1.0

8

1.2

Observed Mean

0.25

0.35 Rsquare

0.45

6.5

7.0 7.5 Sigma

8.0

Figure 8. Bootstrap distributions for R2 and residual standard deviation in stepwise regression.

use residual variance calculated using a divisor of (n − p − 1) instead of n, where p is the number of terms in the model. But in this case, it is not only the values of the coefficients that are optimized, but which terms are included in the model. This result is not reflected in the usual formulae. As a result, the residual standard error obtained from the stepwise procedure is biased downward, even using a divisor of (n − p − 1). 6.0.3 Bootstrapping Rows or Residuals. Two basic ways exist to bootstrap linear regression models—to resample rows, or residuals (3). To resample
residuals, we fit the initial model yˆ i = βˆ0 + βˆj xij , calculate the residuals ri = yi − yˆ i , then create new bootstrap samples as y∗i = yˆ i + r∗i

(5)

for i = 1, . . . , n, where r∗i is sampled with replacement from the observed residuals {r1 , . . . , rn }. We keep the original x and yˆ values fixed to create new bootstrap y* values. Resampling rows corresponds to a random effects sampling design—in which x and y are both obtained by random sampling from a joint population. Resampling residuals corresponds to a fixed effects model, in which the xs are fixed by the experimental design and ys are obtained conditional on the xs.

So at first glance, it would seem appropriate to resample rows when the original data collection has random xs. However, in classic statistics we commonly use inferences derived using the fixed effects model, even when the xs are actually random. We do inferences conditional on the observed x values. Similarly, in bootstrapping we may resample residuals even when the xs were originally random. In practice, the difference matters most when there are factors with rare levels or interactions of factors with rare combinations. If resampling rows, it is possible that a bootstrap sample may have none of the level or combination, in which case the corresponding term cannot be estimated and the software may give an error. Or, what is worse, there may be one or two rows with the rare level, enough so the software would not crash, but instead quietly give garbage answers. That are imprecise because they are based on few observations. Hence, with factors with rare levels or small samples more generally, it may be preferable to resample residuals. Resampling residuals implicitly assumes that the residual distribution is the same for every x and that there is no heteroskedasticity. A variation on resampling residuals that allows heteroskedasticity is the wild bootstrap, which in its simplest form adds either plus or minus the original residual ri to each

BOOTSTRAP

fitted value,

y∗i = yˆ i ± ri

(6)

with equal probabilities. Hence the expected value of y∗i is yˆ i , and the standard deviation is proportional to ri . For more discussion see Ref. 3. Other variations on resampling residuals exist, such as resampling studentized residuals or weighted error resampling for nonconstant variance (3). 6.0.4 Prediction Intervals. The idea of resampling residuals provides a way to obtain more-accurate prediction intervals. To capture both variation in the estimated regression line and residual variation, we may resample both. Variation in the regression line may be obtained by resampling either ˆ∗ residuals or rows to generate random
β val∗ ˆ ˆ ues and corresponding yˆ = β0 + βj x0j , for predictions at x0 . Independently, we draw random residuals r* and add them to the yˆ *. After repeating this many times, the range of the middle 95% of the (ˆy* + r*) values gives a prediction interval. For more discussion and alternatives see Ref. 3. 6.1 Logistic Regression

observation to the prediction from another yields values anywhere between −1 and 2. Instead, we keep the xs fixed and generate y values from the estimated conditional distribution given x. Let pˆ i be the predicted probability that yi = 1 given xi . Then y∗i



1 with probability pˆ 0 with probability 1 − pˆ

(7)

The kyphosis dataset (4) contains observations on 81 children who had corrective spinal surgery, on four variables: Kyphosis (a factor indicating whether a postoperative deformity is present), Age (in months), Number (of vertebrae involved in the operation), and Start (beginning of the range of vertebrae involved). A logistic regression using main effects gives coefficients:

(Intercept) Age Start Number

Value

Std. Error

t value

−2.03693352

1.449574526 0.006446256 0.067698863 0.224860819

−1.405194

0.01093048 −0.20651005

0.41060119

1.695633 −3.050421

1.826024

1.0 0.8 0.0

0.2

0.4

Kyphosis

original bootstrap

0.6

1.0

which suggest that Start is the most important predictor. The left panel of Fig. 9 shows Kyphosis versus Start, together with predicted curve for the base case with Age = 87 (the median)

0.8 0.6 0.4 0.2 0.0

Kyphosis

In logistic regression, it is straightforward to resample rows of the data, but resampling residuals fails—the y values must be either zero or one, but adding the residual from one

5

10 Start

15

11

5

10 Start

Figure 9. Bootstrap curves for predicted kyphosis, for Age = 87 and Number = 4.

15

12

BOOTSTRAP

Figure 10. Bootstrap distributions for logistic regression coefficients.

and Number = 4 (the median). This graph is a sunflower plot (5,6), in which a flower with k > 2 petals represents k duplicate values. The right panel of of Fig. 9 shows predictions from 20 bootstrap curves. Figure 10 shows the bootstrap distributions for the four regression coefficients. All distributions are substantially non-normal. It would not be appropriate to use classic normal-based inferences. Indeed, the printout of regression coefficients above, from a standard statistical package (S-PLUS) includes t values but omits p values. Yet, it would be tempting for a package user to interpret the t coefficients as developing from a t distribution; the bootstrap demonstrates that it

would be improper. The distributions are so non-normal as to make the use of standard errors doubtful. The numerical bootstrap results are:

(Intercept) Age Start Number

Observed

Mean

Bias

−2.03693

−2.41216

−0.375224

SE

1.737216 0.01093 0.01276 0.001827 0.008017 −0.20651 −0.22991 −0.023405 0.084246 0.41060 0.48335 0.072748 0.274049

The bootstrap standard errors are larger than the classic (asymptotic) standard errors by 20–24%. The distributions are also ex-

BOOTSTRAP

tremely biased, with absolute bias estimates that range from 0.22 to 0.28 standard errors. These results are for the conditional distribution bootstrap, which is a kind of parametric bootstrap. Repeating the analysis with the nonparametric bootstrap (resampling observations) yields bootstrap distributions that are even longer-tailed, with larger biases and standard errors. This result reinforces the conclusion that classic normal-based inferences are not appropriate here. 7 ACCURACY OF BOOTSTRAP DISTRIBUTIONS How accurate is the bootstrap? This inquiry entails two questions: • How accurate is the theoretical boot-

strap? • How accurately does the Monte Carlo

implementation approximate the theoretical bootstrap? We begin this section with a series of pictures intended to illustrate both questions. We conclude this section with a discussion of cases where the theoretical bootstrap is not accurate, and remedies. In the section on ‘‘How many bootstrap samples are needed,’’ we return to the question of Monte Carlo accuracy. The treatment in this section is mostly not rigorous. Much literature examines the first question rigorously and asymptotically; we reference some of that work in other sections, particularly in the Section. About confidence intervals, and also refer the reader to (7,8) and some sections of (Ref. 3), as well as the references therein. Figure 11 shows a population, and five samples of size 50 from the population in the left column. The middle column shows the sampling distribution for the mean, and bootstrap distributions from each sample based on B = 1000 bootstrap samples. Each bootstrap distribution is centered at the statistic (x) from the corresponding sample rather than being centered at the population mean µ. The spreads and shapes of the bootstrap distributions vary a bit. This example informs what the bootstrap distributions may be used for. The bootstrap

13

does not provide a better estimate of the population parameter µ, because no matter how many bootstrap samples are used, they are centered at x (plus random variation), not µ. On the other hand, the bootstrap distributions are useful for estimating the spread and shape of the sampling distribution. The right column shows five more bootstrap distributions from the first sample; the first four using B = 1000 resamples and the final using B = 104 . These examples illustrate the Monte Carlo variation in the bootstrap. This variation is much smaller than the variation caused by different original samples. For many uses, such as quick and dirty estimation of standard errors or approximate confidence intervals, B − 1000 resamples is adequate. However, there is noticeable variability, particularly in the tails of the bootstrap distributions, so when accuracy matters, B = 104 or more samples should be used. Note the difference between using B = 1000 and B = 104 bootstrap samples. These examples correspond to drawing samples of size 1000 or 104 observations, with replacement, from the theoretical bootstrap distribution. Using more samples reduces random Monte Carlo variation, but it does not fundamentally change the bootstrap distribution— it still has the same approximate center, spread, and shape. Figure 12 is similar to Fig. 11, but for a smaller sample size, n = 9 (and a different population). As before, the bootstrap distributions are centered at the corresponding sample means, but now the spreads and shapes of the bootstrap distributions vary substantially because the spreads and shapes of the samples also vary substantially. As before, the Monte Carlo variation is small, and it may be reduced using B = 104 or more samples. It is useful to compare the bootstrap distributions to classic statistical inferences. With √ classic t intervals of the form x ± tα/2 s/ n, the confidence interval width varies substantially in small samples as the sample standard deviation s varies. Similarly, the classic √ standard error s/ n varies. The bootstrap is no different in this regard—bootstrap standard errors and widths of confidence intervals for the mean are proportional to s.

14

BOOTSTRAP Population mean = mu Sample mean = x Sampling distribution

Population

−3

mu

0

3

6

0

mu

3

x

0

x

3

3 0

x

Bootstrap distribution 3 for Sample 1

0

3

x

x

3

x

0

0

x

3

0

Bootstrap distribution 5 for Sample 1

x

3 0

3

x

B=10^4

Bootstrap distribution for Sample 5

Sample 5

3

x

3 0

x

Bootstrap distribution 4 for Sample 1

B=1000

Bootstrap distribution for Sample 4

3

x

B=1000

3 0

x

Sample 4

0

3 0

Bootstrap distribution for Sample 3

0

3

x

B=1000

Sample 3

0

Bootstrap distribution 2 for Sample 1

Bootstrap distribution for Sample 2

Sample 2

0

B=1000

Bootstrap distribution for Sample 1

Sample 1

0

3

Bootstrap distribution 6 for Sample 1

x

3

Figure 11. Bootstrap distribution for mean, n = 50. The left column shows the population and five samples. The middle column shows the sampling distribution and bootstrap distributions from each sample. The right column shows five more bootstrap distributions from the first sample, with B = 1000 or B = 104 .

Where the bootstrap does differ from classic inferences is how it handles skewness. The bootstrap percentile interval, and other bootstrap confidence intervals discussed below in the next Section, are in general asymmetrical with asymmetry depending on the sample. They estimate the population skewness from the sample skewness. In contrast, classic t intervals assume that the population is zero. In Bayesian terms, the bootstrap uses

a noninformative prior for skewness, whereas classic procedures use a prior with 100% of its mass on skewness = 0. Which method is preferred? In large samples, it is clearly the bootstrap. In small samples, the classic procedure may be preferred. If the sample size is small, then skewness cannot be estimated accurately from the sample, and it may be better to assume skewness

BOOTSTRAP Population

15

Population mean = mu Sample mean = x Sampling distribution

−3

mu

3

-3

mu

3

Bootstrap distribution for Sample 1

Sample 1

−3

x

3

Bootstrap distribution 2 for Sample 1

−3

x

Sample 2

−3

3

Bootstrap distribution for Sample 2

x

3

−3

x

3

Bootstrap distribution for Sample 3

Sample 3

−3

x

3

x

3

−3

x

3

x

3

−3

Bootstrap distribution 3 for Sample 1

−3

−3

x

3

−3

x

3

−3

3

B=1000

x

3

B=1000

x

3

B=10^4

Bootstrap distribution 6 for Sample 1

x

3

B=1000

Bootstrap distribution 5 for Sample 1

Bootstrap distribution for Sample 5

Sample 5

−3

−3

x

Bootstrap distribution 4 for Sample 1

Bootstrap distribution for Sample 4

−3

−3

B=1000

x

3

Figure 12. Bootstrap distribution for mean, n = 9. The left column shows the population and five samples. The middle column shows the sampling distribution and bootstrap distributions from each sample. The right column shows five more bootstrap distributions from the first sample, with B = 1000 or B = 104 .

= 0 despite the bias, rather than to use an estimate that has high variability. Now turn to Fig. 13, where the statistic is the sample median. Here, the bootstrap distributions are poor approximations of the sampling distribution. In contrast, the sampling distribution is continuous, but the bootstrap distributions are discrete, with the only possible values being values in the original

sample (here n is odd). The bootstrap distributions are very sensitive to the sizes of gaps among the observations near the center of the sample. The bootstrap tends not to work well for statistics such as the median or other quantiles that depend heavily on a small number of observations out of a larger sample.

16

BOOTSTRAP

−4

M

10

−4

M

10

Bootstrap distribution for Sample 1

Sample 1

−4

M

10

−4

M

10

M

10

−4

M

10

M

10

−4

M

10

M

10

−4

M

Sample 5

−4

M

10

M

10

−4

M

M

10

10

Bootstrap distribution 4 for Sample 1

−4

M

10

Bootstrap distribution 5 for Sample 1

−4

M

Bootstrap distribution for Sample 5

−4

10

Bootstrap distribution 3 for Sample 1

Bootstrap distribution for Sample 4

Sample 4

−4

−4

Bootstrap distribution for Sample 3

Sample 3

−4

Bootstrap distribution 2 for Sample 1

Bootstrap distribution for Sample 2

Sample 2

−4

Population median = M Sample median = m

Sampling distribution

Population

10

Bootstrap distribution 6 for Sample 1

−4

M

10

Figure 13. Bootstrap distribution for median, n = 15. The left column shows the population and five samples. The middle column shows the sampling distribution and bootstrap distributions from each sample. The right column shows five more bootstrap distributions from the first sample.

7.1 Systematic Errors in Bootstrap Distributions

Second, in many applications there is a relationship between the statistic and its standard error [‘‘acceleration’’ in the termi-

We note three ways that bootstrap distributions are systematically different than sampling distributions. First, as noted above, bootstrap distributions are centered at the statistic θˆ (plus bias) rather than at the parameter θ (plus bias).

nology of Efron (9)]. For example, the standard error of a binomial proportion  p(1 ˆ − p)/n ˆ depends on p. ˆ Similarly, when sampling from a gamma distribution, the variance of the sample mean depends on the

BOOTSTRAP

underlying mean. More generally when sampling the mean from positively skewed distributions, samples with larger means tend to give larger standard errors. When is acceleration occurs, the bootstrap standard error reflects the standard error ˆ not the true standard corresponding to θ, deviation of the sampling distribution (corresponding to θ ). Suppose the relationship is positive; when θˆ < θ , it tends to be true that the estimated standard error is also less than the true standard deviation of the sampling distribution, and confidence intervals tend to be too short. This is true for t intervals, whether using a formula or bootstrap standard error, and also to a lesser extent for bootstrap percentile intervals. The more accurate intervals discussed in the next section correct for acceleration. The third systematic error is that bootstrap distributions tend to be too narrow. 7.2 Bootstrap Distributions Are Too Narrow In small samples, bootstrap distributions tend to be too narrow. Consider the case of a sample mean from a single population; in this case, the
standard √ theoretical bootstrap error is σˆ / n where σˆ 2 = (1/n) (xi − x)2 . In contrast to the usual sample standard deviation s, this method uses a divisor of n rather than n − 1. The reason the distributions are too narrow relates to the plug-in principle; when plugging in the empirical distribution Fˆ n for use as the population, we are drawing samples from a population with standard deviation σˆ . The result is that bootstrap  standard errors are too small, by√a factor 1 − 1/n relative to the usual s/ n; the errors are about 5% too small when n = 10, about 1% too small when n = 50, and so on. In stratified bootstrap situations, the bias depends on the strata sizes rather than on the total sample size. Some easy remedies can be implemented. The first is to draw bootstrap samples of size n − 1, with replacement from the data of size n. The second, bootknife sampling (10), is a combination of jackknife and bootstrap sampling—first create a jackknife sample by omitting an observation, then draw a bootstrap sample of size n with replacement from

17

the n − 1 remaining observations. The omission can be random or systematic. A third remedy is the smoothed bootstrap. Instead of drawing random samples from the discrete distribution Fˆ n , we draw
from a kernel density estimate Fˆ h (x) = n−1 ((x − xi )/h), where  is the standard normal density (other densities may be used). The original motivation (11,12) was to draw samples from continuous distributions, but it can also be used to correct for narrowness (10). The variance of an observation from Fˆ h is σˆ 2 + h2 . Using h2 = s2 /n results in the theoretical bootstrap standard error matching the usual formula (10). For multidimensional data x the kernel covariance can be 1/n times the empirical covariance matrix. For non-normal data, it may be appropriate to smooth on a transformed scale; for example, for failure time data, take a log transform of the failure times, add normal noise, then transform it back to the original scale. 8

BOOTSTRAP CONFIDENCE INTERVALS

Several bootstrap confidence intervals have been proposed in the literature. Reviews of confidence intervals are found in Refs. 13–15. Here we focus on five examples: t intervals with either bootstrap or formula standard errors, bootstrap percentile intervals (16), bootstrap t intervals (17) bootstrap BCa intervals (9), and bootstrap tilting (18–21). Note that ‘‘t intervals with bootstrap standard errors’’ and ‘‘bootstrap t intervals’’ are different. Percentile and t intervals are quick-anddirty intervals that are relatively simple to compute, but they are not very accurate except for very large samples. They do not properly account for factors such as bias, acceleration, or transformations. They are first-order correct under fairly general circumstances (basically, for asymptoticallynormal statistics) the one-sided noncoverage levels √ for nominal (1√− α) intervals are α/2 + O(1/ n). The O(1/ n) errors decrease to zero very slowly. The BCa, tilting, and bootstrap t intervals are second-order correct, with coverage errors O(1/n).

18

BOOTSTRAP

The percentile, BCa, and tilting intervals are transformation invariant—they give equivalent results for different transformations of a statistic, for example, hazard ratio and log-hazard ratio, or relative risk and log relative risk. t intervals are not transformation invariant. Bootstrap t intervals are less sensitive to transformations than are t intervals; the use of different (smooth) transformations has coverage√effects of order O(1/n), compared with O(1/ n) for t intervals. Our focus is on one-sided errors because few practical situations are truly two-sided. A nominal 95% interval that misses 2% of the time on the left and 3% of the time on the right should not be considered satisfactory. It is a biased confidence interval—both endpoints are too low, so it gives a biased impressions about where the true parameter may be. The appropriate way to add one-sided coverage errors is by adding their absolute values, so the 2%/3% interval has a total coverage error of |2 − 2.5| + |3 − 2.5| = 1%, not 0%. 8.1 t Intervals A t interval is of the form θˆ ± tα/2,ν sθˆ

(8)

where sθˆ is a standard error computed using a formula or using the bootstrap, and ν is degrees of freedom, which is typically set to n − 1 (although other values would be better for non-normal distributions). The bootstrap standard error may be computed using the techniques in the section ‘‘Bootstrap distributions are too narrow’’— bootknife, sampling with reduced size, or smoothed bootstrap. This results in slightly wide intervals that are usually more accurate in practice. These techniques have an O(1/n) effect on one-sided coverage errors, which is unimportant for large samples but is important in small samples. For example, for a sample of independent identically distributed observations from a normal distribution, a nominal 95% t interval for the mean using a bootstrap standard error without these corrections would have one-sided coverages errors:

n 10 20 40 100

Non-coverage

Error

0.0302 0.0277 0.0264 0.0256

0.0052 0.0027 0.0014 0.0006

8.2 Percentile Intervals In its simplest form, a 95% bootstrap percentile interval is the range of the middle 95% of a bootstrap distribution. More formally, bootstrap percentile intervals are of the form ˆ −1 (1 − α/2)) ˆ −1 (α/2), G (G

(9)

ˆ is the estimated bootstrap distribuwhere G tion of θˆ ∗ . Variations are possible that improve finitesample performance. These examples have received little attention in the bootstrap literature, which tends to focus on asymptotic properties. In particular, the simple bootstrap percentile intervals tend to be too narrow, and the variations give wider intervals with better coverage. First, the bootknife or other techniques cited previously may be used. Second, the percentiles may be adjusted. In a simple situation, like the sample mean from a symmetric distribution, the interval is similar to the t interval in Equation (8) but using quantiles of a normal distribution rather than t distribution, zα/2 rather than tα/2,n−1 . As a result, the interval tends to be too narrow. A correction is to adjust the quantiles based on the difference between a normal and t distribution, ˆ −1 (1 − α  /2)) ˆ −1 (α  /2), G (G

(10)

−1 (α/2) and where  is where −1 (α  /2) = Ft,n−1 the standard normal distribution and F t,n−1 is the t distribution function with n − 1 degrees of freedom. This method gives wider intervals. Extensive simulations (not included here) show that this gives smaller coverage errors in practice, in a wide variety of applications. The effect on coverage errors is O(1/n), which is the same order as the

BOOTSTRAP

bootknife adjustment, but the magnitude of the effect is larger; for example, the errors caused by using z rather than t quantiles in a standard l interval for a normal population are as follows:

n

Non-coverage

Error

0.0408 0.0324 0.0286 0.0264

0.0158 0.0074 0.0036 0.0014

10 20 40 100

A third variation relates to how quantiles are calculated for a finite number B of bootstrap samples. Hyndman and Fan (22) give a family of definitions of quantiles for finite samples, governed by a parameter 0 ≤ ∗ is the (b δ ≤ 1. The bth order statistic θˆ(b) − δ)/(B + 1 − 2δ) quantile of the bootstrap distribution for b = 1, . . . , B. Linear interpolation between adjacent bootstrap statistics is used if the desired quantile is not of the form (b − δ)/(B + 1 − 2δ) for some integer b. For bootstrap confidence intervals δ = 0 is preferred, as other choices result in lower coverage probability. The effect on coverage errors is O(1/B).

The standard error may be calculated either by formula or bootstrap sampling; in the latter case, calculating each sθˆ ∗ requires a second level of bootstrap sampling, with secondlevel bootstrap samples drawn from each first-level bootstrap sample. Figure 14 shows the bootstrap distribution for the t statistic for means arsenic concentration, where t is the ordinary t statistic √ (x − µ)/(s/ n). In contrast to Fig. 2, in which the bootstrap distribution for the mean is positively skewed, the distribution for the t statistic is negatively skewed. The reason is that there is positive correlation between x* and s*, as seen observed in the right panel of Fig. 14, so that a negative numerator in Equation (12) tends to occur with a small denominator. The bootstrap t interval is based on the identity θˆ − θ < G−1 t (1 − α/2)) = 1 − α sθˆ (13) where Gt is the sampling distribution of t [Equation (11)]. Assuming that t* [Equation (12)] has approximately the same distribution as t, we substitute quantiles of the bootstrap distribution for t*; then solving for θ yields the bootstrap t interval P(G−1 t (α/2) <

−1 ˆ (θˆ − G−1 t∗ (1 − α/2)sθˆ , θ − Gt∗ (α/2)sθˆ )

8.3 Bootstrap t The difference between t intervals (possibly using bootstrap standard errors) and bootstrap t intervals is that the former assume that a t statistic follows a t distribution, whereas the latter estimate the actual distribution using the bootstrap. Let θˆ − θ (11) t= sθˆ where t is statistic. Under certain conditions, the t statistic follows a t distribution. Those conditions are rarely met in practice. The bootstrap analog of t is

t∗ =

θˆ ∗ − θˆ sθˆ ∗

(12)

19

(14)

Note that the right tail of the bootstrap distribution of t* is used in computing the left side of the confidence interval, and conversely. The bootstrap t and other intervals for the mean arsenic concentration example described in the introduction are shown in Table 1. It is not appropriate to use bootknife or other sampling methods in the section ‘‘Bootstrap distributions are too narrow’’ with the bootstrap t. The reason we use those methods with the other intervals is because those intervals are too narrow if the plug-in population is narrower, on average, than the parent population. The sampling distribution of a t statistic, in contrast, is invariant under changes in the scale of the parent population. This method gives it an automatic correction for the plug-in population being too narrow, and to add bootknife sampling would overcorrect.

BOOTSTRAP

S

20

M

Figure 14. Bootstrap distribution for t statistic, and relationship between boot strap means and standard deviations, of arsenic concentrations. Table 1. Confidence Intervals for Mean Arsenic Concentration, Based on 100,000 Bootstrap Samples, Using Ordinary Nonparametric and Bootknife Resampling 95% Interval Formula t t w boot SE percentile bootstrap t BCa tilting t w boot SE percentile BCa tilting

Asymmetry ±35.7

(88.8, 160.2) Ordinary bootstrap (88.7, 160.2) (91.5, 162.4) (94.4, 172.6) (95.2, 169.1) (95.2, 169.4) Bootknife (88.7, 160.3) (91.5, 162.6) (95.4, 169.3) (95.2, 169.4)

±35.8 (−33.0, 38.0) (−30.1, 48.1) (−29.3, 44.6) (−29.3, 44.9) ±35.8 (−32.9, 38.1) (−29.1, 44.8) (−29.3, 45.0)

The ‘‘asymmetry’’ column is obtained by subtracting the observed mean. The ‘‘t w boot SE’’ interval is a t interval using a bootstrap standard error.

8.4 BCa Intervals

where 

The bootstrap BCa (bias-corrected, accelerated) intervals are quantiles of the bootstrap distribution, like the percentile interval, but with the percentiles adjusted depending on a bias parameter z0 and acceleration parameter a. The interval is

(G−1 (p(α/2)), G−1 (p(1 − α/2)))

(15)

p(c) −
z0 +

z0 +
(−1) (c) 1 − a(z0 +
(−1) (c))

 (16)

is the adjusted probability level for quantiles; it simplifies to c when z0 = a = 0. The BCa interval is derived by assuming a smooth transformation h exists such that h(θˆ ) ∼ N(h(θ ) + z0 σh , σh2 )

(17)

BOOTSTRAP

where σ h = 1 + ah(θ ) and that the same relationship holds for bootstrap samples (subˆ and θˆ for θ ). Some algebra stitute θˆ ∗ for θ, yields the BCa confidence interval. The transformation h cancels out, so it does not be estimated. For the nonparametric bootstrap, the parameter z0 is usually estimated using the fraction of bootstrap observations that fall below the original observed value, z0 =
(−1) (#(θˆ ∗ < θˆ )/B)

(18)

and acceleration parameter based on the skewness of the empirical influence function. One estimate of that skewness is obtained from jackknife samples; let θˆ(i) be the statistic calculated from the original sample but excluding observation i, and θ (i) be the average of those values, then − a=



i=1

n  6 (θˆ(i) − θ (i) )2

3/2

probability 2.5% of exceeding the observed ˆ = 0.025. Bootstrap tilting value, Pθleft (θˆ ∗ > θ) borrows this idea. The idea behind bootstrap tilting is to create a one-parameter family of populations that includes the empirical distribution function to find the member of that family that has 2.5% (or 97.5%) of the bootstrap distribution exceeding the observed value. Let the left (right) endpoint of the interval be the parameter of interest calculated from that population. The family is restricted to have support on the empirical data with varying probabilities on the observations. For example, given i.i.d. observations (x1 , . . . , xn ) when the parameter of interest is the population mean, one suitable family is the exponential tilting family, which places probabilities Pi = c exp(τ xi )

n  (θˆ(i) − θ (i) )3

(19)

i=1

Davison and Hinkley (3) also give expressions for a in the case of stratified sampling, which includes two sample applications. For the arsenic data, z0 = 0.0438 (based on 100,000 replications) and a = 0.0484. The 95% interval is then the range from the 0.0436 to 0.988 quantiles of the bootstrap distribution. The BCa interval has greater Monte Carlo error than the ordinary percentile interval because Monte Carlo error in estimating z0 propagates into the endpoints and because typically one of the quantiles is farther in the tail than for the percentile interval. For example, here the 98.8% quantile is used instead of the 97.5% quantile. In the best case, that a = z0 = 0, this method requires a bit more than twice as many bootstrap samples as the percentile interval for comparable Monte Carlo accuracy. 8.5 Bootstrap Tilting Intervals In parametric statistics, the left endpoint of a confidence interval for a parameter θ is the value such that the sampling distribution has

21

(20)

on observation i, where τ is a tilting parameter, and c is a normalizing constant (depend
ing on τ ) such that i pi = 1. τ = 0 gives equal probabilities pi = 1/n, which corresponds to the empirical distribution function, and about half of the bootstrap distribution is below the observed x. τ < 0 places higher probabilities on smaller observations; sampling with these probabilities is more likely to give samples with smaller observations, and smaller bootstrap means, so more of the bootstrap distribution is below x. We find the values of τ for which only 2.5% of the bootstrap distribution is above x. The left endpoint of the confidence interval is the mean of the corresponding weighted population n  pi xi . θleft = i=1

Similarly, the right endpoint is n 

pi xi

i=1

when pi is computed using the τ that puts 97.5% of the bootstrap distribution to the right of x.

22

BOOTSTRAP

Another suitable family is the maximum likelihood family, with probability pi =

c 1 − τ (xi − x)

(21)

on observation i. 8.6 Importance Sampling Implementation Conceptually, finding the right value of τ requires trial and error; for any given τ , we calculate p = (p1 , . . . , pn ), draw bootstrap samples with those probabilities, calculate the bootstrap statistics, and calculate the fraction of those statistics that are above θˆ , then repeat with a different τ until the fraction is 2.5%. This method is expensive, and the fraction varies because of random sampling. In practice, we use an importance sampling implementation. Instead of sampling with unequal probabilities, we sample with equal probabilities, then reweight the bootstrap samples by the relative likelihood of the sample under weighted and ordinary bootstrap sampling. The likelihood for a bootstrap sample is l(x1∗ , . . . , xn∗ ) = wi∗

(22)

compared with (1/n)n for ordinary bootstrap sampling. Let wb = wi∗ /(1/n)n = nwi∗ be the relative likelihood for bootstrap sample b. We estimate the probability by Pˆ p (θˆ ∗ > θˆ ) = B−1

B 

wb I(θˆb∗ > θˆ ) = B−1

b=1



 b∈R

 b∈L

Ui (p) = lim ε−1 (θ (p + ε(δi − p)) − θ (p)) (24) →0

where δ i is the vector with 1 in position i and 0 elsewhere. When evaluated at p0 these derivatives are known as the empirical influence function, or infinitesimal jackknife. Four least-favorable families found in the tilting literature are: F1 : pi = c exp(τ Ui (p0 )) F2 : pi = c exp(τ Ui (p))

wb

F4 : pi = c(1 − τ Ui (p))−1

(25)

b∈R

wb + (1.2)



wb .

b∈E

Similar calculations are done for the τ used for the right endpoint; solve 0.025B =

8.6.1 Tilting for Nonlinear Statistics. The procedure can be generalized to statistics other than the mean using a least-favorable single-parameter family, one for which inference within the family is not easier, asymptotically, than for the original problem (18). This method is best done in terms of derivatives. Let F p denote a weighted distribution with probability pi on original data point xi , θ (p) = θ (F p ) be the parameter for the weighted distribution (e.g., weighted mean, or weighted regression coefficient), and p0 = (1/n, . . . , 1/n) correspond to the original equal-probability empirical distribution function. The gradient of θ (p) is

F3 : pi = c(1 − τ Ui (p))−1

(23) where R is the subset of {1, . . . , B} with θˆb∗ > θˆ . In practice we also worry about ties, on ˆ let E be the subset where cases with θˆ ∗ = θ; ˆ We numerically find τ to solve with θˆb∗ = θ. 0.025B =

In any case, after finding τ , the endpoint of the interval is the weighted mean for the empirical distribution with probabilities calculated using τ .

wb + (1/2)



wb

b∈E

where L is the subset of {1, . . . , B} with θˆb∗ < θˆ .

each indexed by a tilting parameter τ , where each c normalizes the probabilities to add to 1. F1 and F2 are well known as ‘‘exponential tilting,’’ and the coincide with Equation (20) if θ is a mean. Similarly F3 and F4 are maximum likelihood tilting and coincide with Equation (21) for a mean. F2 and F4 minimize the backward and forward Kullback-Leibler distances between p and p0 , respectively, subject to pi ≥ 0, pi = 1, and θ (p) = A; varying A results in solutions of the form given in Equation (25). F4 also maximizes the likelihood pi subject to the same constraints.

BOOTSTRAP

As in the case of the sample mean, having selected a family, we find the value of τ for which 2.5% (95%) of the bootstrap distribution is to the right of the observed θˆ ; the left (right) endpoint of the confidence interval is then the parameter calculated for the weighted distribution with probability pi on xi . All four families result in second-order accurate confidence intervals (19), but the finite-sample performance differs, sometimes dramatically for smaller samples. The fixedderivative versions F1 and F3 are easier to work with, but they have inferior statistical properties. They are shorter, have actual coverage probability lower than the nominal confidence, and for sufficiently high nominal confidence levels the actual coverage can decrease as the nominal confidence increases. The maximum likelihood version F4 gives the widest intervals with highest and usually most accurate coverage. The differences in coverage between the four families are O(1/n). The maximum likelihood family has better statistical properties, which produces wider confidence intervals with closer to the desired coverage levels. The exponential tilting family is more convenient numerically. 8.7 Confidence Intervals for Mean Arsenic Concentration Table 1 shows 95% confidence intervals for the mean arsenic concentration example described in the Introduction. The intervals vary dramatically, particularly in the degree of asymmetry. The t intervals are symmetric about x. The bootstrap t interval reaches much farther to the right, and it is much wider. The percentile interval is asymmetrical, longer on the right side, to a lesser extent than other asymmetrical intervals. Whereas it is asymmetrical, it is so haphazardly rather than by design, and the amount of asymmetry is too small for good accuracy. Although preferable to the t intervals, it is not as accurate as the second-order accurate procedures. The t intervals assume that the underlying population is normal, which is not true here. Still, the common practice with a sample size as large as 271 would be to use t intervals anyway. The bootstrap can help answer whether

23

Table 2. Actual Noncoverage of Nominal 95% t Intervals, as Estimated from Second-Order Accurate Intervals

bootstrap t BCa

Left

Right

0.0089 0.0061

0.062 0.052

The actual noncoverage should be 0.025 on each side. A t interval would miss more than twice too often on the right side.

that is reasonable, by giving an idea what the actual noncoverage is for a 95% t interval. Table 2 shows the what nominal coverage levels would be needed for the bootstrap t and BCa intervals to coincide with the actual endpoints of the t interval— in other words, what the bootstrap t and BCa intervals think is the actual noncoverage of the l intervals. The discrepancies are striking. On the left side, the t interval should miss 2.5% of the time; it actually misses only about a third or fourth that often, according to the bootstrap t and BCa intervals. On the right side, it should miss 2.5% of the time, but it actually misses somewhere between 5.2% and 6.2%, according to the BCa and bootstrap t procedures. This finding suggests that the t interval is severely biased, with both endpoints systematically lower than they should be. 8.8 Implications for Other Situations The t intervals are badly biased in the arsenic example. What does this finding imply for other situations? On the one hand, the arsenic data are skewed relative to most data observed in practice. On the other hand, the sample size is large. What can we say about other combinations of sample size and population skewness? For comparison, samples of size 47 from an exponential population are comparable with the arsenic data, in the sense that the sampling distribution for the mean is equally skewed. A quick simulation with 106 samples of exponential data with n = 47 shows that the actual noncoverage of 95% t intervals is 0.0089 on the left and 0.0567 on the right, comparable with the bootstrap estimates above. This example shows that for a distribution with only moderate skewness,

24

BOOTSTRAP

like the exponential distribution, n = 30 is not nearly enough to use t intervals; that even n = 47 results in noncoverage probabilities that are off by factors of about 3 and 2, on the two sides. Reducing the errors in noncoverage to a more reasonable 10% of the desired value, i.e. that the actual one-sided noncoverage probabilities are between 2.25% and 2.75% on each side for a nominal 95% interval would require around n = 7500 for an exponential distribution. Even for distributions that are not particularly skewed, say 1/4 the skewness of an exponential distribution (e.g., a gamma distribution with shape = 16), the sample size would need to be around 470 to reduce the errors in noncoverage to 10% of the desired values. To obtain reasonable accuracy for smaller sample sizes requires the use of more accurate confidence intervals, either a secondorder accurate bootstrap interval or a comparable second-order accurate nonbootstrap interval. Two general second-order accurate procedures that do not require sampling are ABC (23) and automatic percentile (24) intervals, which are approximations for BCa and tilting intervals, respectively. The current practice of statistics, using normal and t intervals with skewed data, systematically produces confidence intervals with endpoints that are too low (for positivelyskewed data). Similarly, hypothesis tests are systematically biased; for positively skewed data, they reject H 0 : θ = θ 0 too often for cases with θˆ < θ0 and too little for θˆ > θ0 . The primary reason is acceleration— when θˆ < θ0 then acceleration makes it likely that s < σ , and the t interval does not correct for this, so it improperly rejects H 0 . 8.9 Comparing Intervals t intervals and bootstrap percentile intervals are quick-and-dirty intervals that are suitable for rough approximations, but these methods should not be used where accuracy is needed. Among the others, I recommend the BCa in most cases, provided that the number of bootstrap samples B is very large. In my experience with extensive simulations, the bootstrap t is the most accurate in

terms of coverage probabilities. However, it achieves this result at a high cost—the interval is longer on average than the BCa and tilting intervals, often much longer. Adjusting the nominal coverage level of the BCa and tilting intervals upward gives comparable coverage to bootstrap t with shorter length. And the lengths of bootstrap t intervals vary much more than the others. I conjecture that this length difference occurs because bootstrap t intervals are sensitive to the kurtosis of the bootstrap distribution, which is hard to estimate accurately from reasonablesized samples. In contrast, BCa and tilting intervals depend primarily on mean, standard deviation, and skewness of the bootstrap distribution. Also, the bootstrap t is computationally expensive if the standard error is obtained by bootstrapping. If sθˆ is calculated by bootstrapping, then sθˆ ∗ is calculated using a second level of bootstrapping—drawing bootstrap samples from each first-level bootstrap sample (requiring a total of B + BB2 bootstrap samples, if B2 second-level bootstrap samples from each of B first-level bootstrap samples). The primary advantage of bootstrap tilting over BCa is that it requires many fewer bootstrap replications, typically by a factor of 37 for a 95% confidence interval. The disadvantages of tilting are that the small-sample properties of the fixed-derivative versions F1 and F3 are not particularly good, whereas the more rigorous F2 and F4 are harder to implement reliably. 9 HYPOTHESIS TESTING An important point in bootstrap hypothesis testing is that sampling should be performed in a way that is consistent with the null distribution. We describe here three bootstrap hypothesis testing procedures: pooling for two-sample tests, bootstrap tilting, and bootstrap t. The first is for two-sample problems, such as comparing two means. Suppose that the null hypothesis is that θ 1 = θ 2 , and that one is willing to assume that if the null hypothesis is true, then the two populations are the same. Then one may pool the data, draw samples of size n1 and n2 with replacement

BOOTSTRAP

from the pooled data, and compute a test statistic such as θˆ1 − θˆ2 or a t statistic. Let T* be the bootstrap test statistic, and led T 0 be the observed value of the test statistic. The P-value is the fraction of time that the T* exceeds T 0 . In practice, we add 1 to the numerator and denominator when computing the fraction—the one-sided P-value for the onesided alternative hypothesis θˆ1 − θˆ2 > 0 is

(26)

#(T ∗ > T0 ) + 1 . B+1

The lower one-sided P-value is #(T ∗ < T0 ) + 1 , B+1 and the twosided P-value is the two times the smaller of the one-sided P-values. This procedure is similar to the twosample permutation test, which pools the data and draws n1 observations without replacement for the first sample and allots the remaining n2 observations to be the second sample. The permutation test is preferred. For example, suppose one outlier exists in the combined sample; every pair of permutation samples has exactly one copy of the outlier, whereas the bootstrap samples may have 0, 1, 2, . . . copies. This result adds extra variability not present in the original data, and it detracts from the accuracy of the resulting P-values. Now suppose that one is not willing to assume that the two distributions are the same. Then bootstrap tilting hypothesis testing (3,25,26) may be suitable. Tilting may also be used in one-sample and other contexts. The idea is to find a version of the empirical distribution function(s) with unbreakequal probabilities that satisfy the null hypothesis (by maximizing likelihood or minimizing Kullback-Leibler distance subject to the null hypothesis), then draw samples from the unequal-probability empirical distributions, and let the P-value be the fraction of times the bootstrap test statistic exceeds the observed test statistic. As in the case of confidence intervals, importance sampling may

25

be used in place of sampling with unequal probabilities, see the section on ‘‘Bootstrap confidence intervals.’’ This method shares many close connections to empirical likelihood (27). Bootstrap tilting hypothesis tests reject H 0 if bootstrap tilting confidence intervals exclude the null hypothesis value. The third general-purpose bootstrap testing procedure is related to bootstrap t confidence intervals. A t statistic is calculated for the observed data, and the P-value for the statistic is calculated not by reference to the Students t distribution, but rather by reference to the bootstrap distribution for the t statistic. In this case, the bootstrap sampling need not be done consistently with the null hypothesis, because t statistics are approximately pivotal—their distribution is approximately the same independent of θ . 10

PLANNING CLINICAL TRIALS

The usual bootstrap procedure is to draw samples of size n from the empirical data, or more generally to plug in an estimate for the population and draw samples using the sampling mechanism actually used in practice. In planning clinical trials, we may modify this procedure in two ways: • try other sampling procedures, such as

different sample sizes or stratification, and/or • plug in alternate population estimates. For example, given training data of size n, to estimate standard errors or confidence interval width that would result from a possible clinical trial of size N, we may draw bootstrap samples of size N with replacement from the data. Similarly, we may estimate the effects of different sampling mechanisms, such as stratified sampling, or case-control allocation to arms, even if pilot data were obtained other ways. For example, we consider preliminary results from a clinical trial to evaluate the efficacy of maintenance chemotherapy for acute myelogenous leukemia (28,29). After achieving remission through chemotherapy,

26

BOOTSTRAP Table 3. Leukemia Data Group

Length of Complete Remission (in weeks)

Maintained Nonmaintained

9, 13, 13+ 18, 23, 28+ 31, 34, 45+ 48, 161+ 5, 5, 8, 8, 12, 16+ 23, 27, 30, 33, 43, 45

the patients were assigned to a treatment group that received maintenance chemotherapy and a control group that did not. The goal was to examine whether maintenance chemotherapy prolonged the time until relapse. The data are in Table 3. In all, 11 subjects were in the treatment group, and 12 subjects were in the control group. A Cox proportional hazards regression, using Breslow’s method of breaking ties, yields a log-hazard ratio of 0.904 and standard error 0.512: coef exp(coef) se(coef) z p group 0.904 2.47 0.512 1.77 0.078 An ordinary bootstrap with B = 104 resulted in 11 samples with complete separation— where the minimum observed relapse time in the treatment group exceeds the maximum observed relapse in the control group, which yields an infinite estimated hazard ratio. A stratified bootstrap reduces the number of samples with complete separation to three. Here, stratification is preferred (even if the original allocation were not stratified) to condition on the actual sample sizes and prevent imbalance in the bootstrap samples. Omitting the three observations results in a slightly long-tailed bootstrap distribution, with standard error 0.523, which is slightly larger than the formula standard error. Drawing 50 observations from each group results in a bootstrap distribution for loghazard ratio that is nearly exactly normal with almost no bias, no samples with separation (they are still possible, but unlikely), and a standard error of 0.221—about 14% less than would be obtained by dividing the formula standard error by two, 0.512/2 = 0.256. Similar results are obtained using Efron’s method for handling ties and from a smoothed bootstrap with a small amount of noise added to the remission times. The fact that the reduction in standard error was 14% greater than expected based on the usual

√ O(1/ n) rate may be because censored observations have a less-serious impact with larger sample sizes. 10.1 ‘‘What If’’ Analyses—Alternate Population Estimates In planning clinical trials, it may often be of interest to do ‘‘what if’’ analyses, perturbing various inputs. For example, how might the results differ under sampling from populations with a log hazard ratio of zero, or 0.5? This method should be done by reweighting observations (30,31). This method is a version of bootstrap tilting (18,20,30,32) and is closely related to empirical likelihood (33). Consider first a simple example—sampling the difference in two means, θˆ = x1 − x2 . To sample from populations with different values of θ , it is natural to consider perturbing the data, shifting one or both samples, for example, adding θ − θˆ to each value in sample 1. Perturbing the data does not generalize well to other situations. Furthermore, perturbing the data would often give incorrect answers. Suppose that the observations represent positive skewed observations such as survival times, with a mode at zero. Shifting one sample to the left would give negative times; a shift to the right would make the mode nonzero. More subtle, but very important, is that shifting ignores the mean– variance relationships for skewed populations—increasing the mean should also increase the variance. Instead, we use a weighted version of the empirical data, which maximizes the likelihood of the observed data subject to the weighted distributions that satisfy desired constraints. To satisfy µ1 − µ2 = θ 0 , for example, we maximize n1  i=1

w1i

n2  i=1

w2i

(27)

BOOTSTRAP

subject to constraints on weights (given here for two samples): w1i > 0, i = 1, . . . , n1

(28)

w2i > 0, i = 1, . . . , n2 n1 

w2i = 1

i=1

and the constraint specific to comparing two means: n1  i=1

w1i x1i −

n2 

w2i x2i = θ0

(29)

i=1

For other statistics we replace Equation (28) with the more general θ (Fˆ n,w ) = θ0

distribution. In this case, both bootstraps are nearly normal, and the standard errors are very similar—0.221 for the ordinary bootstrap and 0.212 for the weighted bootstrap, both with 50 observations per group. 11 HOW MANY BOOTSTRAP SAMPLES ARE NEEDED

w1i = 1

i=1 n2 

27

(30)

where Fˆ n,w is the weighted empirical distribution (with obvious generalization to multiple samples or strata). The computational tools used for empirical likelihood (33) and bootstrap tilting (18,20) are useful in determining the weights. The bootstrap sampling is from the weighted empirical distributions, that is, the data are sampled with unequal probabilities. Figure 15 shows this idea applied to the leukemia data. The top left shows KaplanMeier survival curves for the original data, and the top right shows the bootstrap distribution for the log hazard ratio, using 50 observations in each group. The bottom left shows weights chosen to maximize Equation (26), subject to Equation (28) and a log hazard ratio equal to 0.5. To reduce the ratio from its original value of 0.904, the treatment group gets high weights early and low weights later (the weighted distribution has a higher probability of early events) whereas the control group gets the converse. Censored observations get roughly the average weight of the remaining noncensored observations in the same group. The middle left shows the resulting weighted survival estimates, and the middle right shows the corresponding bootstrap

We suggested in the Section ‘‘Accuracy of Bootstrap definitions’’ that 1000 bootstrap samples is enough for rough approximations, but that more are needed for greater accuracy. In this section, we give details. The focus here is on Monte Carlo accuracy— how well the usual random-sampling implementation of the bootstrap approximates the theoretical bootstrap distribution. A bootstrap distribution based on B random samples corresponds to drawing B observations with replacement from the theoretical bootstrap distribution. Quantities such as the mean, standard deviation, or quantiles of the bootstrap distribution converge to their √ theoretical counterparts at the rate O(1/ B), in probability. Efron and Tibshirani (1) suggest that B = 200, or even as few as B = 25, suffices for estimating standard errors, and that B = 1000 is enough for confidence intervals. We argue that larger sizes are appropriate, on two grounds. First, those criterion were developed when computers were much slower; with faster, computers, it is much easier to take large samples. Second, those criteria were developed using arguments that combine the random variation caused by the original sample with the random variation caused by bootstrap sampling. For example, Efron and Tibshirani (1) indicate that  E( ) + 2 1/2 . ˆ ∞ )2 + ˆ B )= cv(se cv(se 4B where cv is coefficient of variation, cv(Y) = σ Y /E(Y), seB and se∞ are bootstrap standard errors using B or ∞ replications, respectively, and relates to the kurtosis of the bootstrap distribution; it is zero for normal distributions. Even relatively small values of ˆ ∞ ) not much ˆ B )/cv(se B make the ratio cv(se larger than 1.

28

BOOTSTRAP

Kaplan Meier curves 1.0

Ordinary bootstrap

0.5

1.0

Density

0.6 0.4

0.0

0.0

0.2

Proportion Surviving

0.8

1.5

Observed Mean

0

50

100

150

0.0

Weighted bootstrap

1.5

Observed Mean

Density

0.6

0.5

0.4

1.0

1.5

0.8

1.0

2.0

Weighted Kaplan Meier curves

0.0

0.0 0

50

100

0.0

150

0.5

Observation Weights

0.10

0.12

Maintained Control

0.08

⊕

0.06

⊕

⊕

0.02

0.04

⊕

0.0

Maintained/censored ⊕ Control/censored 0

50

100

1.0

Log Hazard Ratio

Survival Time in Weeks

Observation Weights

1.0

Log Hazard Ratio

0.2

Proportion Surviving

0.5

Survival Time in Weeks

43

150

Log Hazard Ratio

Figure 15. Perturbed bootstrap distribution for survival.

1.5

BOOTSTRAP

We feel more relevant is the variation in bootstrap answers conditional on the data. This finding is particularly true in clinical trial applications, where • reproducibility is important—two peo-

ple analyzing the same data should get (almost exactly) the same results, with random variation between their answers minimized, and • the data may be very expensive—there is little point in wasting the value of expensive data by introducing extraneous variation using too small B. Given the choice between reducing variation in the ultimate results by gathering more data or by increasing B, it would be cheaper to increase B, at least until B is large. .

ˆ B) =  Conditional on the data, cv(se (δ + 2)/(4B), where δ is the kurtosis of the theoretical bootstrap distribution (conditional on the data). When δ is zero (usually approximately √true), this equation simplifies . ˆ B )=1/ 2B. to cv(se To determine how large B should be, we consider the effect on confidence intervals. Consider a t interval of the form θˆ ± tα/2 seB . Suppose that such an interval using se∞ would be approximately correct, with onesided noncoverage α/2. Then the actual noncoverage using seB in place of se∞ would −1 (α/2)). For large n be Ft,n−1 ((seB /se∞ )Ft,n−1 and α = 0.05, to have the actually one-sided noncoverage fall within 10% of the desired value (between 0.0225 and 0.0275) requires that seB /se∞ be between −1 (0.025*1.1)/−1 (0.025) = 0.979 and −1 (0.025*0.9)/−1 (0.025) = 1.023. To have 95% confidence of no√more than 10% error requires that 1.96/ 2B ≤ 0.022, or B ≥ 0.5(1.96/0.022)2 = 3970, or about 4000 bootstrap samples. To satisfy the more stringent criterion of 95% confidence that the noncoverage error is less than 1% of 0.025 would require approximately 400,000 bootstrap samples. With modern computers, this number is not unreasonable, unless the statistic is particularly slow to compute. Consider also bootstrap confidence intervals based on quantiles. The simple bootstrap percentile confidence interval is the range

29

from the α/2 to 1 − α/2 quantiles of the bootstrap distribution. Let G−1 ∞ (c) be the c quantile of the theoretical bootstrap distribution, and the number of bootstrap statistics that fall below this quantile is approximately binomial with parameters B and c (the proportion parameter may differ slightly because the discreteness of the bootstrap distribution). For finite B, the one-sided  error has standard error approximately c(1 − c)/B. For c = 0.025, to reduce 1.96 standard errors to c/10 requires B ≥ (10/0.025)2 1.962 0.025*0.975 = 14980, about 15,000 bootstrap samples. The more stringent criterion of a 1% error would require approximately 1.5 million bootstrap samples. The bootstrap BCa confidence interval has greater Monte Carlo error because it requires estimating a bias parameter using the proportion of bootstrap samples that fall below the original θˆ (and  the variance of a binomial proportion p(1 − p)/B is greatest for . p = 0.5). It requires B about twice as large as the bootstrap percentile interval for equivalent Monte Carlo accuracy— 30,000 bootstrap samples to satisfy the 10% criterion. On the other hand, the bootstrap tilting interval requires about 17 times fewer bootstrap samples for the same Monte Carlo accuracy as the simple percentile interval, so that about 1000 bootstrap samples would suffice to satisfy the 10% criterion. In summary, to have 95% probability that the actual one-sided non-coverage for a 95% bootstrap interval falls within 10% of the desired value, between 0.0225 and 0.0275, conditional on the data, requires about 1000 samples for a bootstrap tilting interval, 4000 for a t interval using a bootstrap standard error, 15,000 for a bootstrap percentile interval, and 30,000 for a bootstrap BCa interval. Figure 16 shows the Monte Carlo variability of several bootstrap confidence interval procedures, for various combinations of sample size, statistic, and underlying data; these samples are representative of a larger collection of examples in a technical report (21). The panels show the variability caused by Monte Carlo sampling with a finite bootstrap sample-size B, conditional on the data.

30

BOOTSTRAP

Figure 16 is based on 2000 randomly generated datasets for each sample size, distribution, and statistic. For each dataset, and for each value of B, two sets of bootstrap samples are created and intervals calculated using all methods. For each method, a sample variance is calculated using the usual unbiased sample variance (based on two observations). The estimate of Monte Carlo variability is then the average across the 2000 datasets of these unbiased sample variances. The result is the ‘‘within-group’’ component of variance (caused by Monte Carlo variability) and excludes the ‘‘between-group’’ component (caused by differences between datasets). 11.1

Assessing Monte Carlo Variation

To assess Monte Carlo variation in practice, two options can be employed. The first is to use asymptotic formulae. For example, the bootstrap estimate of bias in Equation (1) depends on the sample mean of the bootstrap statistics; the usual formula √ for standard error of a sample mean is seB / B, in which seB is the sample standard deviation of the bootstrap statistics. The standard error of a  ˆ − p)/B. ˆ The bootstrap proportion pˆ is p(1 standard  error of a bootstrap standard error seB is seB (δ + 2)/(4B). The other alternative is to resample the bootstrap values. Given B i.i.d. observations θˆ1∗ , θˆ2∗ , . . . , θˆB∗ from the theoretical bootstrap distribution, and a summary statistic Q (e.g., standard error, bias estimate, or endpoint of a confidence interval), we may draw B2 bootstrap samples of size B from the B observations, and calculate the summary statistics Q∗1 , Q∗2 , . . . , Q∗B . The sample standard devia2 tion of the Q*s is the Monte Carlo standard error. 11.2

Variance Reduction

Several techniques can be used to reduce the Monte Carlo variation. The balanced bootstrap (34), in which each of the n observations is included exactly B times in the B bootstrap samples, is useful for bootstrap bias estimates but is of little value otherwise. Antithetic variates (35) is moderately helpful for bias estimation but is of little value otherwise.

Importance sampling (36,37) is particularly useful for estimating tail quantiles, as for bootstrap percentile and BCa intervals. For nonlinear statistics, one should used a defensive mixture distribution (38,39). Control variates (35,38,40,41) is moderately to extremely useful for bias and standard error estimation and can be combined with importance sampling (42). They are most effective in large samples for statistics that are approximately linear. Concomitants (41,43) are moderately to extremely useful for quantiles and can be combined with importance sampling (44). They are most effective in large samples for statistics that are approximately linear; linear approximations tailored to a tail of interest can dramatically improve the accuracy (45). Quasi-random sampling (46) can be very useful for small n and large B; the convergence rate is O(log(B)n B−1 ) compared with O(B−1/2 ) for Monte Carlo methods. Analytical approximations for bootstrap distributions are available in some situations, which include analytical approximations for bootstrap tilting and BCa intervals (19,23), and saddle-point approximations (47–51).

12 ADDITIONAL REFERENCES In this section we give some additional references. Hesterberg et al. (52) is an introduction to the bootstrap written for introductory statistics students. It is available free at http://bcs. whfreeman.com/pbs/cat 160/PBS18.pdf Efron and Tibshirani (1) is an introduction to the bootstrap written for upper-level undergraduate or beginning graduate students. Davison and Hinkley (3) is the best general-purpose reference to the bootstrap for statistical practitioners. Hall (7) looks at asymptotic properties of various bootstrap methods. Chernick (53) has an extensive bibliography, with roughly 1700 references related to the bootstrap.

BOOTSTRAP

0.025

Correlation, Normal, n = 80

0.020 0.015 0.010 0.005

Monte Carlo standard dev

empir BCa boot-t exp-tilt ml-tilt

0.0

0.01

0.02

0.03

empir BCa boot-t exp-tilt ml-tilt

0.0

Monte Carlo standard dev

Mean, Exponential, n = 40

100

140

200 300 500 Bootstrap sample size

1K

2K

100

200 300 500 Bootstrap sample size

1K

2K

1K

2K

0.08 0.06 0.04

Monte Carlo standard dev

0.10

empir BCa boot-t exp-tilt ml-tilt

0.0

0.02

0.005

0.015

0.025

140

Variance, Exponential, n = 20

empir BCa boot-t exp-tilt ml-tilt

0.0

Monte Carlo standard dev

Variance, Normal, n = 10

100

140

200 300 500 Bootstrap sample size

1K

2K

100

200 300 500 Bootstrap sample size

Ratio of Means, Exponential, n = 80

0.06 0.04

Monte Carlo standard dev

0.06 0.04

0.0

0.02 0.0

empir BCa boot-t exp-tilt ml-tilt

0.02

empir BCa boot-t exp-tilt ml-tilt

0.08

140

0.08

Ratio of Means, Normal, n = 10

Monte Carlo standard dev

31

100

140

200 300 500 Bootstrap sample size

1K

2K

100

45

140

200 300 500 Bootstrap sample size

Figure 16. Monte Carlo variability for confidence intervals.

1K

2K

32

BOOTSTRAP

The author’s website www.insightful.com/ Hesterberg/bootstrap has resources for teaching statistics using the bootstrap and for bootstrap software in S-PLUS. Some topics that are beyond the scope of this article include bootstrapping dependent data (time series, mixed effects models), cross-validation and bootstrap-validation (bootstrapping prediction errors and classification errors), Bayesian bootstrap, and bootstrap likelihoods. References 1 and 3 are good starting points for these topics, with the exception of mixed effects models. REFERENCES 1. B. Efron and R. J. Tibshirani, An Introduction to the Bootstrap. Boca Raton, FL: Chapman and Hall, 1993. 2. L. Breiman, Random forests. Mach. Learn. 2001; 45(1): 5–32. 3. A. Davison, and D. Hinkley, Bootstrap Methods and their Applications. Cambridge, UK: Cambridge University Press, 1997. 4. J. Chambers and T. Hastie, Statistical Models in S. Beimont, CA: Wadsworth, 1992. 5. J. M. Chambers, W. S. Cleveland, B. Kleiner, and P. A. Tukey, Graphical Methods for Data Analysis. Beimont, CA: Wadsworth, 1983. 6. A. Ruckstuhl, W. Stahel, M. Maechler, and T. Hesterberg, sunflower. 1993. Statlib: Availables: http://lib.stat.emu.edu/S. 7. P. Hall, The Bootstrap and Edgeworth Expansion. New York: Springer, 1992. 8. J. Shao and D. Tu, The Jackknife and Bootstrap. New York: Springer-Verlag, 1995. 9. B. Efron, Better bootstrap confidence intervals (with discussion). J. Am. Stat. Assoc., 1987; 82: 171–200. 10. T. C. Hesterberg, Unbiasing the Bootstrap Bootknife Sampling vs. Smoothing. In Proceedings of the Section on Statistics & the Environment, Alexandria, VA: American Statistical Association, 2004, pp. 2924–2930. 11. B. Silverman and G. Young, The bootstrap: to smooth or not to smooth. Biometrika 1987; 74: 469–479. 12. P. Hall, T. DiCiccio, and J. Romano, On smoothing and the bootstrap. Ann. Stat., 1989; 17: 692–704. 13. T. J. DiCiccio and J. P. Romano, A review of bootstrap confidence intervals (with discussion). J. Royal Stat. Soc. Series B, 1988; 50(3): 338–354.

14. P. Hall, Theoretical comparison of bootstrap confidence intervals (with discussion). Ann. Stat., 1988; 16: 927–985. 15. T. DiCiccio and B. Efron, Bootstrap confidence intervals (with discussion). Stat. Sci., 1996; 11(3): 189–228. 16. B. Efron, Bootstrap methods: another look at the jackknife (with discussion). Ann. of Stat., 1979; 7: 1–26. 17. B. Efron, The Jackknife, the Bootstrap and Other Resampling Plans. National Science Foundation–Conference Board of the Mathematical Sciences Monograph 38. Philadelphia, PA: Society for Industrial and Applied Mathematics, 1982. 18. B. Efron, Nonparametric standard errors and confidence intervals. Canadian J. Stat., 1981; 9: 139–172. 19. T. J. DiCiccio and J. P. Romano, Nonparametric confidence limits by resampling methods and least favorable families. Internat. Stat. Rev., 1990; 58(1): 59–76. 20. T. C. Hesterberg, Bootstrap tilting confidence intervals and hypothesis tests. In: K. Berk and M. Pourahmadi, (eds.), Computer Science and Statistics: Proceedings of the 31st Symposium on the Interface, vol. 31. Fairfax Station, VA: Interface Foundation of North America, 1999, pp. 389–393. 21. T. C. Hesterberg, Bootstrap Tilting Confidence Intervals. Research Department 81, Seattle, WA: MathSoft, Inc., 1999. 22. R. J. Hyndman and Y. Fan, Sample quantiles in statistical packages. Am. Stat., 1996; 50: 361–364. 23. T. DiCiccio and B. Efron, More accurate confidence intervals in exponential families. Biometrika, 1992; 79: 231–245. 24. T. J. DiCiccio, M. A. Martin, and G. A. Young, Analytic approximations to bootstrap distribution functions using saddlepoint methods. Technical Report 356. Palo Alto, CA: Department of Statistics, Stanford University, 1990. 25. B. Efron, Censored data and the bootstrap. J. Am. Stat. Assoc., 1981; 76(374): 312–319. 26. D. V. Hinkley, Bootstrap significance tests. Bull. Internat. Stat. Inst., 1989; 65–74. 27. A. Owen, Empirical likelihood ratio confidence intervals for a single functional. Biometrika 1988; 75: 237–249. 28. S. H. Embury, L. Elias, P. H. Heller, C. E. Hood, P. L. Greenberg, and S. L. Schrier, Remission maintenance therapy in acute

BOOTSTRAP myelogenous leukemia. Western J. Med. 1977; 126: 267–272. 29. Insightful. S-PLUS 8 Guide to Statistics. Seattle, WA: Insightful, 2007. 30. T. C. Hesterberg, Bootstrap tilting diagnostics, Proceedings of the Statistical Computing Section, 2001. 31. T. C. Hesterberg, Advances in Importance Sampling. PhD thesis, Statistics Department, Stanford University, Palo Alto, California, 1988. 32. T. C. Hesterberg, Resampling for Planning Clinical Trials-Using S + Resample. Paris: Statistical Methods in Biopharmacy, 2005. Available: http:// www.insightful.com/Hesterberg/ articles/Paris05-ResampleClinical.pdf. 33. P. Hall and B. Presnell, Intentionally biased bootstrap methods. J. Royal Stat. Soc., Series B, 1999; 61(1): 143–158. 34. A. Owen, (2001). Empirical Likelihood. Boca Raton− FL: Chapman & Hall/CRC Press, 2001. 35. J. R. Gleason, Algorithms for balanced bootstrap simulations. Am. Statistic., 1988; 42(4): 263–266. 36. T. M. Therneau, Variance reduction techniques for the bootstrap, PhD thesis, Department of Statistics, Stanford University, Palo Alto, California, 1983. Technical Report No. 200. 37. M. V. Johns, Importance sampling for bootstrap confidence intervals. J. Am. Stat. Assoc., 1988; 83(403): 701–714. 38. A. C. Davison, Discussion of paper by D. V. Hinkley. J. Royal Stat. Soc. Series B, 1986; 50: 356–57. 39. T. C. Hesterberg, Weighted average importance sampling and defensive mixture distributions. Technometrics 1995; 37(2): 185–194. 40. A. C. Davison, D. V. Hinkley, and E. Schechtman, Efficient bootstrap simulation. Biometrika, 1986; 73: 555–566. 41. B. Efron, More efficient bootstrap computations. J. Am. Stat. Assoc., 1990; 85(409): 79–89. 42. T. C. Hesterberg, Control variates and importance sampling for efficient bootstrap simulations. Stat. Comput., 1996; 6(2): 147–157. 43. K. A. Do and P. Hall, Distribution estimation using concomitants of order statistics, with application to Monte Carlo simulations for the bootstrap. J. Roy. Stat. Soc., Series B, 1992; 54(2): 595–607.

33

44. T. C. Hesterberg, Fast bootstrapping by combining importance sampling and concomitants. In: E. J. Wegmand and S. Azen, (ed.), Computing Science and Statistics: Proceedings of the 29th Symposium on the Interface, volume 29. Fairfax Station, VA: Interface Foundation of North America, 1997, pp. 72–78. 45. T. C. Hesterberg, Tail-specific linear approximations for efficient bootstrap simulations. J. Computat. Graph. Stat., 1995; 4(2): 113–133. 46. K. A. Do and P. Hall, Quasi-random sampling for the bootstrap. Stat. Comput., 1991; 1(1): 13–22. 47. M. Tingley and C. Field, Small-sample confidence intervals. J. Am. Stat. Assoc., 1990; 85(410): 427–434. 48. H. E. Daniels and G. A. Young, Saddlepoint approximation for the studentized mean, with an application to the bootstrap. Biometrika 1991, 78(1): 169–179. 49. S. Wang, General saddlepoint approximations in the bootstrap. Stat. Prob. Lett., 1992; 13: 61–66. 50. T. J. DiCiccio, M. A. Martin, and G. A. Young, Analytical approximations to bootstrap distributions functions using saddlepoint methods. Statistica Sinica 1994; 4(1): 281. 51. A. J. Canty and A. C. Davison, Implementation of saddlepoint approximations to bootstrap distributions, In L. Billard, and N. I. Fisher, (eds.), Computing Science and Statistics; Proceedings of the 28th Symposium on the Interface, vol. 28. Fairfax Station, VA: Interface Foundation of North America, 1997, pp. 248–253. 52. T. Hesterberg, S. Monaghan, D. S. Moore, A. Clipson, and R. Epstein, Bootstrap methods and permutation tests In: D. Moore, G. McCabe, W. Duckworth, and S. Sclore, The Practice of Business Statistics. New York: W. H. Freeman, 2003. 53. M. R. Chernick, Bootstrap Methods: A Practitioner’s Guide. New York: Wiley, 1999.

CARDIAC ARRHYTHMIA SUPPRESSION TRIAL (CAST)

1

OBJECTIVES

The major objective of CAST was to test whether post MI patients with left ventricular dysfunction (defined by reduced ejection fraction) who have either symptomatic or asymptomatic ventricular arrhythmias suppressed by antiarrhythmic agents would be less likely to experience an arrhythmic death over 3 years 5.

LEMUEL A. MOYE´ University of Texas School of Public Health

The Cardiac Arrhythmia Suppression Trial (CAST) was the National Heart, Lung, and Blood Institute-(NHLBI)-sponsored, randomized, controlled clinical trial designed to confirm the arrhythmia suppression hypothesis. Instead, it revealed a 2.5-fold increase (96% CI 1.6 to 4.5) in the mortality rate of patients assigned to either encainide or flecainide, which resulted in the early termination of the study. Its findings altered the development program of antiarrhythmic agents and demonstrated the value of objective evidence in formulating treatment guidance. After a myocardial infarction (MI), patients have an increased risk of death from arrhythmia and nonarrhythmic cardiac causes. Although many factors contribute to this risk, ventricular premature contractions (VPCs) that occur post MI confer an independent risk for both arrhythmic death and death from all cardiac causes (1,2). These patients are commonly treated with antiarrhythmic drugs 3. During the 1980s the available antiarrhythmic agents (quinidine and procainimide) were supplemented with the development of the three new agents, encainide, flecanide, and moricizine. Initial reports from case studies suggested that these drugs successfully suppressed ventricular arrhythmias 4. After numerous discussions, but in the absence of a clinical trial, these drugs were approved by the FDA for use in patients with serious ventricular arrhythmias, in the belief that the suppression of asymptomatic or mildly symptomatic ventricular arrhythmias would reduce the arrhythmic death rate. This theory became known as the arrhythmia suppression hypothesis. After FDA approval, the National Heart, Lung, and Blood Institute (NHLBI) designed a clinical trial to test this concept.

1.1 Primary Endpoint The single primary endpoint for CAST was death from arrhythmia. This definition included (1) witnessed instantaneous death in the absence of severe congestive heart failure or shock, (2) unwitnessed death with no preceding change in symptoms and for which no other cause was identified, and (3) cardiac arrest. 1.2 Secondary Endpoints Additional, prospectively declared endpoints were (1) new or worsened congestive heart failure, (2) sustained ventricular tachycardia, (3) recurrent myocardial infarction, (4) cardiac procedures, and (5) quality of life. 2

STUDY DESIGN

CAST was a randomized, double-blind, placebo-controlled clinical trial. 2.1 Patient Eligibility Patients were eligible for CAST screening if they had an MI between 6 days and 2 years before the screening. Patients whose MI was within 90 days before the screening Holter monitor examination had to have ejection fractions of 55% or less. Patients whose MI occurred more than 90 days before the qualifying Holter were required to have an ejection fraction of 40% or less. To be eligible, patients must have had a ventricular arrhythmia defined as at least six or more ventricular premature contractions per hour and, in addition, must have had documented sustained ventricular tachycardia

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CARDIAC ARRHYTHMIA SUPPRESSION TRIAL (CAST)

to be randomized. However, patients whose ventricular ectopy caused either presyncope or syncope were excluded. In addition, patients who had no contraindications to the therapy or were poor compliers were also denied entry to the study. The protocol was approved by the institutional review board of each clinical center. 2.2 Titration Patients who met the above entry criteria then underwent on open titration period, receiving up to three drugs (encainide, flecanide, and moricizine) at two oral doses. The goal of the titration was to suppress the ventricular arrhythmia. Titration ceased when the arrhythmia was suppressed, defined by a greater than 60% reduction in premature ventricular contractions and at least a 90% reduction in runs of unsustained ventricular tachycardia. Subjects with ejection fractions less than 30% were not given flecainide. 135 patients (6%) had incomplete arrhythmia suppression. Patients whose arrhythmia was aggravated by the titration therapy or proved to be intolerant to the drugs were not randomized. The titration exercise produced 447 patients who either died or were intolerant to the drug. 2.3 Randomization and Follow-Up Patients who 1) met the inclusion and exclusion criteria and 2) had their ventricular ectopy suppressed during the titration phase were randomized by telephone call through the CAST Coordinating Center. Patients were randomized to receive either encainide, flecainide, moricizine, or placebo therapy. Randomization was stratified by clinical center, ejection fraction (either ≤ 30% or > 30%), and time between the MI and the qualifying Holter examination (< 90 days or ≥ 90 days). Follow-up examinations were scheduled at 4-month intervals. The expected duration of follow-up was anticipated to be 3 years. 2.4 Sample Size The primary study hypothesis was whether the use of antiarrhythmic agents (encainide, flecanide, and moricizine) would reduce the 3-year cumulative mortality rate from sudden death. Notably, CAST was designed as a

one-tailed test to assess whether drug therapy was beneficial or had no benefit. It was not designed with statistical rigor to demonstrate harm. The anticipated 3-year cumulative mortality rate from death because of arrhythmia was 11%. Treatment was anticipated to reduce this by 30%. The overall alpha error was 0.025 (one-tailed), and the desired power was 85%. These computations produced a sample size of 4400 patients for the trial. 2.5 Analysis Plan Baseline comparisons were performed by ttests and chi-square tests. Primary and secondary endpoint comparisons that involve time-to-event analyses were executed using log-rank tests. The significance levels for the individual drug comparisons were increased three-fold to adjust for multiple comparisons among the three drug groups. Confidence intervals were estimated by the method of Cornfield. The relative risk of treatment in clinically defined subgroups was calculated to evaluate the consistency of the effects of the drugs on the study endpoints across subgroups. 2.6 Monitoring Procedures The Data and Safety Monitoring Board met twice yearly to review the unblinded CAST results. This group approved the interim monitoring protocol in September 1988 before it evaluated any data. This protocol included a conservative boundary for stopping the study because of demonstrated benefit, a symmetric boundary for advising stopping the study because of adverse effects, and a stochastic curtailment boundary to permit stopping CAST for a low probability of demonstrating a beneficial effect. The total number of events anticipated in the trial was 425, but the data as of March 1989 indicated that less than 300 events would occur. 3 RESULTS 3.1 Screening As of March 30, 1989, 2309 patients had been recruited and either were undergoing or had completed the open-label titration phase of the study. Of these, 1727 patients met the

CARDIAC ARRHYTHMIA SUPPRESSION TRIAL (CAST)

arrhythmia requirement, had their arrhythmias suppressed, and completed randomization. Assignments to therapy were as follows: 1455 received either encainide, flecainide, or placebo; 272 received either moricizine or placebo. Overall, 730 patients were randomly assigned to encainide and flecainide, and 725 patients received placebo therapy. 3.2 Baseline Characteristics The baseline characteristics of patients randomized to encainide or flecainide were compared with patients recruited to placebo therapy. The average age was 61 years old, and 82% of participants were male. Placebo and treatment groups were similar with respect to all characteristics, including age, ejection fraction, time elapsed since myocardial infarction, and use of beta-blockers, calcium-channel blockers, digitalis, or diuretics at baseline. The mean left ventricular ejection fraction was 0.40 in patients treated with encainide or flecainide. Approximately 2.2% of patients had an ejection fraction below 0.20; 20% had an ejection fraction between 0.30 and 0.55. The mean frequency of VPCs was 127 per hour in drug-treated patients, and 20.6% of the patients had a least one run of unsustained ventricular tachycardia (≥ 120 beats per minute) during baseline Holter recording. 3.3 Compliance As of March 30, 1989, 8.4% of patients randomized to encainide or flecainide therapy had their therapy discontinued. More than half of the withdrawals exited because of protocol-defined reasons (e.g., major adverse events or symptoms). Placebo therapy was removed in 8.6% of patients. Of the patients still taking active therapy or placebo, 79% were taking at least 80% of their medication. Of the patients who died of arrhythmia or were resuscitated after a cardiac arrest, 88% were following the study regimen at the time of the fatal event. The average exposure to therapy was 295 days in the encainide/flecainide group and 300 days in the placebo group.

3

3.4 Mortality Findings 730 patients were recruited to encainide and flecainide, and 725 patients were recruited to placebo. 78 deaths occurred in the combined encainide/flecainide-placebo groups, 56 deaths in the encainide/flecainide group and 22 deaths in the placebo group. The total mortality relative risk for encainide or flecainide therapy when compared with placebo was 2.5 (95% CI 1.6 to 4.5). The relative risks of encainide or flecainide were indistinguishable (2.7 and 2.2, respectively). 33 deaths occurred because of either arrhythmia or cardiac arrests in the encainide/flecainide group, and 9 deaths occurred in the placebo group. The relative risk of death from arrhythmia for encainide or flecainide was 3.6 (95% CI 1.7–8.5). The relative risks of death from arrhythmia or cardiac arrest for patients receiving encainide or flecainide considered separately were not different (3.4 and 4.4, respectively). 14 deaths occurred for other cardiac reasons in the encainide/flecainide group compared with 6 deaths in the placebo group. 9 noncardiac deaths, unclassified deaths, or other cardiac arrest occurred in the encainide/ flecainide group, and 7 occurred in the placebo group. 3.5 Subgroup Analyses The effect of the therapy was consistent across subgroups. In all subgroups, patients treated with encainide or flecainide had higher rates of both total mortality and death because of arrhythmia than patients treated with placebo. The observed increased risk from either of these two agents was present regardless of age, prolonged QRS interval, or use of beta-blockers, calcium-channel blockers, digitalis, or diuretic at baseline. Notably, patients with an ejection fraction ≥ 30% in which use of encainide and flecainide were stratified had essentially equal relative risks (4.6 and 4.4, respectively) 3.6 Early Termination On April 16 and 17, 1989, the CAST Data and Safety Monitoring Board reviewed the data available as of March 30, 1989. An evaluation of the data for all randomized patients

4

CARDIAC ARRHYTHMIA SUPPRESSION TRIAL (CAST)

revealed that the interim monitoring statistic (Z = − 3.22) had crossed the lower advisory boundary for harm (Z = − 3.11). In addition, the conditional power for a beneficial effect (< 0.27) was well below the minimum conditional power established (0.55). The DSMB therefore recommended that the encainide and flecainide arms of the study be discontinued. 4

CONCLUSIONS

The scheduled termination of CAST was preempted by an unexpected increase in mortality, which prompted the termination of the encainide and flecainide limbs of the study. Given that millions of patients received prescriptions for approved antiarrhythmic agents during the conduct of CAST, the public health implications of its finding of antiarrhythmic therapy-induced harm were profound 6. The use of these drugs was sharply curtailed as the practicing community struggled to understand the staggering findings. No confounding factors that could explain the finding of harm were identified. In fact, the consistency of the excessive mortality risk of encainide and flecainide in all subgroups was a noteworthy feature of this study. The implications of CAST for the development of new antiarrhythmic drugs was profound. Its demonstration of hazard in the face of VPC suppression removes this suppression as an informative, predictive surrogate endpoint for clinical endpoints. In addition, the results had regulatory implications, as the FDA grappled with the correct combination of the number of trials, the type of patients, the sample size, and the goals of these trials to view the risk–benefit balance of antiarrhythmic therapy clearly. Finally, perhaps no point more clearly reveals the expectations of the CAST investigators than their acceptance of the onetailed statistical hypothesis test, specifically designed not to focus on harm. The demonstration of excess mortality overwhelmed the ability of the CAST analysis that was designed simply to demonstrate benefit versus lack of benefit. The use of one-tailed research continues to be debated in the clinical literature (7,8). The surprise results of CAST reinforced

the time-tested need for the study investigators not just to consider the possibility of harm, but also to weave this concern into the fabric of the study 9. Fortunately, their wise decision to include a placebo group not only distinguished CAST from the many preliminary studies that led to FDA approval of drugs subsequently shown to be harmful but also was the one essential characteristic of the study that permitted a clear demonstration of the damage stemming from the use of these drugs. REFERENCES 1. J. T. Bigger, Jr. et al., Multicenter PostInfarction Research Group, The relationships among ventricular arrhythmia, left ventricular dysfunction, and mortality in the 2 years afgter myocardial infarction. Circulation. 1984; 69: 250–258. 2. J. Mukharjil et al., Risk factors for sudden death afgter acute myocardial inarction; two year follow-up. Am J Cardiol. 1984; 54: 31–36. 3. S. Vlay, How the university cardiologist treats ventrcicular premature beats; a nationwide survey of 65 university medical centers. Am Heart J. 1985; 110: 904–912. 4. The CAPS Investigators, The Cardiac Arrhythmia Pilot Study. Am J Cardiol. 1986; 57: 91–95. 5. The CAPS Investigators, The CAST Investigators (1989) Preliminary Report: Effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. New England J Medicine. 1989; 3212: 406–412. 6. L. K. Hines, T. P. Gross, and D. L. Kennedy, Outpatient arrhythmic drug use from 1970 to 1986. Arch Intern Med. 1989; 149: 1524–1527. 7. J. A. Knottnerus and L. M. Bouter, (2001) Commentary: The ethics of sample size: Twosided testing and one-sided thinking. J Clinical Epidemiology. 2001; 54: 109–110. 8. L. A. Moy´e and A. Tita, Defending the Rationale for the Two-Tailed Test in Clinical Research. Circulation. 2002; 105: 3062–3065. 9. L. A. Moy´e, Statistical Reasoning in Medicine. The Intuitive P-value Primer. New York: Springer-Velag, 2006.

FURTHER READING T. Moore, Deadly Medicine. New York: Simon and Schuster, 1995.

CARDIAC ARRHYTHMIA SUPPRESSION TRIAL (CAST)

CROSS-REFERENCES Placebo controlled clinical trial Run in period Interim analyses Trial termination

5

Table 1. 2 × 2 Contingency Table for Treatment and Success

CATEGORICAL RESPONSE MODELS GERHARD TUTZ

Response

¨ Statistik Munich Institut Fur Germany Treatment

1

INTRODUCTION

Drug (Treatment 1) Control (Treatment 2)

In clinical trials, categorical data occur if the response is measured in categories. The simplest experimental design compares two treatment groups, usually a treatment and a control group, on a binary response that may be generally described by the categories of success or failure. Thus, data are given in a 2 × 2 table (see Table 1) where the marginals n1+ , n2+ denote the number of observations in treatment groups 1 and 2. With the binary response given by y = 1 for success and y = 0 for failure, and T denoting the treatment, investigation concerns the conditional probabilities of success in treatment group t

Failure 2

Marginals

n11

n12

n1+

n21

n22

n2+

The dependence of y on treatment group may be given in the form of a binary logit model log

πt1 = β0 + βT(t) πt2

(1)

where βT(t) is a parameter connected to treatment group t. As πt1 + πt2 = 1, only two probabilities, namely π11 and π21 , are free to vary. Therefore, the model has too many parameters and an additional restriction is needed. By setting βT(1) = βT , βT(2) = 0, one immediately obtains

πt1 = P(y = 1|T = t). The effect of the treatment is measured by comparing π11 and π12 (i.e., the probabilities of success in group 1 and 2) or functions thereof. Instead of comparing the probabilities themselves, it is often preferable to compare the odds, which for treatment group t are defined by

log

π11 = β0 + βT , π12

log

π21 = β0 . π22

Simple calculation shows that βT may be interpreted as the log odds ratio βT = log

P(y = 1|T = t) πt1 = πt2 P(y = 0|T = t)

π11 /π12 π21 /π22

and

where πt2 = 1 − πt1 = P(y = 0|T = t). Comparison of treatments may be based on the odds ratio (also called cross-product ratio) θ=

Success 1

eβT =

π11 /π12 π21 /π22

is the odds ratio. Thus, the case of no effect where the odds ratio equals (1) corresponds to βT = 0. For general designs, it is instructive to give the model in Equation (1) in the usual form of a regression model. With the restriction βT(2) = 0, Equation (1) is equivalent to the logit model

π11 /π12 π21 /π22

which is a directed measure of association between treatment and success. If θ = 1, the odds in both treatment groups are the same, and therefore treatment has no effect on the probability of success. If θ > 1, odds in treatment group 1 are larger than in treatment group 2; for 0 < θ < 1, treatment group 2 is superior to treatment group 1.

log

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

πt1 = β0 + xt βT πt2

(2)

2

CATEGORICAL RESPONSE MODELS

Table 2. m × 2 × 2 Contingency Table with Binary Response Response Center 1 2 · · m

Treatment

Success

Failure

Drug Control Drug Control · · · · Drug Control

n111 n121 n211 n221 · · · · nm11 nm21

n112 n122 n212 n222 · · · · nm12 nm22

n11+ n12+ n21+ n22+ · · · · nm1+ nm2+

where x1 = 1 (coding T = 1) and x2 = 0 (coding T = 2). 2 BINARY RESPONSE MODELS WITH COVARIATES

Often treatments are compared by using data from different sources. If, in a multi-center study, the success of treatment is investigated, one obtains count data as given in Table 2. As the effect of a treatment may depend on the specifics of a center, it is necessary to include the information where the data come from. With πst1 = P(y = 1|T = t, C = s) denoting the conditional probability of success given treatment t and center s, a logit model that uses center as covariate is given by πst1 = β0 + xt βT + βC(s) πst2

πs11 /πs12 πs21 /πs22 =

P(y = 1|T = 1, C = s)/P(y = 0|T = 1, C = s) P(y = 1|T = 2, C = s)/P(y = 0|T = 2, C = s)

= eβT which does not depend on the center. Thus, the effect of the treatment (measured by odds ratio) does not depend on the center, although centers might differ in efficiency. If the effect of treatment is modified by centers, a more complex model including interactions applies. 2.2 The Binary Logit Model with Covariates

2.1 Observations from Several Strata

log

odds. If βT = 0, variation of success might exist across centers but treatment is without effect. If Equation (3) holds, the association between treatment and response for center s is given by the odds ratio

(3)

where πst2 = 1 − πst1 = P(y = 0|T = t, C = s), and x1 = 1 (for T = 1) and x2 = 0 (for T = 2). The parameter βT specifies the effect of treatment whereas βC(s) represents the center effect. The effect on the odds is seen from the form πst1 = eβ0 · (eβT )xt · eβC(s) . πst2 Thus, eβ0 and eβC(s) represent the multiplicative effects of treatment and center on the

The multi-center study is just one example in which observations come from several strata that should be included as control variables. Alternative covariates that influence the response may be levels of age or severity of the condition being treated. If the covariates are categorical, as, for example, age levels, one again obtains a contingency table of the form given in Table 2. In the more general case where covariates also may be metric, as, for example, age in years, data are no longer given in contingency tables. Let yi with yi = 1 for success and yi = 0 for failure denote the response of the ith individual (or observed unit). Together with yi one has information about treatment (xT,i ) and a vector of covariates (zi ), which may contain age, gender, condition, and so on. With πi = P(yi = 1|xi , zi ), the corresponding binary logit model is given by log

πi = β0 + x
T,i βT + zi βz . 1 − πi

(4)

For two treatment groups, xT,i is a dummy variable as in Equation (3), given by xT,i = 1 if treatment 1 is applied on the ith observation and xT,i = 0 if treatment 2 is applied on the ith observation. It should be noted that i refers to observations not treatments.

CATEGORICAL RESPONSE MODELS

In the case with m treatment groups, xT,i represents a vector of dummy variables, x
T,i = (xT(1),i , . . . , xT(m−1),i ) where xT(s),i

⎧ ⎨1 if treatment s is used on individual i = ⎩ 0 otherwise.

log

πi = x
i β 1 − πi

exp(x
i β) . 1 + exp(x
i β)

The last form has the general structure πi = F(x
i β)

Estimation of parameters may be based on the concept of maximum likelihood. The loglikelihood for Equation (5) is given by n 

yi log(πi ) + (1 − yi ) log(1 − πi ) (6)

i=1

where πi is specified by πi = F(x
i β). Maximization of l(β) yields the maximum likeˆ In the general case, no lihood estimator β. explicit form of βˆ is available, therefore the estimator is computed iteratively [e.g., McCullagh & Nelder (2)]. For the accuracy of the maximum likelihood estimator, the Fisher or expected information matrix F(β) = E(−∂ 2 l(β)/∂β ∂β
) has to be computed. One obtains F(β) =

n 

F
(x
i β) xi − x
j − F(x
i β))

F
(x
i β)(1 i=1

where F
(η) is the first derivative F
(η) = ∂F(η)/∂η. Under regularity conditions [see Fahrmeir & Kaufmann (3)], one obtains asymptotic existence of the maximum likelihood estimate, consistency, and asymptotic normality, β˜ ∼ N(β, F(β)−1 ).

or equivalently πi =

2.3 Inference in Binary Response Models

l(β) =

The variables xT(s),i are dummy variables that code if treatment s is used or not by 0, 1 coding. Alternative coding schemes are given, for example, in Fahrmeir and Tutz (1). More generally, xT,i may contain dummy variables for treatment groups as well as metric variables referring to treatment, as, for example, dosage of drug. The same holds for the covariates zi . In a multi-center study, zi is a vector of dummy variables coding centers, but it may also contain metric variables. Equation (4) accounts for the control variables given in zi but still assumes that the effect of treatment is not modified by these variables. The model is a main effect model because it contains treatment as well as covariates as separate additive effects within the logit model. By collecting intercept, treatment, and control variables into one vector x
i = (1, x
T,i , z
i ) and defining β
= (β0 , βT
, βz
), one obtains the general form of the logit model

3

(5)

where F is the logistic distribution function F(η) = exp(η)/(1 + exp(η)). The logit model is a special case of a generalized linear model [compare Kauermann & Norris in EoCT and McCullagh & Nelder (2)], with F as the (inverse) link function. Alternative (inverse) link functions in use are based on the exponential distribution F(η) = 1 − exp(−η), the extreme minimal-value distribution F(η) = 1 − exp(− exp(η)), or the normal distribution function. The latter yields the so-called probit model [see McCullagh & Nelder (2)].

If only categorical covariates are present and therefore data are given in a contingency table, estimation may also be based on the weighted least squares approach suggested by Grizzle et al. (4). If the linear predictor ηi = x
i β is partitioned into treatment effects xi,T βT and covariates zi βz , the focus of interest is on the null hypothesis H0 : βT = 0, although the effect of covariates may also be investigated by considering the null hypothesis H0 : βz = 0. These hypotheses are special cases of the linear hypothesis H0 : Cβ = ξ

against

H1 : Cβ = ξ

where C is a specified matrix and ξ is a given vector.

4

CATEGORICAL RESPONSE MODELS

Let βˆ denote the maximum likelihood estimate for the model and β˜ denote the maximum likelihood estimate for the submodel under the additional restriction Cβ = ξ . Test statistics for the linear hypothesis are the likelihood ratio test ˜ − l(β)) ˆ λ = −2(l(β) where l(β) is the log-likelihood given in Equation (6), the Wald test ˆ
)(Cβˆ − ξ ) w = (Cβˆ − ξ )
(CF −1 (β)C and the score test

parameters than models in which ordinality of the response is ignored. For categorical data, parsimony of parameters is always recommended. In the following, let Y take values from ordered categories 1, . . . , k. 3.1 Cumulative Type Models A simple concept for treating the ordinal response is to use a model for binary responses in which the binary response corresponds to a split between category r and r + 1 yielding binary response categories {1, . . . , r}, {r + 1, . . . , k}. The corresponding binary response model in Equation (5) with linear predictor β0r + x
β is

˜ −1 (β)s( ˜ β) ˜ u = s
(β)F

P(Y ≤ r|x) = F(β0r + x
β).

where s(β) = ∂l(β)/∂β is the score function. With rg(C) denoting the rank of matrix C, all three test statistics, λ, w, and u, have the same limiting χ 2 -distribution

Although the resulting models for r = 1, . . . , k − 1 seem to be separate models, they are linked by the common response Y. A consequence is that the restriction β01 ≤ . . . ≤ β0,r+1 must hold because P(Y ≤ r) ≤ P(Y ≤ r + 1). Moreover, because β does not depend on the category in Equation (7), it is assumed that the effect of x is the same for all splits into categories {1, . . . , r}, {r + 1, . . . , k}. As in Equation (7), categories are cumulated, it is referred to as a cumulative response model. It may be motivated by considering Y as a coarser version of an underlying unobservable variable Y˜ = −x
β + with β01≤...≤β0,r+1 denoting the category boundaries on the latent continuum that defines the levels of Y. The link between observable Y and underlying variable Y˜ is specified by Y = r if γ0,r−1 < Y˜ ≤ γ0r where has distribution function F. The most common model is the proportional odds model that results from choosing F as the logistic distribution function. One obtains

a

λ, w, u ∼ χ 2 (rg(C)) under regularity conditions, which are similar to the conditions needed for asymptotic results for maximum likelihood estimation (5). An advantage of the Wald statistic is that only one fit is necessary. Although for likelihood ration and score test the restricted estimate β˜ has to be computed, the Wald test uses only the maximum likelihood estimate ˆ For testing single coefficients H0 : βT = 0, β. the Wald test is the square (w = t2 ) of the ‘‘t-value’’ βˆT t= √ aˆ rT where aˆ rT = v ar(βˆT ) is the rth diagonal element of the estimated covariance matrix ˆ Most program packages use t when F −1 (β). investigating the effect of single variables. 3

ORDINAL RESPONSES

In many cases, the response is measured in ordered categories; for example, patient condition may be given by good, fair, serious, or critical. The use of the ordering of categories allows for models that have less

log

(7)

P(Y ≤ r|x) = β0r + x
β, P(Y > T|x)

r = 1, . . . , k − 1. The model is extensively discussed in McCullagh (6). The name proportional odds is caused by the property that the cumulative odds are proportional, for example, for two values of explanatory variables x1 , x2 one obtains log

P(Y ≤ r|x1 )/P(Y > |x1 ) = (x
1 − x
2 )β P(Y ≤ r|x2 )/P(Y > |x2 )

CATEGORICAL RESPONSE MODELS

which does not depend on r. Therefore, the proportion of cumulative odds only depends on the explanatory variables. 3.2 Sequential Type Models For ordered categories, it may often be assumed that they are reached successively. The sequential model may be seen as a process model with start in category 1 and consecutive modeling of binary transitions to higher categories. The transition is specified by P(Y = r|Y ≥ r, x) = F(β0r + x
β) r = 1, . . . , k − 1. The binary model specifies the probability that the process stops in category r given category r is reached. In most common use is the sequential logistic model with F(η) = exp(η)/(1 + exp(η)). An advantage of the sequential model over the cumulative model is that no ordering restriction for the intercept is necessary. Moreover, it may be estimated by software that handles binary regression models. The relationship to ¨ a¨ cumulative models is investigated in La¨ ar and Matthews (7) and Greenland (8). 4

NOMINAL RESPONSE

For nominal response variable Y ∈ {1, . . . , k}, the most common model is the multinomial logit model P(Y = r|x) =

exp(x
βr ) , k−1 1 + s=1 exp(x
βs )

r = 1, . . . , k − 1, where k is the reference category with P(Y = k) = 1 − p(Y = r) − . . . − P(Y = k − 1). The alternative form log

P(Y = r|x) = x
βr P(Y = k|x))

(8)

shows the similarity to the binary logit model. It should be noted that when comparing P(Y = r|x) to P(Y = k|x), which is basically parameterized in Equation (8), the parameter β r has to depend on r. As Y is assumed to be on nominal scale level, the interpretation of parameters in

5

Equation (8) is similar to the binary logit model with β r referring to the effect of x on the logits between category r and the reference category k. The model is easily reparameterized if not k but k0 {1, . . . , k} is considered as the reference category by considering log

P(Y = r|x) = x
(βr − βk0 ) − x
β˜r P(Y = k0 |x)

where β˜r = βr − βk0 are the parameters corresponding to reference category k0 .

5 INFERENCE FOR MULTICATEGORICAL RESPONSE For data (Yi , xi , i = 1, . . . , n), the multicategorical models from Sections 3 and 4 may be written as multivariate generalized linear models πi = h(Xi β) where πi
= (πi1 , . . . , πi,k−1 ), πi,r = P(Yi = r|xi ), h is a link function depending on the model, and Xi is a design matrix. The multinomial model, for example, uses the block design matrix Xi = Diag(x
i , x
i , . . . , x
i ), β
= (β1
, . . . ,
), and link function h = (h1 , . . . , hk ) with βk−1  hr (η1 , . . . , ηk−1 ) = exp(ηr )/(1 + k−1 s=1 exp(ηs )). For more details, see, for example, Fahrmeir and Tutz (1). The log likelihood has the form

l(β) =

n  k−1 

yij log(πij /(1 − πi1 − . . . − πiq ))

i=1 j=1

+ log(1 − πi1 − . . . − πiq ), The Fisher matrix F(β) = E(−∂ 2 l/∂β∂β
) is given by F(β) =

n 

Xi
Wi (β)Xi

i=1

where Wi (β) = {

∂g (πi ) ∂g(π ) i (β) ∂π i }−1 ∂πi
i

with g de-

noting the inverse function of h, g = h−1 , and

6

CATEGORICAL RESPONSE MODELS

i (β) = Diag(πi ) − πi πi
denoting the covariance matrix. Under regularity conditions, one asymptotically obtains β˜ ∼ N(β, F(β)

−1

).

Table 3. Clinical Trial for Curing an Infection (16) Response Center Treatment Success Failure Odds Ratio

Linear hypotheses H0 : Cβ = ζ may be tested by likelihood ratio tests, Wald tests, or score tests in the same way as for binary response models.

2

6

4

3

FURTHER DEVELOPMENTS

The models in the previous sections use the linear predictor ηi = x
i β. This assumption may be weakened by assuming a more flexible additive structure ηi = f(1) (xi1 ) + . . . + f(p) (xip ) where f(j) are unspecified functions. Several methods have been developed to estimate the components f(j) ; see Hastie and Tibshirani (9) and Fan and Gijbels (10) for localizing approaches and Green and Silverman (11) for penalizing approaches. For small sample sizes, asymptotic inference may not be trustworthy. Exact inference, for example, in case control studies is considered in Agresti (12). A short overview on exact inference is found in Agresti (13). In medical research, often continuous variables are converted into categorical variables by grouping values into categories. By using models with flexible predictor structure, categorization may be avoided. Categorization always means loss of information (14), and the choice of cutpoints may yield misleading results [e.g., Altman et al. (15)]. 7

1

AN EXAMPLE

Beitler and Landis (16) considered a clinical trial in which two cream preparations, an active drug and a control, are compared on their success in curing an infection [see also Agresti (12)]. The data were collected at eight centers. The present analysis is restricted to four centers and the data are given in Table 3. By considering the centers as fixed (not as a random sample), the most general logistic model incorporates interactions between treatment (T) and centers (C). Testing the main effect model 1 + T + C against the

Drug Control Drug Control Drug Control Drug Control

11 10 16 22 14 7 2 1

25 27 4 10 5 12 14 16

1.19 1.82 4.80 2.29

Table 4. Estimated Effects for Main Effects Model

1 C2 C3 C4 T

estimate

std error

t-value

tail probability

−1.265 2.028 1.143 −1.412 0.676

0.325 0.419 0.423 0.622 0.340

−3.89 4.83 2.70 −2.133 1.98

.000090 .000001 .00681 .03295 .04676

model with interaction 1 + T + C + T.C yields λ = 2.66 on 4 degrees of freedom. Therefore, the interaction effect may be omitted. Further simplification does not seem advisable as the likelihood ratio test for the relevance of the main effect treatment yields λ = 4.06 on 1 degree of freedom and, for the main effect center, λ = 48.94 on 3 degrees of freedom. The estimates for the main effect model with center 1 as reference category are given in Table 4. The Wald test for the effect of treatment shows significance at the 0.05 level, confirming the result of the likelihood ratio test. REFERENCES 1. L. Fahrmeir and G. Tutz, Multivariate Statistical Modelling based on Generalized Linear Models, 2nd ed. New York: Springer, 2001. 2. P. McCullagh and J. A. Nelder, Generalized Linear Models, 2nd ed. New York: Chapman & Hall, 1989. 3. L. Fahrmeir and H. Kaufmann, Consistency and asymptotic normality of the maximum

CATEGORICAL RESPONSE MODELS likelihood estimator in generalized linear models. Ann. Stat. 1985; 13: 342–368. 4. J. E. Grizzle, C. F. Starmer, and G. G. Koch, Analysis of categorical data by linear models. Biometrika 1969; 28: 137–156. 5. L. Fahrmeir, Asymptotic likelihood inference for nonhomogeneous observations. Statist. Hefte (N.F.) 1987; 28: 81–116. 6. P. McCullagh, Regression model for ordinal data (with discussion). J. Royal Stat. Soc. B 1980; 42: 108–127. ¨ a¨ and J. N. Matthews, The equivalence 7. E. La¨ ar of two models for ordinal data. Biometrika 1985; 72: 206–207. 8. S. Greenland, Alternative models for ordinal logistic regression. Stat. Med. 1994; 13: 1665–1677. 9. T. Hastie and R. Tibshirani, Generalized Additive Models. London: Chapman & Hall, 1990. 10. J. Fan and I. Gijbels, Censored regression: local linear approximation and their applications. J. Amer. Statist. Assoc. 1994; 89: 560–570. 11. D. J. Green and B. W. Silverman, Nonparametric Regression and Generalized Linear Models: A Roughness Penalty Approach. London: Chapman & Hall, 1994. 12. A. Agresti, Categorical Data Analysis. New York: Wiley, 2002. 13. A. Agresti, A survey of exact inference for contingency tables. Stat. Sci. 1992; 7: 131–177. 14. J. Cohen, The cost of dichotomization. Appl. Psycholog. Meas. 1983; 7: 249–253. 15. D. G. Altman, B. Lausen, W. Sauerbrei, and S. Schumacher, Dangers of using ‘‘optimal’’ cutpoints in the evaluation of prognostic factors. J. Nat. Cancer Inst. 1994; 86: 829–835. 16. P. Beitler and J. Landis, A mixed-effects model for categorical data. Biometrics 1985; 41: 991–1000.

7

CAUSAL INFERENCE

Since it is not possible to observe the value of Y t (u) and Y c (u) at the same time, one cannot observe the causal effect of the experiment medicine on subject u. This is considered to be the Fundamental Problem of Causal Inference. To solve this problem, one turns to using a set of units with similar characteristics to infer the average causality. The SUTVA (stable unit treatment value assumption) is essential for ensuring the equivalence between the effect at a unit and the effect in the set of units (the Population). SUTVA assumes (1) no interference between units; and (2) no hidden version of treatment; i.e., no matter how unit u receives a treatment, the observed outcome would be Yt (u) or Y c (u). When special assumptions are added to the above model, causal effect can be inferred. Some of those assumptions are described below, again, in the context of clinical trials.

YILI L. PRITCHETT Abbott Laboratories Abbott Park, Illinois

Causal inference refers to making statistical inference with respect to causality. An effect of a cause is always defined in the context with respect to another cause. Cause is different from attributes; that is, causal relationships are different from associational relationships. Associational inferences use observed data to estimate how different outcomes from the same experimental unit are related. A typical example is that, correlation, the most commonly used statistic to quantify the relation between two variables, does not infer any causation. To evaluate causal effect, an experimental setting is the desired and the simplest setup. Ronald A. Fisher (1) and Jerzy Neyman (2) pioneered the design of experiments for causal inference. Fisher’s sharp null hypothesis testing, using P-values to quantify the strength of the evidence when rejecting a null hypothesis, and his exact test for 2 × 2 tables were among his many significant contributions to causal inference. Neyman first came up with the notation of potential outcome, which provided the foundation for a general framework developed later by Donald B. Rubin (3). Rubin’s causal model can be described in the setting of clinical trials as follows, where ‘‘treatment’’ is the cause of interest. Assume there is a population U of subjects. Let S be a binary variable of treatment assignment. S(u) = t indicates that the subject u receives the experimental medicine, and S(u) = c indicates the control medicine. S(u) for each subject could have been different. Let Y(u) be the outcome measure of the clinical trial. On each subject u, both Yt (u) and Yc (u) are potential outcomes. However, only one of those will be observed in the study, depending on which treatment the subject is exposed to. The causal effect of t on subject u is defined as Yt (u) − Yc (u)

1. Temporal Stability and Causal Transience Assumptions. Temporal stability assumes consistency of potential outcomes over time; that is, a subject’s outcome is independent from when the treatment is given. Causal transience assumes that the previous treatment effect under c is transient so that it has no impact on the potential outcome under current treatment of t, and vise versa. Under these two assumptions, it is plausible that one can expose a subject to different treatments in sequence, t and c, and observe outcomes under each treatment. The causal effect can then be estimated by Y t (u) − Yc (u). These two assumptions are the fundamental assumptions supporting the validity of using crossover design to evaluate causal effect. 2. Unit Homogeneity Assumption. Assume that Yt is homogeneous over different subjects, and so is Yc . Thus, the causal effect of t can be estimated as Yt (u1 ) and Yc (u2 ). Case-control studies are operated under this assumption (4). In those types of studies, to evaluate a cause effect, subjects with similar (if not identical) demographic and base-

(1)

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CAUSAL INFERENCE

line characteristics are matched with only an exception for the existence of a condition, which is under investigation as a potential cause. The outcome measures are compared between these two groups of subjects, and the difference will be attributed to the only difference of possessing that specific condition. Unit homogeneity assumption implies a weaker assumption than the constant effect assumption, which assumes that the cause effect is a constant over the population. 3. Independence Assumption. Denote the average causal effect of T on population U as T = E(Yt − Yc ) = E(Yt ) − E(Yc )

(2)

If the average potential outcome of Yt and Yc on the population can be estimated using observed data, the average causal effect T can then be estimated. In parallel group design, at least half of Yt (u) and Yc (u) are missing. What one can estimate is E(Yt |S = t), which is the conditional average of outcome Yt over those who receive t. Similarly, it is also estimable for E(Yc |S = c). When subjects are assigned at random to either group by a certain physical mechanism, the determination of which treatment the subject is exposed to is regarded as statistically independent of all other variables over U, including outcomes Yt and Yc . That is, in a randomized clinical trial, if the randomization procedure is carried out correctly, it is plausible that S is independent of Yt and Yc . Therefore, we have E(Yt ) = E(Ys |S = t) E(Yc ) = E(Ys |S = c) The above equations imply

estimated by the between-difference in sample means as has been routinely performed in parallel design, randomized clinical trials. Holland (5) introduced the concept of prima facie causal effect, TPF , an associational parameter for the joint distribution of (YS , S), as TPF = E(YS |S = t) − E(YS |S = c)

(4)

In general, TPF and T are not equal, if the independence assumption is not true. Randomized experiments play an important role in causal inference with the reason as shown above. In nonrandomized experiments, Equation (3) no longer holds true, because the units exposed to one treatment generally differ systematically from the units exposed to the other treatment. That is, the sample means are confounded with selection bias, and they cannot be used directly to infer to causal effect. A solution to this problem is to create a balancing score b(X), which is a function of observed covariates X such that the treatment assignment is independent to X conditioning on b(X). Therefore, causal effect can be evaluated at each stratum characterized by b(X). The average effect over b(X) can be used to estimate the average cause effect on the population. In that approach, the unit-level probability of treatment assignment is called the propensity score (6). Causal inference highly relies on how the treatment is assigned to the experimental units. Using the concept of ‘‘assignment mechanism,’’ where the probability that a specific treatment is assigned to an experimental unit is modeled, randomized, nonrandomized, and response adaptive randomized experiments can be characterized within the same framework. To illustrate, let Pr(S = t|X,YS ,YC ) denote the assignment mechanism for a clinical trial and X the covariates, YS and YC are the potential outcomes, and YOBS are the accumulate observed outcomes. We have

T = E(Ye ) − E(Yc ) (3)

• Pr(S = t|X,YS ,YC ) = Pr(S = t), when this

That is, when the independence assumption holds true, the average causal effect T can be

• Pr(S = t|X,YS ,YC ) = Pr(S = t|X), when

= E[YS |S = t] − E[YS |S = c]

is a classic randomized design. this is a nonrandomized design.

CAUSAL INFERENCE • Pr(S = t|X,YS ,YC ) = Pr(S = t|YOBS ),

when this is a response-adaptive, randomized design. In this type of design, the probability of assigning treatment t is not a constant any more, but it is dynamically adapting based on the observed response data. Of note, causal inference is not the same in concept as causal modeling that has been extensively used in social science. In causal modeling, path diagrams are used to describe the relationships among variables that have plausible causal or associational relationships. For example, studying the relationship between age and job satisfaction (7), a plausible assumption was ‘‘the older the employee gets, the more satisfied he is with his job.’’ When other intervening factors were also taken into consideration, such as the responsibility and the income associated with the job, the relation between age and job satisfaction became more complicated. One could use a path diagram to put those plausible relationships together and could use regression approaches to find the answer. That is, the coefficients between variables estimated from regression models could be used to quantify the relationship from age to Job satisfaction on each potential pathway. The sum of the coefficients among several pathways provided a quantitative measure on the relationship between the two variables. One can see that the two variables of interests in this example were not causal in nature. This example demonstrated that the application of causal modeling could deliver answers for associational inference but not necessary for causal inference. However, if all variables included in the causal modeling are causal factors, then causal modeling can be used as one of the tools to deduct the causal effects. REFERENCES 1. R. A. Fisher, The Design of Experiments. Edinburgh: Oliver & Boyd, 1935. 2. J. Neyman, On the application of probability theory to agricultural experiments. In: Essay on Principles. Section 9 (in Polish). Reprinted in English in Stat. Sci. 1990; 5:463– 480.

3

3. D. B. Rubin, Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 1974; 66:688– 701. 4. D. B. Rubin, Matched Sampling for Causal Effects. Cambridge, UK: Cambridge University Press, 2006. 5. P. W. Holland, Statistics and causal inference (with discussion). J. Am. Stat. Assoc. 1986; 81:945 − 970. 6. P. R. Rosenbaum and D. B. Rubin, The central role of the propensity score in observational studies for causal effects. Biometrika 1983; 70:41 − 55. 7. R. D. Retherford and M. K. Choe, Statistical Models for Causal Analysis. New York: Wiley, 1993.

CELL LINE

grow for long periods in culture if they are fed regularly under aseptic conditions (3). It was not until the 1940s that evidence began to develop to indicate that vertebrate cells could be maintained indefinitely in culture. Wilton Earle, at the National Cancer Institute, isolated single cells of the L cell line and showed that they form clones of cells in tissue culture (4). At around the same time, Gey et al. (5) were attempting to establish continuous cultures from rat and human tissue. One of their successes was the culture of continuous line of cells from human cervical carcinoma, the cell line that later become the well-known HeLa cell line (5). The cultivation of human cells received another stimulus by the development of different serum-free selective media as well as by the greater control of contamination with antibiotics and clean–air equipment.

THERU A. SIVAKUMARAN SUDHA K. IYENGAR Case Western Reserve University Department of Epidemiology and Biostatistics Cleveland, Ohio

Culturing of cells is a process whereby cells derived from various tissues are grown with appropriate nutrients under appropriate environmental conditions. Cell cultures and lines can be used as an alternative approach to the use of live animals for studying models of physiological function. A cell line, which is formed after first subculture of the primary culture, is a permanently established cell culture that can proliferate indefinitely under appropriate conditions. The formation of a cell line from primary culture implies an increase in the total number of cells over several generations as well as the ultimate predominance of the cells or cell lineage with the capacity for high growth, which results in a degree of uniformity in the cell population. Although all the cells in a cell line are very similar, they may have similar or distinct phenotypes. If one cell lineage is selected to have certain specific properties in the bulk of cells in the culture, this becomes cell strain. 1

2

PRIMARY CELL CULTURE

The primary culture is the stage that starts from isolation cells, is applied until the culture reaches confluence, and is split into multiple cultures for the first time. At this stage, the cells are usually heterogeneous but still represent the parent cell types and expression of tissue-specific properties. Primary cell culture can be obtained by one of three methods:

CELL CULTURE: BACKGROUND

2.1 Mechanical Disaggregation The tissue is disaggregated by chopping with a scalpel or by forcing the tissue through a mesh screen or syringe needle. The resulting suspension of cells is allowed to attach to an appropriate substrate to enable growth.

The concept of maintaining the live cell that was separated from original tissue was discovered in the nineteenth century. In late 1885, Wilhelm Roux (1) established the principle of tissue culture by demonstrating that embryonic chick cells can be maintained outside the animal body in a saline solution for several days. The first success in culturing animal cells is attributed to Harrison (2), who showed that the spinal cord from frog embryo could be cultured in a natural medium of clotted frog lymph; he also observed outgrowth of nerve fibers from the explant. The man who was responsible for establishing the idea of the cell culture was Alexis Carrel. He showed that cells could

2.2 Enzymatic Disaggregation Cell–cell and cell–substrate adhesion is generally mediated by three major classes of transmembrane proteins: the cell–cell adhesion molecules, cell–substrate interactions molecules, and transmembrane proteoglycans. Proteolytic enzymes, such as trypsin, collagenase, and so on, are usually added to the tissue fragments to dissolve the cement that holds the cells together, and enzymatic

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CELL LINE

disaggreation is the most widely used method to isolate cells for primary culture. 2.3 Primary Explanation This method was originally developed by Harrison (2) for initiating cell culture, and it is useful when a small amount of tissue is available. The tissue is finely chopped and rinsed, and the small fragments are seeded onto the surface of the culture vessel so it will adhere. Adherence is facilitated by scratching the plastic surface or trapping the tissue under a coverslip, or by use of surface tension created by adding a small volume of medium with a high concentration of serum or clotted plasma. Once adhesion is achieved, the primary culture develops by outgrowth from the explant. The tendency of animal cells in vivo to interact with one another and with the surrounding extracellular matrix is mimicked in their growth in culture. After tissue aggregation or subculture, most normal cells, with the exception of hematopoietic cells, need to spread out on a substrate to proliferate, and the cells are said to be anchorage dependent. Cells that can grow in suspension such as hematopoietic cells, tumor cells, transformed cell lines, and so on, are known as adherent independent. Cell adhesion is known to be mediated by specific cell surface receptor molecules in the extracellular matrix, and it is likely that secretion of extracellular matrix protein and proteoglycans may precede spreading by the cells. These matrix components bind to the culture surface and function as a bridge between the surface and the cells. Hence, the culture surface can be conditioned by treating it with spent medium from another culture or by purified fibronectin or collagen. 3

SUBCULTURE

When cells in culture have grown and filled up the entire available surface, they must be transferred, subcultured, or passaged into fresh culture vessels to give them room for continued growth. The subculture process involves removal of the culture medium and addition of trypsin to dislodge the cells from the substrate and to disrupt the cell–cell

connections. When the primary cell culture is thus subcultured, it generates a secondary culture that becomes known as a cell line. Each cell line can be subcultured several times such that a secondary subculture can then yield to tertiary culture and so on; this nomenclature is seldom used beyond the tertiary culture. With each successive subculture, the components of the cell population with the ability to proliferate most rapidly will gradually predominate, and nonproliferating or slowly proliferating cells will be diluted out. This finding is apparent after first subculture, in which differences in proliferative capacity of cells is compounded by varying abilities and transfer. To assess when a cell line will require subculture or how long cells will take to reach a certain density, it is necessary to become familiar with the cell cycle for each cell line, as cells at different phases of the growth cycle proliferate at different rates and show differences in respiration, synthesis of a variety of products, enzyme activity, and so on. This growth cycle is typically divided into three phases (6). 3.1 The Lag Phase When cells are taken from a stationary culture, a lag occurs before growth begins (this lag is the time that follows reseeding after subculture) (Fig. 1). During this time, cells replace elements of the cell surface, and the extracellular matrix that is lost during trypsinization adheres to the substrate and spreads out. 3.2 The Log Phase The log phase is the phase when the cells grow and the population doubles. The length of the log phase depends on the density at which the cells were seeded (initial culture), the growth rate of the cells, and the final concentration that will inhibit additional cell proliferation. The log phase, when the growth rate is as high as 90–100%, is the optimal time for using cells for experiments as the culture is in its most reproducible form. 3.3 The Plateau Phase At the end of the log phase, the entire available growth surface is occupied, and every

CELL LINE

3

Saturation density 106

Cells/cm2

Cells/ml

105

105

Figure 1. Three different phases of the growth cycle of cultured cells. Reproduced from Freshney’s Culture of animal cells: A manual of basic technique (6).

104

cell is in contact with its surroundings; this phenomenon is known as confluence. As a result, the growth rate slows and eventually may stop; specialized proteins may now be synthesized faster than during other phases. The growth rate in the plateau phase drops to 0–10%. Because the transformed cells are deficient in contact inhibition of cell motility and density limitation of cell proliferation, they will continue to proliferate for several generations even after reaching confluence, and they will reach higher saturation densities at plateau. The plateau phase of these cultures is a balance between increased proliferation and cell loss from apoptosis, which results in a greater turnover than occurs in normal cells at plateau. The construction of a growth curve from cell counts made at regular intervals after initiation of the subculture enables the measurements of parameters, such as lag period or lag time as well as population doubling time and saturation density, that determine the characteristics of the cell line under a given set of culture conditions. When these parameters are consistent, they can be used to calculate variables such as split ratio. The latter is defined as the degree of dilution required to reinitiate a new growth cycle with a short lag period and to achieve the appropriate density for subculture at a convenient time in the future.

104

Doubling time

Seeding concentration Lag 0

4

2

4 6 8 Days from subculture

10

EVOLUTION OF CELL LINES

The cell lines that are derived from normal tissue can divide only for a limited number of times, and their lifespan is determined by the cell type as well as by the environmental conditions at the first subculture. If the medium, substrate, and other conditions are appropriate, then the cell line may progress through several serial subcultures (Fig. 2). Conversely, if the conditions are not adequate, then the cell line will die out within one or two subcultures. Even for cell lines grown under satisfactory conditions, the maximum number of attainable subcultures (passages) is fixed. After a few passages, cells enter a state where they no longer divide, which is known as replicative senescence. For example, normal human fibroblasts typically divide only 25–40 times in culture before they stop. Senescence is a genetic event determined by the inability of terminal DNA sequences in the telomeres of each chromosome to replicate at each cell division, which results in progressive shortening of telomeres, until the cell cannot divide any more (7). Human somatic cells have turned off the enzyme, called telomerase, that normally maintains the telomeres and that is why their telomeres shorten with each cell division. The cell lines that only provide a limited number of population doublings are called finite cell lines (6). Some cell lines, notably those from rodents and most tumors, are not limited by a finite

4

CELL LINE

Transformation

1020

Cumulative Cell Number

1018

Primary Culture

Continuous Cell Line

Finite Cell line

1016

1st Subculture 2nd Subculture

1014

Senescence and Cell Death

1012 1010

Subculture Interval 108 Serial Passage 106

0

Explantation

2

4

6 8 10 Weeks in Culture

12

14

100

Figure 2. Evolution of a cell line. Reproduced from Freshney’s Culture of animal cells: A manual of basic technique (6).

lifespan, and they can continue to divide indefinitely. These cell lines are called continuous cell lines. The ability of growing continuously is reflective of the action of specific genetic variations, such as deletions or mutations of the p53 gene and overexpression of the telomerase gene. 5 DEVELOPMENT OF IMMORTALIZED CELL LINES The use of normal human cells in biomedical research is restricted in part by the limited proliferative potential of primary cultures. Thus, researchers need to re-establish fresh cultures from explanted tissue frequently, which is a tedious process. To use the same cells throughout an ongoing research project, primary cells need to extend their replicative capacity or undergo immortalization. Some cells immortalize spontaneously by passing through replicative senescence and thus adapt easily to life in culture. However, these spontaneously immortalized cells invariably have unstable genotypes and are host to numerous genetic mutations, which renders them less reliable representatives of their starting tissue’s phenotype. Many applications would require having essentially unlimited numbers of carefully characterized cells

with the desired phenotypes. Therefore, the ideal immortalization protocol would produce cells that are capable not only of extended proliferation, but also possess the same genotype and tissue markers of their parental tissue. The most commonly used method to attain this objective is the transduction of normal cells, before they enter senescence, with genes from DNA tumor virus, such as simian virus 40 (SV40), a human papillomavirus (HPV), or Epstein-Barr virus (EBV). 5.1.1 SV40. The major region of SV40 oncoprotein encoded by early region DNA, which is the larger T antigen (LT) gene, inactivates the p53 and Rb proteins. Cells that express these oncoproteins proliferate beyond the point of senescence. Transduction of normal cells with the SV40 early region genes by transfection of an expression plasmid remains a very common immortalization technique for almost any human cell type (8). 5.1.2 HPV. The E6 and E7 oncogenes of high-risk oncogenic risk HPV strains, which are HPV types 16 and 18, have been shown to assist the immortalization of several different cell types. The E6 protein causes degradation of the p53 protein, it upregulates the

CELL LINE

c-myc expression, and it also partially activates the telomerase (9). The HPV16 E7 protein induces degradation of the Rb protein via ubiquitin-proteasome pathway, and this mechanism explains the immortalization ability of E7 gene independently of E6 gene (10). 5.1.3 EBV. This herpes virus has the unique ability of transforming B lymphocytes into permanent, latently infected lymphoblastoid cell lines. Every infected cell carries multiple extrachromosomal copies of the viral episome and constitutively expresses a limited set of viral gene products, which are called latent proteins, that comprise six EBV nuclear antigens (EBNAs 1, 2, 3A, 3B, 3 C, and -LP) and three latent membrane proteins (LMPs 1, 2A, and 2B). The latent membrane protein 1 and EBNA1 are likely to be required for EBV to immortalize B lymphocytes. They have been shown to mediate the increase in p53 levels via activation of the NF-kB transcription factor (11). The process of transfection or infection extends the proliferative capacity of cells to 20–30 population doublings before entering a state called crisis. In some cultures, very few cells (about 1 per 107 ) recommence proliferation after a variable period of crisis, probably because of additional genetic changes, and form an immortalized cell line. The limitation of transformation with DNA tumor viruses is that the proteins they encode may cause undesirable changes in the resulting immortalized cell lines (e.g., loss of some differentiated properties and loss of normal cellcycle checkpoint controls). These unwanted changes can be avoided by direct induction of telomerase activity. This activity is done by transfecting the cells with the telomerase gene htrt, which thereby extends the life span of the cell line such that a proportion of these cells become immortal but not malignantly transformed (7). 5.2 Characterization and Authentication of Cell Lines Research that involves the use of cell lines demands the precise knowledge of the purity and species of origin of the working cell lines. In the facilities where multiple cell

5

lines are maintained, the possibility of cross contamination can occur. In some cases, the contaminating colony of cells in the culture may be identified visually. In other cases, this unintentional coculture cannot be determined by visual inspection. Therefore, it is important to verify the identity and purity of cell cultures periodically. In the absence of such a monitoring system, it is possible that interspecies and intraspecies cell line contamination may occur, which results in the generation of false conclusions. Apart from this cross contamination, most human cell lines are prone to various sorts of genetic rearrangements, such as chromosomal aneuploidy, deletion of chromosomal regions, and so on, which affect the biochemical, regulatory, and other phenotypic features of cells during their cultivation. Therefore, it would be very useful to monitor the authenticity of the cell lines permanently and/or to have a comprehensive set of standard tests for confirming their cellular identity. Finally, instances occur when hybrid cells lines from two species are created, which are called somatic cell hybrids (e.g., culture lines that are developed with a mix of human and mouse cells through Sendai virus fusion). These types of hybrids predominantly retain the genome of one of the species but also keep some chromosomal complement from the alternate species. Herein, it would be necessary to determine which species contributed which element of the genomic complement. The standard approach that is generally followed is derivation of multiple immunological and genetic marker system profiles from each cell culture to be tested. Once an early molecular signature of the cells is obtained, it can later be compared with all future cell lines for identity validation. Some methods used to derive these profiles are described below. 5.2.1 Isoenzyme Analysis. Isoenzymes are enzymes that exhibit interspecies and intraspecies polymorphisms. Isoenzymes have similar catalytic properties but differ in their structure, and they can be detected based on their differences in electrophoretic mobility. By studying the isoenzymes present in the cell lines, it would be possible

6

CELL LINE

to identify the species from which the cell line was derived. 5.2.2 Chromosomal Analysis. Chromosome content is one of the best-defined criteria for differentiating cell lines derived from more than one species and sex. Chromosomal banding patterns (e.g., trypsin-Giemsa banding or G-banding) can distinguish between normal and malignant cells and can also distinguish between human and mouse chromosomes. Thus, this technology is used frequently in cell line characterization. Fluorescence in situ hybridization is another powerful and rapid method for detecting aneuploidy, chromosomal deletions, and so on. Using different combinations of chromosomeand species-specific probes to hybridize to individual chromosomes, it would be possible to identify chromosomal fragments in metaphases of interspecies hybrid cells. 5.2.3 DNA Profiling. This method is based on the existence of dispersed hypervariable regions of tandem-repetitive nucleotide sequences in the genome (12). The original approach is based on using multilocus Variable Number of Tandem Repeats (VNTR) probes and Southern blotting, which is a DNA analysis technique that works on the principle that smaller fragments of DNA electrophorese at rates faster than larger fragments of DNA. When the DNA from a cell line analyzed by Southern blotting with probes targeted to single locus, many unique size fragments characteristic of many different alleles encoding that particular locus are identified. Given the high degree of heterogeneity at every locus, each cell line produces two specific bands. After examining many such VNTR loci, application of multiple single-locus probes to a single enzyme yields a unique cell line profile. This profile is cell line specific unless more than one cell line is derived from the same individual or if highly inbred donor subjects have been used. More recently, other techniques that can produce profiles more rapidly have been advocated. Thus, the second approach is a more rapid method that involves amplification of microsatellite markers using species-specific primers by polymerase chain reaction; this method is used only for human cell lines. The

banding pattern produced by electrophoretic separation of amplified DNA products can be compared with archived profiles on a database, which is similar to what is done for forensic profiling. Like multilocus VNTR analysis, this technique can also be used to demonstrate continuity between master and working banks and to detect contamination and instability in cell lines. 5.3 Cell Banks Advances in cell technology enable scientists to isolate and cultivate any tissue cells from any species of interest. Alternatively, wellcharacterized cell lines can be obtained from cell banks. Several cell banks such as American Type Culture Collection (ATCC), European Collection for Animal Cell Cultures, Coriell Institute for Medical Research, and so on, acquire, preserve, authenticate, and distribute reference cell lines to the scientific communities. Currently, the ATCC house over 4000 cell lines derived from over 150 different species (Table 1). Once obtained, cell lines can be preserved in cell banks for future use.

REFERENCES 1. J. Paul, Achievement and challenge. In: C. Barigozzi (ed.), Progress in Clinical and Biological Research, vol. 26. New York: Alan R Liss, Inc., 1977, pp. 3–10. 2. A. G. Harrison, Observations on the living developing nerve fiber. Proc. Soc. Exp. Biol. Med. 1907; 4: 140–143. 3. A. Carrel, Artificial activation of the growth in vitro of connective tissue. J. Exp. Med. 1913; 17: 14–19. 4. W. R. Earle, E. L. Schilling, T. H. Stark, N. P. Straus, M. F. Brown, and E. Shelton, Production of malignacy in vitro; IV: the mouse fibroblast cultures and changes seen in the living cells. J. Natl. Cancer Inst. 1943; 4: 165–212. 5. G. O. Gey, W. D. Coffman, and M. T. Kubicek, Tissue culture studies of the proliferative capacity of cervical carcinoma and normal epithelium. Cancer Res. 1952; 12: 364–365. 6. R. I. Freshney, Culture of Animal Cells: A Manual of Basic Technique. New York: WileyLiss, 2000.

CELL LINE

7

Table 1. List of Some Commonly used Cell Lines Cell Line

Cell Type

Origin

Age

MRC-5 MRC-9 BHK21-C13 HeLa 293 3 T3-A31 CHO-K1 WI-38 ARPE-19 C2 BRL3A NRK49F A2780 A9 B16 MOG-G-CCM SK/HEP-1 Caco-2 HL-60 Friend ZR-75-1 C1300 HT-29 KB Vero

Fibroblast Fibroblast Fibroblast Epithelial Epithelial Fibroblast Fibroblast Fibroblast Epithelial Fibroblastoid Fibroblast Fibroblast Epithelial Fibroblast Fibroblastoid Epithelioid Endothelial Epithelial Suspension Suspension Epithelial Neuronal Epithelial Epithelial Fibroblast

Human lung Human lung Syrian hamster kidney Human cervix Human kidney Mouse BALB/c Chinese hamster ovary Human lung Human retina (RPE) Mouse Skeletal muscle Rat liver Rat kidney Human ovary Mouse subcutaneous Mouse melanoma Human glioma Human hepatoma Human colon Human myeloid leukemia Mouse spleen Human breast, ascites fluid Rat neuroblastoma Human colon Human oral Monkey kidney

Embryonic Embryonic Newborn Adult Embryonic Embryonic Adult Embryonic Adult Embryonic Newborn Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult Adult

7. A. G. Bodnar, M. Ouellette, M. Frolkis, S. E. Holt, C. P. Chiu, G. B. Morin, et al., Extension of life-span by introduction of telomerase into normal human cells. Science 1998; 279: 349–352. 8. L. V. Mayne, T. N. C. Price, K. Moorwood, and J. F. Burke, Development of immortal human fibroblast cell lines. In: R. I. Freshney and N. R. Cooper (eds.), Culture of Immortalized Cells. New York: Wiley-Liss, 1996, pp. 77–93. 9. A. J. Klingelhutz, S. A. Foster, and J. K. McDougall, Telomerase activation by the E6 gene product of human papillomavirus type 16. Nature 1996; 380: 79–82. 10. S. N. Boyer, D. E. Wazer, and V. Band, E7 protein of human papilloma virus-16 induces degradation of retinoblastoma protein through the ubiquitin-proteasome pathway. Cancer Res. 1996; 56: 4620–4624. 11. W. P. Chen and N. R. Cooper, Epstein-Barr virus nuclear antigen 2 and latent membrane protein independently transactivate p53 through induction of NF-kappa B activity. J. Virol. 1996; 70: 4849–4853. 12. A. J. Jeffreys, V. Wilson, and S. L. Thein, Individual-specific fingerprints of human

DNA. Nature 1985; 316: 76–79.

FURTHER READING J. P. Mather and D. Barnes, eds. Animal cell culture methods. In: Methods in Cell Biology, vol. 57. San Diego, CA: Academic Press, 1997.

CROSS-REFERENCES In vitro/in vivo correlation Pharmacokinetic study Toxicity Pharmacogenomics Microarray DNA Bank

CENSORED DATA

1. The presence of censoring may alter the hazard function of the lifetime X i , i.e. the conditional distribution of X i , given that i is alive at t(X i ≥ t) and uncensored at t (U i ≥ t), may be different from what it was in the uncensored case, i.e. just given X i ≥ t (dependent censoring). 2. The observed right-censoring times, U i , may contain information on θ (informative censoring).

PER KRAGH ANDERSEN University of Copenhagen, Copenhagen, Denmark

In classical statistics, the observations are frequently assumed to include independent random variables X 1 , . . . , X n , with X i having the density function

An example of a dependent censoring scheme would be if, in a clinical trial with survival times as the outcome variables, one removed patients from the study while still alive and when they appeared to be particularly ill (or particularly well), so that patients remaining at risk are not representative of the group that would have been observed in the absence of censoring. In other words, dependent censoring represents a dynamic version of what in an epidemiologic context would be termed a selection bias. An example is provided below (Example 1). Mathematical formulations of independent censoring (conditions on the joint distribution of X i and U i ) may be given, and it may be shown that several frequently used models for the generation of the times of right-censoring satisfy these conditions. The difficulty in a given practical context lies in the fact that the conditions may be impossible to verify, since they refer to quite hypothetical situations. The second concept mentioned, noninformative censoring, is simpler and relates to the fact that if censoring is informative, then a more efficient inference on θ may be obtained than the one based on (2); see below.

fiθ (x) = αiθ (x)Sθi (x), where αiθ (x) is the hazard function, Sθi (x) is the survival function, and θ is a vector of unknown parameters. Then inference on θ may be based on the likelihood function, L(θ ) =




fiθ (Xi ),

i

in the usual way. In survival analysis, however, one can rarely avoid various kinds of incomplete observation. The most common form of this is right-censoring where the observations are (X˜ i , Di ),

i = 1, . . . , n,

(1)

where Di is the indicator I{X˜ i = Xi }, and X˜ i = Xi , the true survival time, if the observation of the lifetime of i is uncensored and X˜ i = Ui , the time of right-censoring, otherwise. Thus, Di = 1 indicates an uncensored observation, Di = 0 corresponds to a right-censored observation. Other kinds of incomplete observation will be discussed below. Survival analysis, then, deals with ways in which inference on θ may be performed based on the censored sample (1). We would like to use the function Lc (θ ) =




1

The general definition of independent censoring given by Andersen et al. 2, Section III.2.2 for multivariate counting processes has the following interpretation for the special case of survival analysis with time-fixed covariates. The basic (uncensored) model is that conditional on covariates Z = (Z1 , . . . , Zn ) the lifetimes X 1 , . . . , X n are independent, X i

αiθ (X˜ i )Di Sθi (X˜ i )

i

=




fiθ (X˜ i )Di Sθi (X˜ i )1−Di

INDEPENDENT CENSORING

(2)

i

for inference, but there are two basic problems:

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CENSORED DATA

having the hazard function αiθ (t|Zi ) ≈ Pθ φ (Xi ∈ Idt |Xi ≥ t, Z)/dt.

(3)

Here, Idt is the interval [t, t + dt) and Pθ φ is the joint distribution of X 1 , . . . , X n , Z and the censoring times. Note that the hazard function only depends on θ , i.e. φ is a nuisance parameter. Because of the conditional independence of X i it follows that Pθ φ (Xi ∈ Idt |Ft− ) ≈ αiθ (t|Zi )I{Xi ≥ t}dt, where the history Ft− contains Z and all information on X 1 , . . . , X n from the interval [0, t), i.e. values of X i for i with X i < t and the information that X j ≥ t for j with X j ≥ t. Let there now be given right-censoring times U 1 , . . . , U n and define the enlarged history Gt as the one containing Ft and all information on U 1 , . . . , U n from the interval [0, t], i.e. values of U i ≤ t and the information that U j ≥ t for those j where U j ≥ t. The condition for independent censoring is then that Pθ φ (Xi ∈ Idt |Ft− ) = Pθ φ (Xi ∈ Idt |Gt− ).

(4)

It follows that simple type I censoring, where all U i are equal to a fixed time, u0 , and simple type II censoring, where all U i are equal to the kth smallest lifetime X (k) for some k between 1 and n, are both independent, since the rightcensoring times in these cases give rise to no extra randomness in the model; that is, Ft = Gt . In some models, U 1 , . . . , U n are assumed to be independent given Z and Z1 , . . . , Zn are independent identically distributed (iid). Then the assumption (4) reduces to αiθ (t|Zi ) ≈ Pθ φ (Xi ∈ Idt |Xi ≥ t, Ui ≥ t, Z)/dt (5) and it is fulfilled, e.g. if U i and X i are independent given Zi . This is, for instance, the case in the simple random censorship model where U 1 , . . . , U n are iid and independent of X1, . . . , Xn. Some authors take the condition (5) (which is less restrictive than (4)) as the definition of independent censoring; see, for example, 6, p. 128. However, (4) may be generalized to other models based on counting processes

and both (4) and (5) cover the most frequently used mathematical models for the right-censoring mechanisms. These include both the models already mentioned, i.e. simple type I, type II and random censorship and various generalizations of these (e.g. progressive type I censorship (cf. Example 2, below), general random censorship, and randomized progressive type II censorship; see, [2, Section III.2.2]). Earlier contributions to the definition and discussion of independent censoring are the monographs by Kalbfleisch & Prentice 13, p. 120 and Gill 7, Theorem 3.1.1 and the papers by Cox 5, Williams & Lagakos 16, Kalbfleisch & MacKay 12 and Arjas & Haara 3, all of whom give definitions that are close or equivalent to (5). Another condition for independent censoring, stronger than (5) but different from (4), is discussed by Jacobsen 11. From (4) and (5) it is seen that censoring is allowed to depend on covariates as long as these are included in the model for the hazard function of the lifetime distribution in (3). Thus, an example of a dependent censoring scheme is one where the distribution of U i depends on some covariates that are not included there. This is illustrated in the following example. 1.1 Example 1: Censoring Depending on Covariates Suppose that iid binary covariates, Z1 , . . . , Zn , have Pθ φ (Zi = 1) = 1 − Pθ φ (Zi = 0) = φ, and that X 1 , . . . , X n are iid with survival function S(t). The Kaplan–Meier estima based on the X i then provides a tor S(t) consistent estimate of θ = S(·), the marginal distribution of X i . This may be written as S(t) = φS1 (t) + (1 − φ)S0 (t), where Sj (t), for j = 0, 1, is the conditional distribution given Zi = j. Note that these may be different, e.g. S1 (t) < S0 (t) if individuals with Zi = 1 are at higher risk than those with Zi = 0. Define now the right-censoring times U i by Ui = u0 , if Zi = 1,

Ui = +∞, if Zi = 0.

CENSORED DATA

Then, for t < u0 the Kaplan–Meier estimator will still consistently estimate S(t), while for 
S(u0 ) will estimate S0 (t)/S0 (u0 ). t > u0 , S(t)/ If, however, the covariate is included in the model for the distribution of X i , i.e. θ = [S0 (·), S1 (·)], then S j (t), the Kaplan–Meier estimator based on individuals with Zi = j, j = 0, 1, will consistently estimate the corresponding Sj (t), also based on the right-censored sample (though, of course, no information will be provided about S1 (t) for t > u0 ). It is seen that censoring is allowed to depend on the past and on external (in the sense of conditionally independent) random variation. This means that if, in a lifetime study, sex and age are included as covariates, then a right-censoring scheme, where, say, every year, one out of the two oldest women still alive and uncensored is randomly (e.g. by flipping a coin) chosen to be censored, is independent. However, a right-censoring scheme depending on the future is dependent. This is illustrated in the following example. 1.2 Example 2: Censoring Depending on the Future Suppose that, in a clinical trial, patients are accrued at calendar times T 1 , . . . , T n and that they have iid lifetimes X 1 , . . . , X n (since entry) independent of the entry times. The study is terminated at calendar time t0 and the entry times are included in the observed history, i.e. Zi = T i in the above notation. If, at t0 , all patients are traced and those still alive are censored (at times U i = t0 − T i ) and, for those who have died, their respective lifetimes, X i , are recorded, then this right-censoring is independent (being deterministic, given the entry times, so-called progressive type I censoring). Consider now, instead, the situation where patients are only seen, for instance, every year, i.e. at times T i + 1, . . . , T i + ki ≤ t0 and suppose that if a patient does not show up at a scheduled follow-up time, then this is because he or she has died since last follow-up and the survival time is obtained. Suppose, further, that for the patients who are alive at the time, T i + ki , of their last scheduled follow-up, and who die before time t0 , there is a certain probability, φ, of obtaining information on the failure, whereas for those

3

who survive past t0 nothing new is learnt. If these extra survival times are included in the analysis and if everyone else is censored at ki , then the right-censoring scheme is dependent. This is because the fact that patient i is censored at ki tells the investigator that this patient is likely not to die before t0 and the right-censoring, therefore, depends on the future. To be precise, if the average probability of surviving past t0 , given survival until the last scheduled follow-up time is 1 − π , then the probability of surviving past t0 , given censoring at the time of the last scheduled follow-up, is (1 − π )/[π (1 − φ) + 1 − π ], which is 1 if φ = 1, 1 − π if φ = 0, and between 1 − π and 1, otherwise. If, alternatively, everyone still alive at time T i + ki were censored at ki , then the censoring would be independent (again being deterministic given the entry times). Another censoring scheme that may depend on the future relative to ‘‘time on study’’, but not relative to calendar time, occurs in connection with testing with replacement, see, for example, (8). Let us finally in this section discuss the relation between independent rightcensoring and competing risks. A competing risks model with two causes of failure, d and c, is an inhomogeneous Markov process W(·) with a transient state 0 (‘‘alive’’), two absorbing states d and c and two causespecific hazard functions α 0d (t) and α 0c (t), e.g. Andersen et al. 1. This generates two random variables: X = inf[t : W(t) = d] and U = inf[t : W(t) = c], which are incompletely observed since the observations consist of the transition time  = X∧U and the state W(X)  = d or c reached X at that time. The elusive concept of ‘‘independent competing risks’’ (e.g. [13, Section 7.2]) now states that in a population where the risk c is not operating, the hazard function for d is still given by α 0d (t). This condition is seen to be equivalent to censoring by U being independent. However, since the population where a given cause of failure is eliminated is

4

CENSORED DATA

usually completely hypothetical in a biological context, this formal equivalence between the two concepts is of little help in a practical situation and, as is well known from the competing risks literature (e.g. 4,15, and [13, Chapter 7]), statistical independence of the random variables X and U cannot be tested  W(X)].  from the incomplete observations [X, What can be said about the inference on the parameter θ = α 0d (·) based on these data is that consistent estimation of θ may be obtained by formally treating failures from cause c as right-censorings, but that this parameter has no interpretation as the d failure rate one would have had in the hypothetical situation where the cause c did not operate. For the concept of independent censoring to make sense, the ‘‘uncensored experiment’’ described in the beginning of this section should, therefore, be meaningful. 2 LIKELIHOODS: NONINFORMATIVE CENSORING The right-censored data will usually consist of i , Di , Zi ; i = 1, . . . , n) (X and, under independent censoring, the likelihood can then be written using productintegral notation L(θ , φ) = Pθ φ (Z)



i

αiθ (t)Di (dt)

t0 θφ

× [1 − αiθ (t)dt]1−Di (dt) γi (t)Ci (dt) θφ

× [1 − γi (t)dt]1−Ci (dt) .

(6)

Here, Di (dt) = I{X i ∈ Idt }, Ci (dt) = I{U i ∈ Idt }, θφ and αiθ (t) and γi (t) are the conditional hazards of failure and censoring, respectively, given the past up until t- (including covariates). The likelihood (6) may be written as L(θ , φ) = Lc (θ )L∗ (θ , φ), with Lc (θ ) given by (2) and where the contributions from censoring and covariates are collected in L*(θ , φ). Thus, the function (2), which is usually taken as the standard censored data likelihood, is, under independent

censoring, a partial likelihood on which a valid inference on θ may be based. It is only the full likelihood for θ if L*(θ , φ) does not depend on θ , which is the case if censoring (and covariates) are noninformative. Thus, noninformative censoring is a statistical concept (while the concept of independent censoring is probabilistic) and means that the θφ conditional hazard of censoring γi (t) does, in fact, not depend on θ , the parameter of interest. An example of an informative rightcensoring scheme could be in a study with two competing causes of failure and where only one of the two cause-specific failure rates is of interest; if the two cause-specific failure rates are proportional (as in the so-called Koziol–Green model for random censoring, 14), then the failures from the second cause (the censorings) will carry information on the shape of the hazard function for the failure type of interest. It is, however, important to notice that even if the censoring is informative, then inference based on (2) will still be valid (though not fully efficient) and as it is usually preferable to make as few assumptions as possible about the distribution of the right-censoring times, the (partial) likelihood (2) is often the proper function to use for inference. 3 OTHER KINDS OF INCOMPLETE OBSERVATION When observation of a survival time, X, is right-censored, then the value of X is only known to belong to an interval of the form [U, + ∞). This is by far the most important kind of censoring for survival data, but not the only one. Thus, the observation of X is interval-censored if the value of X is only known to belong to an interval [U, V) and it is said to be left-censored if U = 0. It was seen above that under independent right-censoring a right-censored observation, U i , contributed to the partial likelihood function with a factor Sθ (U i ), which was also the contribution to the full likelihood under noninformative censoring. Similarly, concepts of independent and noninformative interval-censoring may be defined as leading to a contribution of Sθ (U i )–Sθ (V i ) to, respectively, the partial and the full likelihood.

CENSORED DATA

These concepts have received relatively little attention in the literature; however, this way of viewing censoring is closely related to the concept of coarsening at random. Formally, grouped data, where for each individual the lifetime is known only to belong to one of a fixed set of intervals [uk−1 , uk ) with 0 = u0 < u1 < · · · < um = +∞, are also interval-censored. However, the fact that the intervals are the same for everyone simplifies the likelihood to a binomial-type likelihood with parameters pθk = Sθ (uk−1 ) − Sθ (uk ), k = 1, . . . , m. Let us finally remark that while, following Hald [9,10, p. 144], censoring occurs when we are able to sample a complete population but individual values of observations above (or below) a given value are not specified, truncation corresponds to sampling from an incomplete population, i.e. from a conditional distribution. Left-truncated samples, where an individual is included only if his or her lifetime exceeds some given lower limit, also occur frequently in the analysis of survival data, especially in epidemiologic studies where hazard rates are often modeled as a function of age and where individuals are followed only from age at diagnosis of a given disease or from age at employment in a given factory. REFERENCES 1. Andersen, P.K., Abildstrom, S. & Rosthøj, S. (2002). Competing risks as a multistate model. Statistical Methods in Medical Research. 11, 203–215. 2. Andersen, P.K., Borgan, Ø., Gill, R.D. & Keiding, N. (1993). Statistical Models Based on Counting Processes. Springer-Verlag, New York. 3. Arjas, E. & Haara, P. (1984). A marked point process approach to censored failure data with complicated covariates, Scandinavian Journal of Statistics 11, 193–209. 4. Cox, D.R. (1959). The analysis of exponentially distributed life-times with two types of failure, Journal of the Royal Statistical Society, Series B 21, 411–421. 5. Cox, D.R. (1975). Partial likelihood, Biometrika 62, 269–276. 6. Fleming, T.R. & Harrington, D.P. (1991). Counting Processes and Survival Analysis. Wiley, New York.

5

7. Gill, R.D. (1980). Censoring and stochastic integrals, Mathematical Centre Tracts 124, Mathematisch Centrum, Amsterdam. 8. Gill, R.D. (1981). Testing with replacement and the product limit estimator, Annals of Statistics 9, 853–860. 9. Hald, A. (1949). Maximum likelihood estimation of the parameters of a normal distribution which is truncated at a known point, Skandinavisk Aktuarietidsskrift 32, 119–134. 10. Hald, A. (1952). Statistical Theory with Engineering Applications. Wiley, New York. 11. Jacobsen, M. (1989). Right censoring and martingale methods for failure time data, Annals of Statistics 17, 1133–1156. 12. Kalbfleisch, J.D. & MacKay, R.J. (1979). On constant-sum models for censored survival data, Biometrika 66, 87–90. 13. Kalbfleisch, J.D. & Prentice, R.L. (1980). The Statistical Analysis of Failure Time Data. Wiley, New York. 14. Koziol, J.A. & Green, S.B. (1976). A Cram´ervon Mises statistic for randomly censored data, Biometrika 63, 465–474. 15. Tsiatis, A.A. (1975). A nonidentifiability aspect of the problem of competing risks, Proceedings of the National Academy of Sciences 72, 20–22. 16. Williams, J.A. & Lagakos, S.W. (1977). Models for censored survival analysis: constant sum and variable sum models, Biometrika 64, 215–224.

CENTER FOR DEVICES AND RADIOLOGICAL HEALTH (CDRH)

the product’s safety. These specific regulations may include requirements for meeting performance standards recognized by the FDA, postmarket surveillance, patient registries, or other appropriate requirements. Class III: Premarket Approval. Class III devices are life supporting, life sustaining, or important in preventing impairment of human health. Because general controls may be insufficient to provide reasonable assurance of the device’s safety and effectiveness, FDA preapproval is required before it is marketed. Under Class III regulations, devices such as heart valves, breast implants, and cranial electrotherapy stimulators must be reviewed for safety and effectiveness before marketing.

The U.S. Food and Drug Administration’s Center for Devices and Radiological Health (CDRH) ensures the safety and effectiveness of medical devices and the safety of radiologic products. 1

MEDICAL DEVICES

The Food, Drug, and Cosmetic (FD&C) Act defines a medical device as any health-care product that does not achieve its principal intended purposes by chemical action or by being metabolized. Under this definition, a ‘‘device’’ can be as simple as a tongue depressor or a thermometer, or as complex as a kidney dialysis machine. Medical devices are classified and regulated according to their degree of risk to the public.

2

OBTAINING FDA APPROVAL

The CDRH works with both medical device and radiologic health industries. A manufacturer of a Class III device files a PreMarket Approval Application (PMA) to obtain FDA approval to market the product. Like the submission that is filed for the approval of a new drug, a PMA contains clinical and laboratory testing data to demonstrate safety and effectiveness. A Premarket Notification, also known as a 510(k), is an application submitted to the FDA to demonstrate that a medical device is substantially equivalent to (i.e., meaning just as safe and effective as) a legally marketed device that does not require premarket approval.

1.1 Regulatory Classes Because each device is different, the Food and Drug Administration (FDA) establishes three different regulatory classes to ensure that each device is subject to regulations that are appropriate. Class I: General Controls. Class I devices are subject to a set of general regulations that apply to all devices. General controls include the registration of manufacturers, general recordkeeping requirements, and compliance with Good Manufacturing Practice regulations. Class II: Special Controls. Class II devices are those for which general regulations are not enough to guarantee the safety of the device. A Class II device may be subject to specific regulations to ensure

3 GOOD MANUFACTURING PRACTICES (GMPS) The FDA further ensures the safety and effectiveness of medical devices by regulating their manufacture. As with drugs, the FDA has established Good Manufacturing

This article was modified from the website of the United States Food and Drug Administration (http://www.eduneering.com/fda/courses/FDATour3/tourFDA-frames-08.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CENTER FOR DEVICES AND RADIOLOGICAL HEALTH (CDRH)

Practices for medical devices and regularly inspects manufacturers to ensure they comply with these regulations. 4

CONTINUOUS ANALYSIS

After approval of a medical device, the FDA continuously analyzes reports to ensure that products are safe and to watch for dangerous events related to the use of medical devices. The CDRH also monitors certain electronic products to protect the public from unnecessary exposure to radiation. Products that are monitored by the FDA include televisions, microwave ovens, X-ray machines, and devices employing lasers (including laser light shows). The FDA administers the law by setting and enforcing standards to limit unnecessary radiation emissions.

CENTER FOR DRUG EVALUATION AND RESEARCH (CDER)

comprise the single most important factor in the approval or disapproval of a new drug and are the basis for a new drug application (NDA). Once the NDA is approved, the product can be legally marketed in the United States. 2. Postmarket Drug Surveillance. The CDER continuously monitors the safety of drugs that have already been marketed. After the drug is approved and marketed, the FDA uses different mechanisms for ensuring that firms adhere to the terms and conditions of approval described in the application and that the drug is manufactured in a consistent and controlled manner. This is done by periodic unannounced investigations of drug production and control facilities by the FDA’s field investigators and analysts. The Center also ensures that prescription drug information provided by drug firms is truthful, balanced, and accurately communicated. In addition, the CDER follows up with companies on concerns related to medication errors, drug shortages, and ineffective or toxic drugs. 3. Generic Drug Review. Generic drug applications are termed ‘‘abbreviated’’ in that they are generally not required to include preclinical (animal) and clinical (human) data to establish safety and effectiveness. The Abbreviated New Drug Application (ANDA) provides for the review and ultimate approval of a generic drug product. After all components of the application are found to be acceptable, an approval or tentative approval letter is issued to the applicant. Tentative approvals require the manufacturer to delay marketing the generic drug until all patent/exclusivity issues have expired. 4. Over-the-Counter Drug Review. Overthe-counter (OTC) drug products are those that are available to consumers without a prescription, and 6 out of 10 medications bought by consumers are OTC drugs. The CDER’s OTC program establishes drug monographs for each

The U.S. Food and Drug Administration’s Center for Drug Evaluation and Research (CDER) ensures that safe and effective prescription, nonprescription, and generic drugs are made available as quickly as possible by overseeing the research, development, manufacture, and marketing of drugs. The CDER reviews the clinical trial evidence of the safety and effectiveness of new drugs before approving them for marketing and monitors their performance for unexpected health risks. The CDER also ensures that drug labeling, drug information for patients, and drug promotion are truthful, helpful, and not misleading. 1

CDER MISSION

The Center is organized into four main functional areas. 1. New Drug Development and Review . Drug development is a highly complicated, lengthy process. During preclinical drug development, a sponsor evaluates the drug’s toxic and pharmacologic effects through in vitro and in vivo laboratory animal testing. Next, the firm submits an investigational new drug (IND) application to the CDER to obtain U.S. Food and Drug Administration (FDA) approval that the new drug is sufficiently safe to allow clinical trials. During clinical trials, the investigational drug is administered to humans and is evaluated for its safety and effectiveness in treating, preventing, or diagnosing a specific disease or condition. The results of this testing will This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/about/default.htm and http://www.eduneering.com/fda/courses/FDA Tour3/tourFDA-frames-06.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CENTER FOR DRUG EVALUATION AND RESEARCH (CDER)

class of product; these are like a recipe book, covering the acceptable ingredients, doses, formulations, and labeling. Products conforming to a monograph may be marketed without further FDA clearance, and those that do not must undergo separate review and approval through the new drug approval process.

FDA in 1 year or less; priority applications for breakthrough medications are usually approved in 6 months. The PDUFA user fees, however, do not cover the FDA’s expenses connected with generic and nonprescription drugs, plant inspections, postmarket surveillance, or monitoring of drug advertisements. 5 ACCELERATED APPROVAL

2

GOOD MANUFACTURING PRACTICES

To make sure that drugs are manufactured to the same high standards that are required for their approval, the FDA provides a set of regulations called Good Manufacturing Practices (GMPs). The law requires that the Office of Regulatory Affairs (ORA) perform periodic inspections of all drug firms for compliance with GMPs. 3

ADVERSE EVENT REPORTING

The FDA maintains several reporting systems that alert the Agency to side effects that were not detected during clinical trials but rather emerged when the product became widely used. One of these programs is the CDER’s MedWatch, which encourages health professionals to report serious adverse events involving any medical product (including drugs, devices, and biologics). If necessary, the FDA can take regulatory actions to protect consumers. Regulatory actions may include restrictions on the product’s use or its withdrawal from the market. About 1% to 3% of products approved each year must be removed later because of rare but serious side effects. 4

PRESCRIPTION DRUG USER FEE ACT

In the Prescription Drug User Fee Act of 1992 (PDUFA) the U.S. Congress, pharmaceutical industry, and FDA agreed on specific review goals for certain drugs and biologics, which are to be achieved with the help of user fees paid to the FDA by the products’ manufacturers. The program has been instrumental in reducing the FDA’s median drug-review times by more than half. As a result, typical drug applications are processed by the

Many of the drugs currently used to treat lifethreatening conditions such as cancer were approved through an accelerated FDA review process. In accelerated approval, the FDA approves the drug on the condition that the applicant will study and report the findings of the clinical benefit of the drug. The FDA continues to review new information and data about these drugs as the data become available; if the findings are negative, the appropriate actions are taken.

CENTRAL NERVOUS SYSTEM (CNS)

nutritional deficiencies, environmental toxicities, or cerebrovascular disorders, neurodegenerative conditions, headache syndromes, seizure disorders, movement disorders, or autoimmune-mediated diseases other than those listed. Finally, we restrict our discussion to RCTs and do not review epidemiological or observational studies.

DAVID B. SOMMER LARRY B. GOLDSTEIN Duke University Medical Center Durham, North Carolina

1

INTRODUCTION

2 CNS TRIALS: GENERAL INCENTIVES AND CONSTRAINTS

Clinical neurology has a rich history based on careful patient observation. Over the last quarter century, however, a major shift toward evidence-based medicine has occurred, and that shift includes the practice of neurology. Although many diagnostic and therapeutic approaches employed in neurology today have never been formally evaluated, clinicians, patients, and payers now seek informative results from well-designed and conducted clinical trials to help guide therapeutic decisions. The randomized controlled trial (RCT) is the gold standard for evaluating putative interventions. The scope of central nervous system (CNS) diseases is vast, and a comprehensive review would be lengthy. The number of published trials has nearly doubled since the last such report was published in 2001 (1). This article provides an overview of issues related to CNS clinical trials focusing on six selected conditions (ischemic stroke, Alzheimer’s disease, migraine headache, epilepsy, Parkinson’s disease, and multiple sclerosis) that primarily involve the CNS and about which an increasing number of trials have been published over the last few decades (Table 1). These conditions were chosen both because they are associated with significant morbidity (Fig. 1) (2) and because they illustrate the diversity of diseases affecting the CNS. It should be noted that each of these topics could support an extensive report and that, for the topics that are discussed, only selected studies are cited. We do not address trials of treatments for CNS effects of systemic illness, psychiatric diseases, infections of the CNS, sleep disorders, traumatic brain injury, CNS neoplasms, inborn errors of metabolism and congenital malformations,

A RCT can be conducted ethically only when clinical equipoise exists between the interventions being considered, the trial can provide an answer to the question being studied, and the rights of subjects can be preserved. RCTs are often expensive and therefore require appropriate levels of funding. In 2005, public funding for brain research in the United States was estimated at 6.1 billion euros. Private funding for brain research in the United States was 8.4 billion euros during the same period. In Europe, the totals were 0.9 billion and 3.3 billion euros, respectively (3). Public granting agencies generally fund clinical trials in CNS disease based on the quality of the science, but in the United States only about 15% of applications are currently funded, with costly clinical trials competing for funds with a variety of other types of research projects (4). Because of the nature of the peer-review process, great effort can be extended in developing successive versions of a proposal with no guarantee of success. Industry-funded trials are undertaken primarily to provide evidence of the efficacy and safety of a company’s product. Research investment is generally limited to interventions that are likely to be profitable. Economic analyses show that device trials have the highest return on investment because of the lower regulatory hurdles for governmental approval and the high market prices (5,6). Suggesting the roles of incentives and constraints other than potential public health impact, Figs. 1 and 2 illustrate the discor-

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CENTRAL NERVOUS SYSTEM (CNS)

Table 1. Number of Randomized Controlled Trials in Selected Neurological Diseases Indexed in Medline Year of Publication

Stroke

Alzheimer’s Disease

Migraine Headache

Epilepsy

Parkinson’s Disease

Multiple Sclerosis

Before 1981 1981–1986 1987–1991 1992–1996 1997–2001 2002–2006 Total

5 16 35 49 281 674 1060

1 18 62 155 201 335 772

44 52 75 135 193 240 739

29 50 81 181 236 199 776

45 21 67 134 179 262 708

14 26 62 108 163 230 603

Source: PubMed search for articles indexed with major subject heading (MeSH) of Stroke, Alzheimer Disease, Migraine Disorders, Epilepsy, Parkinson Disease, and Multiple Sclerosis (searches limited to randomized controlled trials and by publication year).

Global Burden of Disease (Million Quality Adjusted Life Years) Multiple Sclerosis, 1.5 Parkinson's Disease, 1.6

Stroke, 49.2

Epilepsy, 7.3

Migraine, 7.7

Figure 1. Source: World Health Report 2002 (2).

Alzheimer's (and other dementias), 10.4

Number of Randomized Controlled Trials (1965-2006) Multiple Sclerosis, 603

Stroke, 1060

Parkinson's, 708

Alzheimer's, 772 Epilepsy, 776 Migraine, 739

Figure 2. Source: PubMed search for articles indexed with major subject heading (MeSH) of Stroke, Alzheimer Disease, Migraine Disorders, Epilepsy, Parkinson Disease, and Multiple Sclerosis (searches limited to randomized controlled trials and by publication year).

CENTRAL NERVOUS SYSTEM (CNS)

dance between the burdens of different groups of CNS diseases and the numbers of clinical trials conducted in each area. 3

CNS TRIALS: COMMON CHALLENGES

Adequate and timely subject enrollment and retention are challenges faced by all prospective clinical trials. Trials involving CNS diseases can be particularly difficult. CNS disease can affect cognition and/or communication and therefore impair the subject’s capacity to give informed consent. In these circumstances, investigators must obtain consent from a surrogate decision maker. This requirement is particularly challenging in clinical trials of interventions for acute stroke in which the onset of disease is abrupt and unexpected, and the time window for treatment is limited. Because many neurological diseases evolve over several years (e.g., clinical worsening in Alzheimer’s disease, disability in multiple sclerosis, evolution of Parkinson’s disease, etc.), long periods of observation may be necessary to detect a treatment effect. This need presents a particular challenge for patient retention, increases the problem of competing comorbidities, and raises trial costs. In the case of industry-developed treatments in which a period of patent protection is limited, the latter two issues may make a trial unfeasible. A short follow-up period also limits a trial’s capacity to detect delayed harm. The majority of CNS trials end at or before 24 months, yet many drugs used to treat CNS disease are taken for a lifetime. Another historic problem faced by clinical trials of CNS disease has been a lack of standardized disease definitions or reliable, validated outcome measures. For example, a 1987 review of all 28 articles including clinical assessments published in the journal Stroke in 1984 and 1985 found that no single neurological scale was used more than once (7). This problem was not unique to stroke research and has been at least partially addressed not only for stroke but also for a variety of other conditions as described for each disease reviewed below. Agreement on useful outcome measures has been challenging for many CNS diseases because of

3

the lack of readily available pathologic data (brain biopsy being generally considered the most invasive of monitoring procedure). Neuroimaging technologies [e.g., magnetic resonance imaging (MRI)] have been employed to provide a measure of disease activity in diseases such as multiple sclerosis, but their use as a primary surrogate endpoint remains controversial (8). No valid or practical biologic endpoints exist for many CNS diseases for which endpoints must be constructed from clinician assessments and/or patient/caregiver report. 4

ISCHEMIC STROKE—PREVENTION

Stroke has the greatest health impact of all neurological diseases (Fig. 1). About 80% of strokes are ischemic, and consequently, interventions that prevent or ameliorate ischemic stroke have the greatest potential cumulative impact of all CNS treatments. The advent of thrombolytic therapy for selected patients with acute ischemic stroke has done little to reduce the overall burden of the disease, and primary and secondary prevention remain the most important interventions from the public health standpoint (9). Numerous clinical trials have shaped current medical therapy for stroke prevention. The benefit of lowering blood pressure in patients with hypertension has been demonstrated in at least 25 controlled trials of a variety of antihypertensive drugs (10). Treatment with dose-adjusted warfarin reduces the risk of stroke by 60% in patients with nonvalvular atrial fibrillation, whereas antiplatelet drugs such as aspirin reduce the risk by approximately 20% (11). Antiplatelet therapy represents a cornerstone of secondary stroke prevention, and several studies inform current practice. The Canadian Cooperative Study was among the first RCTs to demonstrate a benefit of aspirin in secondary stroke prevention (12). Supported by numerous other trials, aspirin use is associated with an approximate 13% reduction in the risk of serious vascular events in patients with prior stroke or transient ischemic attack (TIA) (13). The role of clopidogrel is somewhat more controversial. The Clopidogrel versus Aspirin for the Prevention of Recurrent Ischemic Events (CAPRIE) trial found a reduction in

4

CENTRAL NERVOUS SYSTEM (CNS)

a composite endpoint of myocardial infarction, stroke, or vascular death in patients with a history of myocardial, infarction, or symptomatic peripheral arterial disease with clopidogrel as compared with aspirin, but because of sample size limitations, it was not clear whether patients with stroke benefited (14). A meta-analysis of available data in 2002 demonstrated similar degrees of risk reduction with ASA and clopidogrel (15). The Management of Atherothrombosis with Clopidogrel in High Risk Patients (MATCH) trial found that the combination of aspirin plus clopidogrel was associated with a higher risk of bleeding without a reduction in recurrent ischemic events in patients with cerebrovascular disease risk when compared to clopidogrel alone (16). The Clopidogrel for High Atherothrombotic Risk and Ischemic Stabilization, Management, and Avoidance Trial (CHARISMA) found no benefit for the combination as compared with aspirin alone in high-risk patients (17). In contrast, the European Stroke Prevention Study-2 (ESPS2) found that the combination of aspirin and sustained-release dypiridamole was more efficacious than aspirin alone, a result that now has been supported by the European/ Australasian Stroke Prevention in Reversible Ischemia Trial (ESPIRIT) (18,19). Negative RCTs are as important as positive ones in that they show which treatments are not efficacious or safe. The Warfarin–Aspirin Recurrent Stroke Study (WARSS) found no benefit for warfarin over aspirin for secondary prevention in patients with noncardioembolic stroke (20). A substudy of WARSS evaluating patients with cryptogenic stroke who had a patent foramen ovale (PFO) similarly found no benefit for warfarin as compared with aspirin (21). The Warfarin–Aspirin Symptomatic Intracranial Disease (WASID) trial found no benefit of warfarin over aspirin in patients with a symptomatic intracranial stenosis (22). Postmenopausal hormone replacement therapy was once thought to reduce the risk of stroke and other cardiovascular events and was widely prescribed, but RCTs now have found either no benefit or increased stroke risk for women with coronary heart disease (23) or prior stroke history (24) or for otherwise healthy post-menopausal women (25).

Trials in subjects with established coronary heart disease or certain risk factors show an approximate 20% reduction in the risk of stroke associated with the use of HMG-CoA reductase inhibitors (statins). Studies with combined vascular endpoints had demonstrated lower stroke rates with statins versus placebo in patients with vascular risk factors (9). The Stroke Prevention with Aggressive Reductions in Cholesterol Levels (SPARCL) trial also found a reduction in recurrent stroke and other vascular events associated with a statin in patients with a recent stroke or TIA, no known coronary heart disease, and a low-density lipoprotein cholesterol (LDL-C) level between 100 to 190 mg/dL (26). A randomized trial first drastically altered surgical intervention for stroke prevention in 1985 with the publication of the negative Extracranial– Intracranial Bypass Study (27). The role of carotid endarterectomy (CEA) in severe symptomatic carotid stenosis for stroke prevention now has been firmly established by RCTs (28–30). The Asymptomatic Carotid Atherosclerosis Study (ACAS) (31) and the Asymptomatic Carotid Surgery Trial (ACST) (32) also demonstrate a smaller benefit in asymptomatic patients with high-grade carotid stenosis. Several trials comparing endarterectomy with carotid angioplasty/stenting have been published (33–35), but the results are inconsistent and the procedure has only been approved in the United States for symptomatic patients deemed at high endarterectomy risk. Several interventions recommended for stroke prevention (e.g., smoking cessation, moderation of alcohol consumption, and diet and weight management) are based on epidemiologic data and have not been examined in an RCT, either because it would be unethical or impractical to do so. Ongoing stroke prevention trials sponsored by the National Institutes of Health (NIH)/NINDS or other public agencies include evaluation of aspirin versus aspirin plus clopidogrel for secondary prevention of small subcortical strokes (NCT00059306), evaluation of warfarin versus aspirin for primary stroke prevention in patients with a cardiac ejection fraction less than

CENTRAL NERVOUS SYSTEM (CNS)

35% (NCT00041938), PFO closure versus anticoagulation versus antiplatelet therapy in recurrent stroke with PFO (NCT00562289), endarterectomy versus carotid stenting in symptomatic carotid stenosis (NCT00004732), and multiple trials of cholesterol-lowering agents in primary prevention (additional information on these trials and other ongoing trials can be accessed at ClinicalTrials.gov by typing the ID number into the search box). Industrysponsored trials include two large Phase III trials of oral direct thrombin inhibitors compared with warfarin in the prevention of stroke in atrial fibrillation (NCT00403767 and NCT00412984), a large Phase III trial of aspirin for primary prevention in patients with moderate cardiovascular risk (NCT00501059), and evaluation of an endovascularly deployed device to prevent embolism from the left atrial appendage (NCT00129545). A large industrysponsored Phase III randomized trial is comparing carotid stenting versus endarterectomy for asymptomatic severe carotid stenosis (NCT00106938), and multiple industry-sponsored trials evaluate PFO closure devices for patients with cryptogenic stroke (e.g., NCT00289289 and NCT00201461). 5

ISCHEMIC STROKE—ACUTE TREATMENT

The outcomes to be measured in stroke prevention trials are relatively straightforward (i.e., the numbers of strokes or other vascular events prevented). The associated costs and impact of therapy on quality of life are secondary measures. In contrast, the primary outcomes of interest for trials of acute stroke treatments are functional measures of stroke severity and are reflected in scales that assess neurological impairments (e.g., the NIH Stroke Scale and NIHSS), disability (e.g., the Barthel Index), and handicap [e.g., the Modified Rankin Scale, (mRS)]. Outcomes of acute stroke interventions are generally assessed 90 days poststroke. The optimal way of analyzing functional outcome data remains a point of discussion. No single trial has had a greater impact in revolutionizing the medical approach to

5

acute ischemic stroke than the NINDS trial of tissue plasminogen activator (tPA) (36). For patients treated within 3 hours of symptom onset, a 13% absolute (32% relative) increase occurred in proportion of patients with little or no functional deficit after 3 months. Because of the limited time window for treatment and because the likelihood of benefit increases the sooner the drug can be given, the NINDS tPA trial has led to a major change in the approach to acute ischemic stroke (37). A pilot RCT suggested that transcranial ultrasound could augment the effectiveness of systemic tPA, however, this finding has not yet been replicated or adopted in practice (38). Trials of several neuroprotective agents, heparin, and abciximab for the treatment of acute stroke have all yielded disappointing results (37). Currently, the NINDS is sponsoring three Phase III trials in acute stroke management. Albumin in Acute Ischemic Stroke (ALIAS, NCT00235495) and Field Administration of Stoke Therapy–Magnesium (FAST–MAG, NCT00059332) are placebocontrolled RCTs that evaluate the potential neuroprotective effects of human albumin and magnesium sulfate, respectively. The Interventional Management of Stroke III Trial (NCT00359424) is randomizing patients presenting with acute stroke within 3 hours to standard therapy with intravenous tPA versus partial-dose intravenous tPA followed by angiography and individualized intravascular intervention in eligible candidates. Currently, industry is sponsoring Phase III neuroprotective trials with citicoline (NCT00331890); ONO-2506, a homolog of valproic acid (NCT00229177); and Tanakan, a standardized ginkgo biloba extract (NCT00276380). Industry is also conducting a placebo-controlled double-blinded evaluation of ancrod, an enzyme derived from snake venom, for the treatment of ischemic stroke presenting within 6 hours (NCT00141011). The manufacturers of the NeuroThera topical phototherapy system and the NeuroFlo partial aortic occlusion devises are sponsoring Phase III efficacy trials (NCT00419705 and NCT00119717, respectively).

6

6

CENTRAL NERVOUS SYSTEM (CNS)

ALZHEMIER’S DISEASE

Alzheimer’s disease (AD) is the most common cause of dementia and leads to a significant burden of disease for both patients and caregivers. Histopathology provides the gold standard diagnostic test for AD, but this test cannot be used to identify subjects for enrollment in clinical trials and must be interpreted in the context of the patient’s clinical status. Standardized clinical criteria for the diagnosis of AD have been developed that have a correlation of 80–100% as compared with histopathology and provide a means for identifying subjects for therapeutic trials (39). The National Institute of Neurological and Communicative Diseases and Stroke/Alzheimer’s Disease and Related Disorders Association (NINCDS–ADRDA) criteria are among the best validated and most widely used for identifying potential subjects with possible or probable AD (40). Objective cognitive assessments, clinician or caregiver subjective global assessments, and functional assessments are all used to measure outcome in AD trials. The MiniMental State Examination (MMSE) (41) is widely used in trials as a cognitive instrument, but the Alzheimer’s Disease Assessment Scale–Cognitive Portion (ADAS–Cog) (42) is a more robust measure and has been used in several clinical trials. Multiple disease-specific instruments are available for assessing functional status including the Disability Assessment for Dementia (DAD) (43) and the Alzheimer’s Disease Cooperative Study Activities of Daily Living inventory (ACDS–ADL) (44). In addition to clinical assessment, several groups are actively seeking to identify biomarkers that can be used to aid in the diagnosis and monitoring of AD. This finding might help in early identification of cases, when patients stand to benefit the most from treatments intended to slow the course of the disease (45). Despite sustained efforts at finding disease-modifying therapies for AD, none have been identified to date, and the available symptomatic therapies have limited clinical benefit (46). Tacrine was the first cholinesterase inhibitor shown to have beneficial effects in a randomized trial, but its use was limited by side effects (47). The

pivotal efficacy studies of donepezil, which had the major advantages of daily dosing and a favorable adverse effects profile, were published in 1998 (48,49). This was followed by publication of positive efficacy trials of rivastigmine (50) and galantamine (51). The N-methyl-D-aspartate (NMDA) receptor antagonist memantine has been shown to improve function compared with placebo in severe patients with AD (45,52). Publicly funded trials currently are evaluating the effectiveness of selenium and vitamin-E (NCT00040378) as well as celecoxib and naproxen (NCT00007189) for the primary prevention of AD. The National Institute on Aging (NIA) is also sponsoring trials of simvastatin (NCT00053599) and docosahexanoic acid (NCT00440050) to slow AD progression. Currently, industry is funding Phase III trials of 3 novel agents targeted at the beta amyloid protein, which are as follows: MCP-7869 (NCT00105547), 3APS (NCT00088673), and bapineuzumab (NCT00574132). Rosiglitazone is also being evaluated as a disease-modifying agent in AD (NCT00348309). 7 MIGRAINE Migraine headache is defined for clinical and research purposes by the International Headache Society (IHS) criteria (53). The prevalence of migraine in the United States and Western Europe has been estimated at 9% of the general population, and about one third of migraneurs have headaches severe enough to interfere with activities of daily living (52). Migraine headaches have long been the subject of RCTs—propranolol was first shown to be effective versus placebo for migraine prophylaxis in 1974 (54). Clinical trials have shaped current practice in both the acute treatment of migraines and the prevention of recurrent migraines. Outcomes in migraine trials rely entirely on subjective patient report. The IHS has recommended the following standardized outcomes for acute therapy trials: percentage of patients pain-free at 2 hours, sustained painfree state at 48 hours, intensity of headache rated on a categorical scale, percentage of patients with reduction in pain from

CENTRAL NERVOUS SYSTEM (CNS)

moderate–severe to none–mild, and functional outcome rated on a categorical scale. For prophylaxis trials, monthly frequency of migraine attacks is the IHS-recommended outcome (55). In 2000, the American Academy of Neurology reviewed the available evidence for the treatment of migraine. Convincing effectiveness data from RCTs in acute treatment existed for all of the serotonin agonists (triptans), dihydroergotamine (DHE) nasal spray, aspirin, ibuprofen, naproxen sodium, butorphanol nasal spray, and opiates. Convincing efficacy data from RCTs in migraine prophylaxis were available for divalproex sodium, amitriptyline, propranolol, timolool, and methysergide (56). Clinical trial data supporting the efficacy of gabapentin (57) and topiramate (58) for migraine prophylaxis have since become available. RCTs have continued to refine our understanding of migraine treatment by evaluation of combination therapy (59), by direct comparisons of different agents (60), and through rigorous testing of widely used but unproven therapies. A recent placebo-controlled trial of intravenous dexamethasone administered in the emergency department failed to find benefit in acute migraine (61). Although parenteral DHE is widely used and believed to be helpful, its efficacy has never been proven in an RCT (62). The Chilean government currently is sponsoring a direct comparison trial of pregabalin and valproate in migraine prophylaxis (NCT00447369). Ongoing industrysponsored trials include a Phase III placebocontrolled evaluation of botulinum toxin type A for migraine prophylaxis (NCT00168428) and evaluations of an implantable neurostimulator (NCT00286078), transcranial magnetic stimulation (NCT00449540), and a cardiac atrial septal repair device (NCT00283738). 8

EPILEPSY

Epilepsy affects nearly 1% of the population and has a cumulative incidence of 3% by age 74 years. Before 1990, only six drugs were available for the treatment of seizures, but the range of available options

7

has since more than doubled (63). Definitions for seizure types and epilepsy syndromes used in research are maintained and periodically updated by the International League Against Epilepsy (www.ilae.org). Outcomes in antiepileptic drug (AED) trials are generally based on patient/caregiver reported seizure frequency. A common practice is to dichotomize subjects into responders (at least 50% reduction in seizure frequency) and nonresponders. The proportion of subjects remaining seizure-free has also been reported. The development of novel AEDs has been spurred because some patients remain refractory to all available therapies. These patients usually comprise the subjects for efficacy trials of new AEDs because the new AED can be ethically tested against placebo as add-on therapy and because an effect is more likely to be demonstrated in a short period of time in a population with frequent seizures. To evaluate the efficacy of newer AEDs in nonrefractory patients, randomized trials with active comparison groups rather than placebo-controlled trials are required because it would be unethical to deprive nonrefractory patients of therapy with proven benefit. The first placebo-controlled RCTs of AEDs as add-on therapy in refractory patients were published in 1975. Carbamazepine (64) and valproate (65) were evaluated in 23 and 20 patients, respectively. In a typical modern add-on trial, levetiracetam was recently shown to reduce mean seizure frequency and have a higher responder rate than placebo in subjects with uncontrolled idiopathic generalized epilepsy (66). The 45% responder rate among placebo patients in this study illustrates the importance of randomized placebo-controlled trials in the evaluation of therapeutic efficacy. Comparative trials of AEDs are less common than add-on trials. This situation is unfortunate because comparative trials have greater value in clinical decision making because little RCT evidence is available to guide AED choices (67). One example is a trial comparing lamotrigine and carbamazepine in subjects with both new onset partial and generalized seizures (68). This trial found

8

CENTRAL NERVOUS SYSTEM (CNS)

no difference in efficacy but improved tolerability with lamotrigine. The trial was not powered to detect a subtle difference in efficacy. The only Phase III NIH-sponsored trial currently in progress is a randomized parallel assignment trial comparing ethosuximde, lamotrigine, and valproic acid in the treatment of childhood absence epilepsy (NCT00088452). Industry-sponsored trials include a comparison of zonisamide versus carbamazepine in newly diagnosed partial epilepsy (NCT00477295) and a comparison of pregabalin versus levetiracetam (NCT00537238). Ongoing placebocontrolled add-on trials of new agents include evaluations of rufinamide (NCT00334958), SPM 927 (NCT00136019), retigabine (NCT00235755), and RWJ-333369 (NCT00433667). 9

PARKINSON’S DISEASE

Parkinson’s Disease (PD) is a neurodegenerative disorder with cardinal motor manifestations of tremor, bradykinesia, and rigidity. Levodopa was found to be the first efficacious symptomatic therapy for PD in the 1960s, and several RCTs have since demonstrated that this and several other agents are efficacious in ameliorating PD symptoms. No therapy has yet been proven to alter the progressive natural history of the disease (69). PD is usually diagnosed for research purposes using the United Kingdom Parkinson’s Disease Brain Bank criteria (70). Disability stage is measured using the classic five-stage scale of Hoehn and Yahr (71), but disease progression now is usually measured using the more comprehensive and detailed Unified Parkinson’s Disease Rating Scale (UPDRS) (72). Although the combination of levodopa and carbidopa has long been used for the treatment of PD, its effect on the natural history of the disease was uncertain. A landmark RCT, published in 2004, randomized patients to placebo or 3 different doses of levodopa and demonstrated a dose-related improvement in function with treatment that partially persisted 2 weeks after discontinuation of drug (73). A stable formulation

of levodopa/carbidopa for continuous duodenal infusion was shown to be efficacious in advanced PD (74). The dopamine agonist bromocriptine was shown to offer symptomatic benefit over placebo in 1975 (75), and pergolide was proven efficacious in 1985 (76). Pergolide has since been removed from the market because of toxicity. In the late 1990s, the dopamine agonists pramipexole (77) and ropinirole (78) were proven efficacious in RCTs and have been widely adopted into practice. Transdermal rotigotine has also recently been adopted as an effective dopaminergic therapy (79). Subcutaneous injection of apomorphine has been shown to treat refractory ‘‘off periods’’ effectively (80). Compounds that alter the metabolism of dopamine, such as the monoamine oxidase inhibitor selegiline, can improve symptom control (81). More recently, rasagiline (82) and an orally disintegrating formulation of selegiline (83) have been reported to improve motor symptoms. The catechol O-methyl transferase (COMT) inhibitors entacapone (84) and tolcapone (85) have been shown to reduce motor fluctuations in PD, but the latter has had limited use because of hepatotoxicity. Amantadine has been suggested to be efficacious in several small randomized trials, each with fewer than 20 patients (86). A large NINDS trial of vitamin E for neuroprotection in PD was negative (87). A trial of coenzyme Q10 yielded intriguing but nonsignificant results (88). Surgical treatment of advanced PD with deep brain stimulation (DBS) has been shown to improve quality of life and motor symptoms compared with medical therapy alone (89). The NINDS currently is sponsoring a Phase III evaluation of creatine as a diseasemodifying therapy in PD (NCT00449865). Two novel compounds currently in Phase III industry-sponsored trials are as follows: E2007 (NCT00286897) and SLV308 (NCT00269516). Another industry-sponsored Phase III evaluation of continuous infusion duodenal levodopa-carbidopa gel is in progress (NCT00357994).

CENTRAL NERVOUS SYSTEM (CNS)

10

MULTIPLE SCLEROSIS

Multiple sclerosis (MS) is a multifocal autoimmune demyelinating disease of the CNS associated with axonal injury with a typical onset in early adulthood. MS can have a variable course but can lead to significant disability. MS is nosologically divided into relapsing remitting MS (RRMS), secondary progressive MS (SPMS), and primary progressive MS (PPMS). Before 1990, no treatments proven to alter the disease or its clinical manifestations were available (90). MS is commonly diagnosed for clinical trial purposes using the so-called revised McDonald criteria (91), which incorporate both clinical and imaging findings. Commonly employed clinical outcomes in MS studies are relapse rates (RRMS) and disability progression using the Expanded Disability Scale (EDS). Several short-term imaging endpoints have been employed, but their correlation to the more important clinical endpoints has been debated (92). Four injectable drugs now have been proven efficacious for reducing disability and relapse rates in patients with RRMS. RCT results were published for interferon (INF) beta 1-b (93) and glatriamer acetate (94) in 1995; INF beta 1-a in 1996 (95); and natalizumab in 2006 (96). The first three compounds now are commonly used in patients with RRMS, whereas natalizumab’s use has been limited because of associated cases of progressive multifocal leukoencephalopathy (PML). In 2002, a higher dose three times weekly formulation of INF beta 1-a was shown to be superior to the previously approved once weekly formulation (97) and is now commonly in use. Promising results of preliminary RCTs for two oral agents, fingolimod (98) and laquinimod (99), in RRMS have been published. Administration of high-dose oral steroids has been shown to be efficacious versus placebo in reducing short-term disability in acute MS exacerbations (100). A small RCT found no difference between treatment of acute exacerbations with high-dose oral versus intravenous methylprednisolone (101). Nonetheless, many clinicians avoid the use of oral steroids for exacerbations because of the results of the Optic Neuritis Treatment

9

Trial in which the group that received lowdose oral steroids fared worse than the group that received a placebo (102). Trials in the treatment of progressive disease (PPMS and SPMS) have been less encouraging. Trials of azathioprine, cladribine, cyclophosphamide, cyclosporine, and methotrexate all have demonstrated a lack of efficacy or overwhelming toxicity (103). Positive results have been published for mitoxantrone (104,105), and this agent is frequently used in refractory patients. Currently, the NINDS is sponsoring a trial comparing INF beta 1-a versus glatriamer acetate versus the combination of the two agents (NCT00211887). Other sources of public funding currently are sponsoring comparative evaluations of cyclophosphamide versus methylprednisolone in secondary progressive disease (NCT00241254) and intravenous versus high-dose oral steroids in acute exacerbations of RRMS (NCT00418145). Industry currently is sponsoring Phase III evaluations of three novel oral agents in the treatment of RRMS, as follows: laquinimod (NCT00509145), BG00012 (NCT00420212), and fingolimod (NCT00420212). All three of these trials have target enrollments of at least 1000 patients. Industry is also sponsoring Phase III evaluations of alemtuzumab as add-on therapy to INF beta 1-a in RRMS (NCT00530348), MBP8298 in SPMS (NCT00468611), mitoxantrone in SPMS (NCT00146159), and rituximab in PPMS (NCT00087529).

11

CONCLUSION

This brief overview of clinical trials for selected diseases or conditions primarily affecting the CNS serves as an introduction to some issues involved in their conduct and interpretation. Only a few example trials for a few selected conditions are discussed and underscore the breath of clinical trials being conducted in this area. Therapeutic trials for CNS diseases remain in their infancy, but they already have had a dramatic impact on the practice of clinical neurology and will likely play an evermore important role in the future.

10

CENTRAL NERVOUS SYSTEM (CNS)

REFERENCES 1. R. J. Guiloff (ed.), Clinical Trials in Neurology. London: Springer, 2001. 2. C. Murry and A. Lopez (eds.), The World Health Report 2002—Reducing Risks, Promoting Health Life. Geneva, Switzerland: World Health Organization, 2002. 3. P. Sobocki, I. Lekander, S. Berwick, J. Olesen, and B. J¨onsson, Resource allocation to brain research in Europe (RABRE). Eur. J. Neurosci., 24(10): 2691–2693, 2006. 4. NINDS Funding Strategy—FY 2007. Available: http://www.ninds.nih.gov/funding/ ninds funding strategy.htm. Accessed December 14, 2007. 5. H. Moses, 3rd, E. R. Dorsey, D. H. Matheson, and S. O. Thier, Financial anatomy of biomedical research. JAMA. 294(11):1333–1342, 2005. 6. L. B. Goldstein, Regulatory device approval for stroke: fair and balanced? Stroke. 38: 1737–1738, 2007. 7. K. Asplund, Clinimetrics in stroke research. Stroke. 18: 528–530, 1987. 8. D. S. Goodin, E. M. Frohman, G. P. Garmany, Jr., et al., Disease modifying therapies in multiple sclerosis: report of the Therapeutics and Technology Assessment Subcommittee of the American Academy of Neurology and the MS Council for Clinical Practice Guidelines. Neurology. 58(2): 169–178, 2002. 9. L. B. Goldstein, R. Adams, M. J. Alberts, et al., Primary prevention of ischemic stroke: a guideline from the American Heart Association/American Stroke Association Stroke Council. Stroke. 37: 1583–1633, 2006. 10. C. M. Lawes, D. A. Bennett, V. L. Feigin, and A. Rodgers, Blood pressure and stroke: an overview of published reviews. Stroke. 35: 1024, 2004. 11. R. G. Hart, L. A. Pearce, and M. I. Aguilar, Meta-analysis: antithrombotic therapy to prevent stroke in patients who have nonvalvular atrial fibrillation. Annals of Internal Medicine. 146: 857–867, 2007. 12. The Canadian Cooperative Study Group, A randomized trial of aspirin and sulfinpyrazone in threatened stroke. New Engl. J. Med. 299: 53–59, 1978. 13. A. Algra and J. van Gijn, Cumulative metaanalysis of aspirin efficacy after cerebral ischaemia of arterial origin. J. Neurol. Neurosurg. Psychiatry. 66: 255, 1999. 14. CAPRIE Steering Committee, A randomized, blinded, trial of Clopidogrel Versus Aspirin in

Patients at Risk of Ischemic Events (CAPRIE). Lancet. 348: 1329–1339, 1996. 15. Antithrombotic Trialists’ Collaboration, Collaborative meta-analysis of randomised trials of antiplatelet therapy for prevention of death, myocardial infarction, and stroke in high risk patients. BMJ. 324: 71–86, 2002. 16. H. C. Diener, J. Bogousslavsky, L. M. Brass, et al., Aspirin and clopidogrel compared with clopidogrel alone after recent ischaemic stroke or transient ischaemic attack in highrisk patients (MATCH): randomised, doubleblind, placebo-controlled trial. Lancet. 364: 331–337, 2004. 17. D. L. Bhatt, K. A. Fox, W. Hacke, et al., Clopidogrel and aspirin versus aspirin alone for the prevention of atherothrombotic events. N. Engl. J. Med. 354: 1706–1717, 2006. 18. H. C. Diener, L. Cunha, C. Forbes, J. Sivenius, P. Smets, A. Lowenthal, European Stroke Prevention Study, 2: dipyridamole and acetylsalicylic acid in the secondary prevention of stroke. J. Neurol. Sci. 143: 1–13, 1996. 19. ESPRIT Study Group, Aspirin plus dipyridamole versus aspirin alone after cerebral ischaemia of arterial origin (ESPRIT): randomised controlled trial. Lancet. 367: 1665–1673, 2006. 20. J. P. Mohr, J. L. Thompson, R. M. Lazar, et al., A comparison of warfarin and aspirin for the prevention of recurrent ischemic stroke. N. Engl. J. Med. 345: 1444–1451, 2001. 21. S. Homma, R. L. Sacco, M. R. Di Tullio, R. R. Sciacca, and J. P. Mohr, for the PFO in Cryptogenic Stroke Study (PICSS) Investigators, Effect of medical treatment in stroke patients with patent foramen ovale. Circulation. 105: 2625–2631, 2002. 22. M. I. Chimowitz, M. J. Lynn, H. HowlettSmith, et al., Comparison of warfarin and aspirin for symptomatic intracranial arterial stenosis. N. Engl. J. Med. 352: 1305–1316, 2005. 23. S. Hulley, D. Grady, T. Bush, et al., Heart and Estrogen/progestin Replacement Study (HERS) Research Group, Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. JAMA. 280: 605–613, 1998. 24. C. M. Viscoli, L. M. Brass, W. N. Kernan, P. M. Sarrel, S. Suissa, and R. I. Horwitz, A clinical trial of estrogen-replacement therapy after ischemic stroke. N. Engl. J. Med. 345: 1243–1249, 2001.

CENTRAL NERVOUS SYSTEM (CNS) 25. J. E. Rossouw, G. L. Anderson, R. L. Prentice, et al., Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women’s Health Initiative randomized controlled trial. JAMA. 288: 321–333, 2002. 26. P. Amarenco, J. Bogousslavsky, A. Callahan, 3rd, et al., Stroke Prevention by Aggressive Reduction in Cholesterol Levels (SPARCL) Investigators. High-dose atorvastatin after stroke or transient ischemic attack. N. Engl. J. Med. 355: 549–559, 2006. 27. The EC/IC Bypass Study Group, Failure of extracranial-intracranial arterial bypass to reduce the risk of ischemic stroke: results of an international randomized trial. N. Engl. J. Med. 313: 1191–1200, 1985. 28. North American Symptomatic Carotid Endarterectomy Trial Collaborators, Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. N. Engl. J. Med. 325: 445–453, 1991. 29. European Carotid Surgery Trialists’ Collaborative Group, MRC European Carotid Surgery Trial: interim results for symptomatic patients with severe (70–99%) or with mild (0–29%) carotid stenosis. Lancet. 337: 1235–1243, 1991. 30. M. R. Mayberg, S. E. Wilson, F. Yatsu, et al., Carotid endarterectomy and prevention of cerebral ischemia in symptomatic carotid stenosis: Veterans Affairs Cooperative Studies Program. JAMA. 266: 3289–3294, 1991. 31. Executive Committee for the Asymptomatic Carotid Atherosclerosis Study, Endarterectomy for asymptomatic carotid artery stenosis. JAMA. 273: 1421–1428, 1995. 32. A. Halliday, A. Mansfield, J. Marro, et al., Prevention of disabling and fatal strokes by successful carotid endarterectomy in patients without recent neurological symptoms: randomised controlled trial. Lancet. 363: 1491–1502, 2004. 33. J. S. Yadav, M. H. Wholey, R. E. Kuntz, et al., Protected carotid-artery stenting versus endarterectomy in high risk patients. N. Engl. J. Med. 351: 1493–1501, 2004. 34. The SPACE Collaborative Group, 30 day results from the SPACE trial of stent-protected angioplasty versus carotid endarterectomy in symptomatic patients: a randomised non-inferiority trial. Lancet. 368: 1239–1247, 2006. 35. J. L. Mas, G. Chatellier, B. Beyssen, et al., Endarterctomy versus stenting in patients

11

with symptomatic severe carotid stenosis. N. Engl. J. Med. 355: 1660–1167, 2006. 36. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group, Tissue plasminogen activator for acute ischemic stroke. N. Engl. J. Med. 333: 1581–1587, 1995. 37. H. P. Adams, Jr., G. del Zoppo, M. J. Alberts, et al., Guidelines for the early management of adults with ischemic stroke. Stroke. 38(5): 1655–1711, 2007. 38. A. V. Alexandrov, C. A. Molina, J. C. Grotta, et al., Ultrasound-enhanced systemic thrombolysis for acute ischemic stroke. N. Engl. J. Med. 351: 2170–2178, 2004. 39. O. L. Lopez, I. Litvan, K. E. Catt, et al., Accuracy of four clinical diagnostic criteria for the diagnosis of neurodegenerative dementias. Neurology. 53: 1292–1299, 1999. 40. G. McKhann, D. A. Drachman, M. F. Folstein, R. Katzman, D. L. Price, and E. Stadlan, Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of the Department of Health and Human Services Task Force on Alzheimer’s disease. Neurology. 34: 939–944, 1984. 41. J. R. Cockrell and M. F. Folstein, Mini-Mental State Examination (MMSE). Psychopharmacol. Bull. 24: 689–692, 1988. 42. R. C. Mhos, The Alzheimer’s disease assessment scale. Int. Psychogeriatr. 8: 195–203, 1996. 43. I. Gelinas, L. Gauthier, M. McIntyre, and S. Gauthier, Development of a functional measure for persons with Alzheimer’s disease: the disability assessment for dementia. Am. J. Occup. Ther. 53: 471–481, 1999. 44. D. Galasko, D. Bennett, M. Sano, et al., An inventory to assess activities of daily living for clinical trials in Alzheimer’s disease. Alzheimer Dis. Assoc. Disord. Suppl. 2:S33–S39, 1997. 45. A. M. Fagan, C. A. Csernansky, J. C. Morris, D. M. Holtzman, The search for antecedent biomarkers of Alzheimer’s disease. J. Alzheimers Dis. 8(4): 347–358, 2005. 46. R. S. Doody, J. C. Stevens, C. Beck, et al., Practice parameter: management of dementia (an evidence-based review). Report of the Quality Standards Subcommittee of the American Academy of Neurology. Neurology. 56(9): 1154–1166, 2001. 47. W. K. Summers, L. V. Majovski, G. M. Marsh, K. Tachiki, and A. Kling, Oral tetrahydroaminoacridine in long-term treatment of

12

48.

49.

50.

51.

52.

53.

54.

55.

56.

57.

58.

59.

60.

CENTRAL NERVOUS SYSTEM (CNS) senile dementia, Alzheimer type. N. Engl. J. Med. 315: 1241–1245, 1986. S. Rogers, R. Doody, R. Mohs, et al., and the Donepezil Study Group, Donepezil improved cognition and global function in Alzheimer’s disease. Arch. Intern. Med. 158: 1021–1031, 1998. S. Rogers, M. Farlow, R. Doody, et al., and the Donepezil Study Group, A 24-week, doubleblind, placebo-controlled trial of donepezil in patients with Alzheimer’s disease. Neurology. 50: 136–145, 1998. M. Rosler, R. Anand, A. Cicin–Sain, et al., Efficacy and safety of Rivastigmine in patients with Alzheimer’s disease: international randomised controlled trial. B. Med. J. 318: 633–638, 1999. M. Raskind, E. Peskind, T. Wessel, et al., and the Galantamine Study Group, Galantamine in AD. A 6-month randomized, placebocontrolled trial with a 6-month extension. Neurology. 54: 2261–2268, 2000. J. L. Brandes, Global trends in migraine care: results from the MAZE survey. CNS Drugs. 16 Suppl. 1: 13–18, 2002. Headache Classification Committee of the International Headache Society, Classification and diagnostic criteria for headache disorders, cranial neuralgias and facial pain. Cephalalgia. 8 Suppl. 7: 1–96, 1998. TE Wideroe and T. Vigander, Propranolol in the treatment of migraine. Br. Med. J. 2: 699–701, 1974. P. Tfelt-Hansen, G. Block, C. Dahl¨of, et al., Guidelines for controlled trials of drugs in migraine: second edition. Cephalalgia. 20: 765–786, 2000. S. D. Silberstein, Practice parameter: evidence-based guidelines for migraine headache (an evidence-based review): report of the Quality Standards Subcommittee of the American Academy of Neurology. Neurology. 55: 754–762, 2000. N. T. Mathew, A. Rapoport, J. Saper, et al., Efficacy of gabapentin in migraine prophylaxis. Headache. 41: 119, 2001. J. L. Brandes, J. R. Saper, M. Diamond, et al., Topiramate for migraine prevention: a randomized controlled trial. JAMA. 291: 965, 2004. J. L. Brandes, D. Kudrow, S. R. Stark, et al., Sumatriptan-naproxen for acute treatment of migraine: a randomized trial. JAMA. 297: 1443, 2007. B. W. Friedman, J. Corbo, R. B. Lipton, et al., A trial of metoclopramide vs sumatriptan

for the emergency department treatment of migraines. Neurology. 64: 463, 2005. 61. B. W. Friedman, P. Greenwald, T. C. Bania, et al., Randomized trial of IV dexamethasone for acute migraine in the emergency department. Neurology. 69: 2038–2044, 2007. 62. I. Colman, M. D. Brown, G. D. Innes, E. Grafstein, T. E. Roberts, and B. H. Rowe, Parenteral dihydroergotamine for acute migraine headache: a systematic review of the literature. Ann. Emerg. Med. 45: 393–401, 2005. 63. J. A. French, A. M. Kanner, J. Bautista, et al., Efficacy and tolerability of the new antiepileptic drugs I: treatment of new onset epilepsy: report of the Therapeutics and Technology Assessment Subcommittee and Quality Standards Subcommittee of the American Academy of Neurology and the American Epilepsy Society. Neurology. 62: 1252–1260, 2004. 64. H. Kutt, G. Solomon, C. Wasterlain, H. Peterson, S. Louis, and R Carruthers, Carbamazepine in difficult to control epileptic out-patients. Acta. Neurol. Scand. Suppl. 60: 27–32, 1975. 65. A Richens and S. Ahmad, Controlled trial of sodium valproate in severe epilepsy. Br. Med. J. 4: 255–256, 1975. 66. S. F. Berkovic, R. C. Knowlton, R. F. Leroy, J. Schiemann, and U. Falter, Levetiracetam N01057 Study Group, Placebo-controlled study of levetiracetam in idiopathic generalized epilepsy. Neurology. 69: 1751–1760, 2007. 67. J. A. French and R. J. Kryscio, Active control trials for epilepsy: avoiding bias in head-tohead trials. Neurology. 66: 1294–1295, 2006. 68. M. J. Brodie, A. Richens, and A. W. Yuen, Double-blind comparison of lamotrigine and carbamazepine in newly diagnosed epilepsy. UK Lamotrigine/ Carbamazepine Monotherapy Trial Group. Lancet. 345: 476–479, 1995. 69. E. Tolosa and R. Katzenshclager, Pharmacological management of Parkinson’s disease. In: J Jankovic and E Tolosa (eds.), Parkinson’s Disease and Movement Disorders. Philidephia, PA: Lippincott Williams and Wilkins, 2007. 70. A. J. Hughes, S. E. Daniel, L. Kilford, et al., Accuracy of clinical diagnosis of idiopathic Parkinson’s disease: a clinico-pathological study of 100 cases. J. Neurol. Neurosurg. Psychiatry. 55: 181–184, 1992. 71. M. M. Hoehn, and M. D. Yahr, Parkinsonism: onset, progression and mortality. Neurology. 17: 427–442, 1967.

CENTRAL NERVOUS SYSTEM (CNS) 72. S. Fahn and R. Elton, Members of the UPDRS Development Committee, Unified Parkinson’s Disease Rating Scale. In: S. Fahn, C. D. Marsden, D. B. Calne, and M. Goldstein (eds.), Recent Developments in Parkinson’s Disease, vol. 2. Florham Park, NJ: MacMillan, 1987. 73. Parkinson Study Group, Levodopa and the progression of Parkinson’s disease. N. Engl. J. Med. 351: 2498–2508, 2004. 74. D. Nyholm, A. I. Nilsson Remahl, N. Dizdar, et al., Duodenal levodopa infusion monotherapy vs oral polypharmacy in advanced Parkinson disease. Neurology. 64(2): 216–223, 2005. 75. P. F. Teychenne, P. N. Leigh, J. L. Reid, D. B. Calne, J. K. Greenacre, A. Petrie, and A. N. Bamji, Idiopathic parkinsonism treated with bromocriptine. Lancet. 2(7933): 473–476, 1975. 76. J. I. Sage and R. C. Duvoisin, Pergolide therapy in Parkinson’s disease: a double-blind, placebo-controlled study. Clin. Neuropharmacol. 8(3): 260–265, 1985. 77. A. Lieberman, A. Ranhosky, and D. Korts, Clinical evaluation of pramipexole in advanced Parkinson’s disease: results of a double-blind, placebo-controlled, parallelgroup study. Neurology. 49: 162–168, 1997. 78. A. Lieberman, C. W. Olanow, K. Sethi, et al., A multicenter trial of ropinirole as adjunct treatment for Parkinson’s disease, Ropinirole Study Group, Neurology. 51: 1057–1062, 1998. 79. R. L. Watts, J. Jankovic, C. Waters, A. Rajput, B. Boroojerdi, and J. Rao, Randomized, blind, controlled trial of transdermal rotigotine in early Parkinson disease. Neurology. 68(4): 272–276, 2007. 80. R. B. Dewey, Jr., J. T. Hutton, P. A. LeWitt, and S. A. Factor, A randomized, double-blind, placebo-controlled trial of subcutaneously injected apomorphine for parkinsonian offstate events. Arch. Neurol. 58: 1385–1392, 2001. 81. B. Sivertsen, E. Dupont, B. Mikkelsen, P. Mogensen, C. Rasmussen, F. Boesen, and E. Heinonen, Selegiline and levodopa in early or moderately advanced Parkinson’s disease: a double-blind controlled short- and longterm study. Acta. Neurol. Scand. Suppl. 126: 147–152, 1989. 82. Parkinson Study Group, A controlled, randomized, delayed-start study of rasagiline in early Parkinson disease. Arch. Neurol. 61: 561–566, 2004. 83. C. H. Waters, K. D. Sethi, R. A. Hauser,

13

E. Molho, and J. M. Bertoni, Zydis selegiline reduces off time in Parkinson’s disease patients with motor fluctuations: a 3-month, randomized, placebo-controlled study. Mov. Disord. 19: 426–432, 2004. 84. Parkinson Study Group, Entacapone improves motor fluctuations in levodopatreated Parkinson’s disease patients. Ann. Neurol. 42: 747–755, 1997. 85. M. C. Kurth, C. H. Adler, M. S. Hilaire, et al., Tolcapone improves motor function and reduces levodopa requirement in patients with Parkinson’s disease experiencing motor fluctuations: a multicenter, double-blind, randomized, placebo-controlled trial, Tolcapone Fluctuator Study Group I. Neurology. 48(1): 81–87, 1997. ´ 86. F. P. da Silva-Junior, P. Braga-Neto, F. Sueli Monte, and V. M. de Bruin, Amantadine reduces the duration of levodopainduced dyskinesia: a randomized, doubleblind, placebo-controlled study. Parkinsonism Relat. Disord. 11(7): 449–452, 2005. 87. The Parkinson Study Group, Effects of tocopherol and deprenyl on the progression of disability in early Parkinson’s disease. N. Engl. J. Med. 328: 176–183, 1993. 88. C. W. Shults, D. Oakes, K. Kieburtz, et al., Effects of coenzyme Q10 in early Parkinson disease: evidence of slowing of the functional decline. Arch. Neurol. 59: 1541–1550, 2002. 89. G. Deuschl, C. Schade-Brittinger, P. Krack, et al., A randomized trial of deep-brain stimulation for Parkinson’s disease. N. Engl. J. Med. 355(9): 896–908, 2006. 90. M. J. Olek and D. M. Dawson, Multiple sclerosis and other inflammatory demyelinating diseases of the central nervous system. In: W. G. Bradley, R. B. Daroff, G. M. Fenichel, and J. Jankovic (eds.), Neurology in Clinical Practice. Philidelphia, PA: Butterworth Heinemann, 2004. 91. C. H. Polman, S. C. Reingold, G. Edan, et al., Diagnostic criteria for multiple sclerosis: 2005 revisions to the ‘‘McDonald Criteria’’. Ann. Neurol. 58(6): 840–846, 2005. 92. D. S. Goodin, E. M. Frohman, G. P. Garmany, Jr., et al., Disease modifying therapies in multiple sclerosis: report of the Therapeutics and Technology Assessment Subcommittee of the American Academy of Neurology and the MS Council for Clinical Practice Guidelines. Neurology. 58(2): 169–178, 2002. 93. The IFNB Multiple Sclerosis Study Group and the UBC MS/MRI Analysis Group, Interferon beta-1b in the treatment of MS: final outcome

14

CENTRAL NERVOUS SYSTEM (CNS)

of the randomized controlled trial. Neurology. 45: 1277–1285, 1995. 94. K. P. Johnson, B. R. Brooks, J. A. Cohen, et al., Copolymer 1 reduces relapse rate and improves disability in relapsing-remitting multiple sclerosis: results of a phase III multicenter, doubleblind, placebo-controlled trial. Neurology. 45: 1268–1276, 1995. 95. L. D. Jacobs, D. L. Cookfair, R. A. Rudick, et al., Intramuscular interferon beta-1a for disease progression in exacerbating remitting multiple sclerosis. Ann. Neurol. 39: 285–294, 1996. 96. C. H. Polman, P. W. O’Connor, E. Havrdova, et al., A randomized, placebo-controlled trial of natalizumab for relapsing multiple sclerosis. N. Engl. J. Med. 354: 899, 2006. 97. H. Panitch, D. S. Goodin, G. Francis, et al., Randomized, comparative study of interferon beta-1a treatment regimens in MS: The EVIDENCE Trial. Neurology. 59: 1496, 2002. 98. L. Kappos, J. Antel, G. Comi, et al., Oral fingolimod (FTY720) for relapsing multiple sclerosis. N. Engl. J. Med. 355: 1124, 2006. 99. C. Polman, F. Barkhof, M. SandbergWollheim, et al., Treatment with laquinimod reduces development of active MRI lesions in relapsing MS. Neurology. 64: 987, 2005. 100. F. Sellebjerg, J. L. Frederiksen, P. M. Nielsen, and J. Olesen, Double-blind, randomized, placebo-controlled study of oral, highdose methylprednisolone in attacks of MS. Neurology. 51: 529, 1998. 101. D. Barnes, R. A. Hughes, R. W. Morris, et al., Randomised trial of oral and intravenous methylprednisolone in acute relapses of multiple sclerosis. Lancet. 349: 902, 1997. 102. R. W. Beck, P. A. Cleary, M. M. Anderson, Jr., et al., the Optic Neuritis Study Group, A randomized, controlled trial of corticosteroids in the treatment of acute optic neuritis. N. Engl. J. Med. 326: 581, 1992. 103. M. J. Olek, Treatment of progressive multiple sclerosis in Adults. In: B. D. Rose (ed.), UpToDate. Waltham, MA:, 2007. 104. G. Edan, D. Miller, M. Clanet, et al., Therapeutic effect of mitoxantrone combined with methylprednisolone in multiple sclerosis: A randomised multicentre study of active disease using MRI and clinical criteria. J. Neurol. Neurosurg. Psychiatry. 62: 112, 1997. 105. H. P. Hartung, R. Gonsette, N. Konig, et al., Mitoxantrone in progressive multiple sclerosis: a placebo-controlled, double-blind, randomised, multicentre trial. Lancet. 360: 2018, 2002.

CFR 21 PART 11

is important to note that the predicate rules encompass a broad range of areas and apply not only to Good Clinical Practice, but also to good laboratory practices and good manufacturing practices (GMP). Records maintained or submitted to the agency in these areas are subject to Part 11. One consequence of Part 11 was to provide a framework for electronic data capture and record keeping in clinical trials. As clinical trials increase in number and complexity, electronic solutions claim a growing presence in the effort to improve data collection, transmission, storage, and retrieval. Sponsors are moving toward electronic alternatives to introduce speed and greater reliability into these data-driven processes that historically, have been paper based. Additionally, the capability to execute electronic signatures at the investigative site (electronic case report forms and Case Books) is accompanied by increased responsibilities for sponsors and technology providers to adhere to and provide solutions that are compliant with 21CFR11 and the current FDA guideline(s). Significant challenges lie ahead, most notably, how to implement electronic solutions so the resulting electronic records are considered equivalent to paper records, and electronic signatures are considered equal to handwritten ones. Since its effective date, Part 11 has had a significant impact on clinical research, and specifically clinical trials with regard to the implementation of technology, validation, data collection, data management, and regulatory submissions.

MICHAEL P. OWINGS and PAUL A. BLEICHER Phase Forward, Waltham, Massachusetts

One consequence of Part 11 was to provide a framework for electronic data capture and electronic record keeping in clinical trials; as such, it has had a significant impact on clinical research, and specifically clinical trials with regard to the implementation of technology, validation, data collection, data management, and regulatory submissions. Several guidances remain in effect for the interpretation of 21 CFR Part 11, specifically the Guidance for Computerized Systems Used in Clinical Trials and a Scope and Application Guidance. The latter guidance indicates that the FDA is in the process of reinterpreting, and possibly redefining 21 CFR Part 11 in line with a risk management approach. On March 20, 1997, The U. S. FDA issued an important regulation under the Code of Federal Regulations entitled 21CFR11: ELECTRONIC RECORDS; ELECTRONIC SIGNATURES, which is commonly referred to as ‘‘Part 11.’’ The purpose of the regulation is to provide criteria under which the FDA will consider electronic records to be equivalent to paper records, and electronic signatures equivalent to traditional handwritten signatures. Part 11 applies to persons who, in fulfillment of a requirement in a statute, have chosen to maintain records or submit designated information electronically to FDA, and to records in electronic form that are created, modified, maintained, archived, retrieved, or transmitted under any records requirements set forth in Agency regulations. Part 11 also applies to electronic records submitted to the Agency under the Federal Food, Drug, and Cosmetic Act (the Act) (3) and the Public Health Service Act (the PHS Act) (4), even if such records are not specifically identified in Agency regulations (§ 11.1). The underlying requirements set forth in the Act, PHS Act, and FDA regulations (other than part 11) are referred to as predicate rules (2). It

1

BACKGROUND

Part 11 was developed by the FDA in concert with other government initiatives that began in the early 1990s and continue to the present. The final rule is the result of a six-year process, which began in 1991, when members of the pharmaceutical industry met with the FDA to explore how they could accommodate paperless record systems

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CFR 21 PART 11

Table 1. Selected Information Specified in Subpart A Related to Scope and Definitions 21CFR11 Subpart A—General Provisions §11.1 – Scope (a) The regulations in this part set forth the criteria under which the agency considers electronic records, electronic signatures, and handwritten signatures executed to electronic records to be trustworthy, reliable, and generally equivalent to paper records and handwritten signatures executed on paper. (b) This part applies to records in electronic form that are created, modified, maintained, archived, retrieved, or transmitted, under any records requirements set forth in agency regulations. However, this part does not apply to paper records that are, or have been, transmitted by electronic means. (c) Where electronic signatures and their associated electronic records meet the requirements of this part, the agency will consider the electronic signatures to be equivalent to full handwritten signatures, initials, and other general signings as required by agency regulations, unless specifically excepted by regulation(s) effective on or after August 20, 1997. (e) Computer systems (including hardware and software), controls, and attendant documentation maintained under this part shall be readily available for, and subject to, FDA inspection. §11.3 – Definitions (4) Closed system means an environment in which system access is controlled by persons who are responsible for the content of electronic records that are on the system (6) Electronic record means any combination of text, graphics, data, audio, pictorial, or other information representation in digital form that is created, modified, maintained, archived, retrieved, or distributed by a computer system. (7) Electronic signature means a computer data compilation of any symbol or series of symbols executed, adopted, or authorized by an individual to be the legally binding equivalent of the individual’s handwritten signature. (9) Open system means an environment in which system access is not controlled by persons who are responsible for the content of electronic records that are on the system.

under the current good manufacturing practice (cGMP) regulations in 21 CFR Parts 210 and 211. From of these meetings came a task force (a working sub-group), who developed a 1992 notice in the Federal Register that the FDA was considering the use of electronic identification and signatures, a proposed rule in 1994 and ultimately, the final rule in 1997. In addition to 21 CFR Parts 210 and 211, Part 11 was complimentary with other initiatives to eliminate paper that were already underway at the federal government, such as the Paperwork Reduction Act of 1995 (5) and later the Government Paperwork Elimination Act (GPEA) of 1998 (6). The purpose of the Paperwork Reduction Act of 1995 as stated is to minimize the paperwork burden for individuals, small businesses, educational and nonprofit institutions, Federal contractors, State, local and tribal governments, and other persons resulting from the collection of information by or for the Federal Government.

Part 11 and GPEA shared the combined purpose of eliminating paper from complex and costly processes and promoting the use of technology with the creation and management of electronic records and use of electronic signatures. This period was an inflection point as the U.S. Government began to encourage and even require the use of technology. GPEA was enacted on October 21, 1998, as title XVII of P.L. 105-277, and contains specific requirements in the use of technology for electronic records such as: the technology shall be compatible with standards and technology for electronic signatures that are generally used in commerce and industry . . . and shall ensure that electronic signatures are as reliable as is appropriate for the purpose in question and keep intact the information submitted.

As will be shown later in the article, these requirements are consistent with, and in some cases almost identical to, the basic tenets contained in Part 11.

CFR 21 PART 11

3

Table 2. Information Specified in Subpart B Related to the Manifestation of the Electronic Signature and Signature Linking 21CFR11 Subpart B–Controls for Closed Systems §11.10 - Controls for closed systems Persons who use closed systems to create, modify, maintain, or transmit electronic records shall employ procedures and controls designed to ensure the authenticity, integrity, and, when appropriate, the confidentiality of electronic records, and to ensure that the signer cannot readily repudiate the signed record as not genuine. Such procedures and controls shall include the following: (a) Validation of systems to ensure accuracy, reliability, consistent intended performance, and the ability to discern invalid or altered records. (b) The ability to generate accurate and complete copies of records in both human readable and electronic form suitable for inspection, review, and copying by the agency. (c) Protection of records to enable their accurate and ready retrieval throughout the records retention period. (d) Limiting system access to authorized individuals. (e) Use of secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records. Record changes shall not obscure previously recorded information. Such audit trail documentation shall be retained for a period at least as long as that required for the subject electronic records and shall be available for agency review and copying. §11.50 - Signature manifestations. (a) Signed electronic records shall contain information associated with the signing that clearly indicates all of the following: (1) The printed name of the signer; (2) The date and time when the signature was executed; and (3) The meaning (such as review, approval, responsibility, or authorship) associated with the signature. §11.70 - Signature/record linking. Electronic signatures and handwritten signatures executed to electronic records shall be linked to their respective electronic records to ensure that the signatures cannot be excised, copied, or otherwise transferred to falsify an electronic record by ordinary means.

2

ORGANIZATION

Part 11 is divided into three subparts: General Provisions, Electronic Records, and Electronic Signatures. Within these three sections, Part 11 states the scope of the rule; provides definitions; and states that individuals using the system(s) are to employ procedures and controls designed to ensure the authenticity, integrity, and when appropriate, confidentiality of electronic records. 2.1 General Provisions Subpart A; General Provisions of Part 11 contains three sections: 11.1—Scope, 11.2— Implementation, and 11.3—Definitions. In essence, these three sections establish a basic framework for which to employ electronic records and electronic signatures in lieu of paper records, and the criteria under which

the FDA consider electronic records and electronic signatures to be equivalent to paper records and signatures executed on paper. Additionally, the applicability of Part 11 is briefly addressed: . . . applies to records in electronic form that are created, modified, maintained, archived, retrieved, or transmitted, under any records requirements set forth in agency regulations. However, this part does not apply to paper records that are, or have been, transmitted by electronic means.

Thus, this document indicates that Part 11 would not apply to a facsimile copy of a paper record or signature required by the agency. The definitions section contains nine definitions related to ERES. Definitions are included for the following terms: Electronic Record, Electronic Signature, and Open and

4

CFR 21 PART 11

Table 3. A Selected Portion of the Signature General Requirements and Procedures Contained in Subpart C 21CFR11 Subpart C–Electronic Signatures §11.100 - General requirements (a) Each electronic signature shall be unique to one individual and shall not be reused by, or reassigned to, anyone else. (b) Before an organization establishes, assigns, certifies, or otherwise sanctions an individual’s electronic signature, or any element of such electronic signature, the organization shall verify the identity of the individual. (c) Persons using electronic signatures shall, prior to or at the time of such use, certify to the agency that the electronic signatures in their system, used on or after August 20, 1997, are intended to be the legally binding equivalent of traditional handwritten signatures. (2) Persons using electronic signatures shall, upon agency request, provide additional certification or testimony that a specific electronic signature is the legally binding equivalent of the signer’s handwritten signature. §11.200 - Electronic signature components and controls. (a) Electronic signatures that are not based upon biometrics shall: (1) Employ at least two distinct identification components such as an identification code and password. (2) Be used only by their genuine owners; and (3) Be administered and executed to ensure that attempted use of an individual’s electronic signature by anyone other than its genuine owner requires collaboration of two or more individuals. (b) Electronic signatures based upon biometrics shall be designed to ensure that they cannot be used by anyone other than their genuine owners. §11.300 - Controls for identification codes/passwords Persons who use electronic signatures based upon use of identification codes in combination with passwords shall employ controls to ensure their security and integrity. Such controls shall include: (a) Maintaining the uniqueness of each combined identification code and password, such that no two individuals have the same combination of identification code and password. (d) Use of transaction safeguards to prevent unauthorized use of passwords and/or identification codes, and to detect and report in an immediate and urgent manner any attempts at their unauthorized use to the system security unit, and, as appropriate, to organizational management.

Table 4. An Overview of the Topics Contained in the Guidance for Industry: Computerized Systems Used in Clinical Trials Overview–Guidance for Industry: Computerized Systems Used in Clinical Trials • General Principles • Standard Operating Procedures • Data Entry • System Features • Security • System Dependability • System Controls • Training of Personnel • Records Inspection •Certification of Electronic Signatures

Closed Systems. Although all definitions in this section should be noted, the definitions provided for Electronic Record, Electronic Signature, and Open and Closed Systems have historically played a very important role

in how practitioners have defined systems and approached implementation and validation of systems for clinical applications in an effort to comply with the rule. Validation and controls of the system are paramount,

CFR 21 PART 11

given that the General Provisions also state that ‘‘Computer systems (including hardware and software), controls, and attendant documentation maintained under this part shall be readily available for, and subject to, FDA inspection.’’. 2.2 Subpart B—Electronic Records Subpart B—Electronic Records contains four sections: 11.10—Controls for closed systems, 11.30—Controls for open systems, 11.50— Signature Manifestations, and 11.70— Signature and record linking. These sections address how applicable systems are controlled, how specific requirements are related to the manifestation of an electronic signature on an electronic record, linking of that signature to the electronic record, what information must be included, and the need to link the signature to all respective electronic records. The preamble of 21CFR Part11, which is essential to interpreting the final rule, is flexible on the method of linkage. It states, While requiring electronic signatures to be linked to their respective electronic records, the final rule affords flexibility in achieving that link through use of any appropriate means, including use of digital signatures and secure relational database references.

Part 11 Subpart B is designed such that the controls delineated under ‘‘Controls for Closed Systems’’ also apply to open systems. This subpart includes important topics such as: Validation ‘‘to ensure accuracy, reliability, consistent intended performance, and the ability to discern invalid or altered records’’, system access and security, copies for inspection, record retention, and the ‘‘use of secure, computer-generated, time-stamped audit trails to independently record the date and time of operator entries and actions that create, modify, or delete electronic records.’’ In addition, procedures and controls must be in place so that the signer cannot easily repudiate the signed record as not genuine. 2.3 Subpart C—Electronic Signatures Subpart C contains three sections: 11.100— General requirements, 11.200—Electronic

5

signature components and controls, and 11.300—Controls for identification codes/ passwords. It addresses electronic signatures and defines them as ‘‘a computer data compilation of any symbol or series of symbols, executed, adopted, or authorized by an individual to be the legally binding equivalent of the individual’s handwritten signature.’’ This section explains the components of electronic signatures and distinguishes between signatures that are based on nonbiometrics and those that are. It specifies that a signature must ‘‘employ at least two distinct identification components such as an identification code and password.’’ According to the rule, whether an electronic signature is based on biometrics, steps must be taken to ensure that it cannot be used by anyone but the owner of that signature. Biometrics refers to the use of automated methods to verify an individual’s identity based on measurement of that individual’s physical features and/or repeatable actions that are unique to that individual. Examples include fingerprints, voice patterns, retinal or iris scanning, facial recognition, temperature, or hand geometry. Electronic signature integrity and detection of unauthorized use is emphasized in this section also, as it requires the ‘‘use of transaction safeguards to prevent unauthorized use of passwords and/or identification codes,’’ and it allows detection and reporting of attempts at their unauthorized use to the system security unit as appropriate. In the event that an individual or company is going to employ the use of electronic signatures, the FDA requires that the party submit an affidavit to ‘‘certify to the agency that the electronic signatures in their system, used on or after August 20, 1997, are intended to be the legally binding equivalent of traditional handwritten signatures.’’ 3

PART 11 GUIDANCE

Following the issuance of Part 11 in 1997, significant discussions ensued among industry, system providers, and the Agency concerning the interpretation and the practicalities of complying with the new regulations. In particular, concerns were raised in the areas

6

CFR 21 PART 11

of validation, audit trails, record retention, record copying, and legacy systems. These concerns suggested the need for additional official guidance in the interpretation of Part 11 from the FDA. In an effort to address these concerns and to assist in implementation of the regulation, the FDA published a compliance policy guide in 1999, (CPG 7153.17: Enforcement Policy: 21 CFR Part 11; Electronic Records; Electronic Signatures) (7), and began to publish several Guidance for Industry documents that included the following: • 21 CFR Part 11; Electronic Records;

Electronic Signatures, Computerized Systems Used in Clinical Trials (Final Rule: April 1999) (8) • 21 CFR Part 11; Electronic Records;

Electronic Signatures, (Draft: August 2001) (9)

Validation

• 21 CFR Part 11; Electronic Records;

Electronic Signatures, Glossary Terms (Draft: August 2001) (10)

of

• 21 CFR Part 11; Electronic Records;

Electronic Signatures, Time Stamps (Draft: February 2002) (11) • 21 CFR Part 11; Electronic Records;

Electronic Signatures, Maintenance of Electronic Records (Draft: July 2002) (12) • 21 CFR Part 11; Electronic Records;

Electronic Signatures, Electronic Copies of Electronic Records (Draft: August 2002) (13) The stated purpose of these guidances is to describe the Agency’s current thinking on a topic; therefore, it should be noted that the FDA states that ‘‘guidance documents do not establish legally enforceable responsibilities . . . and should be viewed only as recommendations.’’ Computerized Systems Used in Clinical Trials (GCSUCT): A Guidance for Industry was originally released in April 1999 (8). ‘‘The Guidance’’ (as it is often referred to in the electronic clinical trials arena) provides additional detail in support of Part 11 and a commentary on implementation. According to the GCSUCT, it was developed

for two related purposes: to address requirements of 21 CFR Part 11 and to provide the agency’s current thinking on issues that pertain to computerized systems used to create, modify, maintain, archive, retrieve, or transmit clinical data intended for submission to the FDA. Although the original version of the GCSUCT is no longer in effect, a new updated guidance has been issued to replace the original guidance (see discussion below). The GCSUCT has been useful and pertinent as a basic guide to Part 11 compliance and implementation for clinical systems. 3.1 Re-Examination of Part 11 and the Scope and Application Guidance After release of the GCSUCT in 1999, the industry awaited additional guidances in the hope that they would provide greater clarity as to how the FDA expected the industry to comply with Part 11. The releases of each of five additional DRAFT guidances were followed by many comments, re-examinations, discussion, and some confusion within the industry. By the end of 2002, five years after Part 11 was released, industry personnel, system providers, and the FDA still found themselves struggling with the interpretation and implementation of Part 11. Many persons within the industry suggested that full compliance with Part 11 could lead to monumental additional costs for sponsors. This statement spurred the FDA to re-examine the progress of Part 11 to date and to discuss how to proceed. At the time, the FDA was also re-examining the CGMP regulation under an initiative called: Pharmaceutical CGMPs for the 21st Century: A Risk-Based Approach; A Science and Risk-Based Approach to Product Quality Regulation Incorporating an Integrated Quality Systems Approach. It only made sense to also re-examine Part 11. On February 20, 2003 the FDA released a new Draft Guidance 21 CFR Part 11; Electronic Records; Electronic Signatures: Scope and Application (14). Accompanying the guidance was the withdrawal of CPG.7153.17 Part 11 Compliance Policy and all previous Part 11 draft Guidance(s) (with the exception of the final GCSUCT guidance). This new FDA approach to 21CFR11 included

CFR 21 PART 11

the beginning of an overall re-examination of the rule. The FDA stated that this reexamination may lead to more modifications to Part 11 itself, which would be aligned with the FDA’s risk- and science-based approach to the GMP regulations announced in August of 2002 (Pharmaceutical cGMPs for the 21st century). It is important to note that the issuance of this guidance was not a general withdrawal of Part 11, but a repositioning on the part of FDA. In the Scope and Application guidance, the FDA attempts to narrow the scope and application of Part 11 by closely coupling it with the Predicate Rule. In effect, the FDA re-emphasized the original basis for Part 11, which for the clinical space is Good Clinical Practices. The Scope and Application Final Rule was issued into effect in the federal register in August 2003. The approach outlined in the guidance as stated by FDA was based on three main elements: • Part 11 will be interpreted narrowly

by the agency; we (the FDA) are now clarifying that fewer records will be considered subject to part 11. • For those records that remain subject to Part 11, we (the FDA) intend to exercise enforcement discretion with regard to part 11 requirements for validation, audit trails, record retention, and record copying in the manner described in this guidance and with regard to all part 11 requirements for systems that were operational before the effective date of part 11(known as legacy systems). • We (the FDA) will enforce all predicate rule requirements, which include predicate rule record and recordkeeping requirements.

7

systems under Part 11 requirements based on a documented risk assessment with relation to Safety, Efficacy, and Quality (SEQ) as opposed to arbitrary parameters.

The FDA refers to enforcement discretion throughout the Scope and Application guidance. The FDA expressed their intent to not take regulatory action to enforce compliance with Part 11 regarding validation, audit trail, record retention, and record copying requirements. The guidance states that the FDA intends to exercise enforcement discretion with regard to legacy systems that otherwise met predicate rule requirements prior to August 20, 1997—the date that the original Part 11 went into effect. The definition of legacy systems is not well defined in the guidance. For example, the guidance is not specific about systems in place in 1997, but the FDA is concerned about components or software that has been replaced subsequently. The language in the guidance suggests that the (legacy) system had to be in compliance with the predicate rule prior to 1997 to qualify for enforcement discretion under this guidance. Some persons in the industry take the stance that if the basic system was in place and functions as originally intended, then it would fall into the definition of a legacy system. A stricter interpretation would only allow systems that were implemented entirely prior to 1997 into the category of legacy systems. One interpretation of the guidance would suggest that it reduced the need for validation, but the majority interpretation was that this result was not what the FDA intended. Some basic tenets still hold true regarding validation and Part 11.

Several key points and concepts were introduced and discussed in the Scope and Application guidance; risk assessment was one of the more prominent topics. The FDA states:

1. Validation of is still required for systems covered by Part 11. The basic approach to validation set forth in the Predicate rule and previous guidances is not specific to 21 CFR Part 11 and had been in practice before Part 11.

We recommend that you base your approach on a justified and documented risk assessment and a determination of the potential of the system to affect product quality and safety and record integrity. This allows for narrowing the scope of

2. Although the CPG.7153.17 Part 11 Compliance Policy and all but one previous Part 11 DRAFT Guidance have been withdrawn, 21CFR11 is still in force and compliance is expected within

8

CFR 21 PART 11

the modified expectations of the Scope and Application Guidance. 3. Predicate rules have increased in importance for company management and compliance regarding Part 11, which presents an opportunity for industry to re-examine and refocus their efforts on the truly important aspects of compliance and the predicate rules regarding clinical systems and electronic records.

Computerized Systems Used in Clinical Investigations; Guidance for Industry (May 2007) (16) supplements Scope and Application guidance and serves to unify Part 11 guidances In May 2007, the FDA issued a new final guidance: Computerized Systems Used in Clinical Investigations. This guidance was based on the DRAFT guidance issued in 2004: Computerized Systems Used in Clinical Trials (15). As stated in the introduction, the guidance ‘‘provides to sponsors, contract research organizations (CROs), data management centers, clinical investigators, and institutional review boards (IRBs), recommendations regarding the use of computerized systems in clinical investigations’’, and additionally ‘‘supersedes the guidance of the same name dated April 1999; and supplements the guidance for industry on Part 11, Electronic Records; Electronic Signatures — Scope and Application and the Agency’s international harmonization efforts when applying these guidances to source data generated at clinical study sites.’’ After the Scope and Application guidance was issued, inconsistencies existed between the new guidance and the 1999 GCSUCT guidance that remained in effect. Namely, the areas identified by the FDA as areas they would exercise ‘‘enforcement discretion’’ were addressed explicitly and rather prescriptively in the GCSUCT. This information caused a misalignment in the guidelines and required the FDA to re-examine and revise the GCSUCT. The document titled, ‘‘Computerized Systems Used in Clinical Investigations’’ is the result of that work. The following are 11 key points to note in the guidance:

1. Application of the guidance; this guidance applies to: • ‘‘Computerized systems that contain any data that are relied on by an applicant in support of a marketing application, including computerized laboratory information management systems that capture analytical results of tests conducted during a clinical trial’’ • ‘‘Recorded source data transmitted from automated instruments directly to a computerized system (e.g., data from a chemistry autoanalyser or a Holter monitor to a laboratory information system)’’ • ‘‘Source documentation that is created in hardcopy and later entered into a computerized system, recorded by direct entry into a computerized system, or automatically recorded by a computerized system (e.g., an ECG reading)’’ The guidance states that it does not apply to ‘‘computerized medical devices that generate such data and that are otherwise regulated by FDA.’’ In addition, the guidance states that acceptance of data from clinical trials for decision-making purposes depends on the FDA’s ability to verify the quality and integrity of the data during FDA on-site inspections and audits. (21 CFR 312, 511.1(b), and 812)’’. 2. The introduction references and emphasizes the importance of ‘‘source data’’ to reconstruct the study. The word ‘‘source’’ is used 24 times throughout the document in reference to source data and/or documentation. This usage differs from previous guidance(s) in which ‘‘source data’’ is hardly mentioned in the electronic context. 3. The guidance focuses on electronic source data and reiterates previously stated tenets related to data integrity. ‘‘Such electronic source data and source documentation must meet the same fundamental elements of data quality (e.g., attributable, legible, contemporaneous, original, and accurate) that are expected of paper records and must

CFR 21 PART 11

comply with all applicable statutory and regulatory requirements.’’

changes that systems make automatically to adjust to daylight savings time.’’ 10. Under Section IV (e) on External Security Safeguards, the guidance states that ‘‘you should maintain a cumulative record that indicates, for any point in time, the names of authorized personnel, their titles, and a description of their access privileges. 11. Change Control: The guidance emphasizes the need to preserve data integrity during ‘‘security and performance patches, or component replacement.’’ In addition, it recommends ‘‘maintaining back up and recovery logs’’. The specificity of this section is more explicit than previous guidance(s).

4. The guidance requires that sponsors identify each step at which a computerized system will be used to create, modify, maintain, archive, retrieve, or transmit source data. 5. The recommendations section (section IV) states that: The computerized systems should be designed, ‘‘to prevent errors in data creation, modification, maintenance, archiving, retrieval, or transmission (e.g., inadvertently unblinding a study).’’ This statement is important to note; essentially the FDA is saying that computerized systems can and should be designed and used to improve the data quality and ensure data integrity. 6. Also under Section IV (c) regarding Source Documentation and Retention, the FDA explicitly states that ‘‘when original observations are entered into a computerized system, the electronic record is the source document’’ (eSource). Here the FDA makes clear their position regarding the interpretation, or definition of electronic source data. 7. Overall, the guidance is not that prescriptive, with exception section IV (d) on Internal Security Safeguards, which literally provides suggested system features and workflow on how the user should conduct a data entry session. 8. Audit Trails: The most significant point regarding audit trails is that the FDA explicitly state that ‘‘Audit trails . . . used to capture electronic record activities should describe when, by whom, and the reason changes were made to the electronic record.’’ Historically, it has been debated whether ‘‘reason for change’’ is a necessary system feature, this guidance makes clear the FDA position on ‘‘reason for change’’ as required. 9. Date and Time Stamps: The FDA states that ‘‘Any changes to date and time should always be documented . . . we do not expect documentation of time

9

4

CONCLUSION

21CFR11 has fulfilled its original intent to allow for electronic recording of data and electronic record keeping in all documents intended for submission to the FDA. The FDA’s intention on the scope and enforcement of the rule was clarified in the Scope and Application guidance of 2003. 21CFR11 is currently being ‘‘re-examined’’ by the FDA, but they have not provided a clear timeframe of when this re-examination will conclude or what the expected outcome of this initiative will be. Some industry insiders speculate that it will be sooner rather than later, and that the regulation will either be significantly changed or revoked. Others believe that the re-examination will result in simply incorporating the philosophy of the Scope and Application guidance and the recently updated GCSUCI guidance into the actual regulation. REFERENCES 1. 62 FR 13430, 21 CFR part 11; Electronic Records; Electronic Signatures. Published in the Federal Register, March 20, 1997. 2. Predicate Rule: These requirements include certain provisions of the Current Good Manufacturing Practice regulations (21 CFR Part 211), Current Good Manufacturing Practice in Manufacturing, Processing, Packing, or

10

CFR 21 PART 11 Holding of Drugs (21 CFR Part 210), the Quality System Regulation (21 CFR Part 820), the Good Laboratory Practice for Non clinical Laboratory Studies regulations (21 CFR Part 58), Investigator recordkeeping and record retention (21 CFR 312.62), IND safety reports (21 CFR 312.32), Protection of Human Subjects (21 CFR 50.27). Documentation of informed consent (21 CFR PART 56), Subpart D—Records and Reports, 56.115 IRB records; Subpart C—IRB Functions and Operations, 56.109 IRB review of research, 21 CFR PART 314—APPLICATIONS FOR FDA APPROVAL TO MARKET A NEW DRUG (NDA); Subpart B—Applications.

3. Federal Food, Drug, and Cosmetic Act (the Act). 4. Public Health Service Act (the PHS Act). 5. Paperwork Reduction Act of 1995. 6. Government Act (GPEA), 10/21/98.

Paperwork Title XVII,

Elimination P.L. 105-277,

7. CPG 7153.17: Enforcement Policy: 21 CFR Part 11; Electronic Records; Electronic Signatures. 8. Computerized Systems Used in Clinical Trials, (GCSUCT), A Guidance for Industry (Final Rule: April 1999). 9. 21 CFR Part 11; Electronic Records; Electronic Signatures, Validation (Draft: August 2001). 10. 21 CFR Part 11; Electronic Records; Electronic Signatures, Glossary of Terms (Draft: August 2001). 11. 21 CFR Part 11; Electronic Records; Electronic Signatures, Time Stamps (Draft: February 2002). 12. 21 CFR Part 11; Electronic Records; Electronic Signatures, Maintenance of Electronic Records (Draft: July 2002). 13. 21 CFR Part 11; Electronic Records; Electronic Signatures, Electronic Copies of Electronic Records (Draft: August 2002). 14. 21 CFR Part 11; Electronic Records; Electronic Signatures: Scope and Application (Final Rule: August 2003). 15. Computerized Systems Used in Clinical Trials, (GCSUCT), Guidance for Industry (Draft: September 2004). 16. Computerized Systems Used in Clinical Investigations, (GCSUCI), Guidance for Industry (May 2007).

FURTHER READING R. D. Kush, eClinical Trials, Planning and Implementation. Boston, MA: Thomson Centerwatch, 2003.

CROSS-REFERENCES Case Report Form Clinical Data Management Software for Data Management Software for Genetics/Genomics Clinical Trial/Study Conduct Electronic Submission of NDA EMEA Food and Drug Administration (FDA, USA) Good Clinical Practice (GCP) International Conference on Harmonization (ICH) Investigational New Drug Application (IND) Confidentiality HIPAA

• CHANGE: X − Y is compared by T.

ANALYZING CHANGE FROM BASELINE IN RANDOMIZED TRIALS

• FRACTION: (X − Y) ÷ X is compared

by T. • ANCOVA: Calculate Y = β 1 X + β 2 T + constant.

ANDREW J. VICKERS Memorial Sloan-Kettering Cancer Center, New York, New York

To illustrate the methods, we will use pain scores on a 0–10 scale as an example, making the common assumption that the scores can be treated as a continuous variable, rather than 11 ordered categories. Take the case of a patient with a baseline pain score of 5, which improves to a score of 3 after randomized treatment. The number entered into a between-group two-sample comparison, such as t-test or Mann-Whitney U test, will be 3 for the POST analysis, 2 for the CHANGE analysis, and 40% for the FRACTION analysis. In ANCOVA, 3 would be entered for the dependent variable (Y) and 5 for the covariate (X).

A very common scenario in medicine is for patients to present to a clinician with some sort of a chronic complaint and ask whether the severity of the complaint can be reduced by treatment. Accordingly, many clinical trials involve measuring the severity of a complaint before and after randomly allocated treatment: Patients with chronic headache, obesity, or hypertension are randomized to treatment or control to observe whether the treatment is effective for reducing pain, weight, or blood pressure. This article provides a nontechnical introduction to the statistical analysis of two-arm randomized trials with baseline and follow-up measurement of a continuous outcome.

1.1 Clinical Interpretation of Statistical Results For the sake of argument, assume that our illustrative patient represented the mean in the treatment group, and that there was a change in means from 5 to 4 in the control group. The results of the trial could be stated in the following terms:

1 METHODS FOR ANALYZING TRIALS WITH BASELINE AND FOLLOW-UP MEASURES In the simplest case, an outcome measure such as pain, weight, or blood pressure is measured only twice, once before randomization (‘‘baseline’’) and once at some point after randomization (‘‘follow-up’’), such as shortly after treatment is complete. We will use a simple notation for this introductory paper: X is the baseline measurement, Y is the followup assessment, and T is an indicator variable coded 1 for patients who receive treatment and coded 0 for controls. We will assume that only a single X and Y exist per patient so that, for example, if a weekly pain score is reported for 4 weeks after treatment, a summary statistic such as a mean is calculated and used as each patient’s datum. We will compare four different methods of analysis: POST, CHANGE, FRACTION, ANCOVA (analysis of covariance), which are defined as follows:

POST:

CHANGE:

FRACTION:

ANCOVA:

Mean pain at follow-up was 3 in the treatment group compared with 4 in controls Pain fell by a mean of 2 points in the treatment group compared with 1 point in controls Pain fell by a mean of 40% in the treatment group compared with 20% in controls As for CHANGE, where β 2 is the change score

The ease with which clinicians can understand each of these interpretations will vary with the endpoint. In the case of hypertension, where particular cut-points such as 140 mm Hg are considered important, clinicians will often want to know mean

• POST: Y is compared by T.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

ANALYZING CHANGE FROM BASELINE IN RANDOMIZED TRIALS

posttreatment measure (POST); for obesity research, clinical questions are often expressed in terms of weight loss (CHANGE). Percentage change (FRACTION) is obviously of little relevance for weight or blood pressure, but it is often useful for a symptom score, such as pain or number of hot flashes, which would ideally be close to zero. 2 STATISTICAL POWER OF THE ALTERNATIVE ANALYSIS METHODS

1.5 1 .5

Relative Sample Size

2

Frison and Pocock (1) have published formulae that can be used to estimate the relative power of POST, CHANGE, and ANCOVA. Relative power is dependent on the correlation between the baseline and follow-up measure (ρ xy ). Assuming that the variances of the baseline and postrandomization measures are similar, CHANGE has higher power than POST if correlation is greater than 0.5 and lower power otherwise. This measure can be thought in terms of whether the baseline measure adds variation (correlation less than 0.5) or prediction (correlation greater than 0.5). ANCOVA has superior power to both POST and CHANGE irrespective of the correlation between baseline and postrandomization measure. However, if correlation is either very high or low, the power of ANCOVA is not importantly higher than either CHANGE in the former case and POST in the latter. These relationships are shown in Fig. 1.

A slight complication in the use of Frison and Pocock’s formulae is that they require data on correlation between baseline and follow-up for the data set in the randomized trial. This correlation will be different to the correlation observed in any pretrial data set because of the effects of treatment. For example, imagine that a group of researchers has a large database of pain scores taken before and after treatment with a standard analgesic agent, and the correlation between the baseline and follow-up (ρ xy ) is 0.5. In a randomized trial that compares a new drug to the standard, ρ xy will be close to 0.5 in the control arm as well as in the trial as a whole if the new drug and standard analgesic are have similar analgesic effects. However, if the new agent was completely effective, with pain scores of 0 for all patients, ρ xy would be 0 in the experimental arm, and therefore somewhere between 0 and 0.5 for the trial as a whole. However, for the moderate effect sizes observed in most clinical trials, the difference between ρ xy in the control arm and ρ xy for the whole trial will be small: For example, if ρ xy is 0.5 in the control arm, and there is an effect size of 0.5 standard deviations, then ρ xy will be approximately 0.485 for control and treatment group patients combined. No analytic formulae have been published for the power of FRACTION. Accordingly, the power of the four analytic approaches can best be compared by simulation. Our base

0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Correlation between baseline and follow-up measures

1.0

Figure 1. Relative sample size required for different methods of analysis. Dashed line: POST. Solid grey line: CHANGE. Solid dark line: ANCOVA.

ANALYZING CHANGE FROM BASELINE IN RANDOMIZED TRIALS

scenario is a trial with a 0.5 standard deviation difference between groups and a total sample size of 60 equally divided between two arms. This sample size was given to yield a power close to 50% for the POST analysis. The correlation between baseline and follow-up measures within each group separately was varied between 0.2 and 0.8. Two other parameters were varied: whether the effect of treatment was a shift (e.g., treatment reduced scores by two points) or a ratio (e.g., treatment reduced scores by 40%), and whether change was observed in the control arm. The results are shown in the tables. Table 1 is similar to a previously published result (2). Power is highest for ANCOVA; the relative power of POST and CHANGE are as shown in Fig. 1. FRACTION has lower power than CHANGE. Table 2 is a result that has not been previously reported: The scores are reduced in the control group (e.g., the ‘‘placebo effect’’) but are reduced more in the treatment group. FRACTION has higher power than CHANGE; however, if scores increase in both groups (e.g., pain worsening over time), although less so in treated patients, the power of FRACTION is reduced. Clearly, the relative power of FRACTION and

3

CHANGE depends on the difference between baseline and follow-up measures in the control group. Tables 3 and 4 show the results of the simulation using a ratio treatment effect. The relative powers of the different methods of analysis are similar to those for a shift treatment effect. 2.1 Which Method to Use? The relative power of POST, CHANGE, and FRACTION seems to vary in complex ways depending on the correlation, type of treatment effect, and the degree of change in the treatment group. Nonetheless, the key message remains the same: ANCOVA has the greatest power irrespective of the scenario. Yet, as pointed out above, occasionally other methods may provide results of greater clinical relevance. A simple solution is based on the principle that presentation of statistical results need not be determined solely by the principal inferential analysis. The most obvious application of this principle is when outcome is measured on a continuous scale, but a binary outcome is used as an additional method to present trial data. For example, in a trial on headache pain (3), the primary outcome was a difference between days

Table 1. Power of Methods for Analyzing Trials with Baseline and Follow-up Measures where Treatment Effect is a Shift Correlation between baseline and follow-up Method POST CHANGE FRACTION ANCOVA

0.20 0.475 0.322 0.300 0.488

0.35 0.484 0.387 0.367 0.527

0.50 0.482 0.470 0.454 0.589

0.65 0.481 0.626 0.609 0.701

0.80 0.475 0.847 0.838 0.874

The trial parameters are 30 patients per group, no change between baseline and follow-up in the control group, and a mean 0.5 standard deviation decrease in the treatment group.

Table 2. Power of FRACTION for Analyzing Trials with Baseline and Follow-up Measures, Varying Change in the Control Group Correlation between baseline and follow-up Change in controls 1 SD decrease 2 SD decrease 3 SD decrease 1 SD increase

0.20 0.362 0.422 0.461 0.251

0.35 0.436 0.492 0.515 0.303

0.50 0.531 0.570 0.561 0.379

0.65 0.678 0.675 0.611 0.510

0.80 0.859 0.788 0.655 0.727

The trial parameters are 30 patients per group, and a 0.5 standard deviation decrease in the treatment group. Power for other methods is invariant to the change in controls; accordingly, Table (1) can be used as a comparison.

4

ANALYZING CHANGE FROM BASELINE IN RANDOMIZED TRIALS

Table 3. Power of Methods for Analyzing Trials with Baseline and Follow-up Measures where Treatment Effect is a Ratio Correlation between baseline and follow-up Method POST CHANGE FRACTION ANCOVA

0.20 0.513 0.337 0.307 0.525

0.35 0.524 0.402 0.370 0.570

0.50 0.520 0.488 0.457 0.630

0.65 0.518 0.646 0.608 0.743

0.80 0.513 0.860 0.829 0.903

The trial parameters are 30 patients per group and no change between baseline and follow-up in the control group. Scores in the treatment group are reduced by a ratio equivalent to a mean 0.5 standard deviation.

Table 4. Power of Methods for Analyzing Trials with Baseline and Follow-up Measures where Treatment Effect is a Ratio and Scores Improve in the Control Group Correlation between baseline and follow-up Method POST CHANGE FRACTION ANCOVA

0.20 0.523 0.340 0.378 0.537

0.35 0.534 0.406 0.448 0.579

0.50 0.530 0.491 0.543 0.640

0.65 0.527 0.649 0.688 0.754

0.80 0.522 0.862 0.871 0.908

The trial parameters are 30 patients per group and a 1 standard deviation change between baseline and follow-up in the control group. Scores in the treatment group are reduced by a ratio equivalent to a mean 0.5 standard deviation.

with headache, but the authors also reported the proportion of patients who experienced a 50% or greater reduction in headache days. Accordingly, one can report the results of the ANCOVA and then, if it would be helpful, give percentage change by dividing mean follow-up score in the control group by β 2 , which is the difference between treatments estimated by ANCOVA. Note that choice of ANCOVA as the method of analysis has implications for sample size calculation. Such calculations require an estimate of the correlation between baseline and follow-up measures (1). Also, note that sample size was not varied in the simulations: ANCOVA involves the loss of an additional degree of freedom, which can hurt power if sample sizes are very small (such as fewer than five patients per group). 2.2 Additional Advantages of ANCOVA ANCOVA has several advantages additional to statistical power in comparison with CHANGE, POST, or FRACTION. The first concerns chance imbalance between groups at baseline, for example, if pain scores at randomization were higher in the treatment group. Of course, all methods of analysis are

unconditionally unbiased, that is, their power (true positive rate) and size (false positive rate or Type I error) are close to nominal levels when averaged over a large number of trials, even if a chance baseline imbalance is observed in some trials. However, baseline imbalance may compromise any specific trial. In the case where the baseline measure is higher in the treatment group, a POST analysis will underestimate the true treatment effect and, because of regression to the mean, a CHANGE analysis will overestimate treatment benefit [empirical examples of this phenomenon have been published (4)]. ANCOVA provides a good estimate of treatment effect irrespective of chance baseline imbalance. The second advantage of ANCOVA is related to stratification. If a trial incorporates one or more stratification factors in randomization, then failure to incorporate these factors in analysis will inflate P values (5). Power can be returned to nominal levels by adding randomization strata as covariates in ANCOVA. For example, if a hypertension trial was randomized using age and weight as stratification variables, then analysis should use Y = β 1 X + β 2 T + β 3 age + β 4 weight + constant, where age and weight are entered

ANALYZING CHANGE FROM BASELINE IN RANDOMIZED TRIALS

in the same form as for stratification (e.g., as continuous variables for minimized randomization; as binary variables for randomization by stratified blocks). ANCOVA is extended to more complex designs and analyses more easily than twosample comparisons. These designs include trials that assess treatment effects that diverge over time, for example, where the effects of a treatment are cumulative, those with more than two arms (e.g., psychotherapy vs. relaxation therapy vs. no treatment), or analyses that examine whether treatment response differs by baseline severity. Although these topics are outside the scope of this article, it can be noted briefly that trials with divergent treatment effects can be analyzed either by an ANCOVA using the slope of follow-up measurements as the dependent variable or by a generalized estimating equations approach with terms for both time and time by treatment interaction. Multiarm trials can be analyzed by ANCOVA using dummy variables for treatments (e.g., ‘‘contact’’ and ‘‘talking therapy’’ would be coded 1, 1 for psychotherapy; 1, 0 for relaxation therapy; 0, 0 for controls); analyses that examine whether treatment response differs by baseline severity can use an interaction term such that Y = β 1 X + β 2 T + β 3 T.X + constant. 3 EFFECT OF NON-NORMAL DISTRIBUTIONS ON ANALYSIS ANCOVA is a parametric analysis, and it has become an article of faith that parametric statistics require either normally distributed data or sample sizes large enough to invoke the ‘‘Central Limit Theorem.’’ As an example, one popular statistics textbook states that, ‘‘Parametric methods require the observations in each group to have an approximately Normal distribution . . . if the raw data do not satisfy there conditions . . . a non-parametric method should be used’’ (6). Moreover, a considerable number of simulation studies have shown that although Type I errors of parametric methods such as the t-test are unaffected by the sample distribution, such methods are underpowered for the analysis of nonsymmetric data (7). Accordingly perhaps, the New England Journal of

5

Medicine currently includes the following in its instructions to authors. ‘‘For analysis of [continuous] measurements, nonparametric methods should be used to compare groups when the distribution of the dependent variable is not normal’’ (note that, at the time of writing, these recommendations are under revision). That said, for a linear regression, it is not the distribution of the dependent variable Y that is of interest, but the distribution of Y conditional on the independent variables. This statement means that, for an ANCOVA, the distribution of interest is not Y but X − Y, in other words, the change score. As it happens, if X and Y are both drawn from non-normal distributions, X − Y will often be distributed normally, because change scores are a linear combination and the Central Limit Theorem therefore applies. As a straightforward example, imagine that baseline and post-treatment measure were represented by a single throw of a die. The posttreatment measure has a flat (uniform) distribution, with each possible value having an equal probability. The change score has a more normal distribution: A peak is observed in the middle at zero; the chance of a zero change score is the same as the chance of throwing the same number twice, that is 1 in 6—with more rare events at the extremes—there is only a 1 in 18 chance of increasing or decreasing score by 5. A similar result can be demonstrated for a wide variety of distributions, which include uniform, asymmetric, and what is probably most common in medical research, positive skew. Distributions for baseline and follow-up measures may show extreme deviations from the normal, yet the change score tends toward normality (7). Simulation studies have shown that ANCOVA retains its power advantages over CHANGE and POST for data sampled from distributions other than the normal (7).

REFERENCES 1. L. Frison and S. J. Pocock, Repeated measures in clinical trials: analysis using mean summary statistics and its implications for design. Stat. Med. 1992; 11: 1685–1704.

6

ANALYZING CHANGE FROM BASELINE IN RANDOMIZED TRIALS

2. A. J. Vickers, The use of percentage change from baseline as an outcome in a controlled trial is statistically inefficient: a simulation study. BMC Med. Res. Methodol. 2001; 1: 6. 3. D. Melchart, A. Streng, A. Hoppe, et al., Acupuncture in patients with tension-type headache: randomised controlled trial. BMJ 2005; 331: 376–382. 4. A. J. Vickers, Statistical reanalysis of four recent randomized trials of acupuncture for pain using analysis of covariance. Clin. J. Pain 2004; 20: 319–323. 5. L. A. Kalish and C. B. Begg, The impact of treatment allocation procedures on nominal significance levels and bias. Control. Clin. Trials 1987; 8: 121–135. 6. D. G. Altman, Practical Statistics for Medical Research. London: Chapman and Hall, 1991. 7. A. J. Vickers, Parametric versus nonparametric statistics in the analysis of randomized trials with non-normally distributed data. BMC Med. Res. Methodol. 2005; 5: 35.

FURTHER READING L. J. Frison and S. J. Pocock, Linearly divergent treatment effects in clinical trials with repeated measures: efficient analysis using summary statistics. Stat. Med. 1997; 16: 2855–2872.

L. Frison and S. J. Pocock, Repeated measures in clinical trials: analysis using mean summary statistics and its implications for design. Stat. Med. 1992; 11: 1685–1704. Y. K. Tu, A. Blance, V. Clerehugh, and M. S. Gilthorpe, Statistical power for analyses of changes in randomized controlled trials. J. Dent. Res. 2005; 84: 283–287. A. J. Vickers, Analysis of variance is easily misapplied in the analysis of randomized trials: a critique and discussion of alternative statistical approaches. Psychosom. Med. 2005; 67: 652–655.

CROSS-REFERENCES Longitudinal data (study design Phase III) ANCOVA (methodology / analysis) Generalized estimating equations (methodology / analysis) Linear model (methodology / analysis)

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

should be addressed to the appropriate clinical division (8) and directed to the document control center at the appropriate address indicated below.

NORMAN R. SCHMUFF 1. For drug products regulated by CDER, send the IND submission to the Central Document Room, Center for Drug Evaluation and Research, Food and Drug Administration, 5901–B Ammendale Rd., Beltsville, MD 20705–1266. 2. For biological products regulated by CDER, send the IND submission to the Central Document Room, Center for Drug Evaluation and Research, Food and Drug Administration, 5901–B Ammendale Rd., Beltsville, MD 20705–1266. 3. For biological products regulated by CBER, send the IND submission to the Document Control Center (HFM–99), Center for Biologics Evaluation and Research, Food and Drug Administration, 1401 Rockville Pike, Suite 200 N, Rockville, MD 20852–1448.

U.S. Food and Drug Administration Silver Spring, Maryland

DAVID T. LIN Biologics Consulting Group, Inc. Alexandria, Virginia

1

OVERVIEW

Formerly, the U.S. Food and Drug Administration (FDA) used the term ‘‘Chemistry, Manufacturing, and Controls’’ (CMC) to refer to what other countries and regions called the ‘‘Quality’’ portion of a new drug submission. The terms are now used interchangeably at FDA. CMC/Quality requirements for INDs are spelled out briefly in the Code of Federal Regulations (CFR) (1). Note that under FDA’s Good Guidance Practices (2,3), items that are obligatory are referred to as ‘‘requirements.’’ Requirements are contained either in laws or legislation (4) (e.g., The Food, Drug and Cosmetic Act (5) or the Public Health Service Act (6) for biological products) or regulation, which, in this case, is a formal FDA-drafted elaboration and interpretation of the legislation, contained in Title 21 of the CFR. These requirements comprise what sponsors and applicants ‘‘must’’ do, whereas recommendations, which are generally contained in guidance documents, are not mandatory. 2

Quality aspects of these applications are assessed either by the Office of New Drug Quality Assessment (ONDQA) or the Office of Biotechnology Products (OBP); however, sponsors should rely on the clinical division as their primary contact point. ONDQA currently handles all of the traditional small molecule and hormone protein (e.g., the insulins and growth hormones) products. OBP handles most of the remaining biological products under CDER’s review jurisdiction.

ADMINISTRATIVE ASPECTS 3

INDs for vaccines, blood products, and gene therapy are under the review jurisdiction of CBER products. CDER currently has jurisdiction over small molecule drugs and biological drugs derived from natural sources, whether chemically synthesized or from recombinant DNA technology. For CDER or CBER, electronic INDs can be sent via FDA’s secure electronic gateway (7) that was established in 2007. Paper-based IND applications and electronic submission on physical media

PRE-INDS

CDER offers a Pre-Investigational New Drug Application (pre-IND) consultation program to foster early communications between sponsors and new drug review divisions to provide guidance on the data necessary to be submitted in INDs. The review divisions are organized generally along therapeutic class and can each be contacted using the designated ‘‘Pre-IND Consultation List’’ (9).

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

4

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

QUALITY FOR IND TYPES

4.1 IND Categories The two IND categories, both of which are standard INDs not distinguished by the IND regulations, are as follows: • Commercial • Research (noncommercial)

‘‘Commercial’’ INDs are filed by pharmaceutical firms intending to file subsequent New Drug Applications (NDAs)/BLAs to commercialize their product. IND filings from large research institutions, such as the National Institutes of Health are also considered to be in this category. ‘‘Research’’ INDs may be submitted by a physician who both initiates and conducts an investigation, and under whose immediate direction the investigational drug is administered or dispensed. A physician might submit a research IND to propose studying an unapproved drug or studying an approved product for a new indication or in a new patient population. The quality data expectations for these applications are the same and are described in general in 21 CFR 312 (See Code of Federal Regulation (CFR) article). 4.2 Special IND Types The three special IND types are as follows: • Emergency Use • Treatment IND • Exploratory IND

An ‘‘Emergency Use’’ IND allows the FDA to authorize use of an experimental drug in an emergency situation that does not allow time for submission of an IND in accordance with 21 CFR 312.23 or 21 CFR 312.34. It is also used for patients who do not meet the criteria of an existing study protocol or for cases in which an approved study protocol does not exist. Typically, these INDs involve single patients. In nearly all cases for quality data, the application references other existing applications.

A ‘‘Treatment IND’’ is submitted for experimental drugs that show promise in clinical testing for serious or immediately lifethreatening conditions, while the final clinical work is conducted and the FDA review takes place. (For additional information see Treatment IND article) In January of 2006, CDER issued ‘‘Guidance for Industry, Investigators, and Reviewers Exploratory IND Studies.’’ An ‘‘exploratory IND study’’ is defined as a clinical trial that • is conducted early in Phase 1, • involves very limited human exposure,

and • has no therapeutic nor diagnostic intent

(e.g., screening studies or microdose studies). The extent of CMC-related IND information is stated to be ‘‘similar to that described in current guidance for use of investigational products.’’ For CMC purposes, little difference is found in the data expectations for this study and a traditional Phase 1 study. 4.3 IND Phases The regulations at 21 CFR 312.23(a)(7)(i) emphasize the graded nature of manufacturing and controls information. Although in each phase of the investigation sufficient information should be submitted to ensure the proper ‘‘identity strength quality and purity’’ of the investigational drug, the amount of information needed to make that assurance will vary with the phase of the investigation, the proposed duration of the investigation, the dosage form, and the amount of information otherwise available. 5 QUALITY-BY-DESIGN AND INDS As part of the Pharmaceutical Quality for the 21st Century Initiative (10), FDA is encouraging a more systematic approach to pharmaceutical development and manufacturing that generally falls under the concept of Quality-by-Design (QbD). The QbD approach suggests that product quality should be built in from the ground up by designing each

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

manufacturing step for optimal product performance at every stage of the development process. QbD is defined in the Step 2 ICH Annex to Q8 (11) as: A systematic approach to development that begins with predefined objectives and emphasizes product and process understanding and process control, based on sound science and quality risk management.

ONDQA Director Moheb Nasr described the QbD approach this way in his November 2006 presentation (12) at the American Association of Pharmaceutical Scientists Annual Meeting: In a Quality by Design system: • The product is designed to meet patient

needs and performance requirements • The process is designed to consistently

meet product critical quality attributes • The impact of starting raw materials

and process parameters on product quality is understood • The process is continually monitored, evaluated and updated to allow for consistent quality over time • Critical sources of process variability are identified and controlled • Appropriate control strategies are developed As described below, much of the QbD aspects of an IND or NDA/BLA will be included in the P.2 Pharmaceutical Development section of the application. Although QbD is not a statutory requirement, FDA believes this approach to be beneficial to both sponsors/applicants and to the public. 6

ELECTRONIC SUBMISSIONS FOR INDS

Submissions of INDs in the electronic Common Technical Document (eCTD) format (13) have been strongly encouraged, although not required. However, as of January 1, 2008, the only acceptable electronic submissions to FDA are submissions in the eCTD format (14). Advantages to FDA include less required storage space for paper submissions, expedited routing, and simultaneous

3

availability to reviewers and managers. An advantage to both sponsors and the agency is the ‘‘lifecycle’’ features built into the eCTD. These features allow for an historical view of changes within a given section. It is also noteworthy that submission redundancies can be eliminated (e.g., submitting the same study report to both an IND and an NDA/BLA) because subsequent applications, both IND and NDA/BLA, simply can include links to previously submitted files. In 2007, an FDA electronic gateway (15) was established to facilitate timely submission and eliminate the need for submissions on physical media. Note that for purposes of eIND submissions (but not NDAs/BLAs), it is not necessary to comply with the ‘‘Granularity’’ Annex (16) to ICH M4. That is, it would be acceptable to submit a single file for the Module 2 Quality Overall Summary (consistent with the ‘‘Granularity’’ document), a single file for the ‘‘S’’ drug substance section, and a single file for the ‘‘P’’ drug product section. These latter two options are in conflict with the ‘‘Granularity’’ Annex but nonetheless acceptable for eINDs. 7

GENERAL CONSIDERATIONS

For the proposed use of a U.S. marketed product in unmodified form and in its original packaging, the sponsors need only to indicate this use. For products that are modified (e.g., over-encapsulated tablets), the IND should include some evidence of equivalent performance of the original to the modified product for the duration of the proposed trial. For repackaged products, data should be included to demonstrate that the product is stable for the duration of the trial. For placebo-controlled trials, component and composition data for the placebo could be included in a separate drug product section. A brief rationale should be included for why the blinding is adequate; it should address, for example, issues relating to appearance, color, and taste. Foreign-sourced comparators present special challenges. Generally, a U.S.-sourced comparator product is preferred. The use of FDA-approved drug products provides assurance of drug quality. Where this use is not

4

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

possible and local products are used, documentation should be provided to show that the drug product is comparable in quality to the U.S. product. This documentation could involve, for example, comparing impurity, dissolution profiles and content uniformity. Without an adequate showing of comparability, study results would be considered inconclusive and would likely not be adequate for support of approval of a new agent, or a new indication for an existing product. In certain cases, a showing of comparability might involve an in vivo bioequivalence study. For cases in which it is desired to use a foreign comparator, when no U.S. approved product is available, discussions with FDA should be occur at an early development stage before filing the IND. 8

CMC CLINICAL HOLD CONSIDERATIONS

A clinical hold is as follows: An order issued by FDA to the sponsor to delay a proposed clinical investigation or to suspend an ongoing investigation. The clinical hold order may apply to one or more of the investigations covered by an IND. When a proposed study is placed on clinical hold, subjects may not be given the investigational drug. When an ongoing study is placed on clinical hold, no new subjects may be recruited to the study and placed on the investigational drug; patients already in the study should be taken off therapy involving the investigational drug unless specifically permitted by FDA in the interest of patient safety (17)

Every effort is made by FDA to facilitate the initiation of clinical trials. When serious deficiencies are identified before the 30-day date, sponsors are generally notified by telephone or e-mail. It is especially important in this period to respond promptly and completely to FDA requests. It is advantageous to have the primary contact available during FDA working hours during this time. Any CMC deficiencies should be addressed through a formal submission to the IND. The previously mentioned FDA electronic gateway (18) was established to facilitate timely submission and reviewer availability of such IND amendments.

For all disciplines, including CMC, the review focus is on the safety of the proposed clinical trial. Consequently, issues relating to commercial manufacture are generally not pertinent. However, inadequate attention to product variability can be a safety issue and may compromise the results of a clinical trial. For ‘‘Research’’ INDs, the most common CMC reasons for a clinical hold relates to lack of administrative information, such as a reference to an existing IND, and a letter of authorization from that IND’s sponsor. Issues may also develop relating to the performance or stability of a repackaged or reformulated commercial product. Any manipulation of a sterile product is also of special concern. For ‘‘Commercial’’ INDs, quoting the Phase 1 guidance: The identification of a safety concern or insufficient data to make an evaluation of safety is the only basis for a clinical hold based on the CMC section. Reasons for concern may include, for example: 1) a product made with unknown or impure components; 2) a product possessing chemical structures of known or highly likely toxicity; 3) a product that cannot remain chemically stable throughout the testing program proposed; or 4) a product with an impurity profile indicative of a potential health hazard or an impurity profile insufficiently defined to assess a potential health hazard; or 5) a poorly characterized master or working cell bank.

Regarding impurities, it is critical that the material proposed for use in the clinical trial have a similar impurity profile to that used in nonclinical studies. For example, the same drug substance lot used in nonclinical studies might be used to manufacture clinical trial lots. If impurities appear only in the drug product, then it is imperative that these, too, should be appropriately qualified. 9 SUBMISSION FORMAT AND CONTENT The content, but not the format, of CMC section for INDs is prescribed by 21 CFR 312.23(a)(7). In 1995, CDER and CBER issued ‘‘Guidance for Industry: Content and Format of Investigational New Drug Applications (INDs) for Phase 1 Studies of Drugs,

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

Including Well- Characterized, Therapeutic, Biotechnology-derived Products’’ (19). This predated work on the International Conference on Harmonisation (20) (ICH) topic M4, the Common Technical Document (CTD), which reached Step 4 of this FDA guidance in November of 2000. Although the content is still pertinent, the submission should now be formatted according to the CTD format as described below. In May of 2003, CDER issued ‘‘Guidance for Industry: INDs for Phase 2 and Phase 3 Studies Chemistry, Manufacturing, and Controls Information’’ (21). Like the Phase 1 guidance described above it includes no references are to the CTD. Nevertheless, as for Phase 1 submissions, these, too, should be submitted in the CTD format, preferably as an eCTD. 9.1 Module 1 Module 1 contains region-specific data, largely administrative, that falls outside the scope of the ICH CTD. For eCTD applications, the format and content of Module 1 is described by FDA (22), Japanese MHLW/PMDA (23), and European EMEA (24). Quality-specific data for FDA should include, if appropriate, a request for categorical exclusion from or submission of an environmental assessment (25). Also, this module should include a copy of all labels and labeling to be provided to each investigator (26). A mock-up or printed representation of the proposed labeling that will be provided to investigator(s) in the proposed clinical trial should be submitted. Investigational labels must carry a ‘‘caution’’ statement as required by 21 CFR 312.6(a). That statement reads: ‘‘Caution: New Drug Limited by Federal (or United States) law to investigational use.’’ Although INDs are generally exempt from expiration dating, where new drug products for investigational use are to be reconstituted at the time of dispensing, their labeling must bear expiration information for the reconstituted drug product (27). 9.2 Module 2 Unlike Efficacy, and Safety, Quality has a single Module 2 heading known as the ‘‘Quality Overall Summary’’ (QOS). FDA has issued

5

no guidance on IND expectations for the QOS or other Module 2 summaries. Consequently, the inclusion of a QOS in an IND can be considered optional. However, if submitting a QOS, it seems appropriate to limit its length to five or fewer pages. The format could be as described in M4-Q (28), or it might be abbreviated as that guidance applies to NDAs/BLAs and ANDAs. 9.3 Module 3 If a QOS is not included, then provide a summary of the application in Module 3 (29). The file should be submitted under eCTD (v 3.2) element ‘‘m3-quality.’’ Although it is not obvious from the eCTD specification, it is permissible to submit documents under this heading (or element, to use XML terminology). A look at the eCTD technical specification will reveal that this possibility also exists for many other headings with no content other than subordinate headings. As suggested in the Phase 1 guidance (30), the sponsor should state whether it believes that either 1) the chemistry of either the drug substance or the drug product, or 2) the manufacturing of either the drug substance or the drug product, presents any signals of potential human risk. If so, then these signals of potential risks should be discussed, and the steps proposed to monitor for such risks should be described, or the reasons why the signals should be dismissed should be discussed. In addition, sponsors should describe any chemistry and manufacturing differences between the drug substance/product proposed for clinical use and the drug substance/product used in the animal toxicology trials that formed the basis for the sponsor’s conclusion that it was safe to proceed with the proposed clinical study. How these differences might affect the safety profile of the drug product should also be discussed. If no differences are found, then this fact should be stated explicitly. 10

S DRUG SUBSTANCE

A short summary, a few paragraphs in length, describing the drug substance should be provided. If the IND proposes a U.S.-approved

6

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

drug, then the entire drug substance section can be omitted. If it is proposed to use a U.S.approved drug manufactured by an alternate supplier, then complete CMC information should be included.

10.2.1 Manufacturer(s)P1 . Provide the complete name and street address for the drug substance manufacturer and its FDA Establishment Identifier number (FEI) and/ or Central File Number (CFN) if known.

10.1 General InformationP1 10.1.1 NomenclatureP1 . Provide all names (including, e.g., company code, INN, and USAN) by which the substance has been known. Pick one of these names and use it consistently throughout the application. If legacy reports use alternate names, then it will be helpful to the reviewer if you can modify these reports so that a single standard name is used across the application. Follow a similar practice for naming impurities. Also, include a systematic name, preferably the name used by the Chemical Abstracts Service (CAS). Include the CAS Registry Number (CAS RN) for the drug and any relevant salt forms.

10.2.2 Description of Manufacturing Process and Process ControlsP1 . A brief narrative description should be included along with a detailed flow diagram. The latter should indicate weights, yields, solvents, reagents, in-process controls, and isolated intermediates. Critical steps should be identified, if known. Include milling or size-reduction steps. More information may be needed to assess the safety of biotechnology-derived drugs or drugs extracted from human or animal sources. Such information includes the cell bank, cell culture, harvest, purification and modification reactions, and filling, storage, and shipping conditions. The general description of the synthetic and manufacturing process (e.g., fermentation and purification) provided to support the Phase 1 studies should be updated from a safety perspective if changes or modifications have been introduced. Reprocessing procedures and controls need not be described except for natural sourced drug substances when the steps affect safety (e.g., virus or impurity clearance). For sterile drug substances, updates on the Phase 1 manufacturing process should be submitted. The Phase 2 information should include changes in the drug substance sterilization process (e.g., terminal sterilization to aseptic processing). Information related to the validation of the sterilization process need not be submitted at this time.

10.1.2 StructureP1 . Provide a legible graphical structure and clearly indicate any relevant stereochemical features. Include the molecular mass. For biotechnological materials, provide schematic amino acid sequence and indicate secondary and tertiary structure, if known, and sites of posttranslational modifications. It may be helpful to provide a machine-readable representation of the structure, such as a SMILES (31) string, MDL MOLFILE (32), or InChI (33). 10.1.3 General PropertiesP1 . Include melting point, and other physico-chemical properties, such as log P, and a pH solubility profile. Indicate which polymorph is proposed for use, as appropriate. Also, include a short summary of biological activity. 10.2

ManufactureP1

After the initial IND submission, if changes have been made in manufacturing sites or processes, then provide a short summary in the following sections. Amend the IND only if such changes involve lots used for stability studies or preclinical or clinical testing. P1 These sections should be included in submissions

for Phase 1 studies.

10.2.3 Control of MaterialsP1 . Indicate the quality standards for the starting material, solvents, and reagents. For fermentation products or natural substances extracted from plant, human, or animal sources, the following information should be provided in Phase 1 submissions: (1) origin (e.g., country), source (e.g., pancreas), and taxonomy (e.g., family, genus, species, and variety) of the starting materials or strain of the microorganism, (2) details of appropriate screening procedures for adventitious agents, if relevant, and (3) information to support the

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

safe use of any materials of microbial, plant, human, or animal origin (e.g., certification, screening, and testing). Any updates to the information submitted in Phase 1 and any new information to support the safety of materials of human or animal origin should be provided in fater submissions. 10.2.4 Controls of Critical Steps and Intermediates. The general strategy for identifying critical steps and the nature of controls for these steps, if identified, should be discussed briefly. For isolated intermediates, provide a tentative specification. To the extent possible in Phase 2, sponsors should provide information on controls of critical steps and intermediates and tentative acceptance criteria to ensure that the manufacturing process is controlled at predetermined points. Although controls of critical steps and intermediates still can be in development, information on controls for monitoring adventitious agents should be provided for fermentation and natural sourced (human or animal) drug substances, as appropriate. 10.2.5 Process Validation and/or EvaluationP1 . Generally this section is not needed in an IND submission. However, it might be appropriate, for example, to describe criteria used to assess clearance of adventitious agents. 10.2.6 Manufacturing Process Development. Although only safety-related aspects need to be reported for Phase 1 studies, this section provides an opportunity for a brief description of the sponsor’s ‘‘Quality-byDesign’’ approach. 10.3 CharacterizationP1 10.3.1 Elucidation of Structure and other Characteristics. Evidence to support the proposed chemical structure of the drug should be submitted. It is understood that the amount of structure information will be limited in the early stage of drug development. Data should be summarized in tabular form and indicate, for example, peak assignments and coupling constants for 1 H NMR spectra. Infrared and 1 H NMR spectra should be submitted. For the latter, provide expansions of regions of interest. For biotechnological

7

materials, information should be provided on the primary, secondary, and higher-order structure. In addition, information on biological activity, purity, and immunochemical properties (when relevant) should be included. 10.3.2 ImpuritiesP1 . This section should provide an HPLC that clearly indicates impurities and should provide structural identification when possible. If structures have not been assigned, then spectra and/or spectral data should be included. Although Q3A does not apply to drug substances for investigational studies, it is advisable follow its general principles in that thresholds should be established for reporting, identification, and qualification. Subsequently, new impurities (e.g., from a change in synthetic pathway) should be qualified, quantified, and reported, as appropriate. Procedures to evaluate impurities to support an NDA/BLA (e.g., recommended identification levels) may not be practical at this point in drug development. Suitable limits should be established based on manufacturing experience, stability data, and safety considerations. 10.4

Control of Drug SubstanceP1

10.4.1 SpecificationP1 . ICH Q6A (34) and Q6B (35) define specification as . . . a list of tests, references to analytical procedures, and appropriate acceptance criteria which are numerical limits, ranges, or other criteria for the tests described.’’ A tentative specification should include the ‘‘Universal Tests / Criteria’’ defined in Q6A as well as the appropriate ‘‘Specific Tests / Criteria.’’ An individual test and acceptance criteria should be in place for impurities above an identification threshold defined by the sponsor. The specification should also include an acceptance criterion for total impurities. 10.4.2 Analytical ProceduresP1 . Analytical procedures used to assure the identity, strength, quality, and purity of the drug product must be included. Except for compendial procedures, a general description of each test method should be provided.

8

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

10.4.3 Validation of Analytical Procedures. Although data should support that the analytical procedures are suitable for their intended purpose, validation data ordinarily need not be submitted at the initial stage of drug development. However, for some well-characterized, therapeutic biotechnology-derived products, validation data may be needed in certain circumstances to ensure safety in Phase 1. For Phase 2/3 studies, the analytical procedure (e.g., high-pressure liquid chromatography) used to perform a test and to support the tentative acceptance criteria should be described briefly, and changes should be reported when the changes are such that an update of the brief description is warranted. A complete description of analytical procedures and appropriate validation data should be available for the analytical procedures that are not from an FDA-recognized standard reference (e.g., official compendium and AOAC International Book of Methods), and this information should be submitted on request. 10.4.4 Batch AnalysesP1 . Batch analyses for all relevant batches should be included. Where possible, test results should be given as numeric values, not just ‘‘pass’’ or ‘‘fail.’’ The use of each batch should be indicated, with a clear indication of whether the proposed clinical trial batch is the same as that used in preclinical testing. 10.4.5 Justification of Specification. Justification of the interim specification should rely on all relevant data, including, for example, batches used in preclinical testing and development. 10.5

Reference Standards or MaterialsP1

Some information should be provided on reference standards. For biological and biotechnological products, if the working reference standard is based on a national or international standard (e.g., World Health Organization), then information should be provided on that reference standard. 10.6

Container Closure SystemP1

A general brief description of the container closure system should be provided. Include,

if known, whether the components in direct contact with the drug substance comply with the indirect food additive regulations in 21 CFR 174.5. 10.7 StabilityP1 Sufficient information should be submitted to support the stability of the drug substance during the toxicologic studies and the proposed clinical studies (see stability analysis and stability study design chapters). 10.7.1 Stability Summary and ConclusionsP1 . Summarize the data from section S.7.3. 10.7.2 Post-approval Stability Protocol and Stability Commitment. This section is not appropriate to an IND and can be deleted. Note that in the eCTD, most sections (elements) are optional. If sections are deleted, then it is important, however, not to change any of the element names that include the section numbers (i.e., if a section like this one is not included, then do not renumber subsequent sections). 10.7.3 Stability DataP1 . Provide the stability data in tabular format. A graphical display of the data can be helpful. 11 DRUG PRODUCT Provide a short drug product summary here. A separate Drug Product section should be provided for any comparator products. If a placebo is used in the trial, then it, too, can be in a separate Drug Product section, but to facilitate comparisons, it is preferable to include this information in the same section as the drug product. If the IND is for a U.S.-approved drug, and the drug product is used in unmodified form in its original container, then this situation should be clearly indicated. If the product is modified (e.g., the marketed tablet is over-encapsulated) or packaged in something other than the marketed container, then information assuring equivalent product performance should be included in the appropriate section.

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

11.1 Description and Composition of the Drug ProductP1 A description of the drug product and its composition should be provided. The information provided should include, for example: • Description of the dosage form, • Composition, for example, list of all com-

ponents of the dosage form and their amount on a per-unit basis (including overages, if any), the function of the components, and a reference to their quality standards (e.g., compendial monographs or manufacturer’s specifications), • Description of accompanying reconstitution diluent(s), and • Type of container and closure used for the drug product and accompanying reconstitution diluent, if applicable. Note that this section includes the components and composition of the unit dose but not the batch formula that is in section P.3.2. 11.2 Pharmaceutical Development This section provides an opportunity to describe Quality-by-Design efforts that may have been built into the drug product. The expectations for a Phase 1 IND are quite modest, and most information indicated in M4Q (R1) can be considered optional. For more extensive P.2 guidance, consult the ICH Q8 ‘‘Pharmaceutical Development’’ guidance (36). Note however that the foregoing documents strictly apply to NDAs, and BLAs. 11.2.1 Components of the Drug Product 11.2.1.1 Drug Substance At minimum, the compatibility of the drug substance with excipients listed in P.1 should be discussed. Likewise, for fixed dose products, containing more than one active ingredient the compatibility of drug substances with each other should be addressed. 11.2.1.2 Excipients The choice of excipients listed in P.1, their concentration, and their characteristics that can influence the drug product performance should be discussed relative to their respective functions.

9

11.2.2 Drug Product 11.2.2.1 Formulation Development A brief summary that describes the development of the drug product should be provided, taking into consideration the proposed route of administration and usage. 11.2.2.2 Overages Any overages in the formulation(s) described in P.1 should be justified. 11.2.2.3 Physico-Chemical and Biological Properties Parameters relevant to the performance of the drug product, such as pH, ionic strength, dissolution, redispersion, reconstitution, particle size distribution, aggregation, polymorphism, rheological properties, biological activity or potency, and/or immunological activity, should be addressed. 11.2.3 Manufacturing Process Development. A general discussion of the approach to process development could be included here, with any specifics on what has been done so far, and what is planned. 11.2.4 Container Closure System. A brief description of the suitability of the container closure for its intended purpose could be discussed, considering, for example, choice of materials, protection from moisture and light, compatibility of the materials of construction with the dosage form (including sorption to container and leaching), safety of materials of construction, and performance (such as reproducibility of the dose delivery from the device when presented as part of the drug product). 11.2.5 Microbiological Attributes. Where appropriate, the microbiological attributes of the dosage form should be discussed, as well as the selection and effectiveness of preservative systems in products that contain antimicrobial preservatives. If applicable, then include the rationale for not performing microbial limits testing or not including preservatives in, for example, multiple-use liquid products. 11.2.6 Compatibility. The compatibility of the drug product with reconstitution diluents or dosage devices (e.g., precipitation of drug substance in solution, sorption on injection vessels, and stability) should be addressed.

10

11.3

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

ManufactureP1

11.3.1 Manufacturer(s)P1 . The name, address, and responsibility of each manufacturer, including contractors, and each proposed production site or facility involved in manufacturing and testing should be provided, as well as the Central File Number (CFN)/ Firm Establishment Identifier (FEI) identifying number. 11.3.2 Batch FormulaP1 . A batch formula should be provided that includes a list of all components of the dosage form to be used in the manufacturing process, their amounts on a per batch basis, including overages, and a reference to their quality standards. 11.3.3 Description of Manufacturing Process and Process ControlsP1 . A diagrammatic presentation and a brief written description of the manufacturing process should be submitted, including sterilization process for sterile products. Flow diagrams are suggested as the usual, most effective, presentations of this information. If known, then critical steps could be indicated. 11.3.4 Controls of Critical Steps and Intermediates. If critical steps are identified in P.3.3, then corresponding controls should be briefly identified. 11.3.5 Process Validation and/or Evaluation. Although process validation is not expected for drug product manufactured during the IND stages of development, information appropriate to the development stage should be available to demonstrate that each process step accomplishes its intended purpose. 11.4

Control of ExcipientsP1

Generally, a reference to the current USP is adequate in this section. For atypical use of compendial excipients (e.g., lactose in a dry-powder inhaler), control of other attributes may be appropriate. Quality standards for noncompendial excipients should be described briefly. If subsections of P.4 are not needed, then the corresponding headings can be deleted.

11.4.1 Specifications. This section need only be included if appropriate, as described above. 11.4.2 Analytical Procedures. This section need only be included if appropriate, as described above. 11.4.3 Validation of Analytical Procedures. This section need only be included if appropriate, as described above. 11.4.4 Justification of Specifications. This section need only be included if appropriate, as described above. 11.4.5 Excipients of Human or Animal OriginP1 . For excipients of human or animal origin, information should be provided regarding adventitious agents (e.g., sources, specifications; description of the testing performed; and viral safety data). 11.4.6 Novel ExcipientsP1 . For excipients used for the first time in a drug product or by a new route of administration, details of manufacture, characterization, and controls, with cross references to support safety data (nonclinical and/or clinical), should be provided directly in the IND or by reference to a DMF, another IND, or a NDA/BLA. 11.5 Control of Drug ProductP1 11.5.1 Specification(s)P1 . The ICH Q6A defines specification as ‘‘ . . . a list of tests, references to analytical procedures, and appropriate acceptance criteria which are numerical limits, ranges, or other criteria for the tests described.’’ A tentative specification should include the ‘‘Universal Tests / Criteria’’ defined in Q6A as well as the appropriate ‘‘Specific Tests / Criteria.’’ An individual test and acceptance criteria should be in place for degradation products above an identification threshold defined by the sponsor. The specification should also include an acceptance criterion for total impurities. 11.5.2 Analytical ProceduresP1 . The analytical procedures used for testing the drug product should be described briefly.

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

11.5.3 Validation of Analytical Procedures. Validation data ordinarily need not be submitted at the initial stage of drug development. However, sponsors should have data in hand to demonstrate that the procedures are suitable for their intended use. For some well-characterized, therapeutic biotechnology-derived products, method qualification (37) data may be needed to ensure safety in Phase 1.

11

of the components in the container closure system should be provided. Additional information may be requested for atypical delivery systems such as metered dose inhalers and disposable injection devices. Include, if known, whether the components in direct contact with the drug substance comply with the indirect food additive regulations in 21 CFR 174.5. 11.8

StabilityP1

P1

11.5.4 Batch Analyses . A description of batches, their use (including study numbers), and results of batch analyses should be provided. The batch proposed for clinical use should be clearly identified. 11.5.5 Characterization of Impurities. Information on the characterization of impurities should be provided, if not previously provided in ‘‘S.3.2 Impurities.’’ 11.5.6 Justification of Specification. Justification for the proposed interim drug product specification should be provided. It is understood that the specification will evolve during the drug’s development. For degradants, although ICH Q3B does not apply to drug products for investigational studies, it is advisable to follow its general principles in that thresholds should be established for reporting, identification, and qualification. As with Q3B, acceptance criteria should be set for individual degradants that are present above the identification threshold. A general acceptance criterion for unspecified impurities should be set at less than or equal to the identification threshold.

(See also stability analysis and stability study design chapters.) 11.8.1 Stability Summary and ConclusionP1 . Summarize the data provided in section 8.3. Note that although expiry dating of IND products is not required, it is necessary to obtain sufficient data to support the stability of product for the duration of the clinical trial. However, where new drug products for investigational use are to be reconstituted at the time of dispensing, their labeling must bear expiration information for the reconstituted drug product. 11.8.2 Post-approval Stability Protocol and Stability Commitment. This section is not appropriate for an IND. 11.8.3 Stability DataP1 . For Phase 1, provide available stability data in tabular format. For Phase 2 and 3, provide data on clinical product used in Phase 1 and 2, respectively, in tabular format. 12 MEETINGS AND OTHER COMMUNICATIONS

11.6 Reference Standards or MaterialsP1 Some information should be provided on reference standards. For biological and biotechnological products, if the working reference standard is based on a national or international standard (e.g., World Health Organization), then information should be provided on that reference standard. 11.7 Container Closure SystemP1 The container closure system is defined as all packaging components that together contain and protect the product. A brief description

Both CFR sections (38) and FDA guidance (39) relate to meetings. Firms have generally been encouraged to hold CMC-specific meetings at the end–of-Phase 2 (EOP2). However, for pre-IND questions, a preference is expressed for written questions, which will receive written answers. Many times questions can be clarified by reference to existing guidance documents. Sometimes a follow-up telephone conference can clear up any unresolved issues. For matters that require extensive discussion, a meeting may be appropriate.

12

13

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

GMPs FOR CLINICAL TRIALS

A direct final rule exempting most Phase 1 products, including biological products, from the GMP regulations (21 CFR 211) was issued on January 17, 2006 (40). A proposed rule with language similar to the direct final rule and a companion draft guidance (41) were issued on the same date. These documents provided FDA’s recommendations on approaches to complying with current good manufacturing practice as required in the 501(a)(2)(B) statute of the Food, Drug, and Cosmetic Act (FD&C Act). Although the final rule exempting Phase 1 products from the GMP regulations was withdrawn on May 2, 2006 (42), the approaches in the proposed rule and the draft guidance can be used until both documents are finalized.

14

OTHER REGIONS

Recently, extensive regulations (43) relating to clinical trials have been put in place in the European Union. One quality-specific aspect is that Directive 2001/83/EC (44) requires that a ‘‘qualified person’’ provide certain oversight functions, such as certifying individual batch release. Another recent European Union guideline (45), EMEA/CHMP/SWP/ 28367/07, entitled ‘‘Guideline on Strategies to Identify and Mitigate Risks for First-inHuman Clinical Trials with Investigational Medicinal Products,’’ includes a section (4.2) on mitigating quality-associated risks in clinical trials. The Japanese Pharmaceutical and Medical Device Agency has information on clinical trials on their English language website (46). The United Kingdom’s MHRA has a web page devoted to clinical trials (47). The web site for the Official Newspaper of the Government of Canada also has a page devoted to clinical trials (48).

REFERENCES 1. Code of Federal Regulations (CFR). Available: http://www.gpoaccess.gov/cfr/index. html.

2. Good Guidance Practices 21 CFR 10.115. Available: http://www.accessdata.fda.gov/ scripts/cdrh/cfdocs/cfCFR/CFRSearch.cfm? fr=10.115. 3. Federal Register Notice on Administrative Practices and Procedures; Good Guidance Practices. Available: http://www.fda. gov/OHRMS/DOCKETS/98fr/091900a.pdf. 4. Laws Enforced by the FDA and Related Statutes. Available: http://www.fda.gov/ opacom/laws/. 5. Federal Food, Drug, and Cosmetic Act. Available: http://www.fda.gov/opacom/laws/ fdcact/fdctoc.htm. 6. Public Health Service Act. Available: http:// www.fda.gov/opacom/laws/phsvcact/phsvcact. htm. 7. FDA Electronic Submissions Gateway. Available: http://www.fda.gov/esg/. 8. CDER Reference Guide. Available: http:// www.fda.gov/cder/directories/reference guide.htm. 9. Pre-IND Consultation Contacts. Available: http://www.fda.gov/cder/ode4/preind/PreINDConsultationList.pdf. 10. FDA’s Pharmaceutical Quality for the 21st Century. Available: http://www.fda.gov/ oc/cgmp/. 11. Pharmaceutical Development: Annex to Q8. Available: http://www.ich.org/cache/compo/ 363-272-1.html#Q8. 12. Office of New Drug Quality Assessment (ONDQA): Presentations. Available: http:// www.fda.gov/cder/Offices/ONDQA/presentations.htm. 13. eCTD Specification and Related Files. Available: http://estri.ich.org/ectd. 14. Memo regarding Docket Notice on eCTDs as the only acceptable electronic submission format. Available: http://www.fda.gov/ ohrms/dockets/dockets/92s0251/92s-0251m000034-vol1.pdf. 15. FDA Electronic Submissions Gateway. Available: http://www.fda.gov/esg/. 16. Guidance for Industry Granularity Document Annex to M4: Organization of the CTD. Available: http://www.fda.gov/cder/guidance/ 7042fnl.htm. 17. INDs: Clinical holds and requests for modification. Available: http://www.accessdata.fda. gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm? fr=312.42. 18. FDA Electronic Submissions Gateway. Available: http://www.fda.gov/esg/.

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

13

19. Guidance for Industry: Content and Format of Investigational New Drug Applications (INDs) for Phase 1 Studies of Drugs, Including Well-Characterized, Therapeutic, Biotechnology-derived Products, Nov 1995. Available: http://www.fda.gov/cder/guidance/ clin2.pdf.

30. Guidance for Industry: Content and Format of Investigational New Drug Applications (INDs) for Phase 1 Studies of Drugs, Including Well-Characterized, Therapeutic, Biotechnology-derived Products, Nov 1995. Available: http://www.fda.gov/cder/guidance/ clin2.pdf.

20. International Conference on Harmonisation. Available: http://www.ich.org.

31. Simplified Molecular Input Line Entry Specification. Available: http://en.wikipedia.org/ wiki/Simplified molecular input line entry specification.

21. Guidance for Industry: INDs for Phase 2 and Phase 3 Studies Chemistry, Manufacturing, and Controls Information. Available: http://www.fda.gov/cder/guidance/3619fnl. pdf. 22. The eCTD Backbone Files Specification for Module 1. Available: http://www.fda.gov/cder/ regulatory/ersr/Module1Spec.pdf. 23. Pharmaceuticals and Medical Devices Agency [Japan]. Available: http://www.pmda.go.jp. In the search box on the home page, enter as a search term ‘‘0527004.’’ This retrieves eCTD related PDF files, including the Module 1 Schema: http://www.pmda.go.jp/ich/m/ m4 ectd toriatsukai 04 5 27.pdf. 24. Telematic Implementation Group for Electronic Submission and ICH Implementation. Available: http://esubmission.emea. eu.int/tiges. 25. a) IND content and format 21 CFR 312.23(a)(7)(iv)(e). Available: http://www. accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/ CFRSearch.cfm?fr=312.23. b) Guidance for Industry: Environmental Assessment of Human Drug and Biologics Applications, July 1998. Available: http://www.fda.gov/cder/ guidance/1730fnl.pdf. 26. IND content and format 21 CFR 312.23(a)(7)(iv)(d). Available: http://www. accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/ CFRSearch.cfm?fr=312.23. 27. CGMPs, Packaging and Labeling Control, Expiration Dating. Available: http:// www.accessdata.fda.gov/scripts/cdrh/cfdocs/ cfCFR/CFRSearch.cfm?fr=211.137.

32. Connation Table File (MDL MOLFILE). Available: http://en.wikipedia.org/wiki/MDL Molfile. 33. International Chemical Identifier. Available: http://en.wikipedia.org/wiki/International Chemical Identifier. The IUPAC International Chemical Identifier. Available: http:// iupac.org/inchi/index.html. 34. International Conference on Harmonisation; Guidance on Q6A Specifications: Test Procedures and Acceptance Criteria for New Drug Substances and New Drug Products: Chemical Substances. Available: http://www.fda.gov/ OHRMS/DOCKETS/98fr/122900d.pdf. 35. Guidance for Industry Q6B Specifications: Test Procedures and Acceptance Criteria for Biotechnological/Biological Products. Available: http://www.fda.gov/cder/guidance/ Q6Bfnl.PDF. 36. Guidance for Industry Q8 Pharmaceutical Development. Available: http://www.fda.gov /cder/guidance/6746fnl.pdf. 37. 37. N. Ritter, S. J. Advant, J. Hennessey, H. Simmerman, J. McEntire, A. Mire-Sluis, and C. Joneckis, What is test method qualification? Bioprocess Intern. 2004; 2(8): 32—46. Available:http://www.bioprocessintl.com/ default.asp? page=articles&issue=9%2F1%2F2004 (site requires registration). 38. 21 CFR 312.47 Investigational New Drug Application: Meetings. Available: http://www. accessdata.fda.gov/scripts/cdrh/cfdocs/cfCFR/ CFRSearch.cfm?fr=312.47.

28. a) Guidance for Industry: M4Q: The CTD—Quality. Available: http://www.fda. gov/cder/guidance/4539Q.PDF. b) Guidance for Industry: M4: The CTD—Quality Questions and Answers/ Location Issues. Available: http://www.fda.gov/cder/guidance/5951fnl. pdf.

39. a) Guidance for Industry: IND Meetings for Human Drugs and Biologics Chemistry, Manufacturing, and Controls Information, May 2001. Available: http://www.fda.gov/ cder/guidance/3683fnl.pdf. b) Guidance for Industry: Formal Meetings With Sponsors and Applicants for PDUFA Products. Available: http://www.fda.gov/cder/guidance/2125fnl. pdf.

29. In Module 3, the information that should be included to support a Phase 1 IND submission is noted by a superscript P1.

40. Current Good Manufacturing Practice Regulation and Investigational New Drugs. Federal Register. 2006; 71(10): 2458–2462.

14

CHEMISTRY, MANUFACTURING AND CONTROLS (CMC)

41. Guidance for Industry: INDs—Approaches to Complying with CGMP During Phase 1. Available: http://www.fda.gov/cder/guidance/ 6164dft.pdf. 42. 42. Current Good Manufacturing Practice Regulation and Investigational New Drugs; Withdrawal. Federal Register. 2006; 71(84): 25747. 43. EUDRALEX Volume 10—Clinical trials: Medicinal Products for human use in clinical trials (investigational medicinal products). Available: http://ec.europa.eu/enterprise/pharmaceuticals/eudralex/homev10.htm. 44. Directive 2001/83/EC of the European Parliament and of the Council of 6 November 2001 on the Community Code Relating to Medicinal Products for Human Use. Available: http://ec.europa.eu/enterprise/pharmaceuticals/eudralex/vol-1/consol 2004/human code. pdf. 45. Guideline on Strategies to Identify and Mitigate Risks for First-in-Human Clinical Trials with Investigational Medicinal Products. Available: http://www.emea.europa. eu/pdfs/human/swp/2836707enfin.pdf. 46. Pharmaceutical and Medical Device Agency: Clinical Trial Related Operations. Available: http://www.pmda.go.jp/english/clinical.html. 47. MHRA: Clinical Trials for Medicinal Products. Available: http://www.mhra.gov.uk/home/ idcplg?IdcService=SS GET PAGE&nodeId =101. 48. Canada Gazette: Regulations Amending the Food and Drug Regulations (1024–Clinical Trials). Available: http://canadagazette.gc.ca/ partII/2001/20010620/html/sor203-e.html.

CITIZEN PETITION Anyone may request or petition the U.S. Food and Drug Administration (FDA) to change or create an Agency policy or regulation under 21 Code of Federal Regulations (CFR) Part 10.30. Requests should be directed to FDA’s Dockets Management Branch. When submitting a petition, keep these points in mind: • Clearly state what problem you think

the Agency needs to address. • Propose specifically what the Agency’s

action should be. Your proposal should be based on sound, supportable facts. • Submit the petition, an original, and three (3) copies, unless otherwise stipulated in the Federal Register announcement, to: Food and Drug Administration Dockets Management Branch Room 1-23 12420 Parklawn Drive Rockville, MD 20857 827-6860 The FDA carefully considers every petition and must respond within 180 days by either approving or denying it (in whole or in part), or by providing a tentative response indicating why the FDA has been unable to reach a decision. All petitions will be subject to public examination and copying as governed by the rules in 21 CFR 10.20(j). If the FDA approves the petition, it may be published in the Federal Register. Any petition could eventually be incorporated into Agency policy.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/ora/fed state/Small business/ sb guide/petit.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

CLINICAL DATA COORDINATION

converting it into a high-quality database ‘‘. . .enabling the data to be efficiently analysed and displayed’’ (1, p. 69). The processes outlined are assumed for a regulatory study (i.e., a study intended for submission to a regulatory agency for approval of a drug or device) using a paper-based case report form. Data management activities are divided into the following phases:

CHERYL KIOUS 1

INTRODUCTION

The goal of clinical research trials is to generate data that proves the safety and effectiveness of a drug or device, leading to marketing approval by a regulatory authority. The acquisition, validation, and integration of this clinical data, known as clinical data management (CDM), are an integral component of the drug development process. CDM is the process of preparing the clinical data for statistical analysis; the conclusions from this analysis form the basis of a regulatory submission for approval and subsequent marketing. The discipline of CDM includes case report form design, clinical study database design and programming, data entry, data validation, coding, quality control, database finalization, and data archiving. CDM begins in the study design phase and ends when the clinical database is finalized and the data are considered ready for statistical analysis. CDM processes are greatly enhanced through the development of a comprehensive data management plan, adherence to good CDM practices, and by ensuring that quality steps are built into all stages of data handling. CDM is a comprehensive series of activities performed by a diverse team of people, ranging from data entry operators, database programmers, and clinical data coordinators to coding specialists. The core of the CDM team is the lead data manager (LDM), who serves as the lead clinical data coordinator and primary CDM contact for a clinical research study. The LDM is responsible for developing the data management plan, providing direction and leadership to the data management team, coordinating the data management activities, and ensuring that a high-quality database is provided for subsequent statistical data analysis. This article provides an overview of the role of the LDM, including best practices and considerations in the CDM process. The CDM process is responsible for taking the raw data provided by investigative sites and

• study initiation • study conduct • study closure

Best practices for the LDM to follow will be outlined for each data management phase. 2

STUDY INITIATION

Careful planning is one of the most important activities in the study initiation phase. ‘‘CDM need to ensure that they have the specific procedures and systems in place to handle the data when they arrive’’ (1, p. 73). It is important to set up good lines of communication within CDM and with other functional groups to allow for optimal data handling. Even with good planning, unexpected things happen during the course of the study. ‘‘If we plan for what is expected it will give us more time to devote to the unexpected and unplanned events’’ (1, p. 86). Good planning processes can facilitate the collection, processing, and reporting of high quality data in a timely manner (1, p. 87). The following activities occur in the study initiation phase: • case report form design • development of the data management

plan • design study database and program edit

checks 2.1 Case Report Form Design A well-designed case report form (CRF) can facilitate accurate data collection, analysis, and reporting. ‘‘Traditionally CRFs have been very extensive, capturing many thousands of

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

CLINICAL DATA COORDINATION

data points of which only 50-60% may actually be used. Quantity versus quality needs to be reconciled and only essential data collected and computerized’’ (1, p. 82). Data are collected for clinical studies for two primary purposes: (1) ‘‘To provide the answer to the question posed by the protocol’’ and (2) ‘‘to provide the required safety data regarding the drug being studied’’ (1, p. 134). A CRF should be designed to accomplish three basic purposes: the collection of raw data, ease in monitoring or auditing, and to facilitate processing of the data. It is challenging to optimally meet all of these purposes; compromises may need to be made. In that case, the investigative site’s perspective should take precedence over other groups’ needs (2). For detailed information and example CRFs, see Reference 2. The principles outlined below should be incorporated during the CRF development phase. 1. Document the procedures used for the creation, approval, and version control of the CRF. 2. Design the CRF to collect only the key safety and efficacy data specified in the protocol. The CRF can be drafted early in the protocol development phase because most of the data to be collected are known before the protocol is finalized (2). The CRF design process can help ‘‘. . .identify attributes of a protocol that are inappropriate or infeasible, requiring additional protocol development’’ (2, p. 4). The statistical analysis plan (SAP) can provide guidance on what data should be collected. If the SAP is not available during the CRF design phase, the statistical methods section of the protocol is a good reference to identify what data to include when developing the CRF. 3. Use standard forms whenever possible to facilitate integration of data across studies, such as the standards defined by the Clinical Data Interchange Standards Consortium (CDISC) Operational Data Model (ODM). Database setup can be achieved more efficiently and reliably with the use of standard CRF modules (1).

4. Obtain cross-functional input on the draft CRF. Representatives from clinical, biostatistics, and CDM programming departments should review the CRF to ensure it is adequate from their respective group’s view. Depending on the type of study and data being collected, other reviewers may need to be included, such as investigators and subject matter consultant experts. 5. If using a validated data collection instrument, such as rating scales or questionnaires, it should not be modified without consulting the author to ensure the validity of the tool is not compromised by the change (3). 6. Avoid collecting duplicate data whenever possible. If redundancy is needed to validate data, ensure the duplicate data are obtained through independent measurements (3). In some cases, some data may need to be collected to confirm other data, but ‘‘. . .it is essential that no one piece of data is asked for twice, either directly or indirectly’’ (1). 7. Include the investigator’s signature in the CRF to document the investigator’s review and approval of the recorded data. 8. Finalize the CRF in conjunction with the finalization of the protocol, ensuring each reviewer’s feedback is incorporated during each review cycle. Document approval of key reviewers on a CRF approval form, noting the version information on the approval form. 9. Ensure CRF completion instructions and training are provided to site personnel. All efforts should be made to ensure the CRF is available to the site before the first subject is enrolled in the study.

2.2 Development of the Data Management Plan A Data Management Plan (DMP) is a comprehensive document developed by the LDM to describe the data handling and data cleaning processes to be used during the study. The content for each section of the DMP is

CLINICAL DATA COORDINATION

suggested below or described in the corresponding section that follows. Each component of the DMP should have input, review, and approval from appropriate data management and cross-functional team members. The DMP should be created at the beginning of the study and updated to reflect any process decisions or changes as needed throughout the study. Each document in the DMP should, at a minimum, contain the following: • document title • version number or date • protocol identification • company identification (and sponsor

name, if study managed by a contract research organization) • author, reviewer, and authorizer names • distribution list • change history section

The following major components are recommended for the DMP: • data

management personnel: this section includes a list of key CDM team members, such as LDM, Database Programmer, Coding Lead, Data Entry Lead

• data management status reports: this

section includes samples of the status reports to be provided, the frequency, and distribution list • database design (see section below) • data tracking and entry (see section

below) • data validation (see section below) • coding (see section below) • external data processing (see section

below) • quality control (see section below) • serious adverse event (SAE) reconcilia-

tion (see section below) • data transfer: if applicable, this section

includes specifications for the transfer of data, such as from a CRO to the sponsor • data handling report (see section below)

3

• standard operating procedures (SOPs):

this section includes a list of applicable SOPs, including the dates they became effective for the study The relevant section of the DMP should be finalized before the corresponding activities begin. For example, data entry should not begin until the Data Tracking and Entry sections of the DMP are finalized. The foundation of the DMP is based on SOPs. SOPs identify what procedures are needed and why, when they occur, how procedures are implemented, and the roles or positions responsible for each procedure (1). The SOPs provide the core framework on which study-specific processes are documented in detail in the DMP. 2.3 Design Study Database and Program Edit Checks The focus of this section is the design and validation of a project-specific clinical database using a validated clinical data management system (CDMS). It does not address the requirements for software applications development and validation or installation and functional-level testing of a packaged CDM application. For further information, see References 4 and 5, which document on computerized systems in clinical trials and provide information on how ‘‘. . .expectations and regulatory requirements regarding data quality might be satisfied where computerized systems are being used to create, modify, maintain, archive, retrieve, or transmit clinical data’’ (5). Ensure that adequate procedures as specified in an SOP are in place to control user accounts and access within the CDMS. Granting and revoking of study access for each user should be documented in the data management study file. Access should be revoked for users who no longer need access to the study (e.g., not working on study, resignation from company). In the design of a project-specific clinical database, the database programmer should follow good programming practices, such as use of a header block, completion of informal development testing, and use of sound programming code. Whenever possible, use

4

CLINICAL DATA COORDINATION

standard data structures, such as CDISC ODM (6). Input from clinical and biostatistics are important during the database design phase ‘‘. . .as the more thought at the start about what is required at the end will result in a study that can be processed efficiently and effectively. It must be remembered that the computer is a tool and that the output from it is only as good as the ‘human’ instructions it receives’’ (1). A database should not be released into production until the study protocol and CRF are finalized. Database design activities consist of the following components: 1. Computer system validation (CSV) plan: The database programmer should create a clinical study database CSV plan to document the testing strategy for the database structure, data entry screens, edit checks, external data imports, and data transfers. The CSV plan outlines the scope of testing, methodology to be used, format for the test data, and documentation and resolution of errors in the testing process. 2. Database annotated CRF: Upon finalization of the CRF, the database programmer annotates the CRF with data file names, item names, code lists, derived items, and other relevant information for each data variable, including version information. 3. Database structure and data entry screens: Using the database annotated CRF, the programmer creates the database metadata and data entry screens. 4. Coding: The variables to be coded are set up as part of the database structure and entry screens and the programs used for auto-encoding are mapped to the appropriate dictionaries. The version of dictionaries used for coding should be referenced in the metadata. 5. Edit checks: Edit checks are programmed according to the edit check specifications document. Edit checks may consist of programmed edit checks that periodically run on the

data or programmed listings that report discrepant data based on specified criteria. 6. External data: Programs used to electronically load non-CRF data into the CDMS or system table (such as Oracle) should be tested to ensure that the import process does not affect the integrity of the data. 7. Data transfer: Programs used to export the data from the CDMS should be tested to assure the transfer process does not affect the integrity of the data being transferred. 8. Formal testing of the database may be done by the programmer or the LDM. Hypothetical or dummy subject data is entered to validate the database structure, data entry screens, coding setup, and edit specifications. All errors identified through the testing process, the action taken to correct the errors, and each correction retested to validate modifications should be documented. Once the database has undergone all testing and is considered ready for use, document the release of the database and edit checks to a production environment. Minimum testing requirements consist of: A. Database structure: verify each field on the CRF is present; each data variable’s type, name, length, ranges, and code lists are accurate; derivations calculate and derive to the targeted item correctly; confirm setup of auto encoding and the appropriate dictionary term populates correctly; and the flow of data entry screens are compatible with the CRF. B. Edit checks: both valid (clean) and invalid (unclean) should be entered to verify each check achieves the expected result and performs according to the edit specifications document. C. External data: completing a test import for each data source verifying the number of records matched the raw data file and all data fields mapped correctly to

CLINICAL DATA COORDINATION

the appropriate database variable in the CDMS or designated table (such as Oracle). If any modifications to the file specifications or programs used to load the dataoccur, the changes must be retested and documented. D. Data transfers: completing a test transfer verifying the number of records exported and that all data, data labels, code lists, formats, and calculated variables are exported as intended.

3.

4. 5.

6. 3

STUDY CONDUCT

The following activities occur in the study conduct phase: • data management documents tracking

and data entry • data validation • coding • serious adverse event (SAE) reconcilia-

7.

tion • external data processing • quality control

3.1 Data Management Documents Tracking and Data Entry The CDM documents that are expected to be received and the process and flow for tracking, entry, and filing should be defined at the start of the study and documented in the DMP. CDM documents may consist of CRFs, Data Clarification Forms (DCFs), sitebased DCFs (SDCFs), and source documents (for example, laboratory reports, ECGs, and subject diaries). These documents should be stored in a secure and controlled environment. It is recommended that the following information be documented: 1. Documents that will be received as a hard copy, fax, or other electronic method, such as PDF. 2. Documents that will be received as originals or copies. Define any hard copies of reports that will be received

8.

9.

5

that correspond with any electronically transferred data, such as laboratory reports. Identify whether any reconciliation of electronically transferred data with hard copy reports will occur. The process and systems to be used for any documents that will be tracked (manually or electronically). The filing organization of documents, within a paper file or electronically. The process for processing CRFs received in a language different than expected. The mechanism that will be used to ensure all expected CRFs and other data entry documents are received and entered in the CDMS. A program written within the CDMS to track pages once they are entered is an efficient and accurate method. Such a program can also provide reports for monitors of expected pages that are missing. The mechanism used to verify unscheduled pages have been received and entered, such as unscheduled visits, repeating pages of log-type forms such as concomitant medications and subject diaries. Any manual review that will be completed before data entry, such as verification for complete and correct subject identifiers, measurement of scales. Describe any data conventions that can be applied before entry. The method of data entry, such as whether it is interactive double data entry, independent double data entry with third party reconciliation, or single entry with 100% manual verification. ‘‘The great debate on whether single or double entry should be used to enter clinical data is still unresolved and it should be questioned if there is a right or wrong answer’’ (1). FDA and ICH regulations do not mandate a specific data entry method. ‘‘All clinical trial information should be recorded, handled, and stored in a way that allows its accurate reporting, interpretation, and verification’’

6

CLINICAL DATA COORDINATION

(7, p. 25695). The method can be evaluated based on factors such as the skill level of the entry operators, amount of time to complete the entry process, and the percent of acceptable errors in the data. 10. General and page-specific data entry guidelines to document handling of data, including date formats, subject initials, blank pages, missing pages, missing data fields, text fields, values recorded as not done or unknown, extraneous comments or marginalia, abbreviations, and illegible fields.

3.2 Data Validation Data validation or data cleaning are the processes used to validate the accuracy of data (3). Data validation can identify errors made while entering data from CRFs into the database and transcription errors made by the site when transcribing from source documents to the CRFs. These errors are identified through checks of the data that report inconsistencies within a subject’s data, medically invalid values, data values out of predefined ranges or formats, incorrect protocol procedures, spelling and legibility problems, and missing data. This check can be achieved through manual review of data listings and programmed edit checks that identify discrepant data. ‘‘The ultimate purpose of validation is to ensure that data meet a level of quality such that the inferences and conclusions drawn from any analyses may be correct’’ (1). Comprehensive data validation guidelines should be developed to check CRF and any other external data related to the study. ‘‘When data management locks a database that is still dirty, the data problems are passed off to the statisticians and programmers. The data problems do not go away. In fact, they become more expensive to fix’’ (8) Comprehensive data validation guidelines (DVGs) should be created consistent with the protocol requirements, CRF, and the statististical analysis plan (or key safety and efficacy data). Cross-functional input from clinical and biostatistics on the DVGs is important and will assure a sound data cleaning plan.

All data changes should be documented at the data management center and the investigative site. The DVGs will generally include the following components: 1. DCF process and flow: A. Frequency for running consistency checks and whether they will be run automatically or manually B. Whether DCF updates will be made to the CRF C. Timelines for DCF generation and receipt from investigative sites D. Method of DCF distribution to investigative sites E. Definitions of discrepancy and DCF statuses and audit trail reasons for change and their use F. Process for handling unsolicited comments on CRFs, for example, review by clinical or medical team members or issue a DCF to the investigative site G. Process for handling any duplicate DCFs 2. Edit specifications: A. Edit specifications can encompass pre-entry review, programmed consistency checks, and manual listings review. The edit specifications should be sufficiently detailed that the database programmer can use this document to program consistency checks and data listings. Examples include: a. Checks for sequential dates b. Checks for missing or required data c. Checks to assure data values are consistent with the data expected, such as character, numeric, within a predetermined range d. Checks across different types of data (cross-panel or crossmodule) e. Checks for protocol violations f. Checks to identify outliers, such as medically implausible

CLINICAL DATA COORDINATION

data, outside expected ranges for the study population B. Standardized DCF message text should be developed for both programmed consistency checks and manual queries issued from manual review of data listings. This process can increase consistency and efficiency during processing, especially when multiple CDCs are working on the study. 3. Data conventions A. Data cleaning conventions may be created for changes that can be made without the investigator’s approval on each individual change. These should be determined at the beginning of the study, before data cleaning activities begin, and only used for obvious errors. Data cleaning conventions should be made by trained data management personnel. B. Data conventions should not be applied to key safety or efficacy variables. C. The investigator should receive and acknowledge at least the first and final versions of data cleaning conventions. D. If certain protocol tests are being performed locally by the site and use normal reference ranges, such as a local laboratory, determine whether local reference ranges will be entered in the database and referenced in reporting or whether laboratory reference ranges will be normalized across investigative sites or studies.

3.3 Coding Clinical data, such as diseases, diagnoses, and drugs that are collected on CRFs, are coded using standard dictionaries to facilitate the analysis and reporting of the data. Coding of data provides enhanced data retrieval, flexible reporting capabilities, the ability to group consistent terms, and the capability

7

for different levels of hierarchical or crossreference reporting (1). ‘‘The aim of processing clinical trials data and safety information is to order and structure the data to allow effective reporting and retrieval of information for regulatory, safety, marketing and planning usage’’ (1). Data are typically coded using an auto-encoding system. Auto-encoders vary in complexity from exact verbatim text to dictionary term-matching to use of a thesaurus to manage synonyms that map to a dictionary term. When selecting an auto-encoder, capabilities to consider include the ability to manage synonyms, misspellings, mixed case, word variations, word order, and irrelevant verbatim information (3). Auto-encoders must be validated and comply with FDA regulations on electronic records (4). The International Conference on Harmonization (ICH) Medical Dictionary for Regulatory Activities (MedDRA) Terminology is recommended for coding clinical trial adverse event data. For guidance regarding the use of MedDRA and version updates, see the Maintenance and Support Services Organization (MSSO) website for publications entitled MedDRA Implementation and Versioning for Clinical Trials and MedDRA Term Selection: Points to Consider, Release 3.4 (http://www.meddramsso.com) (9, 10). Medications are coded to facilitate the analysis and reporting of possible drug interactions (3). The WHO Drug Dictionary is commonly used for coding medications (see Uppsala Monitoring Centre) (11). If adverse events and serious adverse events are maintained in two separate databases, ensure that the same dictionaries and versions are used, if possible, and that consistent coding guidelines are used to code serious adverse events. All versions of dictionaries used should remain available for reference. If changes or additions are needed to publish dictionaries, the organizations that maintain the dictionaries have procedures for submitting change requests (3). ‘‘All levels, codes or group assignments for the data should be stored’’ (3). This storing ensures all information is available if recoding is needed or if hierarchy or multi-axial information is needed to fully understand a particular coded term.

8

CLINICAL DATA COORDINATION

The following points should be considered for coding data and detailed coding guidelines should be developed for each study: 1. Identify the variables to be coded, the dictionaries, and the version used. 2. Define the process for terms that do not auto-encode, including misspellings, abbreviations, and irrelevant verbatim information. If coding guidelines allow modifications to be made to facilitate auto-encoding, the modifications should only be made to the corresponding modifiable term and the original verbatim term should remain unchanged in the database. Changes to the original verbatim term should only be made based on a DCF confirmed by the investigator. 3. Develop guidelines for how terms containing multiple concepts will be handled, whether they can be split per coding guidelines or issuing a DCF to investigator. In general, all adverse events reported as multiple events should be issued on a DCF to the investigator for clarification. 4. Any terms that cannot be coded should be documented in a data handling report. All serious adverse events should be coded. 5. Determine how dictionary updates that occur during the study will be managed. If updates will be applied, identify the frequency of planned updates and how updates will be applied (e.g., updates only or reload dictionary). 6. Define the process and frequency for review of coded terms for missing codes, consistency, and medical or clinical judgment. Define reports that will be used for coding review. For further information on dictionary management, the following are good resources. 1. Good Clinical Data Management Practices, Version 3 (3) 2. MedDRA maintenance and the Maintenance Support Services

Organization (MSSO), http://www. meddramsso.com (9, 10) 3. Uppsala Monitoring Centre, http:// www.who-umc.org (11)

3.4 Serious Adverse Event Reconciliation Serious adverse event (SAE) reconciliation is performed for studies in which SAE data and study adverse event data are maintained in two separate databases. SAE data are typically reported to a drug safety group responsible for maintaining the regulatory safety reporting database. Adverse events (AEs) are reported along with CRF data and entered into the clinical database. As some events are entered into both databases, a comparison of the databases for key safety data variables is performed to ensure consistency. When designing the AE and SAE forms, standardize the capture of SAE data variables for consistency in reporting and to facilitate reconciliation. Develop SAE reconciliation guidelines to include: 1. The systems and locations of the safety database and the clinical database. 2. The method for providing safety data between the two groups (i.e., paper or electronic). If electronic, the medium to be used and format of the data, such as ASCII or SAS. 3. The mechanism for comparing data (i.e., manually or programmatically). 4. The time intervals for when comparisons will be performed, including details for comparisons prior to interim analyses or safety data reporting. 5. The coding dictionaries and versions used to code diagnoses among the two databases. If using different dictionaries or a different version of the same coding dictionary, identify how differences in coded terms for the same event will be resolved. 6. Determine cut-off dates, such as database lock, after which no new SAEs will be added to the clinical database.

CLINICAL DATA COORDINATION

7. Identify the variables that will be reconciled and which items must be an exact match and which may be similar. The following items are examples of items that might be reconciled if they are present in both the safety and clinical databases: A. protocol B. investigator C. subject identification (e.g., subject number, randomization number, initials, date of birth, gender, race) D. diagnosis (i.e., verbatim or coded term) E. severity F. onset date G. date of death H. resolution date I. causality assessment J. action taken with drug K. outcome 8. Reconciliation can occur on events where all data to be reconciled have been entered and validated, no outstanding DCFs exist, and the event has been coded. It is recommended that discrepancies be documented consistently, such as on an SAE reconciliation form. 9. Determine the process for documenting and resolving discrepancies. Outline the communication process with the drug safety group and expected turn-around time for resolving discrepancies. Data clarification forms should be issued to the investigator to resolve discrepancies. The drug safety group should be notified as soon as possible of any events that are in the clinical database but not in the safety database. It is possible for events to exist in the safety database but not in the clinical database, reflecting CRFs that have not been retrieved and entered. 10. It is recommended to have written documentation from the drug safety lead and the LDM once all data have been received and reconciled. Any discrepancies that cannot be resolved should be documented in the DMP.

9

3.5 External Data Processing Use of a centralized vendor for processing and reporting of select data is very common in multicenter trials. Centralized vendors are used to standardize the methods of testing and reporting, using consistent reference values for all data collected across study sites. Many sponsors use one vendor for protocol procedures or tests such as automated randomization, clinical laboratory, ECG interpretation, pharmacokinetic data, and subject diaries. The results are created, entered, or processed and quality control checked by the vendor and typically sent as electronic file (or files) to the sponsor or CRO. The sponsor or CRO should ensure the vendor’s personnel, systems, and procedures are adequate and meet regulatory requirements. For further information, see Good Clinical Data Management Practices, Version 3, section Vendor Management (3). Once the vendor has been identified and qualified, it is important to work with a contact person at the vendor to develop transfer specifications, which are then documented. The transfer specifications and detailed guidelines for each type of external data should be developed as part of the DMP and include information, such as: 1. Contact names and phone numbers for both the sponsor or CRO and vendor. 2. Format of data to be received, such as ASCII or SAS, and version identification. 3. Method of data transfer, for example, disk, CD, data tape, Internet. If data are to be transferred through the Internet, ensure appropriate encryption mechanisms are used. 4. Schedule for data transfers. 5. Whether transfers will be incremental or cumulative. 6. Testing strategy, including completing a test transfer to validate the integrity of the data following the import process (see Design Study Database section). 7. Process for loading data, the programs used to manipulate data, if applicable, and loading logs.

10

CLINICAL DATA COORDINATION

8. Location of loaded data, such as CDMS, Oracle table, or SAS datasets. 9. Detailed file structure, including variable names, labels, attributes (size, format), code lists, mapping of external data variables to clinical database structure and position within file, required variables, how missing variables will be identified. 10. Details of normal ranges and process for conversion to different units, if applicable. The vendor should provide a list of reference ranges and effective dates. Ensure reference ranges are up-to-date throughout the study and prior to database lock. Identify how any reference range changes will be handled for reporting purposes. 11. Process for handling data corrections and additions. If an audit trail is not available for corrections at the data management site, the vendor should make all data changes and provide a corrected data file. 12. Process for handling repeat tests, duplicates, and partial records. 13. Process for handling special characters, such as symbols and quotation marks. 14. Mechanism to identify retransmission of corrected data. 15. Communication process with vendor to acknowledge receipt of data transfers, such as e-mail or hard copy form. 16. If external data contains information that would unblind the CDM team, identify the process for handling the data to ensure blind is maintained. Once transfer specifications have been agreed on with the vendor, the external data import program has been set up and tested, external data transfers may occur per the specifications. The following points should be considered in processing external data: 1. Acknowledge receipt of each data transfer. 2. Communicate any problems encountered with the data file and whether the data file was acceptable.

3. Obtain a list of checks that are performed on the data by the vendor before transfer to ensure CDM does not duplicate checks. 4. Identify in the edit specifications the checks that will be performed on the data following each transfer. Some checks to consider include: A. Verify a minimum number of demographic variables to ensure the data correspond to the intended subject, such as subject number, subject initials, and date of birth. B. Verify for each record in the data transfer file that a corresponding CRF record exists in the clinical database and the intended data variables match, such as visit identifier and date of sample or procedure. C. Verify for each CRF record in the clinical database that a corresponding record exists in the data transfer file and the intended data variables match as above. D. If the CRF and clinical database do not capture unscheduled tests or procedures, develop a process to ensure these records are present in the data transfer file and contain the appropriate data. It is recommended that the CRF capture, at a minimum, the date of each unscheduled test or procedure to ensure all expected data is received and reconciled in the transfer file. 5. To avoid duplication of information, transfer only those variables that are needed to accurately match the subject and associated visit with the CRF data in the clinical database. 6. Retain an electronic version of the data transfer file (i.e., each transfer file if incremental and the final transfer if cumulative). 3.6 Quality Control Quality control in the context of CDM are the procedures undertaken to ensure the

CLINICAL DATA COORDINATION

‘‘. . .data contains no errors, inconsistencies or omissions (that can be rectified) and that the data are coded correctly and transferred accurately from the case record form (CRF) to the computer database’’ (1). The ICH GCP guideline states, ‘‘Quality control should be applied to each stage of data handling to ensure that all data are reliable and have been processed correctly’’ (7, p. 25699). Quality is not a single measurement; quality is measured with individual steps of a total process (1). Quality can be influenced by the experience and judgment used by the CDM personnel reviewing discrepancies, ‘‘. . .while queries are usually issued in line with set procedures which broadly define what should and should not be queried, there will always be some discrepancies where an element of judgment is needed in order to arrive at the decision of whether a query is warranted’’ (1). The level of expertise and appropriate judgment used in determining whether to send a DCF to the investigative site can affect the overall quality of the study data. CDM staff responsible for data validation should receive adequate training and quality control of their work product until a reasonable level of accuracy is attained before working independently. Quality is typically measured by setting a certain standard for an allowable number of errors within a defined number of data values. Errors can be weighted differently according to the value of the data (e.g., key efficacy data may need to be error-free). Good data validation is time-consuming to define, develop, and implement. ‘‘It is important to assess the value of the effort put into validations against the resulting improvement in the data’’ (1). A quality control (QC) plan should be developed at the beginning of the study to document the QC procedures that will be performed during the study. QC reviews should consist of a comparison of the selected data variables and corresponding DCFs, site DCFs, and data conventions to ensure no entry, programming, or database update errors occurred in the data management process. The QC plan may provide for initial, ongoing, and final inspections. Initial inspections can provide immediate feedback in areas where investigative site or monitoring training is needed. Ongoing inspections

11

can be beneficial for studies of long duration or large size to ensure that no systematic processes resulting in errors have occurred. The final inspection is performed to provide an error rate between the CRF and the final database used for analysis. All errors identified in inspections should be documented and corrected in the clinical database. ‘‘Error detection methods that are applied to only a sample of the data can be used to obtain estimates of the distribution of undetected errors’’ (3). The Good Clinical Data Management Practices document suggests that the best practice for determining appropriate QC sample size is by statistical sampling (3, p. 82). CDM error rate standards vary across the industry. Errors should be expressed in the form of an error rate rather than a raw count of the number of errors. Error rates should be calculated by taking the number of errors identified divided by the total number of fields inspected. ‘‘The error rate gives a common scale of measurement for data quality’’ (3). Study-specific details should be documented in a QC plan, covering details that are not specified in the relevant QC SOP. Study-specific details that might be included in the QC plan are:

1. Identify any key safety and efficacy variables that are deemed critical for the analysis to undergo 100% QC and be error-free. These variables can be determined from the SAP (or the statistical methods section of the protocol if the SAP is not final) and in discussion with biostatistics. If the study has a very large volume of key safety and efficacy data, the LDM in conjunction with biostatistics determines an acceptable error rate for key variables while maintaining a statistical confidence interval. 2. If inspections will occur other than on the final, clean data, identify the timing and sampling methodology to be used.

12

4

CLINICAL DATA COORDINATION

STUDY CLOSURE

The following activities occur in the study closure phase: • final quality control procedures • database lock • data archiving

4.1 Final Quality Control Procedures Before database lock and when a specified percentage of the study data are considered clean, such as 90%, a final inspection is performed to estimate the error rate between the CRFs and the clinical database. The final inspection is to ensure data management processes resulted in high-quality data suitable for statistical analysis. 4.2 Database Lock Database lock occurs when all study data are considered clean and ready for statistical analysis. For some studies, databases may be locked for interim analyses to support study continuance for safety or efficacy reasons or for regulatory submissions. The locking of the database in the CDMS is completed to prevent inadvertent or unauthorized changes once analysis and reporting have begun (3). Written procedures should be in place for the locking of the database and removal of write access in the CDMS. Use of a database lock checklist is recommended to ensure all appropriate procedures have been completed. Potential items to include on a database lock checklist are mentioned later in the section. Once the database lock process is underway (such as initiation of signatures on a database lock authorization form), no additional changes should be made to the database. Instances occur when a database may need to be unlocked. A discrepancy may be identified during the statistical analysis and reporting phase that may warrant unlocking the database. A discrepancy should be discussed with project team members, including clinical, biostatistics, and data management, to determine any affect the discrepancy might have on the analysis. If the decision is made to unlock the database to make corrections

or query the investigator for additional information, the unlock process should be documented, including authorization signatures, date, time, and the names of individuals given access to the unlocked database. Once discrepancies are resolved or corrections are made to the database, formal database lock procedures should once again be followed, including completion of a database lock checklist. If the decision is made not to unlock the database, the rationale should be documented in the data management study file, preferably a data handling report or comparable document, and as errata in the clinical study report. In some cases, the discrepancy identified post-lock does not warrant unlocking because the data either do not effect or only minimally effect the statistical results. Before database lock, it is good practice to use a database lock checklist to ensure all items have been completed. The following should be considered in evaluating the database for readiness to lock: 1. All study data have been received or accounted for by data management 2. All data have been entered in the CDMS and all entry discrepancies have been resolved 3. All noncoded text fields and unsolicited comments have been reviewed and appropriate actions taken 4. All electronic external data have been received and reconciled with the clinical database and all discrepancies resolved 5. All data validation has been completed per the Data Management Plan and all queries have been received and updated 6. Coding has been completed and reviewed for consistency and accuracy 7. SAE reconciliation has been completed and all discrepancies have been resolved (applicable if AEs and SAEs are maintained in separate databases) 8. Reference ranges, such as laboratory normal ranges, are available in the database and have been verified to map appropriately to corresponding data values

CLINICAL DATA COORDINATION

9. Inspections specified in the QC plan have been completed and an acceptable error rate was achieved 10. All automatic batch jobs have been canceled 11. All deviations and unresolved data issues have been documented, such as in the Data Handling Report The concept of a Data Handling Report is to document deviations in processes defined in the DMP; deviations to SOPs; unresolved data issues, such as DCFs, missing CRF pages or other study data; terms that could not be coded; and any other information that will be useful for the statistical analysis and clinical study report. Write access to the clinical database should be revoked from all users once the database is locked. 4.3 Data Archiving Environmentally protected, secure, and accessible storage for study data documents is usually limited in most CDM organizations. Routine data archiving is good practice to allow space for ongoing studies and ensure the data are easily accessible by CDM personnel throughout the life of the study. Data archiving also ensures the documents are available for audits by independent auditing groups and regulatory authorities. All CDM organizations should have an SOP for archiving paper and electronic records based on at least the minimum requirements specified as essential documents ‘‘. . .those documents that individually and collectively permit evaluation of the conduct of a trial and the quality of the data produced. . .’’ in ICH Guideline, E6 Good Clinical Practice: Consolidated Guidance (7, p. 25705). In many organizations, paper archiving follows a master filing plan or template for all clinical study documents within which CDM documents are filed. A standard master filing plan ensures documents can be easily retrieved if needed for future reference. Study data documents should be retained according to applicable regulations of the country (or countries) where the drug is approved. According to ICH Guideline, ‘‘. . .essential documents should be retained

13

until at least 2 years after the last approval of a marketing application in an ICH region . . .’’ (7, p. 25700). Archiving consists of both paper and electronic records. Electronic data mediums are changing rapidly. ‘‘Media obsolescence can make orphans of data in the clinical world’’ (12). It is important for clinical trial data to have continued readability once archived. As technology advances, the hardware and software to read data files can quickly become obsolete. One strategy to keep data from becoming obsolete and unreadable is to migrate it to newer versions. This process can be time-consuming and costly to ensure the data is properly migrated and validated. The process of migrating data can have risks, such as losing data in the transfer or unintended transformation (12). Mostly recently, an industry-accepted standard for data archiving is the CDISC Operational Data Model (ODM). For further information, please see http://www.cdisc.org. The following should be considered when archiving CDM documents and electronic records: 1. Ensure electronic data are archived in an open format, such as Extensible Markup Language (XML) used in CDISC ODM, because it can be easily accessed, regardless of system software version used. 2. These items should be archived: A. Database metadata and program code B. Raw data, including CRF data and any externally loaded electronic data C. External data electronic files in original format, retaining each file if incremental data files received, or last file only if cumulative data files were received D. Copy of dictionary used to code data E. Each version of laboratory reference ranges used F. Audit trail G. Final data, including any derived data

14

CLINICAL DATA COORDINATION

H. Original documents, including CRFs, DCFs, DMP, projectspecific computer system validation documentation, and any other relevant study documentation, such as correspondence I. Discrepancy management data for data that failed edit checks J. SOPs used and any documented deviations K. Database closure documentation, such as database lock authorization, confirmation of database lock, and removal of access L. Electronic back-up copies 3. The recommended timing to archive data is subjective, but a standardized approach might be to archive once the clinical study report for a trial is finalized. 4.4 CDM’s Role in Training and Education CDM can play a significant role in the training and continuing education of clinical study site personnel and the clinical research associates who are responsible for monitoring the data. CDM input in all phases of the clinical study can improve the quality and integrity of the data. Examples of CDM input are: 1. CDM participates in the review or development of CRF completion guidelines. These guidelines provide detailed instructions to clinical study site personnel on how to complete each page of the CRF. 2. CDM participates in investigator meetings to present CRF completion guidelines and provide sample completed CRFs and DCFs. CDM provides instructions for correcting errors on CRFs on the original and how to handle errors for CRFs that have already been submitted to CDM. They communicate the flow of the data collection and cleaning process and identify potential errors in consistency across data collection forms. 3. CDM provides ongoing feedback on trends in CRF completion and frequency of data errors at clinical team

meetings or teleconferences, investigative site newsletters, and annual investigator meetings.

5 SUMMARY CDM is a crucial step in the drug development process. Good CDM processes can improve the integrity and quality of data when built on the foundation of a welldesigned clinical study and data collection tool and an adequate clinical monitoring plan. CDM is a complex process in which data from a clinical study are acquired, processed, validated according to predefined rules, and integrated into a comprehensive database for subsequent statistical analysis and reporting. Key to successful data management is a knowledgeable LDM, who applies the principles of good CDM practices in all stages of data handling. In the study initiation phase, it is important to proactively plan and document detailed processes in a Data Management Plan; develop a comprehensive data management timeline to ensure a complete and high-quality database within the required company or CRO timelines; lead the cross-functional team providing input on the design of the data collection tool following industry-accepted standards; and coordinate the design and testing of the database and edit checks before processing production data. In the next phase—study conduct—the LDM should assess the adequacy of the Data Management Plan and adapt processes to meet the requirements of the study; provide early and ongoing feedback to monitors and study sites on the quality of the data; ensure the data validation guidelines comprehensively address both anticipated and unanticipated data scenarios; and modify or add edit checks as needed. All of the planning efforts completed at the beginning of the study and the data validation steps that occur while the study is ongoing culminate in the study closure phase. During this final phase, QC procedures are carried out and the final steps to database lock are completed, including documenting

CLINICAL DATA COORDINATION

any deviations to SOPs or processes outlined in the Data Management Plan. Following database lock, completion of the statistical analysis, and a final clinical study report, the paper and electronic study data may be permanently archived. Well-planned and managed CDM processes that follow industry-accepted standards and practices can result in a timely, high-quality database. The conclusions drawn from high-quality databases can prove the safety and effectiveness of a drug or device, providing the foundation for a regulatory submission and subsequent approval and marketing. REFERENCES 1. R. K. Rondel, S. A. Varley, and C. F. Webb (eds.), Clinical Data Management. London: Wiley, 1993. 2. B. Spilker and J. Schoenfelder, Data Collection Forms in Clinical Trials. New York: Raven Press, 1991. 3. Society for Clinical Data Management, Inc., Good Clinical Data Management Practices (GCDMP), Version 3. Milwaukee, WI: SCDM, September 2003. 4. Food and Drug Administration, 21 CFR Part 11, Electronic Records; Electronic Signatures; Final Rule. Fed. Reg. 1997; 62(54): 13429–13466. 5. Food and Drug Administration, Guidance for Industry: Computerized Systems Used in Clinical Trials. Washington, DC: FDA, April 1999. 6. Clinical Data Interchange Standards Consortium (CDISC). Operational Data Model, Final Version 1.2.1. (online). Available: http://www.cdisc.org/models/odm/v1.2.1/ index.html. 7. International Conference on Harmonization, Good Clinical Practice (ICH GCP): Consolidated Guideline. Fed. Reg. 1997; 62(90). 8. K. L. Monti (2001). A statistician shows how to save time and money through data management. Appl. Clin. Trials (online). Available: http://www.actmagazine.com/ appliedclinicaltrials/article/articleDetail.jsp? id=92010. 9. Maintenance and Support Services Organization (MSSO), MedDRA Term Selection: Points to Consider, Release 3.4, November 18, 2004. (online). Available: http://www.meddramsso.com/NewWeb2003/

15

document library/9530-710%20ptc output% 20doc122204.pdf. 10. Maintenance and Support Services Organization (MSSO), Recommendation for MedDRA Implementation and Versioning for Clinical Trials. (online). Available: http://www.meddramsso.com/NewWeb2003/ Docs/clinicaltrialversioning.pdf. 11. Uppsala Monitoring Centre (2005). (online). Availabe: http://www.who-umc.org. 12. P. Bleicher (2002). Diamonds may be forever, but data? Appl. Clin. Trials (online). Available: http://www.actmagazine.com/ appliedclinicaltrials/article/articleDetail.jsp? id=87252.

CLINICAL DATA MANAGEMENT

data. As will be demonstrated in the articles that follow, the accuracy and completeness of data collected for clinical research is a primary focus of many of the activities surrounding clinical data management, whereas the focus for online retailers, airlines, and banking applications is optimization for transaction processing response and security. Clinical data management, then, encompasses the processes involved in transferring data to a computerized database; applying a variety of techniques to make sure that the data are as accurate and complete as possible; tracking all changes, modifications, additions, and deletions to the data; and providing a mechanism for delivering the data for statistical analysis.

RUTH MCBRIDE Axio Research, Seattle, Washington

Even before computers were used in clinical research, we have been collecting data, organizing it, and analyzing it. Since the advent of computers and specialized data management software, the area of clinical data management has matured to the point that clinical investigators can count on having complete, reliable, and current data for interim and final analyses. Clinical data management, as the name implies, is data management focused on clinical data, but not on all clinical data. Clinical data are collected for a variety of purposes, as part of electronic medical records to manage medical or dental practice, as part of billing information, and in the context that this section of the encyclopedia will discuss, to support clinical research and, in particular, clinical trials. In this context, there are some specialized requirements of a data management system or process. First of all, data collected for clinical trials research is primarily study participantcentric (or patient centric). This implies a hierarchical data structure where most data records are tied to participants. Within participant, data are further organized by ‘‘visit.’’ Contrast this data structure with databases designed to support, for example, banking or airlines. These databases are designed to support many transactions and are more typically relational in structure. Although commercial clinical data management systems are built on relational database servers (e.g., Oracle or SQL/Server), in order to retrieve and display data most efficiently, these systems take advantage of the hierarchical structure to build links and keys to optimize access. Second, the volume of data in clinical data management databases is relatively small compared with other commercial applications. A typical database for a phase III study might contain several thousand megabytes of information. Databases for large online retailers contain many terabytes of

1 HOW HAS CLINICAL DATA MANAGEMENT EVOLVED? 1.1 Coronary Drug Project and Early Trials As early as the 1960s computerized systems were being used for clinical data management in NIH-sponsored clinical trials. The Coronary Drug Project, which began collecting data in 1966 (1), describes data management processes that included computer files, keypunched data, and computer programs to scan for possible data problems. Originally, data for this trial were stored in fixed-length patient records on computer tape, indexed by a master patient directory. Each record contained variables or reserved space for each of the possible patient visits. Thus, a lot of precious file space was wasted. Data were keypunched onto 80 column punch cards and 100% verified before being loaded into the database. Text data were separately keyed and stored. Eventually, ‘‘key-to-disk’’ machines replaced the punch cards. Computer programs were written to check for a variety of potential data problems. The range of data edits, remarkably, was consistent with the data edits applied to more contemporary clinical databases: missing responses, where required; values outside expected limits; internal logical consistency; and data values indicating possible protocol deviations.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CLINICAL DATA MANAGEMENT

Without intelligent data entry, however, these early data editing programs had to check to make sure that legitimate values were entered even for categorical responses. More contemporary programs will not allow an invalid entry for categorical responses such as ‘‘Yes’’/‘‘No’’ variables. In the late 1970s, the Coronary Artery Surgery Study (CASS), another NIH-sponsored clinical trial, initiated the use of distributed data entry or remote data entry (RDE). Computer workstations were installed at each of the clinical sites. The workstations allowed the clinical site staff to key study data directly to a computer diskette. Data entry programs checked for some possible data problems, generally univariate data edits. Double-entry verification was enforced. In the beginning, the computer diskettes were mailed to the Data Coordinating Center. Later during the trial, the data were electronically transmitted from the workstation at the clinical site to a similar workstation at the Coordinating Center. The key advantage to this system was that data were received at the Coordinating Center more quickly than if paper CRFs had been sent. Data editing programs could be run sooner and queries sent back to the clinical sites more quickly. A major premise driving this innovation is that the sooner that potential data errors or omissions are detected, the more likely they will be corrected. This was based on the conjecture that the clinic staff would be more likely to remember a patient visit and the medical chart would more likely still be easily available. The key-to-disk software was developed by staff at the Coordinating Center. Even at that time, there were few commercial programs to support clinical data management. The CASS Coordinating Center, like many other data coordinating centers involved in clinical trials at the time, developed its own clinical data management system (2). Separate computer programs were developed to scan the database for potential data problems. Query reports were periodically sent by mail to the clinical sites.

1.2 Development of Commercial Software to Support Clinical Data Management Until the 1980s, most clinical data management was done using ‘‘home-grown’’ software. Data Coordinating Centers at major universities supporting NIH-sponsored research, fledgling Contract Research Organizations, and pharmaceutical companies with their own data centers developed and deployed a wide variety of data systems. These early ‘‘home-grown’’ systems initially had to provide their own underlying file structure since database engines such as Oracle were not yet widely available. A few products emerged, such as the Scientific Information Retrieval System (3), which provided a database system tailored to clinical studies. SIR was a hierarchical data system optimizing for data linked together for cases and visits. It provided a robust storage format for clinical data, but at that time lacked many of the data entry and data cleaning tools of more contemporary systems. By the mid-1980s, a few commercial systems came to the market to support clinical data management. These systems added more features such as data entry, data cleaning, and reporting functions. Most of these systems presumed that paper CRFs would be sent to a central data center for entry and processing. The Internet was in its infancy, and Web-based applications were yet to come. By the mid-1990s, personal computers had penetrated the business world and the Web was quickly becoming a platform for secure business transactions. Clinical data management moved slowly toward adopting this new technology. A few university-based Data Coordinating Centers were experimenting with home-grown systems to allow data entry using Web applications. By the turn of the century, however, the market for Web-based clinical data management systems, primarily ‘‘electronic data capture’’ or eDC systems had emerged. Even into the 1990s, a big decision for any clinical data management group, however, was whether to acquire and implement a commercial software system or to develop a software system more closely tailored to internal processes.

CLINICAL DATA MANAGEMENT

2

ELECTRONIC DATA CAPTURE

Numerous articles have been written on the value of eDC systems over conventional paper CRFs. In the late 1990s, experts were predicting that by now well over half of clinical trials would be conducted using eDC systems. Granted, there has been tremendous growth in the adoption of eDC, reported by some to be as much as 23% per year (4). The number of eDC vendors has grown steadily in the intervening years, with a few systems capturing a sizeable portion of this market. However, at present, the proportion of trials that are managed using paper CRFs remains substantial. Several factors have limited the rate of adoption of eDC, including acceptance by clinical site staff. Although eDC clearly lessens the burden for storage of paper records, it does not necessarily lessen the burden on clinical site staff for collecting and transcribing that information from source documents to the CRF, whether it be paper or electronic. Not all clinics are equipped with sufficient numbers of properly configured PCs to allow research staff access to the eDC system when it would be most efficient. For example, most examination rooms do not yet have high-speed Internet access or wireless Internet access. 3 REGULATORY INVOLVEMENT WITH CLINICAL DATA MANAGEMENT For those clinical trials conducted in support of a new drug application or approval of a medical device, regulatory agencies such as the U.S. FDA, the European EMEA, and Health Canada have issued guidance documents and regulations that govern many aspects of clinical data management. The objective of these guidance documents and regulations is to assure that the information presented to the regulatory body is as complete and accurate as possible, since the safety of the patients who will use these new treatments will depend on its reliability. Perhaps the most far-reaching of these regulations is 21CFR11, Electronic Records and Signature. This regulation was issued in 1997 after extensive consultation with the pharmaceutical industry and other stakeholders. Work began on this regulation in

3

1991 with the FDA forming a Working Group in 1992 to investigate the various issues surrounding digital signature, in particular. A proposed rule was published in 1994 and resulted in a hailstorm of discussion. The final rule was issued in March 1997. The regulation, as its title implies, puts forth requirements for the use of both electronic data and electronic signatures for studies to be submitted to the U.S. FDA. The regulation’s primary focus is to make sure that electronic data are as reliable and attributable as data or information that would have been supplied on paper, and that if an electronic signature is used in place of a physical signature, that the electronic signature carries the same weight as a physical signature, is attributable, and is untamperable. Part 11 also addresses requirements for electronic systems that would support creation, storage, modification, and transmission of electronic records. These regulations lay out some fundamental requirements for clinical data management systems, such as complete electronic audit trail, attributability of data by automatic date/time stamp, and use of security measures such as passwords to prevent unauthorized access. In addition to 21CFR11, the FDA issued a guidance document in April 1999 on the use of computerized systems in clinical trials. This guidance document ‘‘applies to records in electronic form that are used to create, modify, maintain, archive, retrieve, or transmit clinical data required to be maintained, or submitted to the FDA’’ (5). The guidance was updated in May 2007 and contains extensive information about the agency’s expectations regarding the design and validation of computerized systems, such as clinical data management systems. 4

PROFESSIONAL SOCIETIES

The Society for Clinical Trials, founded in 1978 (www.sctweb.org), has always included professionals involved with clinical data management. Their annual meetings include numerous workshops and plenary sessions on various topics related to clinical data management. A supplement to the Society’s journal, Controlled Clinical Trials, published in

4

CLINICAL DATA MANAGEMENT

1995 (6) discusses the fundamentals of clinical data management. Similarly, the Drug Information Association (www.dia.org) supports professional interchange on clinical data management. Their annual meeting includes a specific track on data management, and they sponsor other meetings throughout the year with a particular focus on clinical data management. And the Society for Clinical Data Management (www.scdm.org) was founded to specifically ‘‘advance the discipline of clinical data management.’’ In addition to meetings, the SCDM offers a complete guide to ‘‘Good Clinical Data Management Practices’’ and a certification program for clinical data managers.

5

LOOK TO THE FUTURE

5.1 Standardization In recent years there has been a move toward standardizing the way that data are collected and exchanged. The Clinical Data Interchange Standards Consortium (CDISC, www. CDISC.org) was formed as a working group from the Drug Information Association. This now independent organization has developed several standards for data models that are becoming widely adopted, not just in the United States, but worldwide. The National Cancer Institute began an initiative to standardize the way that clinical data are captured with their Cancer Data Standards Repository (CaDSR) project (http://ncicb.nci.nih. gov/NCICB/infrastructure/cacore overview/ cadsr). This repository contains standardized data element definitions for a wide variety of domains involving both cancer studies and clinical studies in other areas, such as dentistry and cardiology. 5.2 Electronic Data Capture and Electronic Health Records As access to the Web continues to increase and as our society continues to move toward electronic rather than paper records, the proportion of studies conducted using electronic systems such as Web eDC systems is bound to increase. Whether eDC completely replaces paper CRFs is still a topic for debate. ‘‘Going

electronic’’ will require changes in the workflow model to take advantage of the efficiencies that an eDC system can offer. As long as the clinic staff are still recording clinical information on paper (either as clinic notes, source documentation, or paper worksheets), they are not realizing the full advantages of the eDC system since they will need to transcribe the information from paper, rather than recording it once electronically. The next level of efficiency to be gained will come from linking research needs with electronic health records (EHRs). The proportion of in-patient and out-patient facilities using electronic health records is increasing rapidly and has gained acceptance in Europe more than in the United States. A few groups have explored extracting data directly from the electronic health record to an electronic case report form. CDISC partnered with the Duke Clinical Research Institute in 2003 to demonstrate integration of electronic health records with clinical trial data (Starbrite project presented at DIA, www.cdisc.org/pdf/ss DIAJune2004lb.ppt). The objectives for EHR systems differ substantially from the objectives for eDC systems (7). Health records might contain data as discrete data points (e.g., laboratory values or blood pressure readings), or they may contain less-structured data such as progress notes or descriptions of symptoms. The purpose of the EHR is to store and convey to the health-care providers as much information as possible to assist in the diagnosis and treatment of a patient presenting at that health-care facility. The purpose of an eDC system is to collect and organize the information needed for a clinical trial, a much more focused need. The market for EHR systems is huge when compared with the market for eDC systems. Vendors of EHR systems will be driven by their market, in particular by large hospital systems. The number of EHR vendors is expected to be large, and thus, the formats for collecting data will be widely varied. The challenges are large, but the potential benefits are also large in terms of the savings in time and the decrease in transcription errors. Groups such as CDISC and PhRMA are collaborating on addressing issues related to

CLINICAL DATA MANAGEMENT

integrating clinical research with electronic health records (8,9). 6

CONCLUSION

The proliferation of the use of computers and the rapid penetration of the Internet as a means of secure exchange of data has had a profound impact on clinical data management. Where it might have taken months to collect, clean, and analyze data for a clinical trial 30 years ago, we now expect complete, relatively clean data almost as soon as a patient visit has been completed. Changes in technology have had a profound impact on the way that a trial is managed and in the role for data management staff (10). As technology has and continues to advance, the nature of clinical data management has changed, but the objective remains the same: to provide complete, accurate, and attributable data to support clinical studies, in particular, clinical trials. The articles that follow will describe various aspects of clinical data management and various approaches that have been taken to collect, organize, clean, and report clinical data. REFERENCES 1. C. L. Meinert, E. C. Heinz, and S. A. Forman, ‘‘The Coronary Drug Project: Role and Methods of the Coordinating Center’’ and other articles in this supplement. Control Clin Trials. 1983; 4(4): 355–375. 2. L. D. Fisher, M. J. Gillespie, M. Jones, and R. McBride, Design of clinical database management systems and associated software to facilitate medical statistical research. CRC Critical Reviews in Medical Informatics. Vol. I-4. 3. G. D. Anderson, E. Cohen, W. Gazdzik, and B. Robinson, ‘‘Scientific information retrieval system: A new approach to research data management’’, Proceedings of the 5th Annual ACM SIGUCCS Conference on User Services, 1977: 209–212. 4. K. A. Getz, The imperative to support site adoption of EDC. Appl. Clin. Trials. (Jan. 2006). Available: http://appliedclinical trialsonline.findpharma.com/appliedclinical trials/article/articleDetail.jsp?id=283027.

5

5. http://www.fda.gov/cder/guidance/7359fnl. pdf 6. R. McBride and S. W. Singer (eds.), Data management for multicenter studies: Methods and guidelines. Controlled Clin Trials. 1995; 16(2, suppl): 1–179. 7. P. Bleicher, Integrating EHR with EDC: When two worlds collide. Appl. Clin. Trials. (Mar. 2006). Available: http://appliedclinical trialsonline.findpharma.com/appliedclinical trials/IT/Integrating-EHR-with-EDC-WhenTwo-Worlds-Collide/ArticleStandard/Article/ detail/310798. 8. eClinical Forum and PhRMA, The Future Vision of Electronic Health Records as eSource for Clinical Research. Discussion document published Sept. 2006. Available: http://www.cdisc.org/publications/index. html. 9. CDISC, CDISC Standards and Electronic Source Data in Clinical Trials. Discussion document published Nov. 2006. Available: http://www.cdisc.org/publications/index. html. 10. F. Falk, Impact of eDC on clinical staff roles. Appl. Clin. Trials. (June 2007). Available: http://appliedclinicaltrialsonline.findpharma. com/appliedclinicaltrials/Feature+Article/ Impact-of-EDC-on-Clinical-Staff-Roles/ ArticleStandard/Article/detail/431922? contextCategoryId=35507.

CLINICAL HOLD DECISION A clinical hold is the mechanism that the U.S. Food and Drug Administration’s Center for Drug Evaluation and Research (CDER) uses when it does not believe, or cannot confirm, that the study can be conducted without unreasonable risk to the subjects/patients. If this occurs, the Center will contact the sponsor within the 30-day initial review period to stop the clinical trial. The CDER may either delay the start of an early-phase trial on the basis of information submitted in the Investigational New Drug Application (IND), or stop an ongoing study based on a review of newly submitted clinical protocols, safety reports, protocol amendments, or other information. When a clinical hold is issued, a sponsor must address the issue that is the basis of the hold before the order is removed. The CDER’s authority concerning clinical holds is outlined in federal regulations. The regulations specify the clinical hold criteria that CDER applies to various phases of clinical testing. In addition, all clinical holds are reviewed by upper management of CDER to ensure consistency and scientific quality in the Center’s clinical hold decisions.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/clinhold.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

CLINICAL SIGNIFICANCE

may be used by regulatory agencies for drug approval, by clinicians to decide between treatment alternatives, by patients to make informed decisions about treatment, by the health-care industry for formulary and reimbursement decisions, and by health-care policy makers to make policy decisions regarding resource allotment. Early evidence of the clinical implications of QOL is evident in the links between survival and QOL components such as patients’ fatigue levels, social support, and group counseling (14–17). Even a simple, single-item measure of patient global QOL can be related to patient survival (18). Changes in QOL scores can also be linked to positive economic (19, 20) and social (21) outcomes.

CELIA C. KAMATH and JEFFREY A. SLOAN Health Sciences Research Mayo Clinic Rochester, Minnesota

JOSEPH C. CAPPELLERI Pfizer, Inc. Global Research & Development Groton, Connecticut

1

INTRODUCTION

The field of patient-reported outcomes, particularly health-related quality of life (QOL), has burgeoned in the last few years (1, 2). The importance assigned to the study of these outcomes has been attributed to the aging of the population and consequently higher prevalence of chronic diseases, along with the reality that medical treatment often fails to cure the disease but may affect QOL (3). Health-related quality of life has gained attention in research and clinical trial settings (3, 4). The increasingly important role assigned by patients and clinicians to QOL’s role in medical decision-making has resulted in greater attention paid to the interpretation of QOL scores, particularly as it relates to clinical significance (5–7). Clinical significance relates to the clinical meaningfulness of inter-subject or intra-subject changes in QOL scores. Clinical significance has been difficult to determine, in part because of the development of a myriad of QOL instruments over the past decade (8, 9). Some of these developments have had little or no psychometric (1, 2, 6, 10, 11) or clinical validation (9, 12). Moreover, relative to traditional clinical endpoints like survival and systolic blood pressure, QOL as a clinical endpoint is relatively unfamiliar, especially in regard to interpretation and relevance of changes in QOL scores (13). Why is clinical significance of QOL scores important? It aids in the design of studies by helping to determine sample size calculations. Evidence of clinical significance

2

HISTORICAL BACKGROUND

Statistical significance as measured by a Pvalue is influenced by sample size and data variability. Although statistical significance can be considered a prerequisite for clinical significance, only clinical significance assigns meaning to the magnitude of effect observed in any study. Historically, Cohen (22) was responsible for proposing one of the earliest criteria for identifying important change, which can be construed as clinically significant. He suggested that a small ‘‘effect size’’ (defined later in the article) was 0.2 standard deviation units, a medium ‘‘effect size’’ was 0.5, and a large ‘‘effect size’’ was 0.8. Although his intention was to provide guidance for sample size calculations in the social and behavioral science, Cohen’s benchmarks have extended to health-care research to decide whether a change in QOL scores is important. Current research suggests that a moderate effect size of one-half a standard deviation unit (effect size = 0.5) is typically important (23). A more recent and popular definition of clinical significance uses an anchor-based approach based on an external standard that is interpretable and appreciably correlated to the target QOL measure in order to elucidate the meaning of change on the target QOL measure.

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

CLINICAL SIGNIFICANCE

Embedded under the rubric of clinical significance is the minimum important difference, a lower bound on clinical significance. One definition of a minimum important difference (MID) is ‘‘the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side-effects and excessive cost, a change in the patient’s management’’ (24). Some researchers prefer to use the term ‘‘minimally detectable difference’’ (25, 26); other definitions have sprouted (e.g., the ERES method) (27, 28). No single solution to the challenging topic of assessing clinical significance exists. Nevertheless, a series of proposals has engendered understanding and appreciation of the topic. Special issues in Statistics in Medicine (1999, Vol. 18) and the Journal of Consulting and Clinical Psychology (1999, Vol. 67) have been dedicated to the topic of clinical significance of QOL and other clinical measures. Proceedings from a meeting of an international group of about 30 QOL experts were published recently in a special issue of the Mayo Clinic Proceedings (2002, Vol. 77) (29–35), which provides practical guidance regarding the clinical significance of QOL measures.

3

ARTICLE OUTLINE

This article draws largely from recent scientific literature, including the Mayo Clinic Proceedings (29–35) and other sources (4), to provide an overview on the clinical significance of QOL measures. The following section on Design and Methodology covers the different perspectives and existing methods to determine clinical significance. The next section on Examples illustrates trials in which researchers attempted to define the concept on specific QOL measures. Then, the section on Recent Developments highlights new methods to determine clinical significance. Finally, the section on Concluding Remarks discusses some future directions for research.

4 DESIGN AND METHODOLOGY 4.1 Perspectives for Determining and Interpreting Clinical Significance Clinical significance involves assigning meaning to study results. The process of establishing such meaning can be conceptualized in two steps: (1) understanding what changes in score mean to the concerned stakeholder (e.g., patient, clinician, clinical researcher, policy maker) and (2) making results of clinical studies interpretable and comprehensible to such stakeholders or decision makers (30, 36). The term ‘‘clinical’’ in relation to significance has different meanings and implications for different stakeholders such as patients, clinicians, and society. From the patient’s perspective, clinical significance can be defined as the change in QOL scores that patients perceive as beneficial (or detrimental) and important and prompts them to seek health care or request changes in their treatment (33), or that induces patients to determine that the intervention has been successful (24). From the clinician’s perspective, it can be defined as the diagnosis of the clinician as to the amount of change in QOL scores that would mandate some form of clinical intervention (37). From the societal or population perspective, clinical significance is based on the values of the group surveyed, in which importance is defined by the outcomes that are deemed worthy of society’s resources. Any or all of these perspectives for defining clinical significance may be applicable, but they are not always in agreement (4). An equally important issue is the different perspectives for interpreting clinical meaningfulness of changes in reported QOL (35). For example, a clinician may use QOL data to explain the treatment alternatives to a patient, whereas a health-policy maker may describe to elected officials the financial impact on a patient population whose QOL has changed. Similarly, a regulatory agency and pharmaceutical company may ascertain the appropriate level of evidence for a successful research study (35). Thus, QOL results must be framed, analyzed, and presented in a way that is meaningful to the pertinent audience and its respective needs.

CLINICAL SIGNIFICANCE

Only then will the concept be meaningful and gain greater acceptance and use over time. 4.2 Methods to Explain the Clinical Significance of Health Status Measures Two common approaches used to establish the interpretability of QOL measures are termed anchor-based and distributionbased. The characteristics of each approach are described below. Several examples will be given later in the section on Examples. Interested readers are encouraged to read Crosby et al. (4) and Guyatt et al. (30) for an expanded discussion of the concepts presented here. Anchor-based approaches are used to determine clinically meaningful change via cross-sectional or longitudinal methods involve comparing measures of QOL to measures with clinical relevance (4). Crosssectional methods include several forms: (1) comparing groups that are different in terms of some disease-related criterion (38, 39); (2) linking QOL to some external benchmarking criteria (40–42); (3) eliciting preferencebased ratings on a pair-wise basis, where one person’s ratings state serves as an anchor to evaluate the other person’s ratings (43); and (4) using normative information from dysfunctional and functional populations (6). Longitudinal methods involve the comparison of changes in QOL scores across time with the use of (1) global ratings of change as ‘‘heuristics’’ to interpret changes in QOL scores (5, 24, 38, 44); (2) significant future medical events for establishing difference thresholds (45); and (3) comparisons of changes in HRQOL to other diseaserelated measures of outcome across time (46). Anchor-based methods are cataloged in Table 1 (4–6, 24, 38–41, 43–45, 47, 48). Anchor-based methods require two properties (30): (1) Anchors must be interpretable, otherwise they will hold no meaning to clinicians or patients and (2) anchors must share appreciable correlation with the targeted QOL measure. The biggest advantage of anchor-based approaches is the link with a meaningful external anchor (4), akin to establishing the construct validity of the measure (49). Potential problems, however, exist with this approach. These problems include

3

recall biases (50), low or unknown reliability and validity of the anchor measure (51), low correlation between anchor and actual QOL change score (52, 53, 54, 55), and complex relationships between anchors and QOL scores (56). Hays and Wooley (57) recommend caution in the indiscriminate dependence and use of a single minimum important difference (MID) measure. They also list several problems in estimating MIDs: The estimated magnitude could vary depending on the distributional index (57, 58), the external anchor (59), the direction of change (improvement vs. decline) (60), and the baseline value (61). In general, longitudinal methods are preferable because of their direct link with change (4). Distribution-based approaches for determining the importance of change are based on the statistical characteristics of the obtained sample, namely average scores and some measure variability in results. They are categorized as (1) those that are based on statistical significance using P-values (i.e., given no real change, the probability of observing this change or a more extreme change), which include the paired t-statistic (62) and growth curve analysis (63); (2) those that are based on sample variation (i.e., those that evaluate mean change in relation to average variation around a mean value), which include effect size (22, 64), standardized response mean (SRM) (44), and responsiveness statistic (65); and (3) those that are based on the measurement precision of the instrument (i.e., evaluate change in relation to variation in the instrument as opposed to variation of the sample), which includes the standard error of the mean (SEM) (7) and the reliable change index (RC) (6). Distributed-based methods are cataloged in Table 2 (4, 6, 7, 22, 44, 62–65). An advantage of the distribution-based methods is that they provide a way of establishing change beyond random variation and statistical significance. The effect size version of the distribution-based methods is useful to interpret differences at the group level and has benchmarks of 0.20 standard deviations units as a small effect, 0.50 as a moderate effect, and 0.80 as a large effect (22, 64, 66). The measures that seem most promising

4

CLINICAL SIGNIFICANCE

Table 1. Anchor-Based Methods of Determining Change Type

Method

Examples

HRQOL evaluated in relation to:

Advantages

Disadvantages

Crosssectional

Comparison to diseaserelated criteria

References 39 and 47

Disease severity or diagnosis

Can be standardized

May not reflect change

Easy to obtain

Groups may differ in other key variables May no reflect change

Comparison to nondiseaserelated criteria

References 40 and 41

Impact of life events

Easy to obtain

Provides external basis for interpretation

Preference rating

Longitudinal

Reference 43

Pairwise comparisons of health status

All health states are compared

Comparison to known populations

Reference 6

Functional or dysfunctional populations

Uses normative information

Global ratings of change

References 5, 24, 38, 44

Patients’ or clinicians’ ratings of improvement

Easy to obtain

Prognosis of future events

Changes in disease related outcome

Reference 45

Reference 48

Reprinted with permission from Crosby et al. (4).

Those experiencing and not experiencing some future event

Changes in clinical outcome

Best measure from individual perspective Can take into account a variety of information Prospective

Provides evidence of predictive validity Tied to objective outcome measure Known psychometric properties

Groups may differ on other key variables Relationship to HRQOL not clear May not reflect change Hypothetical, artificial Time Consuming Normative information not always available Amount of change needed not specified Does not consider measurement precision Unknown reliability

Influenced by specific rating scale and anchors Does not consider measurement precision Difficult to obtain

Does not consider measurement precision Assumes strong HRQOL-outcome correlation

CLINICAL SIGNIFICANCE

for the purpose of establishing clinical significance at the individual patient level are the SEM and the RC. These measures are based on the measurement precision of the instrument and incorporate the reliability of the instrument (e.g., Cronbach’s alpha or test-retest reliability), and the standard deviation of scores. In principle, SEM and RC are sample invariant. Researchers have favored Cronbach’s alpha over test-retest reliability to calculate reliability for the SEM (7, 30, 67). Distribution methods are particularly helpful when used together with meaningful anchors, which enhances validity and hence meaning to the QOL measure. Some encouragement exists to know that anchor-based measures appear to coincide with distribution-based methods. Researchers have found a correspondence between SEM and anchor-based determinant of a minimum important difference across difference diseases (7, 23, 67, 68). The 1 SEM benchmark corresponds with an effect size of approximately 0.5. Standard error of measurement is moderated by the reliability of the measure, where measures with higher reliability are ‘‘rewarded’’ by lowering the effect size needed to achieve a minimally important difference. A rationale for a SEM as a measure of MID is provided by Norman et al. (23) who assert that Miller’s theory (69) of the limits of human discernment is linked to the threshold of 0.5 standard deviation units. 5

EXAMPLES

This section provides examples of studies used to determine clinical significance and presents general advice for defining and interpreting clinical significance in clinical studies. Table 3 (5, 7, 24, 64, 67, 70–72) includes several examples on the use of both anchorbased methods and distribution-based methods to establish clinical significance across a wide range of QOL measures. These examples span several disease groups, instruments, and methods for determining clinical significance. Readers are encouraged to review the cited papers for further details on these studies. The authors begin with a classic paper by Jaeschke et al. (24), one of the first papers

5

on clinically meaningful differences determined through the anchor-based approach. The magnitude of difference considered minimally significant was an average of 0.5 per item on a 7-point scale, which was confirmed by Juniper et al. (5) on the Asthma Quality of Life Questionnaire (AQLQ). A third study, by Kazis et al. (64), examined the difference between statistical significance and clinical significance. Using several pain studies and the Pain Intensity numerical rating scale (PI-NRS), Farrar et al. (70) found a reduction of 2 points or 30% on the 11-point pain scale to be clinically significant. Using the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) scale for osteoarthritis, Angst et al. (71) compared estimates derived from anchor-based and distribution-based approaches; they determined sample sizes for specified changes signifying worsening and, separately, improvement. Using the Chronic Heart Failure Questionnaire (CHQ), Wyrwich et al (7) also compared anchor- and distribution-based approaches in determining that 1 SEM equals the MCID of 0.5 per item determined by the anchor-based approach. Finally, using the Functional Assessment of Cancer Therapy-Lung (FACTL) questionnaire, Cella et al (72) showed the convergence of three different complementary approaches on clinical significance. Taken from Sprangers et al. (34), Table 4 provides a useful checklist of questions to help in interpretation of longitudinal, patient-derived QOL results presented in clinical trials and the clinical literature. These questions are based on the premise that detecting meaningful change depends on the adequacy of the research design, measurement quality, and data analysis. 6

RECENT DEVELOPMENTS

6.1 The

1 2

Standard Deviation Rule

It would be desirable to simply define, at least initially, what a clinical significant result is likely to be. Emerging research comparing anchor-based and distribution-based estimates provides an evolving standard as to what to use as an initial estimate (23). The anchor-based estimates averaging 0.5 per

6

CLINICAL SIGNIFICANCE

item on a 7-point scale appear to converge with an estimate of 12 standard deviation (SD) units. This latter estimate is derived through distribution-based methods such as the effect size approach (22, 64), the SEM (7, 67, 68), the Jacobson reliable change index (6), and the standardized response mean (73). Potential moderating factors that could impact these estimates upward or downward are the method used to determine minimum difference estimates and whether patients were suffering from acute or chronic conditions (23, 74). 6.2 Empirical Rule Effect Size Sloan et al. (27, 28) have taken this concept one step further in the form of the Empirical Rule Effect Size (ERES) by combining Cohen’s effect size categorization (22) with the empirical rule from statistical theory (75). The ERES is based on Tchebyschev’s Theorem and states that the distribution of any QOL tool is contained within 6 SDs of the observed values. The ERES entails the estimation of QOL change scores in terms of SD estimates, expressed as units on the theoretical range of a QOL instrument. Thus, small, moderate, and large effect sizes for comparing QOL treatment groups turn out to be 3%, 8%, and 13%, respectively, of the theoretical range of any QOL tool. This simple and intuitive rule to identify the magnitude of clinically significant changes is likely to be easy for clinical researchers to comprehend. The rule can facilitate the design of clinical trials in terms of sample size calculations and interim monitoring of clinical trials. The ERES framework for a priori establishment of effect sizes is sample-independent and thus an improvement over sample-dependent methods (5, 21, 76). However, the simplicity of the ERES method gives rise to some challenges and questions. The theoretical range of the instrument is rarely observed in its entirety, necessitating the modification of the theoretical range to more practical limits before calculating the ERES estimate for 1 SD as necessarily 16.7% (i.e., 1/6 of distribution of observed values) of the range. Similarly, truncated distributions, where the patient population is homogeneously ill or uniformly

healthy, can be accommodated by incorporating this knowledge into the definition of the appropriate range. These guidelines for clinical treatments can be used in the absence of other information but will need modification in their application to idiosyncratic or unique clinical settings. More research is needed to examine the generalizability of such benchmarks across baseline patient health, severity of illness, and disease groups. 6.3 Group Change vs. Individual Change Distinctions should be made in determining the significance of change at the group versus the individual level. Every individual in a group does not experience the same change in outcomes (group level outcomes are assigned a mean change value). Higher variability exists in individual responses than those of the group. Depending on the distribution of individual differences, the same group mean can have different implications for an individual (77). The traversing of group- and individuallevel QOL data entails procedures for moving from one level to the other involving two distinctive scientific traditions: deductive and inductive (31). A deductive approach is employed when one addresses the extent to which group data can be used to estimate clinical significance at the individual level. An inductive approach is used when one evaluates the extent to which individual change data can be brought to the group level to define clinical significance. Readers are advised to read Cella et al. (31) for a more detailed account. 6.4 Quality of Life as a ‘‘Soft’’ Endpoint The ‘‘softness’’ of QOL as an endpoint, relative to, say, survival and tumor response, is cited as a particular barrier to implementation and interpretation of results (13). However, methodological and conceptual strides made in defining and measuring QOL, and the growing familiarity with the interpretation and potential utility of QOL data, make those concerns increasingly outdated. Psychometric advances have been made in QOL assessment tools across disease areas (8, 78–81). Funding opportunities to study QOL endpoints have allowed for study designs

CLINICAL SIGNIFICANCE

that are large enough to have power to detect meaningful differences (13). Moreover, accumulated experience with analyzing QOL endpoints has resulted in the recognition that their statistical challenges are no different from those of ‘‘hard’’ endpoints. 7

CONCLUDING REMARKS

Several suggestions on clinical significance are offered. First, the application of multiple strategies for determining clinical significant is recommended. Doing so would enable better interpretability and validity of clinically significant change, would add to existing evidence of the magnitude of change that constitutes clinical significance, and would provide indicators of distributional parameters that create convergence or divergence in estimation of clinical significance. For example, Kolotkin et al. (46) found convergence between anchor-based and distribution-based methods at moderate level of impairment but wide disparities at mild and severe levels of impairment. Second, more research is needed to identify the effect psychometric properties—that is, reliability, validity, and responsiveness of QOL instruments— have in quantifying clinically meaningful change (4, 62, 82). Similarly, research into the psychometric properties of global rating and health transition scales used in anchor-based methods is also needed. Global ratings tend to be single item measures and may therefore fall short in terms of explaining complex QOL constructs. Anchoring assessment also tends to be positively correlated with post-treatment states but with near-zero correlation with pre-treatment states, suggesting a recall bias (83) or response shift (84). More research is needed to address the cognitive process used by patients to retrospectively assess changes in health over time (30). Third, baseline severity results in regression to the mean (RTM), an error-based artifact describing the statistical tendency of extreme scores to become less extreme at follow-up. Failure to take this tendency into account may lead to false conclusions that patients with severe impairments at baseline have shown clinically significant change

7

when, in fact, it was just RTM. RTM also has a greater impact on data when the measure is less reliable (4, 85). More research is also needed into the effect of baseline QOL impairment on magnitude of clinically meaningful change (4, 48, 66, 86, 87). Similar research is needed in terms of the generalizability of the standardized benchmarks for determining clinically meaningful change, especially for distribution-based methods (4, 66). Specifically, how satisfactory are the evolving benchmarks (effect sizes of 0.2, 0.5, and 0.8 for small, moderate, and large change, respectively) across different dimensions of QOL (e.g., mental versus physical), different disease groups (e.g., arthritis versus cancer), respondents (e.g., patients versus clinicians), measures (e.g., generic versus disease-specific), patient populations (e.g., older versus younger), or patient conditions (e.g., improving versus deteriorating)? Finally, care must be taken in presenting results of studies in a way that is familiar to the user of the information. For example, translating clinical significance into a number needed to treat (NNT) and a proportion of patients achieving various degrees of clinical benefit relative to the control may provide a desirable way to present study results (30). REFERENCES 1. N. Aaronson, Methodologic issues in assessing the quality of life of cancer patients. Cancer 1991; 67(3 Suppl): 844–850. 2. D. Cella and A. E. Bonomi, Measuring quality of life. Oncology 1995; 9(11 Suppl): 47–60. 3. R. Berzon, Understanding and Using HealthRelated Quality of Life Instruments within Clinical Research Studies. Quality of Life Assessment in Clinical Trials: Methods and Practice. Oxford, UK: 2000, pp. 3–15. 4. R. D. Crosby, R. L. Kolotkin, and G. R. Williams, Defining clinically meaningful change in health-related quality of life. J. Clin. Epidemiol. 2003; 56: 397–407. 5. E. F. Juniper, G. H. Guyatt, and A. Willan, Determining a minimal important change in a disease-specific quality of life questionnaire. J. Clin. Epidemiol. 1994; 47: 81–87. 6. N. S. Jacobson and P. Truax, Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J. Consult. Clin. Psychol. 1991; 59: 12–19.

8

CLINICAL SIGNIFICANCE 7. K. W. Wyrwich, W. M. Tiemey, and F. D. Wolinsky, Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health related quality of life. J. Clin. Epidemiol. 1999; 52: 861–873. 8. D. Cella, Quality of life outcomes: measurement and validation. Oncology 1996; 10(11 Suppl): 233–246. 9. J. A. Sloan, J. R. O’Fallon, and V. J. Summan, Incorporating quality of life measurements in oncology clinical trials. Proceeding of the Biometrics Section of the American Statistical Association, 1998: 282–287.

10. B. Spilker, Quality of Life and Pharmacoeconomics in Clinical Trials. New York: Lippincott Raven, 1996. 11. D. Osoba, What has been learned from measuring health-related quality of life in clinical oncology. Eur. J. Cancer 1999; 35(11): 1565–1570. 12. J. A. Sloan and T. Symonds, Health-related quality of life measurement in clinical trials: when does a statistically significant change become relavant? unpublished manuscript, 2003. 13. M. H. S. J. Frost, Quality of life measures: a soft outcome - or is it? Amer. J. Managed Care 2002; 8(18 Suppl): S574–S579. 14. L. Degner and J. A. Sloan, Symptom distress in newly diagnosed ambulatory cancer patients as a preditor of survival in lung cancer. J. Pain Symptom Manag. 1995; 10(6): 423–431. 15. H. M. Chochinov and L. Kristjanson, Dying to pay: the cost of end-of-life care. J. Palliat. Care 1998; 14(4): 5–15. 16. R. A. Silliman, K. A. Dukes, and L. M. Sullivan, Breast cancer care in older women: sources of information, social support, and emotional health outcomes. Cancer 1998; 83(4): 706–711. 17. D. Spiegel, J. R. Bloom, and H. Kraemer, Psychological support for cancer patients. Lancet 1989; 2(8677): 1447. 18. J. A. Sloan, C. L. Loprinzi, and S. A. Kuross, Randomized comparison of four tools measureing overall quality of life in patients with advanced cancer. J. Clin. Oncol. 1998; 16: 3662–3673. 19. D. L. Patrick and P. Erickson, Applications of health status assessment to health policy. In: B. Spilker (ed.), Quality of Life and Pharmacoeconomics in Clinical Trials. New York: Lippincott Raven, 1996, pp. 717–727.

20. M. R. Gold, D. L. Patrick, and G. W. Torrance, Identifying and valuing outcomes. In: M. Gold et al. (eds.), Cost Effectiveness in Health and Medicine. New York: Oxford University Press, 1996, pp. 82–134. 21. E. F. Juniper, The value and quality of life in asthma. Eur. Resp. J. 1997; 7: 333–337. 22. J. Cohen, Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates, 1988. 23. G. R. Norman, J. A. Sloan, and K. W. Wyrwich, Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med. Care 2003; 41(5): 582–592. 24. R. Jaeschke, J. Singer, and G. H. Guyatt, Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin. Trials 1989; 10(4): 407–415. 25. P. Jones, Interpreting thresholds for a clinically significant change in health status (quality of life) with treatment for asthma and COPD. Eur. Resp. J. 2002; 19: 398–404. 26. J. G. Wright, The minimally important difference: who’s to say what is important? J. Clin. Epidemiol. 1996; 49: 1221–1222. 27. J. A. Sloan et al., Detecting worms, ducks, and elephants: a simple approach for defining clinically relevant effects in quality of life measures. J. Cancer Integrative Med. 2003; 1(1): 41–47. 28. J. A. Sloan, T. Symonds, D. Vargas-Chanes, and B. Fridley, Practical guidelines for assessing the clinical significance of health-related QOL changes within clinical trials. Drug Inf. J. 2003; 37: 23–31. 29. J. A. Sloan et al., Assessing clinical significance in measuring oncology patient quality of life: introduction to the symposium, content overview, and definition of terms. Mayo Clin. Proc. 2002; 77: 367–370. 30. G. H. Guyatt et al., Methods to explain the clinical significance of health status measures. Mayo Clin. Proc. 2002; 77: 371–383. 31. D. Cella et al., Group vs individual approaches to understanding the clinical significance of differences or changes in quality of life. Mayo Clin. Proc. 2002; 77: 384–392. 32. J. A. Sloan et al., Assessing the clinical significance of single items relative to summated scores. Mayo Clin. Proc. 2002; 77: 479–487. 33. M. H. Frost et al., Patient, clinician, and population perspectives on determining the clinical significance of quality-of-life scores. Mayo Clin. Proc. 2002; 77: 488–494.

CLINICAL SIGNIFICANCE

9

34. M. A. G. Sprangers et al., Assessing meaningful change in quality of life over time: a users’ guide for clinicians. Mayo Clin. Proc. 2002; 77: 561–571.

47. R. A. Deyo et al., Physical and psychosocial function in rheumatoid arthritis: clinical use of a self-adminstered health status instrument. Arch. Intern. Med. 1992; 142: 879

35. T. Symonds et al., The clinical significance of quality-of-life results: practical considerations for specific audiences. Mayo Clin. Proc. 2002; 77: 572–583.

48. R. L. Kolotkin, R. D. Crosby, and G. R. Williams, Integrating anchor-based and distribution-based methods to determine clinically meaningful change in obesity-specific quality of life. Qual. Life Res. 2002; 11: 670.

36. M. A. Testa, Interpretation of quality-of-life outcomes issues that affect magnitude and meaning. Med. Care 2000; 38: II166–II174. 37. C. van Walraven, J. L. Mahon, D. Moher, C. Bohm, and A. Laupacis, Surveying physicians to determine the minimal important difference: implications for sample-size calculation. J. Clin. Epidemiol. 1999; 52: 717–723. 38. R. A. Deyo and T. S. Inui, Toward clinical application of health status measures: sensitivity of scales to clinically important changes. Health Serv. Res. 1984; 19: 278–289. 39. P. A. Johnson, L. Goldman, E. J. Orav et al., Comparison of the medical outcomes study short-form 36-item health survey in black patients and white patients with acute chest pain. Med. Care 1995; 33: 145–160. 40. J. E. Ware, R. H. Brook, A. Davies-Avery et al., Conceptualization and Measurement of Health for Adults in the Health Insurance Study, vol. 1. Model of Health and Methodology. Santa Monica, CA: Rand Corporation, 1979. 41. M. Testa and W. R. Lenderking, Interpreting pharmacoeconcomic and quality-of-life clinical trial data for use in therpeutics. Pharmacoeconomics 1992; 2: 107. 42. M. Testa and D. C. Simonson, Assessment of quality-of-life outcomes. N. Engl. J. Med. 1996; 28: 835–840. 43. H. A. Llewellyn-Thomas, J. I. Williams, and L. Levy, Using a trade-off techniques to assess patients’ treatment preferences for benign prostatic hyperplasia. Med. Decis. Making 1996; 16: 262–272. 44. G. Stucki, M. H. Liang, and A. H. Fossel, Relative responsiveness of condition specific and health status measures in degenerative lumbar spinal stenosis. J. Clin. Epidemiol. 1995; 48: 1369–1378. 45. J. M. Mossey and E. Shapiro, Self-rated health: a predictor of mortaility among the elderly. Amer. J. Public Health 1982; 72: 800–808. 46. R. L. Kolotkin, R. D. Crosby, and K. D. Kosloski, Development of a brief measure to assess quality of life in obesity. Obes. Res. 2001; 9: 102–111.

49. E. Lydick and R. S. Epstein, Interpretation of quality of life changes. Qual. Life Res. 1993; 2: 221–226. 50. N. Schwartz and S. Sudman, Autobiographical Memory and the Validity of Retrospective Reports. New York: Springer-Verlag, 1994. 51. K. W. Wyrwich, S. Metz, and A. N. Babu, The reliability of retrospective change assessments. Qual. Life Res. 2002; 11: 636. 52. B. Mozes, Y. Maor, and A. Shumueli, Do we know what global ratins of health-related quality of life measure? Qual. Life Res. 1999; 8: 269–273. 53. J. R. Kirwan, D. M. Chaput de Sainttonge, and C. R. B. Joyce, Clinical judgment in rheumatoid arthritis. III. British rheumatologists’ judgment of ‘change in response to therapy.’ Ann. Rheum. Dis. 1984; 43: 686–694. 54. D. Cella, E. A. Hahn, and K. Dineen, Meaningful change in cancer-specific quality of life scores: differences between improvement and worsening. Qual. Life Res. 2002; 11: 207–221. 55. G. H. Guyatt and R. Jaeschke, Reassessing quality of life instruments in the evaluation of new drugs. Pharmacoeconomics 1997; 12: 616–626. 56. F. Lydick and B. P. Yawn, Clinical interpretation of health-related quality of life data. In: M. J. Staquet et al. (eds.), Quality of Life Assessment in Clinical Trials: Methods and Practice. Oxford: Oxford University Press, 1998, pp. 299–314. 57. R. D. Hays and J. M. Wooley, The concept of clinically meaningful difference in healthrelated quality-of-life research. How meaningful is it? Pharmacoeconomics 2000; 18(5): 419. 58. J. Wright and N. L. Young, A comparison of different indices of responsiveness. J. Clin. Epidemiol. 1997; 50: 239–246. 59. B. Barber, N. C. Santanello, and R. S. Epstein, Impact of the global on patient perceivable change in an asthma specific QOL questionnaire. Qual. Life Res. 1996; 5: 117–122. 60. J. Ware, K. Snow, M. Kosinski et al., SF-36 Health Survey: Manual and Interpretation

10

61.

62.

63.

64.

65.

66.

67.

68.

69.

70.

71.

72.

CLINICAL SIGNIFICANCE Guide. Boston, MA: The Health Institute, 1993. D. W. Baker, R. D. Hays, and R. H. Brook, Understanding changes in health status: is the floor phenomenon merely the last step of the staircase? Med. Care 1997; 35: 1–15. J. A. Husted, R. J. Cook, and V. T. Farewll, Methods for assessing responsiveness: a critical review and recommendations. J. Clin. Epidemiol. 2000; 53: 459–468. D. C. Speer and P. D. Greenbaum, Five methods for computing significant individual client change and improvement rates: support for an individual growth curve approach. J. Consult. Clin. Psychol. 1995; 63: 1044–1048. L. Kazis, J. J. Anderson, and R. S. Meenan, Effect sizes for interpreting changes in health status. Med. Care 1989; 27(Suppl 3): S178–S189. G. H. Guyatt, C. Bombardier, and P. X. Tugwell, Measuring disease-specific quality of life in clinical trials. CMAJ 1986; 134: 889–895. G. Samsa, D. Edelman, and M. L. Rothman, Determining clinically important differences in health status measures: a general approach with illustration to the Health Utilities Index Mark II. Pharmacoeconomics 1999; 15: 41–55. K. W. Wyrwich, N. A. Nienaber, and W. M. Tiemey, Linking clinical relevance and statistical significance in evaluating intraindividual changes in health-related quality of life. Med. Care 1999; 37: 469–478. K. W. Wyrwich, W. M. Tiemey, and F. D. Wolinsky, Using the standard error of measurement to identify important changes on the Asthma Quality of Life Questionnaire. Qual. Life Res. 2002; 11: 1–7. G. G. Miller, The magic number seven plus or minus two: some limits on our capacity for processing information. Psychol. Rev. 1956; 63: 81–97. J. T. Farrar et al., Clinical importance of changes in chronic pain intensity measured on an 11-point numerical pain rating scale. Pain 2001; 94: 149–158. F. Angst, A. Aeschlimann, and G. Stucki, Smallest detectable and minimal clinically important differences of rehabilitation intervention with their implication for required sample sizes using WOMAC and SF-36 quality of life measurement instruments in patients with osteoarthritis of the lower extremities. Arthrit. Care Res. 2001; 45: 384–391. D. Cella et al., What is a clinically meaningful change on the Functional Assessment of Cancer Therapy-Lung (FACT-L) Questionnaire?

Results from Eastern Cooperative Oncology Group (ECOG) Study 5592. J. Clin. Epidemiol. 2002; 55: 285–295. 73. C. McHorney and A. Tarlov, Individualpatient monitoring in clinical practice: are available health status measures adequate? Qual. Life Res. 1995; 4: 293–307. 74. A. L. Stewart, S. Greenfield, and R. D. Hays, Functional status and well-being of patients with chronic conditions: results from the medical outcomes study. JAMA 1989; 262: 907–913. 75. F. Pukelsheim, The three sigma rule. Amer. Stat. 1994; 48: 88–91. 76. E. F. Juniper, G. H. Guyatt, and D. H. Feeny, Measuring quality of life in childhood asthma. Qual. Life Res. 1996; 5: 35–46. 77. G. Guyatt, E. F. Juniper, S. D. Walter, L. E. Griffith, and R. S. Goldstein, Interpreting treatment effects in randomized trials. BMJ 1998; 316: 690–693. 78. O. Chassany et al., Patient-reported outcomes: the example of health-related quality of life - a European guidance document for the improved integration of health-related quality of life assessment in the drug regulatory process. Drug Inf. J. 2002; 36: 209–238. 79. C. Speilberger, State-Trait Anxiety Inventory:STAI (Form Y). Palo Alto, CA: Consulting Psychologists Press, Inc., 1983. 80. L. Radloff, The CES-D scale: a self-report depression scale for research in the general population. Appl. Psychol. Meas. 1977; 1: 385–481. 81. D. M. McNair, M. Lorr, and L. F. Droppleman, Profile of mood states manual. San Diego, CA: EdiTS, 1992. 82. R. D. Hays, R. Anderson, and D. Revicki, Psychometric considerations in evaluating health-related quality of life measures. Qual. Life Res. 1993; 2: 441–449. 83. G. R. Norman, P. W. Stratford, and G. Regehr, Methodological problems in the retrospective computation of responsiveness to change: the lessons of Cronbach. J. Clin. Epidemiol. 1997; 50(8): 869–879. 84. C. E. Schwartz and M. A. G. Sprangers, Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Sci. Med. 1999; 48: 1531–1548. 85. M. T. Moser, J. Weis, and H. H. Bartsch, How does regression to the mean affect thresholds of reliable change statistics? Simulations and examples for estimation of true change in

CLINICAL SIGNIFICANCE cancer-related quality of life. Qual. Life Res. 2002; 11: 669. 86. C. McHorney, Generic health measurement: past accomplishments and a measurement paradigm for the 21st century. Ann. Int. Med. 1997; 127: 743–750. 87. P. W. Stratford, J. Binkley, and D. L. Riddle, Sensitivity to change of the Roland-Morris Back Pain Questionnaire: part 1. Phys. Ther. 1998; 78: 1186–1196.

11

12

Table 2. Distribution-Based Methods of Determining Change Reference

HRQOL evaluated in relation to:

Paired t-statistic

62

Standard error of the mean change

Growth curve analysis

63

Standard error of the slope

Effect size

22, 64

Pre-test standard deviation

Calculation


x1 − x0  (di − d)2 n(n−1)

B √ V




x1 − x0  (x0 − x0 )2 n−1

Advantages

Disadvantages

None

Increases with sample size

Not limited to pre-test and post-test scores Uses all of the available data

Increases with sample size

Standardized units

Benchmarks for interpretation Independent of sample size Standardized response mean

Responsiveness statistic

44

65

Standard deviation of change

Standard deviation of change in a stable group







x1 − x0  (di − d)2 n−1

x1 − x0  (di stable − dstable )2 n−1

Standardized units

Requires large sample sizes Assumes data missing at random Decreases with increased baseline variability of sample Does not consider variability of change May vary widely among samples Varies as a function of effectiveness of treatment

Independent of sample size Based on variability of change Standardized units

More conservative than effect size Independent of sample size Takes into account spurious change due to measurement error

Data on stable subjects frequently not available

CLINICAL SIGNIFICANCE

Method

Table 2. (continued) Method Standard error of measurement

Reliable change index

Reference 7

6

HRQOL evaluated in relation to: Standard error measurement

Standard error of the measurement difference

Calculation


x1 − x0 

√ (x0 − x0 )2 ( 1 − r) (n − 1)

x1 − x0  2(SEM)2

Advantages

Disadvantages

Relatively stable across populations

Assumes measurement error to be constant across the range of possible scores

Takes into account the precision of the measure Cutoffs based on confidence intervals Relatively stable across populations

Reprinted with permission from Crosby et al. (4).

CLINICAL SIGNIFICANCE

Takes into account precision of measure Cutoffs based on confidence intervals

Assumes measurement error to be constant across the range of possible scores

13

14

CLINICAL SIGNIFICANCE

CLINICAL SIGNIFICANCE

15

16

CLINICAL SIGNIFICANCE

CLINICAL SIGNIFICANCE

17

18

– Is the questionnaire appropriate given the research objective and the rationale for QOL assessment? – Is the questionnaire appropriate given the domains included and in light of the disease and population characteristics? – Is the questionnaire reliable and valid? Is this information reported in the article? – Is the questionnaire responsive to change? Is this information reported in the article?

Is the QOL questionnaire relevant, reliable, valid, and responsive to change?

– What are their disease (e.g., tumor type), treatment (e.g., duration), sociodemographic and cultural (e.g., age, ethnicity), and behavioral (e.g., alcohol use) characteristics? – To what extent are the QOL data applicable to your patients? – Is actual QOL status of individual patients reported (e.g., by providing confidence intervals, standard deviations, subgroup data, individual data plots), thus documenting the amount of individual variation in response to treatment?

What are the characteristics of the population for whom changes in QOL are reported?

– Is the sample size appropriate for the research questions (e.g., by providing a power calculation)? – Is a rationale and/or source for the anticipated effect size specified?

Is the study adequately powered?

– Is QOL assessed at appropriate times to document treatment course, clinical events, and post-treatment effects? – Are standard research design procedures followed (e.g., avoidance of respondent burden, collection of data prior to treatment or consultation)? – Is the timing of the QOL assessments similar across treatment arms?

– Is a baseline assessment included? – Is QOL assessed at appropriate times for determining minimally important change given the natural course of the disease? – Is QOL assessed long enough to determine a clinical effect, taking disease stage into account?

Are the timing and frequency of assessments adequate?

– Are patients’ baseline QOL scores close to the extremes of the response scale? Do the treatment groups differ in baseline QOL?

– Is the questionnaire appropriate given practical considerations (e.g., regarding respondent burden and the availability of different language versions)?

Table 4. Checklist for Assessing Clinical Significance over Time in QOL

– Do the tabular and graphical presentations take the problems inherent in the data into account (e.g., presence of floor and ceiling effects, patient attrition)? – Are the data appropriately analyzed (e.g., are all time points included, are missing data taken into account, are pre-treatment co-variates included)? – Does the article provide sufficient information on the statistical models selected?

– Are the data presented in a meaningful and suitable way enabling an overview of QOL changes over time?

How are multiple time-points handled?

– Is the adopted approach of handling multiplicity explicitly described? – Which approach is taken: limiting the QOL outcomes, use of summary measures, adjustment of p-values, and/or multivariate statistical analysis and modeling? – Did the interpretation of the results take the problem of multiple outcomes into account?

How are multiple QOL outcomes addressed in analyses?

– Does the power calculation take into account: the scale range of the anticipated effect, the score distribution (i.e., magnitude and form), the number of outcome measures, and research hypothesis (i.e., equivalence versus difference)?

19

Reprinted with permission from Sprangers et al. (34).

– Does the article report the reasons for missing questionnaires? – Is there an association between patients’ health status and missing QOL data? – If patients with incomplete data are excluded from the analysis (e.g., by using complete case methods), does the article document that these are nonignorable missing data?

– Are missing data handled adequately? – Does the article indicate how missing items within a questionnaire are handled? – Does the article report the number of missing questionnaires at each scheduled assessment?

Can alternative explanations account for the observed change or lack of observed change? Are dissimilar baseline characteristics adequately accounted for? – Is the baseline QOL score used as a co-variate?

Table 4. (continued)

– Are changes in patient’s internal standards, values, and/or the conceptualization of QOL explicitly measured? – Are insignificant or small changes in QOL reported despite substantial changes in patient’s health status (i.e., deterioration or improvement)?

Did the patient’s QOL perspective change over time?

– Is observed survival difference combined with QOL in evaluating change? – If patients have died in the course of the study, is mortality accounted for in the evaluation of QOL? – Are summary indices (e.g., QALYs, Q-TWiST) or imputation techniques used?

– In cases of non-ignorable missing data, are several analytical approaches presented to address possible bias in conclusions based on this QOL data set?

– To what extent is the statement of clinical importance appropriate and empirically warranted?

– Does the article provide some guidance regarding the clinical importance of the observed change in QOL?

How is statistical significance translated into meaningful change?

– How likely is it that patients have changed their internal standards, values, and/or their conceptualization of QOL as a result of adaptation to deteriorating or improving health?

CLINICAL TRIAL MISCONDUCT

the present situation in the United States, where handling misconduct has become routine, and the United Kingdom, where efforts to establish consensus have been fragmentary and fitful, serves as a sharp reminder of the importance of facing up to and dealing with this unpleasant side of science. It also confirms that scientific institutions are unlikely to deal with the problem, except on a case-by-case basis, unless forced to do so by their governmental paymasters (5).

DRUMMOND RENNIE University of California Institute for Health Policy Studies San Francisco, California

1

THE SCOPE OF THIS ARTICLE

The purpose of clinical trials of interventions is to obtain unbiased, relevant, and reliable information on the value of the interventions (1). Many ethical problems can occur during a trial. It can be designed incorrectly, for example, so that the results of a placebo-controlled trial on a me-too drug are worthless to the clinician who wants to know whether it represents an improvement over the best already available. Randomization can be broken and concealment made inadequate. The trial can be under-powered so that the research participants are put at risk without any hope of obtaining results of value to mankind. The trial may be stopped early by the sponsor for financial reasons (2), the results of the trial can be buried, or the same results can be reported repeatedly by different authors without cross-referencing (3), resulting in publication bias or distortion of the published literature. These and other such actions are, to varying degrees, unethical, and all of them have at one time or another been labeled research or scientific misconduct (4), but in the United States at least, they fall outside the federal definition of research misconduct and no one would be found guilty of misconduct for such acts alone. It is particularly useful to look at the history and present situation of scientific misconduct in the United States to see how this came about, because it was in the United States that the first well-publicized cases occurred, and the United States was the first country to take decisive action to deal with it. This action took the part of forming a consensus that a problem to be solved existed; reaching a definition; developing a process to be followed when allegations arose; and modifying the process in response to experience and new case law. The contrast between

2 WHY DOES RESEARCH MISCONDUCT MATTER? Crooked research, clinical or otherwise, corrupts the record, leading other scientists down false trails. In clinical trials, it inevitably distorts the apparent efficacy of interventions, may falsely raise the expectations of patients and physicians alike, and, when uncovered, leads to public anger, skepticism, and loss of confidence in research in general and of all research institutions. Within trials and within institutions, misconduct, if not dealt with, leads to loss of morale, cynicism, and resignations of honest investigators. It often leads to the ostracism of whistleblowers, even if they are correct in their accusations, and to deep divisions within research teams. Clinical trials are exceedingly expensive, so efforts to correct tainted research may be thwarted because replication may be too costly to organize. Finally, the process of investigation is expensive and involves numerous busy people, with more pressing claims on their time, in extra work. 3

EARLY CASES

The modern history of research misconduct started in 1974 at Sloan-Kettering with Summerlin and his painted mouse (6–12). Over the next 15 years, numerous spectacular cases occurred at major U.S. research universities, later summarized by numerous books and articles. These cases were reported fully by the media, as were the responses of the scientific establishment. In general,

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

CLINICAL TRIAL MISCONDUCT

scientists, accustomed to trusting their colleagues, had great difficulty imagining that colleague would ever break that trust. They denied fraud could occur, or maintained, in the absence of any evidence, that those who committed fraud were extreme aberrations or sick individuals. Few seemed to accept the common sense view that some percent of scientists, as with any other profession, were likely to be fraudulent. ‘‘The response of each research institution varied but was only too often characterised by circling the wagons, denial, and cover up. Under the eyes of the press, each institution would hurriedly patch together its own process, assembling ad hoc panels, sometimes with glaring conflicts of interest. The results were frequently slow, bungled, idiosyncratic, and unfair to almost everyone’’ (5). Politicians became involved in 1981 when then Congressman Al Gore held the first of some dozen hearings at which delinquent scientists were called on the carpet, whistleblowers testified on how they had been abused, and important institutional administrators denied to openly skeptical audiences that any problem existed at all (5, 13–15). At the close of the first hearing, Gore could not ‘‘avoid the conclusion that one reason for the persistence of this type of problem is the reluctance of people high in the science field to take these matters seriously’’ (13). In the face of inertia, denial and opposition on the part of the scientific establishment and an increasingly rancorous atmosphere, Congress, using the justification that it was responsible for overseeing how tax dollars were spent, and that faked research was a fraud against the public purse, forced the setting up of an Office of Scientific Integrity (OSI) in the National Institutes of Health (NIH) in 1989 (6). Rules and a definition of misconduct were promulgated, and the process for investigation and adjudication laid down. Although governmental regulations did not apply to those on non-governmental money, the new regulations applied to all institutions receiving money from the government, so this effectively meant that all biomedical research institutions in the United States had to draw up institutional rules that complied with the government’s definition of misconduct and procedures for responding to it. Although

those funded by the National Science Foundation operated under a slightly different system, the governmental rules became, de facto, the universal rules within biomedicine. After several years, the initiative and the OSI were moved outside the NIH, but operated only within the Public Health Service. The OSI transmogrified into the more lawyerheavy Office of Research Integrity (ORI). It provoked bitter attacks from scientists, politicians, accused persons and their attorneys, whistleblowers, and the press. The procedures were said to be amorphous, inconsistent, illegal, and poorly articulated, the process too quick, too slow, too timid, too aggressive, and often bungled. The accused and whistleblower/accusers both complained of being deprived of their rights (6, 15). The OSI/ORI operated on a collegial ‘‘scientific dialog’’ model, intended to keep the process in the hands of scientists, rather than lawyers. But as cases accumulated and the OSI/ORI lost them when they were appealed to the courts, this was found to be legally flawed and unfair to the accused, and was abandoned for a process that followed the stricter rules of administrative law. Meanwhile, some whistleblowers, exasperated by the confusion, pursued an alternative route provided by the False Claims Act, which qualified them to share in a portion of any grant monies recovered through the courts rather than through the designated institutional routes (6, 15). 4 DEFINITION The process of developing the regulations had been greatly facilitated by a successful series of meetings, endorsed by the American Bar Association and scientific societies, notably the American Association for the Advancement of Science and attended by scientists, administrators, politicians, and lawyers. The most fundamental issue, the definition, was a bone of contention from the start. In general, scientists, well aware of the problems caused by breakdowns in the complex interpersonal relationships formed during team research, and afraid of the devastating effect of malicious and unfounded accusations, attempted to limit the definition to the relatively clear

CLINICAL TRIAL MISCONDUCT

crimes of fabrication, falsification, and plagiarism. Lawyers and politicians, who were mainly lawyers, were concerned to separate definitely criminal acts from uncouth and uncollegial behavior, (to separate, in Tina Gunsalus’ phrase, ‘‘the crooks from the jerks’’) and to separate those acts, such as sexual harassment, which could occur outside the scientific environment and for which laws already existed. Abused whistleblowers, (many of whom, although correct in their accusations, had ended up losing their jobs because of the universal tendency on the part of friends to rally around their accused colleagues, no matter what the evidence) often felt that their rights had been violated and usually sought a wider definition. The precise details of each of their cases differed, so there was pressure from them to include a large number of acts, from duplicate publication to impolite behavior, in the definition of research misconduct. Institutions were concerned that they have control over disciplining their faculty, while at the same time not being made to sink under the weight of elaborate new rules that were expensive to enforce. Everyone wanted clarity because, without a clear and universal definition, everyone had to invent their own rules, leading to confusion and unfairness. It was decided to use the term ‘‘misconduct’’ rather than ‘‘fraud,’’ because a finding of fraud, usually financial, needed several specific requirements to be proved, which would have been impossible to sustain in the case of purely scientific irregularities. The definition of scientific misconduct adopted by the OSI in 1989, was ‘‘Fabrication, falsification, plagiarism or other practices that seriously deviate from those that are commonly accepted within the scientific community for proposing, conducting, or reporting research’’ (16). Although everyone agreed that fabrication, falsification, and plagiarism were antithetical to good science, large scientific organizations immediately objected to the inclusion of ‘‘other practices . . . ’’ on the grounds that this would include, and so inhibit, breakthrough, unconventional science. They wanted the definition limited to fabrication, falsification, and plagiarism, what came to be called ‘‘FF&P.’’ Although this argument

3

seemed spurious to many, underlying the concern was the understandable fear that scientists would be found guilty of practices they had no idea could be wrong. Against this notion, others argued that many cases were not covered by FF&P, for example, stealing work during peer review. There could not be a definition that implied that such behaviors were not wrong. 5

INTENT

Gruber said: ‘‘The power and the beauty of science do not rest upon infallibility which it has not, but on corrigibility without which it is nothing’’ (17). No one could undertake the risky and uncertain business of science if every error were to be construed as misconduct. Distinguishing error from misconduct requires making a judgment about intent. ‘‘Misconduct’’ in law meals the ‘‘willful’’ trangression of a definite rule. Its synonyms are ‘‘misdemeanor, misdeed, misbehavior, delinquency, impropriety, mismanagement, offense, but not negligence or carelessness’’ (18), which is one reason why the definition decided on in 1999 at a consensus conference in Edinbugh, Scotland, (‘‘Behavior by a researcher, intentional or not, that falls short of good ethical and scientific standards.’’) is doomed (19). The law has a long history of judging intent, and that is what, in addition to deciding on the facts, panels looking into scientific misconduct must do. 6 WHAT SCIENTIFIC MISCONDUCT WAS NOT What became clear from the battle was that a great many examples of egregious conduct had to be dealt with, if at all, by different mechanisms. Thus, failure to obtain informed consent, or theft of laboratory funds, or mistreatment of laboratory animals, or accepting money to do fake research while billing the government for the costs, were serious offenses covered by laws concerning conduct of trials, laws against theft, laws governing animal research, or anti-kickback laws. Similarly, inserting the results of fake experiments as supporting evidence in grant applications to the government contravened the

4

CLINICAL TRIAL MISCONDUCT

law that, in the United States at least, it is illegal to lie intentionally to the government. For an excellent discussion of legal issues in research, the reader is referred to Kalb and Koehler (20). All sorts of other unethical behaviors were left to be dealt with by the institutions, the scientific community, and the scientific journals. So, for example, datadredging, duplicate publication, ghost and guest authorship, failure to publish, starting trials in the absence of equipoise, failure to share data, or failure to reveal massive financial conflicts of interest, and many other practices damaging to science and exasperating to scientists were either by implication condoned or left to the community to sanction. In this chapter, such issues or problems in clinical trials such as mismanagement of funds or improper and biased statistical analysis will not be addressed. 7

THE PROCESS

The 1989 regulations detailed a process to be followed whenever an allegation was received, either by an official at the research institution, or by the OSI/ORI (henceforth called the ORI), which stipulated an initial inquiry by the institution to see whether there might be any merit to the accusation, and, if likely, a full investigation. This investigation was to be carried out in the institutions by people with no conflict of interest, with the results to be forwarded to the ORI. In addition, the ORI was given a monitoring function to assure the public that the all research institutions were taking the matter seriously and complying with the regulations. The most important change, made in 1993, was the introduction of an appeals process before a Research Integrity Adjudication Panel appointed by the Departmental Appeals Board of the Department of Health and Human Services (DHHS). The process now is that the ORI reviews the findings and either (rarely) refers to the DHHS Office of Inspector General, or almost always, refers to the Assistant Secretary of Health for accepting or rejecting the institutions’ recommendations. If a finding of misconduct occurs, the ORI would impose sanctions or negotiate an agreement with the

accused. The accused could then appeal to the Departmental Appeals Board, where the accused scientist has a right to be represented by counsel, to discovery of all evidence used to convict, to cross examine witnesses, including the accuser, and to participate in the hearing (21). An independent analysis of policies and procedures at institutions approved by the ORI, done in 2000, has shown considerable variation between institutions in all phases of the handling of cases of suspected misconduct before the report goes to the ORI (22), and low levels of due process. For example, only 32% had policies requiring the accused have access to all the evidence, and only 21% the right to present witnesses (22). Mello and Brennan have criticized the process as providing insufficient safeguards and due process for accused scientists, for whom a finding of misconduct would be devastating (21). 8 THE PAST DECADE Despite an apparently clear and workable system, the controversy continued and, in the early 1990s, several high-profile cases were decided against government science agencies and their ways of proceeding. To try to restore calm, the U.S. Public Health Service set up a Commission on Research Integrity (named the Ryan Commission, after its chair, Kenneth J. Ryan of Harvard Medical School). The Commission heard testimony in 15 meetings across the United States from a large number of witnesses, including scientists and their organizations, whistleblowers, attorneys, institutions, the press, interested citizens, and government officials. Their report was completed in November 1995 (23). The Commission recommended that the definition of research misconduct should be ‘‘based on the premise that research misconduct is a serious violation of the fundamental principle that scientists be truthful and fair in the conduct of research and the dissemination of its results’’ (23). ‘‘Research misconduct is significant misbehavior that improperly appropriates the intellectual property or contributions of others, that intentionally impedes the progress

CLINICAL TRIAL MISCONDUCT

of research, or that risks corrupting the scientific record or compromising the integrity of scientific practices. Such behaviors are unethical and unacceptable in proposing, conducting, or reporting research or in reviewing the proposals or research reports of others’’ (23). The commission specifically included within this definition ‘‘misappropriation,’’ which included plagiarism and making use of another’s ideas and words during peer review; ‘‘interference’’ with another’s research; and ‘‘misrepresentation’’ by reporting scientific data falsely. Its members having the advantage of hearing many actual cases, the Commission also included obstruction of investigations of research misconduct, and noncompliance with research regulations (23). Finally, the Commission, recognizing that whistleblowers represented an invaluable quality assurance mechanism, but that whistleblowers had to receive protection from retaliation lest they suffer damage to their careers as a result of their actions, presented an appendix ‘‘Responsible Whistleblowing: a Whistleblower’s Bill of Rights.’’ The reaction to the report varied from enthusiastic (Nature and the Lancet) to angry. The President of the Federation of American Societies for Experimental Biology (FASEB) wrote to the Secretary for Health and Human Services that the ‘‘Commission’s report is so seriously flawed that it is useless as a basis for policy making and should be disavowed . . . we find the definition to be unworkable, and therefore unacceptable’’ (24). He was quoted in the press as calling the report ‘‘an attack on American science,’’ (25) surprising words given the reliance the commission had put on the National Academy of Science’s own report (26, 27). It seems likely that the cause of this excessive reaction was the Whistleblower’s Bill of Rights, the insertion of which was interpreted not as attention to the plight of the accuser, but as a failure to protect the rights of the already well-protected accused. Once again, the bogeyman that regulation beyond FF&P would inhibit groundbreaking research was raised, this time to attack the Commission, even though in thousands of actual cases this had never once happened. In response to court decisions, changes to the process were introduced to allow more

5

protections for the accused and to bring the entire process more into line with the procedures set down by administrative law. Institutions became used to handling cases, the process was seen to be fairer, and the late 1990s were characterized by a decrease in the shrillness of the debate. On December 6, 2000, the United States issued the new, government-wide regulations defining research misconduct and laying down the rules for investigation and adjudication of allegations of misconduct concerning research done with U.S. federal funds (Fig. 1) (28). As all important universities and research institutions receive such funds, these regulations, broadly speaking, became institutional rules, although institutions are allowed to have their own additional rules if they wish to impose a higher internal standard (28). The definition in the regulations (see Fig. 1) restricted the definition to FF&P; required intent to be taken into account; and stipulated that the allegation must be proven by a ‘‘preponderance of the evidence.’’ Relative quiet has descended on the community as institutions have successfully handled cases in an entirely routine fashion. The ORI now serves chiefly an educational and monitoring function, whereas the accused is now assured of the full protections of the law. 9

LESSONS FROM THE U.S. EXPERIENCE

Among the lessons to be gleaned from the turbulent experience in the United States are the following. Despite numerous very public scandals, and enormous publicity, building consensus on such an emotional topic is hard and takes many years. With notable exceptions, scientists are naturally loath to cede any of their authority, and good scientists often find it hard to imagine that anyone could break the bonds of trust that allow science to function. Unless pushed by those holding the purse strings, they will do little to police their own profession and tend to challenge, resist, and circumscribe attempts from outside to impose rules. They will tend to be suspicious of attempts to protect whistleblowers, who remain by far the most useful source for reporting misconduct and who often, perhaps usually, suffer for their efforts on behalf

6

CLINICAL TRIAL MISCONDUCT

I. Research § Misconduct Defined Research misconduct is defined as fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting research results. • Fabrication is making up data or results and recording or reporting them. • Falsification is manipulating research materials, equipment, or processes, or changing or omitting data or results such that the research is not accurately represented in the research record.‡ • Plagiarism is the appropriation of another person’s ideas, processes, results, or words without giving appropriate credit. • Research misconduct does not include honest error or differences of opinion. II. Findings of Research Misconduct A finding of research misconduct requires that: • There be a significant departure from accepted practices of the relevant research community; and • The misconduct be committed intentionally, or knowingly, or recklessly; and • The allegation be proven by a preponderance of evidence.
No rights, privileges, benefits, or obligations are created or abridged by issuance of this policy alone. The creation or abridgment of rights, privileges, benefits, or obligations, if any, shall occur only on implementation of this policy by the federal agencies. † Research, as used herein, includes all basic, applied, and demonstration research in all fields of science, engineering, and mathematics, which includes, but is not limited to, research in economics, education, linguistics, medicine, psychology, social sciences, statistics, and research involving human subjects or animals. ‡ The research record is the record of data or results that embody the facts resulting from scientific inquiry, and includes, but is not limited to, research proposals, laboratory records, both physical and electronic, progress reports, abstracts, theses, oral presentations, internal reports, and journal articles. § The term ‘‘research institutions’’ is defined to include all organizations using federal funds for research, including, for example, colleges and universities, intramural federal research laboratories, federally funded research and development centers, national user facilities, industrial laboratories, or other research institutes. Independent researchers and small research institutions are covered by this policy.

Figure 1. Federal Policy on Research Misconduct
,‡

of science. Scientists have learned that the same body must not be investigator, prosecutor, and judge. Lastly, although scientific misconduct has to be defined and assessed by scientists, the threats to the livelihood of the accused are so great that no solution will work unless it is seen to be fully in accord with the law, the workings of which they are often ill-informed.

10

OUTSIDE THE UNITED STATES

The most thoughtful national response to the issue of scientific misconduct has been made in Denmark (29, 30). What is particularly interesting is that those who organized this did so before Denmark, in sharp contradistinction to the United States and other countries, had any important cases or scandals with which to deal. Their effort was from the start aimed at prevention as well as at setting down procedures for responding to allegations. For a full description and analysis, the reader is referred to ‘‘Scientific Dishonesty & Good Scientific Practice’’ published by the Danish Medical Research Council in 1992 (29, 30).

In the United Kingdom, the situation still seems similar to that in the United States 20 years ago, few lessons having been learned from abroad. Certain granting agencies (31–33) have well-established rules—as in the United States, their authority based on the need to account for funds disbursed. But little enthusiasm has been expressed for any general response, despite energetic pushing from the editors of the BMJ and Lancet, who, exasperated, founded the Committee on Publication Ethics (COPE). Editors of general medical journals are quite frequently made aware of suspect research, and are then called on to investigate—something they ordinarily do not have the mandate, authority, money, legal standing, time, or expertise to do. In the United States, editors simply convey the allegation to the researcher’s institution and act as witnesses. In the United Kingdom, there may be no one and no system to investigate, adjudicate, and sanction. COPE, which has reported annually since 1998, serves as a sounding board, looks into ethical maters, and, where necessary and possible, refers to the appropriate authority. Although the editors have repeatedly called for a national

CLINICAL TRIAL MISCONDUCT

7

body, very little progress has been made (34, 35). Elsewhere, many other countries, which, like the United Kingdom and South Africa, have poorly defined procedures, are struggling to deal with the problem are sweeping it under the table. Germany has had to contend with long-running and widely publicized cases and is attempting to frame regulations (36). In China, Beijing University has issued broad regulations that include, for example, ‘‘intentionally exaggerating the academic value and economic and social results of a research finding’’ (37).

The team rapidly discovered serious protocol deviations, poor documentation, nonexistent patients, failure to obtain ethical approval for the study from the institution, and no evidence of informed consent. They concluded that Bezwoda’s study was invalid (43). Among the consequences of this behavior were disappointment for thousands of women and their physicians and loss of confidence in clinical research. It is unclear what part Bezwoda’s colleagues played, but the continued day-to-day involvement of colleagues in all aspects of this research would surely have prevented this misconduct.

11 SCIENTIFIC MISCONDUCT DURING CLINICAL TRIALS

12

Many of the early cases of scientific misconduct were perpetrated by physicians under contract to pharmaceutical companies to follow-up patients and to help with post-marketing surveillance. Wells and Brock have described several cases (38, 39). Pharmaceutical companies were left with a quandary. Their audits had revealed clear misconduct, but until the 1990s they did not wish to antagonize the physicianperpetrators or their colleagues by prosecuting (39). Now that the public recognizes that scientific misconduct exists, companies are backed up by the medical community, so their reluctance to pursue cases has diminished. As with misconduct in other settings, distinguished researchers in large medical centers may be found guilty (40). The difficulties of dealing with misconduct during a trial when no standard procedures exist is well illustrated by Hoeksema et al. (41). Of recent cases of misconduct within a clinical trial, the most notorious concerns the South African oncologist Dr. Werner Bezwoda of the University of Witwatersrand in Johannesburg. Bezwoda’s clinical trial was alone in reporting striking benefit from high-dose chemotherapy and peripheral blood stem cell rescue in women with high-risk breast cancer (42). His results, presented in 1999, were so at variance with three other trials reported of the same therapy that the National Cancer Institute sent an audit team to South Africa to investigate.

AUDIT

Audit is an inefficient and costly system for policing science and should not in general be used. Clinical trials are different. In 1978, well before there were standard methods for investigating misconduct, numerous researchers at Boston University Medical Center reported being pressured to falsify and fabricate data in clinical trials in oncology by their chief. He, on the other hand, alleged that the falsifications, later shown to involve 15% of the data, had been perpetrated by his juniors (7). He also maintained that ‘‘there are certain types of studies that are almost beyond the limits of absolute surveillance’’ as they are so complex (7). As a consequence of the wide publicity given this case, the National Cancer Institute mandated on-sight audits by their cooperative groups. Subsequently, in 1985, Shapiro and Charrow, examining the results of U.S. Food and Drug Administration (FDA) audits showed a high prevalence of bungled and fraudulent research (44). In 1993, Weiss et al. reported on their audit system for the Cancer and Leukemia Group B, and found a very low rate (45). However, the finding of one case of misconduct and one of ‘‘gross scientific error’’ was made under conditions where everyone had been put on notice that there would be regular audits; where one of those audited had received a suspended prison sentence for misconduct; where three large centers with adverse audit results had simply been dropped from the study, and where everyone involved had made a very strong commitment

8

CLINICAL TRIAL MISCONDUCT

to the expensive and time-consuming task of audit (46). It was a routine audit that revealed misconduct by one investigator, Dr. Roger Poisson of Montreal, in one of the most significant of trials, the National Surgical Adjuvant Breast and Bowel Project (NSABBP). The ORI, eventually alerted by the National Cancer Institute, discovered 115 well-documented cases of fabrication and falsification, for which Poisson took responsibility, as well as falsifications in other studies (47). This episode was particularly unfortunate because it took years for the audit results to become known to the journals that had published articles that had included Poisson’s data, and journals and public learned of it via the Chicago Tribune (47). The scientists who later asserted that Poisson’s falsifications did not materially affect the results of the trial, so that informing public and journals was unimportant, were missing an essential point. Research is paid for by the public, who are deeply interested in the results of important trials in breast cancer. The legitimacy of researchers and their trials depends on the trust of the public, which this incident, the fact that it was unearthed by a reporter, and the massive publicity it aroused, damaged severely. The result was doubly unfortunate, given that it was the trial’s own audit that had first revealed to its leaders that a problem existed. As Brock has pointed out, the knowledge that audit will be routine and thorough, and that misconduct will be reported and prosecuted is likely to have a deterrant effect (39). Steward et al. have suggested that peer reviewed journals consider publication of the results of trials only when a program of random audits exists (48). Experience has shown that in widely-flung enterprises like multicenter clinical trials, careful attention to audit, conducted by senior investigators involved with the trial, is necessary if trials are to be credible. 13

CAUSES

A great deal of unsupported opinion has been published on the causes. The usual favorite is that scientists are driven to misconduct

because of the ‘‘publish or perish’’ atmosphere of the research life. This facile explanation takes no account of the fact that no correlation, positive or negative, exists between publication and misconduct; that the vast majority of scientists are honest, whatever their publication rates; that the pressure to publish has been shown to be exaggerated; and that at least one prominent scientist found guilty of fabrication went out of his way to deny this was the cause (13). Brock concludes that the motive is often simple greed, which, associated with a desire for advancement, may be a powerful causative factor. We are far from understanding the psychological processes working here, as is evidenced by, for example, the case of Karen Ruggiero, whose extensive fabrications came as a complete shock to her psychologist superior colleague (49). It is to be hoped that the large amount of research stimulated by the ORI’s grants and research conferences will shed light on this topic. In the meantime, it seems likely that ignorance of scientific mores, poorly enunciated standards, poor or absent teaching, and inadequate individual supervision of young researchers are far more important. 14 PREVALENCE We do not know how commonly misconduct or misconduct in trials occurs. The publication of data from audit committees such as that by Weiss et al. (45) is useful, but such audits reflect only the experience of the trials with strict audit and the highest standards. Pharmaceutical companies seem to be increasing their audit of the performance of trials conducted in the community, but their combined experience has not been gathered and reported. The annual reports of the ORI (http://ori.hhs.gov) give some indication of the number of cases of misconduct, not confined to clinical trials, reported to the ORI and their disposition. In 2002, the following types of allegations were reported by institutions in the United States: fabrication: 45; falsification: 58; plagiarism: 27; ‘‘other’’: 33. During that year, there were 67 initial inquiries and 31 full investigations (50). No data exist to show that misconduct is more prevalent in one country than another.

CLINICAL TRIAL MISCONDUCT

It is simplest to assume that clinical scientists everywhere lie on a curve reaching from the obsessively honest through to the serially dishonest, and that as scientific or medical degrees do not come with any guarantee of probity, there will always be those who commit and report fraudulent research. The idea of an experimental, short-term, confidential audit, to be reported only in aggregate, in order to establish the prevalence of gross fabrication was first proposed in 1988 (51). But when the ORI recently funded a survey of misconduct and other forms of poor behavior, there were immediate and forceful protests from the same scientific bodies that had fought so hard to limit the definition of misconduct to FF&P (52, 53). There were legitimate concerns about the scientific quality of the projected survey. But in an editorial, the journal Nature described the defensiveness of the actions of the large scientific societies as ‘‘heads-in-the-sand,’’ and a ‘‘good impersonation of . . . . special interests with something to hide’’ (54). Whether this is true or not, given these official attitudes, it will be a long time before we have any clear idea of the prevalence of misconduct. 15

PEER REVIEW AND MISCONDUCT

A few cases of plagiarism of ideas and words have occurred during peer review. Given the privileged and confidential nature of manuscripts sent to reviewers, the possibility of plagiarism or breaking confidentiality is a particularly disturbing one, and one that editors must do their best to prevent by reminding reviewers of their responsibilities. Editors have a duty to report breaches of confidentiality and act as witnesses to investigatory panels. For a fuller account, the reader is referred to Reference 55. Peer review operates on the assumption that the authors are telling the truth, and is a very poor detector of fraudulent research, unless, as sometimes happens, the reviewer sees his or her own words plagiarized. 16

RETRACTIONS

The case of Poisson mentioned above, and a great many other serious cases, have amply

9

demonstrated that a single incident of misconduct should immediately mean that the validity of all the guilty scientist’s other research cannot be assumed, and all of it must be scrutinized carefully. The University of California San Diego (UCSD) set the standard after the discovery, in 1985, of duplicate data in two studies published by Robert A. Slutsky. Over the next year, investigating panels held Slutsky’s co-authors responsible for the validity of every part of every one of 137 articles published in 7 years (56). Overall, 77, including reviews, were judged valid, 48 were ‘‘questionable’’ (and therefore unciteable), and 12 were deemed at that time ‘‘fraudulent’’ (this event occurred before federal definitions had been promulgated). Nevertheless, most of the journals asked to retract the articles refused to do so, with the result that Slutsky’s fraudulent articles were being cited as if correct years later (57). Moreover, 2 years after a high-profile inquiry in Germany looking into the work of Friedhelm Herrmann and Marion Brach found 94 published papers to include manipulated data, 14 of the 29 journals publishing the articles had not published retraction notices concerning any of the articles (58). Journal editors tend to be cowed by threats either from the guilty scientist or from the innocent co-authors, each of whom has an interest in preventing the appearance of a retraction. The editor of Nature has written feelingly of the problems of retracting seven papers all sharing the first author, Jan Hendrik Sch¨on, who maintained throughout that his work should stand unless faced with hard evidence to the contrary (59). Despite this fact, it is everyone’s duty to correct the literature, and an increased resolve on the parts of editors, backed up by strong policies from editorial societies is in evidence. The International Committee of Medical Journal Editors policy reads: ‘‘The retraction or expression of concern, so labeled, should appear on a numbered page in a prominent section of the journal, be listed in the contents page, and include in its heading the title of the original article. It should not simply be a letter to the editor. Ideally, the first author should be the same in the retraction as in the article, although under certain

10

CLINICAL TRIAL MISCONDUCT

circumstances the editor may accept retractions by other responsible people. The text of the retraction should explain why the article is being retracted and include a bibliographic reference to it’’ (60). It is the duty of research institutions and a guilty scientist’s co-authors to check the scientist’s entire published work. It is also their duty, and that of the relevant journals, to issue retractions when the reports are found to contain fraudulent work (61, 62). The difficulties of doing this are shown in the case of Poehlman, who was sentenced to serve a year in prison for his misconduct (62). These difficulties are much greater when the work has originated in a country with no process for dealing with misconduct. 17

PREVENTION

From the start, it was recognized that many cases of misconduct were associated with apparent ignorance on the part of young researchers of the mores and standards of good research, and others with a complete breakdown in the assumed mentor-trainee relationship, resulting in ineffective supervision, monitoring, or training of young researchers. Add to that the fact that clinical trials demand rigorous adherence to sometimes elaborate protocols on the part of many individuals, some of whom, although quite senior, may be new to research, and many of whom are from different cultural backgrounds. The responsibility on trial leaders to educate, train, and monitor their colleagues is therefore considerable, but unavoidable. Every effort should be made not to put inordinate pressure on people such as clinical coordinators to recruit patients (63) or to distort the whole process with excessive monetary incentives. Audit and putting people on notice about the consequences of misconduct are necessary, but building up close relationships within a well-trained team may well turn out to be more important. REFERENCES 1. I. Chalmers, Unbiased, relevant, and reliable assessments in health care. BMJ 1998; 317: 1167–1168.

2. B. M. Psaty and D. Rennie, Stopping medical research to save money: a broken pact with researchers and patients. JAMA 2003; 289: 2128–2131. 3. D. Rennie, Fair conduct and fair reporting of clinical trials. JAMA 1999; 282: 1766–1768. 4. I. Chalmers, Underreporting research is scientific misconduct. JAMA 1990; 263: 1405–1408. 5. D. Rennie, Dealing with research misconduct in the United Kingdom. An American perspective on research integrity. BMJ 1998; 316(7146): 1726–1728. 6. D. Rennie and C. K. Gunsalus, Scientific misconduct. New definition, procedures, and office—perhaps a new leaf. JAMA 1993; 269(7): 915–917. 7. W. Broad and N. Wade, Betrayers of the Truth—Fraud and Deceit in the Halls of Science. New York: Simon & Schuster, 1982. 8. M. C. LaFollette, Stealing Into Print—Fraud, Plagiarism, and Misconduct in Scientific Publishing. Berkeley, CA: University of California Press, 1992. 9. S. Lock and F. Wells (eds.), Fraud and Misconduct in Biomedical Research. 1st ed. London: BMJ Publishing Group, 1993. 10. S. Lock and F. Wells (eds.), Fraud and Misconduct in Biomedical Research. 2nd ed. London: BMJ Publishing Group, 1996. 11. S. Lock, F. Wells, and M. Farthing (eds.), Fraud and Misconduct in Biomedical Research. 3rd ed. London: BMJ Publishing Group, 2001. 12. A. Kohn, False Prophets. Fraud and Error in Science and Medicine. Oxford: Basil Blackwell Ltd., 1986. 13. Fraud in Biomedical Research. In: Hearings Before the Subcommittee on Investigations and Oversight of the Committee on Science and Technology, 1981. 14. Fraud in NIH Grant Programs. In: Hearings Before the Subcommittee on Energy and Commerce, 1988. 15. D. Rennie and C. K. Gunsalus, Regulations on scientific misconduct: lessons from the US experience. In: S. Lock, F. Wells, and M. Farthing (eds.), Scientific Fraud and Misconduct. 3rd ed. BMJ Publishing Group, 2001, pp. 13–31. 16. U.S. Department of Health and Human Services, Public Health Service, Responsibilities of awardee and applicant institutions for dealing with and reporting possible misconduct

CLINICAL TRIAL MISCONDUCT in science: final rule. Fed. Reg. 1989; 54: 32446–32451. 17. New York Times. July 22, 1975. 18. B. Mishkin, The Investigation of scientific misconduct: some observations and suggestions. New Biologist 1991; 3: 821–823. 19. Joint Consensus Conference on Misconduct in Biomedical Research. In: Royal College of Physicians of Edinburgh, 1999. 20. P. E. Kalb and K. G. Koehler, Legal issues in scientific research. JAMA 2002; 287: 85–91. 21. M. M. Mello and T. A. Brennan, Due process in investigations of research misconduct. N. Engl. J. Med. 2003; 349: 1280–1286. 22. CHPS Consulting, Analysis of Institutional Policies for Responding to Allegations of Scientific Misconduct. Rockville, MD: Office of Research Integrity. CHPS Consulting, 2000. 23. Integrity and Misconduct in Research. Report of the Commission on Research Integrity to the Secretary of Health and Human Services, the House Committee on Commerce and the Senate Committee on Labor and Human Resources.(* the Ryan Commission). (1995). (online). Available: http://gopher.faseb.org/opar/cri.html;. 24. R. A. Bradshaw, Letter to Secretary of Health and Human Services Donna Shalala. January 4, 1996. 25. B. Goodman, Scientists are split over finding of Research Integrity Commission. The Scientist 1996;Sect. 1. 26. Responsible Science—Ensuring the Integrity of the Research Process, vol. I. Washington, DC: National Academy Press, 1992. 27. Responsible Science—Ensuring the Integrity of the Research Process, vol. II. Washington, DC: National Academy Press, 1993. 28. Office of Science and Technology Policy, Federal policy on research misconduct. Fed. Reg. 2000; 76260–76264. 29. D. Andersen, L. Attrup, N. Axelsen, and P. Riis, Scientific Dishonesty & Good Scientific Practice. Copenhagen: Danish Medical Research Council, 1992. 30. P. Riis, The concept of scientific dishonesty: ethics, value systems, and research. In: S. Lock, F. Wells, and M. Farthing (eds.), Fraud and Misconduct in Biomedical Research. 3rd ed. London: BMJ Publishing Group, 2001, p. 268. 31. The Wellcome Trust. (2002). Guidelines on Good Research Practice. (online). Available: http://www.wellcome.ac.uk/en/1/ awtvispolgrpgid.html.

11

32. R. Koenig, Wellcome rules widen the net. Science 2001; 293: 1411–1413. 33. I. Evans, Conduct unbecoming— the MRC’s approach. BMJ 1998; 316: 1728–1729. 34. R. Smith, The need for a national body for research misconduct—nothing less will reassure the public. BMJ 1998; 316: 1686–1687. 35. M. Farthing, R. Horton, and R. Smith, UK’s failure to act on research misconduct. Lancet 2000; 356: 2030. 36. A. Bostanci, Germany gets in step with scientific misconduct rules. Science 2002; 296: 1778. 37. D. Yimin, Beijing U. issues first-ever rules. Science 2002; 296: 448–449. 38. F. Wells, Counteracting research misconduct: a decade of British pharmaceutical industry action. In: S. Lock, F. Wells, and M. Farthing (eds.), Fraud and Misconduct in Biomedical Research. 3rd ed. London: BMJ Publishing Group, 2001, pp. 64–86. 39. P. Brock, A pharmaceutical company’s approach to the threat of research fraud. In: S. Lock, F. Wells, and M. Farthing (eds.), Fraud and Misconduct in Biomedical Research. 3rd ed. London: BMJ Publishing Group, 2001, pp. 89–104. 40. J. H. Tanne, FDA limits research of former AHA president for submitting false information. BMJ 2002; 325: 1377. 41. H. L. Hoeksema et al., Fraud in a pharmaceutical trial. Lancet 2000; 356: 1773. 42. R. Horton, After Bezwoda. Lancet 2000; 355: 942–943. 43. R. B. Weiss et al., High-dose chemotherapy for high-risk primary breast cancer: an on-site review of the Bezwoda study. Lancet 2000; 355: 999–1003. 44. M. F. Shapiro and R. P. Charrow, Scientific misconduct in investigational drug trials. N. Engl. J. Med. 1985; 312: 732–736. 45. R. B. Weiss et al., A successful system of scientific data audits for clinical trials. JAMA 1993; 270: 459–464. 46. D. Rennie, Accountability, audit, and reverence for the publication process. JAMA 1993; 270(4): 495–496. 47. D. Rennie, Breast cancer: how to mishandle misconduct. JAMA 1994; 271(15): 1205–1207. 48. W. P. Steward, K. Vantongelen, J. Verweij, D. Thomas, and A. T. Van Oosterom, Chemotherapy administration and data collection in an EORTC collaborative group—can we trust the results. Eur. J. Cancer 1993; 29A: 943–947.

12

CLINICAL TRIAL MISCONDUCT

49. C. Holden, Psychologist made up sex bias results. Science 2001; 294: 2457. 50. Report on 2002 Institutional Annual Report on Possible Research Misconduct. Washington, DC: Office of Research Integrity, August 2003. 51. D. Rennie (ed.), Mark, Dupe, Patsy, Accessory, Weasel, Flatfoot. In: Ethics and Policy in Scientific Publication. Bethesda, MD: Council of Biology Editors, Inc., 1990, pp. 155–174. 52. C. Holden, Planned misconduct surveys meet stiff resistance. Science 2002; 298: 1549. 53. D. S. Greenberg, Misconduct poll prompts fury among scientists. Lancet 2002; 360: 1669. 54. Soft responses to misconduct. Nature 2002; 240: 253. 55. D. Rennie, Misconduct and journal peer review. In: F. Godlee and T. Jefferson (eds.), Peer Review in Health Sciences. London: BMJ Books, 1999, pp. 90–99. 56. R. L. Engler et al., Misrepresentation and responsibility in medical research. N. Engl. J. Med. 1987; 317: 1383–1389. 57. W. P. Whitely, D. Rennie, and A. W. Hafner, The scientific community’s response to evidence of fraudulent publication. The Robert Slutsky case. JAMA 1994; 272(2): 170–173. 58. A. Abbott and J. Schwarz, Dubious data remain in print two years after misconduct inquiry. Nature 2002; 418: 113. 59. Retractions’ realities. Nature 2003; 422: 1. 60. International Committee of Medical Journal Editors. (2001). Uniform requirements for manuscripts submitted to biomedical journals. (online). Available: http://www.icmje.org/index.html#top. 61. E. Panel Marshall, Extensive Sudbø Fraud. Science 2006; 313: 29. 62. H. C. Sox, D. Rennie. Research Misconduct, Retraction and Cleansing the Medical Literature: Lessons from the Poehlman Case. Ann Intern Med. 2006; 144: 609–613. 63. P. A. Cola, Follow up to scientific misconduct. Clin. Researcher 2002; 2: 26–27.

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

materials were effective and not overly toxic, ultimately lead to clinical trials in humans. By 1960, there was an annual screening of approximately 25 000–30 000 materials sponsored by NCI with only about 10—20 new agents having sufficient effectiveness in animal systems to merit consideration for testing in humans. Peter Armitage, of the London School of Hygiene and Tropical Medicine, was a visiting scientist at NCI in the late 1950s. His background in sequential statistical procedures quickly found direct application in the development of two- and three-stage screening procedures for animal tumor systems that permitted rejection of an agent at any stage but acceptance only at the final stage (3,43). The object was to determine quickly which new compounds should be considered for further study in man. In the late 1950s, optimism was high that this screening program would lead to a new chemotherapeutic treatment that would make large clinical trials unnecessary. Also, there was a belief that different forms of cancer were sufficiently similar so that an agent active in one form of the disease would also be active in another.

MARVIN A. SCHNEIDERMAN∗ EDMUND A. GEHAN Georgetown University Medical Center, Washington, DC, USA

Early developments in controlled clinical trials at the National Institutes of Health (NIH) took place mainly at the National Cancer Institute (NCI) and what was then the National Heart Institute (NHI) [subsequently the National Heart, Lung, and Blood Institute (NHLBI)] beginning in the 1950s. This article reviews the developments from the early 1950s to the late 1960s at both institutes, summarizing the early efforts in clinical trials, the organizations set up to conduct and monitor the clinical trials, and the developments in statistical methodology that have formed the basis for conducting many of the present day randomized controlled trials. The early history of clinical trials at these institutes has been reviewed in more detail at NCIbyGehan & Schneiderman and at NHLBI by Halperin et al. (28,32).

1.1 Early Efforts in Clinical Trials

1 DEVELOPMENTS IN CLINICAL TRIALS AT THE NATIONAL CANCER INSTITUTE (NCI)

Dr C. Gordon Zubrod came to NCI in 1954 at about the time that Dr James Holland departed for Roswell Park Memorial Institute in Buffalo, NY. Drs Emil Frei and E.J. Freireich arrived at NCI in 1955. Under the leadership of Zubrod, this formed the key group of clinicians who initiated the clinical trials program at NCI. When Zubrod was at Johns Hopkins University in the early 1950s, he indicated that there ‘‘were two streams of influence (relating to developments in clinical trials)—infectious disease chemotherapy and comparative studies of analgesics and hypnotic drugs’’ (52). Among those playing an important role in the conduct of clinical trials at Johns Hopkins were Dr James Shannon (later Director of the National Institutes of Health), the pharmacologist E.K. Marshall, Jr and W.G. Cochran. About this time, the studies of streptomycin in pulmonary tuberculosis by the Medical Research Council were

A major advance in the development of chemical agents for the treatment of cancer came from observations of the treatment of children with acute lymphocytic leukemia, which was a rapidly fatal disease until 1948 when Sidney Farber, in a nonrandomized study of methotrexate, observed complete remissions and longer survival among some pediatric patients (21). However, results did not meet with uniform acceptance and questions were raised about diagnosis, selection of patients, and reporting. There was a need for a more organized approach to treatment experimentation that would lead to unbiased evaluations of treatments. At about the same time, animal models of the major forms of cancer— sarcomas, carcinomas, and leukemias—were developed that could be used to screen candidate materials and, if the

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

published and had a profound influence on the Johns Hopkins group (41). The first effort at a randomized trial was a comparison of the efficacy of tetracycline and penicillin in the treatment of lobar pneumonia (5). At the same time, the Veterans Administration began its first randomized controlled trials in tuberculosis (50). 1.2 The Organization of Trials In 1954, the US Congress created the Cancer Chemotherapy National Service Center (CCNSC) to stimulate research in the chemotherapy of cancer. A clinical panel was formed, headed by Dr I. Ravdin, and included among others Drs Zubrod and Holland. At an early meeting, the clinical panel reviewed a paper by Louis Lasagna, which enunciated five principles of the controlled clinical trial, including randomization and the statistical treatment of data (38). Over the next several years, the clinical panel of the CCNSC oversaw the organization of cooperative clinical trials groups for the conduct of clinical trials in cancer. By 1960, there were 11 cooperative clinical study groups (Table 1), each comprised of a number of universities and/or V.A. Hospitals and Medical Centers and a Statistical Coordinating Center (48). The cooperative groups were funded by the NCI through the Chairman and a Statistical Center. Zubrod recruited the chairman of each group and Marvin Schneiderman recruited the biostatisticians and statistical centers. One of the statisticians, W.J. Dixon, had two graduate students who were writing general statistical programs for the analysis of biomedical data. NCI awarded a contract to carry out this work that subsequently became the Biomedical Data Processing Program (BMDP) package of statistical programs. In the establishment of a clinical cooperative group, CCNSC agreed that there should be adherence to the following principles: combination of data from all institutions to accumulate rapidly the necessary number of patients; standard criteria of diagnosis, treatment, and measurement of effect; statistical design of the study, with a randomized assignment of patients to the groups to be compared; and statistical analysis and collaborative reporting of the results.

The clinical trials effort involved more types of clinical studies than randomized trials. There was a sequence of trials with differing objectives: Phase I—to determine the maximum tolerated dose of a regimen that can be used in looking for therapeutic effect; Phase II—to determine whether a particular dosage schedule of an agent is active enough to warrant further study; and Phase III—a comparative trial, usually randomized, to decide whether a new therapy is superior to a standard therapy. The primary objective of the clinical trials program was to provide a means of testing in humans new agents that had previously demonstrated effectiveness in animal tumor systems. 1.3 Some Early Trials Following some preliminary discussions between Dr Zubrod and Jerome Cornfield, a leading statistician at NIH, there was agreement that childhood leukemia was an ideal disease for testing some of the new agents objectively. The first randomized cooperative clinical trial in acute leukemia was planned in 1954, begun in 1955, and reported by Frei et al. in 1958 (23). The trial involved two regimens of combination chemotherapy—6-mercaptopurine and either intermittent or continuous methotrexate in 65 patients. The study had the following features: a uniform protocol at the four participating institutions uniform criteria of response adherence to the principles of the controlled clinical trial, especially the randomization of patients to therapies; and stratification of patients by age, type of leukemia, and history of prior therapy. Statistical methods used were a comparison of median survival times and median duration of remissions between therapies, confidence intervals, and Fisher’s exact test. The first randomized clinical trial in solid tumors was conducted by members of the Eastern Solid Tumor Group and reported by Zubrod et al. in 1960 (53). The trial involved a randomized comparison of two alkylating agents (thiotepa vs. nitrogen mustard) in patients with solid tumors. One objective was to ‘‘study the feasibility and usefulness of collaborative clinical research in cancer

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

3

Table 1. Cooperative Clinical Study Groups in 1960 Group

Chairman Statistician

Acute leukemia, Group A Acute leukemia, Group B Eastern Solid Tumor Group Southeastern Group Western Group Southwestern Group Prostate Group Breast Group A Breast Group B V.A. Groups—various malignancies University Groups—lung, breast, stomach, ovary, colon

M. Lois Murphy E. Frei C.G. Zubrod R.W. Rundles F. Willett H.G. Taylor H. Brendler A. Segaloff G. Gordon J. Wolf et al. A. Curreri et al.

chemotherapy’’. The trial involved 258 randomized patients, and notable features were: blind evaluation of response by vote of clinical investigators objective procedures for measurement of tumors and determination of when a response began and ended; the importance of accounting for type I and type II statistical errors appropriate sample size for detection of differences between treatments and statistical analysis in the reporting of results. A subsequent trial demonstrated the value of combination chemotherapy in acute leukemia and the independent action of drugs to increase the probability that a patient achieves complete remission (24). Freireich et al. (25) reported a prospective, randomized, double-blind, placebo-controlled, sequential study of 6-mp vs. placebo in the maintenance of remissions in pediatric acute leukemia. This study established that 6-mp maintenance treatment leads to substantially longer remissions than placebo and was a forerunner to many adjuvant studies in other forms of cancer, such as breast cancer, in which treatments are administered when the patients are in a disease-free state (25). This study also was a motivation for the development of an extension of the Wilcoxon test for comparing survival distributions subject to censoring (27) and was used as an example by Cox in his, now classic, paper on regression models and life tables (16).

I. Bross M. Schneiderman M. Schneiderman B.G. Greenberg E. MacDonald D. Mainland M. Schneiderman M. Schneiderman M. Patno R. Stiver, G. Beebe, W. Dixon

1.4 Developments in Methodology In the clinical trials program at NCI prior to 1970, there were several developments in methodology that have influenced the conduct of subsequent clinical trials. Before 1960, the clinical testing of new agents often involved as few as five patients, with the agent discarded if no positive response was obtained in at least one patient. In 1961, Gehan proposed a plan for Phase II trials that determined the minimum number of consecutive patients to study when all patients are nonresponders, before one could reject a new agent for further study, at given levels of rejection error (26). This plan, or now more commonly Simon’s modification, continues in use today in Phase II studies (46). Several philosophical issues arose from the drug development program. The practice of human experimentation could be questioned by ‘‘Doesn’t a physician have an implied duty to give his patient the best treatment? If that is the case, how can one justify having the toss of coin (i.e. randomization) decide which treatment a patient should receive?’’ The reply was (and is), ‘‘If the physician really knows what is the best treatment for the patient, the patient must receive that treatment and not be randomized into a trial.’’ The question then becomes, ‘‘How and when does a physician know what is the best treatment for a specific patient?’’ The major ethical issue then becomes one of learning quickly (i.e. with a minimum number of patients) what is the best treatment. There have been several proposals for

4

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

establishing what one ‘‘knows’’ while minimizing the number of patients who will receive the less effective treatment. Armitage proposed closed sequential procedures with paired patients on each treatment, and with the trial terminated as soon as one could establish the superiority of one of the treatments over the other (2). A feature of the plans was an upper limit on the number of patients one could enter. Schneiderman & Armitage later described a family of sequential procedures, called wedge plans because of the shape of the acceptance boundary, which provided a bridge between the open plans derived from Wald’s theory and the restricted procedures of Armitage (44,45). In the 6-mp vs. placebo study for maintaining remissions in pediatric leukemia, patients were paired according to remission status (complete or partial), one patient receiving 6-mp and the other placebo by a random allocation, and a preference was recorded for 6-mp or placebo depending upon the therapy which resulted in the longer remission. The trial was conducted sequentially according to one of the Armitage plans (2) and a sequential boundary favoring 6-mp was reached after 18 preferences had occurred— 15 for 6-mp and 3 for placebo. There were 12 patients still in remission at the time the study was terminated, although one could record a preference for one or the other treatment because the pair-mate had relapsed at an earlier time. It was clear that a more efficient analysis could be obtained by using the actual lengths of remission. Gehan, while working on an NCI fellowship with D.R. Cox at Birkbeck College in London, developed a generalization of the Wilcoxon test for the fixed sample size problem with each sample subject to arbitrary right censoring (27). Halperin had previously developed a generalization of the Wilcoxon test, when all times to censoring were equal to the longest observation time (30). Mantel noticed that one could utilize the chi-square test for comparison of survival data between two or more groups, assuming that one constructs a contingency table of deaths and survivors at each distinct failure time in the groups of patients under study. This chi-square test was appropriate when the risk of failure in one group was a constant multiple of that in the other; this

test was an extension of the earlier test developed by Mantel and Haenszel which measured the statistical significance of an observed association between a disease and a factor under study in terms of an increased relative risk of disease (39,40). This test subsequently became known variously as the Mantel–Haenszel test, the logrank test or the Cox–Mantel test, and has been studied by Cox and Peto, among others (16,42). Another development in the 1960s was the exponential regression model proposed by Feigl & Zelen (22). Dr Robert Levin of NCI was interested in studying the relationship of the survival time of leukemia patients to the concomitant variate of white blood count, separately according to the presence or absence of auer rods and/or significant granulature of leukemia cells in the bone marrow at diagnosis. Feigl & Zelen proposed a model in which an exponential survival distribution is postulated for each patient and the expected value of the survival time is linearly related to the patient’s white blood count. A more general loglinear model was subsequently given by Glasser (29), and there have been numerous subsequent developments in parametric regression models with censored survival data 17, Chapters 5 and 6, pp. 62–90. 2 DEVELOPMENTS IN CLINICAL TRIALS AT THE NATIONAL HEART, LUNG, AND BLOOD INSTITUTE (NHLBI) Prior to 1960, the National Heart Institute (NHI), subsequently to become NHLBI, had little involvement in multicenter clinical trials. In a trial designed in 1951, there was a comparison of ACTH, cortisone, and aspirin in the treatment of rheumatic fever and the prevention of rheumatic heart disease. A total of 497 children were enrolled in 12 centers in the UK, the US, and Canada. Felix Moore, then Chief of the Biometrics Section of NHI, was a statistical consultant. There were no differences in treatment effectiveness in the study, and no statistical or methodologic problems mentioned in the final report (8). Subsequently, there was a multicenter observational study of lipoproteins in

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

atherosclerosis that had substantial impact on the methodology for coordinating studies performed at several sites (47). The Statistical Center was led by Felix Moore and Tavia Gordon at NHI. Careful quality control procedures and standardization of methods across centers were emphasized. 2.1 Early Efforts in Clinical Trials Jerome Cornfield joined the NHI in 1960 and strongly influenced the conduct of clinical trials at NHI and statistical research on methodologic issues arising in clinical trials. In the early 1960s, intensive planning for two clinical trials was begun at NHI to reduce risk factors for coronary heart disease—The Diet Heart Feasibility Study (DHFS) and the Coronary Drug Project (CDP) (14,20). These studies reflected the strong interest in both dietary and drug approaches to the prevention of coronary heart disease and the recurrence of myocardial infarction. For the DHFS, the NHI Biometrics Branch served as the statistical coordinating center, first under the supervision of Joseph Schachter and later of Fred Ederer. Max Halperin rejoined the NHI in 1966 and, upon Cornfield’s retirement in 1968, became Chief of the Biometrics Research Branch until his retirement in 1977. Four areas of clinical trials and methodology can be traced to these early studies and the individuals responsible for them. These areas are: organizational structure for clinical trials at NIH; methodology for the interim analysis of accumulating data, including the Bayesian approach, group sequential and stochastic curtailment methods; design and analysis of clinical trials, including the effects of patient noncompliance on power and the intention to treat principle; and methods for analysis of data from longitudinal clinical trials. 2.2 The Organization of NHLBI Trials The ‘‘NHLBI Model’’ for cooperative clinical trials evolved from discussion during the planning stage of the CDP among outside medical experts and NHI medical and statistical staff. In 1967, a report by a committee appointed by the National Advisory Heart Council and chaired by Bernard Greenberg

5

described this structure (35). The report, subsequently known as the ‘‘Greenberg Report’’, became the basis for a structure of nearly all subsequent NHLBI trials as well as for many other trials sponsored at NIH. The major components of the organizational structure include a Steering Committee, a Policy Advisory Board, a Data Monitoring Committee, and a Statistical or Data Coordinating Center, as well as individual clinics, central laboratories, and various other committees which served the needs of the trial. These might include committees to develop eligibility criteria, to assign cause of death, to define methodology and standards, or to oversee the preparation of manuscripts (for more details of organizational structure). From the biostatistical viewpoint, the Data Monitoring Committee has the responsibility of monitoring accumulating data on a periodic basis and analyzing results for evidence of early benefit or harm. Primary and secondary outcomes measures are reviewed, along with safety data, compliance to the protocol, and subgroup analyses which may identify particular risk groups. The Statistical Coordinating Center and the Data Monitoring Committee work closely together in performing the appropriate data analyses needed for fulfilling the Committee’s responsibilities. The Statistical and Data Coordinating Centers for early trials at the NHLBI are given in Table 2. Personnel at these coordinating centers have played an important role in the development of clinical trials and made numerous contributions to statistical methodology. 2.3 Developments in Methodology These are considered under three headings: data monitoring, design and analysis, and longitudinal studies. 2.3.1 Data Monitoring. Jerome Cornfield was involved in the planning and conduct of two clinical trials—the DHFS and the CDP. Both Cornfield and Halperin served on the Data and Safety Monitoring Committee of the CDP. At least partly motivated by his involvement in these trials, Cornfield

6

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE Table 2. Early NHLBI Coordinating Centers. University of Maryland/Maryland Research Institute Coronary Drug Project University of Texas School of Public Health Hypertension Detection and Follow-up Program University of North Carolina—Chapel Hill, School of Public Health Lipid Research Clinical Program University of Minnesota School of Public Health, Biometry Division Multiple Risk Factor Intervention Trial University of Washington School of Public Health, Biostatistics Department Coronary Artery Surgery Study George Washington University Biostatistics Center Intermittent Positive Pressure Breathing Trial NHLBI Biometrics Research Branch National Diet Heart Feasibility Study Urokinase Pulmonary Embolism Trial Urokinase Streptokinase Pulmonary Embolism Trial

published papers in 1966 on sequential trials, sequential analysis, and the likelihood principle, from a Bayesian perspective (9,10). In 1966, Max Halperin worked jointly with Cornfield and Samuel Greenhouse (then at the National Institute of Mental Health) to develop an adaptive allocation procedure that would assign an increasing proportion of patients to the better of two treatments as evidence accumulated (13). Their approach to the problem was Bayesian and generalized the earlier work of Anscombe and Colton (1,7). At around the same time, Cornfield published a general paper on the Bayesian approach that involved the use of a prior probability distribution with a mass of probability P at the null hypothesis, with a continuous density of total mass 1 − P over a set of alternative hypotheses (11). A key feature of Cornfield’s proposal was the rejection of the null hypothesis when the posterior odds (the relative betting odds or RBO) became small for H0 . The RBO was used in the CDP in the monitoring of mortality differences between the control and each of the drug treatment groups. Subsequently, Canner, of the CDP Coordinating Center, considered the determination of critical values for decision making at multiple time points during the conduct of the clinical trial from the Neyman–Pearson perspective (6). Later, curtailment and stochastic curtailment methods were developed and applied to trials of the NHLBI in the 1970s and early 1980s (19,31,34,37).

Statisticians working with the CDP were aware that, as the data accumulated, repeated testing for treatment differences using conventional statistical significance levels would increase the type I error over the nominal alpha level associated with that critical value. Armitage et al. evaluated the impact of repeated testing on the type I error and demonstrated that multiple tests could increase the type I error substantially (4). Interim analyses of clinical data are necessary for scientific and ethical reasons, but large type I errors are not acceptable. Canner developed a method for the CDP for determining the critical value at each interim analysis so that the overall type I error is close to the desired level (6). Statisticians involved with NHLBI trials developed group sequential methods and applied them to trials starting with the CDP. 2.3.2 Design and Analysis. In the DHFS, it was projected that a reduction in cardiovascular risk would result from a reduction in cholesterol level. The original sample size projection was for the entry of 8000 patients into several treatment arms. Although a special review committee suggested that this sample size might be too large, Cornfield argued that there were too many inconclusive small studies already in the literature. Several aspects of the trial required consideration, including noncompliance with the treatment regimen. It was presumed that the maximum effect on risk would occur only

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

after some period of time on treatment and that failure to adhere to the treatment regimen could mean a return to higher risk levels. Halperin et al. (33) incorporated these considerations into the design of clinical trials by proposing methods for adjusting sample size for noncompliance in the treatment group. Studies were considered with a fixed period of observation and a comparison of proportions as the main analysis. Implicit in this paper is the ‘‘intention to treat’’ principle, i.e. analysis of all randomized patients in their assigned treatment group regardless of compliance. Ultimately, the report of the CDP recognized this point (15). Most primary and secondary prevention trials conducted by the NHLBI since 1970 have made use of sample size adjustments for noncompliance. The Framingham Heart Study was begun in 1948 and has had an important influence on methodologic research at the NHLBI and the design of prevention trials. Over 5000 adult residents of Framingham, Massachusetts, were entered into a longitudinal study with the objective of evaluating the effects of various risk factors on the development of subsequent cardiovascular disease. The study has clarified the roles of high blood pressure, elevated total serum cholesterol, and cigarette smoking on the risk of cardiovascular disease (18,36). Occurrence or not of a cardiovascular event in a 2-year follow-up period is a binary outcome. Cornfield considered a regression approach to deal with the binary outcome variables. The problem was closely related to the discrimination problem between two samples from multivariate normal distributions. For a specific prior probability of belonging or not to a disease group, the posterior probability could be represented as a logistic regression function that was closely related to what could be obtained from a conventional discriminant function analysis (49). Cornfield & Mitchell argued that one could use the logistic model to predict the impact on risk of specified changes in risk factors (12). Subsequently, this logistic model approach was used in the design of several NHLBI prevention trials. 2.3.3 Longitudinal Studies. A methodology for analysis of longitudinal data was needed for the Framingham Study which

7

could be considered both a cohort and a longitudinal study. Cohorts of individuals were followed to observe patterns of morbidity and mortality, and biennial measurements of cardiovascular risk factors provided an opportunity to study patterns relating to aging. Early reports of the Framingham study used simple graphical and descriptive methods to describe patterns of aging. During the 1980s, there was much work on methodology for longitudinal studies, Overview that ultimately led to NHLBI sponsorship of a workshop on methods for analysis of longitudinal and follow-up studies, whose proceedings have appeared as a special issue in Statistics in Medicine (51). REFERENCES 1. Anscombe, F. J. (1963). Sequential medical trials, Journal of the American Statistical Association 58, 365–383. 2. Armitage, P. (1957). Restricted sequential procedures, Biometrika 44, 9–26. 3. Armitage, P. & Schneiderman, M. (1958). Statistical problems in a mass screening program, Annals of the New York Academy of Science 76, 896–908. 4. Armitage, P., McPherson, C. K. & Rowe, B. C. (1969). Repeated significance tests on accumulating data, Journal of the Royal Statistical Society, Series A 132, 235–244. 5. Austrian, R., Mirick, G., Rogers, D., Sessoms, S. M., Tumulty, P. A., Vickers, W. H., Jr. & Zubrod, C. G. (1951). The efficacy of modified oral penicillin therapy of pneumococcal lobar pneumonia, Bulletin of Johns Hopkins Hospital 88, 264–269. 6. Canner, P. L. (1977). Monitoring treatment differences in long-term clinical trials, Biometrics 33, 603–615. 7. Colton, T. (1963). A model for selecting one of two medical treatments, Journal of the American Statistical Association 58, 388–400. 8. Cooperative Clinical Trial of ACTH, Cortisone and Aspirin in the Treatment of Rheumatic Fever and the Prevention of Rheumatic Heart Disease (October 1960). Circulation 22,. 9. Cornfield, J. (1966). Bayesian test of some classical hypotheses—with applications to sequential clinical trials, Journal of the American Statistical Association 61, 577–594. 10. Cornfield, J. (1966). Sequential trials, sequential analysis, and the likelihood principle, American Statistician 20, 18–23.

8

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE

11. Cornfield, J. (1969). The Bayesian outlook and its application, Biometrics 25, 617–657. 12. Cornfield, J. & Mitchell, S. (1969). Selected risk factors in coronary disease. Possible intervention effects, Archives of Environmental Health 19, 382–394. 13. Cornfield, J., Halperin, M. & Greenhouse, S. (1969). An adaptive procedure for sequential clinical trials, Journal of the American Statistical Association 64, 759–770. 14. Coronary Drug Project Research Group (1973). The Coronary Drug Project. Design, methods, and baseline results, Circulation 47, Supplement 1, 179. 15. Coronary Drug Research Group (1980). Influence of adherence to treatment and response of cholesterol on mortality in the Coronary Drug Project, New England Journal of Medicine 303, 1038–1041. 16. Cox, D. R. (1972). Regression models and life tables (with discussion), Journal of the Royal Statistical Society, Series B 34, 187–220. 17. Cox, D. R. & Oakes, D. (1984). Analysis of Survival Data. Chapman & Hall, London. 18. Dawber, T. R., Meadors, G. F. & Moor, F. E. (1951). Epidemiological approaches to heart disease: the Framingham Study, American Journal of Public Health 41, 279–286. 19. DeMets, D. L. & Halperin, M. (1981). Early stopping in the two-sample problem for bounded variables, Controlled Clinical Trials 3, 1–11. 20. Diet–Heart Feasibility Study Research Group (1968). The National Diet–Heart Study Final Report, Circulation 37, Supplement 1, 428. 21. Farber, S., Diamond, L. K., Mercer, R., Sylvester, R. F. Jr. & Wolff, J. A. (1948). Temporary remissions in children produced by folic acid antagonist aminopterin, New England Journal of Medicine 238, 787–793. 22. Feigl, P. & Zelen, M. (1965). Estimation of exponential survival probabilities with concomitant information, Biometrics 21, 826–838. 23. Frei, E., III, Holland, J. F., Schneiderman, M. A., Pinkel, D., Selkirk, G., Freireich, E. J., Silver, R. T., Gold, G. L. & Regelson, W. (1958). A comparative study of two regimens of combination chemotherapy in acute leukemia, Blood 13, 1126–1148. 24. Frei, E., III, Freireich, E. J., Gehan, E. A., Pinkel, D., Holland, J. F., Selawry, O., Haurani, F., Spurr, C. L., Hayes, D. M., James, W., Rothberg, H., Sodee, D. B., Rundles, W., Schroeder, L. R., Hoogstraten, B., Wolman, I. J., Tragis, D. G., Cooper, T., Gendel, B.

R., Ebaugh, F. & Taylor, R. (1961). Studies of sequential and combination antimetabolite therapy in acute leukemia: 6-mercaptopurine and methotrexate, Blood 18, 431–454. 25. Freireich, E. J., Gehan, E. A., Frei, E., III, Schroeder, L. R., Wolman, I. J., Anbari, R., Bergert, O., Mills, S. D., Pinkel, D., Selawry, O. S., Moon, J. H., Gendel, B. R., Spurr, C. L., Storrs, R., Haurani, F., Hoogstraten, B. & Lee, S. (1963). The effect of 6-mercaptopurine on the duration of steroid-induced remissions in acute leukemia: A model for evaluation of other potentially useful therapy, Blood 21, 699–716. 26. Gehan, E. A. (1961). The determination of the number of patients required in a preliminary and follow-up trial of a new chemotherapeutic agent, Journal of Chronic Diseases 13, 346. 27. Gehan, E. A. (1965). A generalized Wilcoxon test for comparing arbitrarily singly-censored samples, Biometrika 52, 203–223. 28. Gehan, E. A. & Schneiderman, M. A. (1990). Historical and methodological developments in clinical trials at the National Cancer Institute, Statistics in Medicine 9, 871–880. 29. Glasser, M. (1967). Exponential survival with covariance, Journal of the American Statistical Association 62, 561–568. 30. Halperin, M. (1960). Extension of the Wilcoxon-Mann-Whitney test to samples censored at the same fixed point, Journal of the American Statistical Association 55, 125–138. 31. Halperin, M. & Ware, J. (1974). Early decision in a censored Wilcoxon two-sample test for accumulating survival data, Journal of the American Statistical Association 69, 414–422. 32. Halperin, M., DeMets, D. L. & Ware, J. H. (1990). Early methodological developments for clinical trials at the National Heart Lung and Blood Institute, Statistics in Medicine 9, 881–882. 33. Halperin, M., Rogot, E., Gurian, J. & Ederer, F. (1968). Sample sizes for medical trials with special reference to long term therapy, Journal of Chronic Diseases 21, 13–24. 34. Halperin, M., Ware, J., Johnson, N. J., Lan, K. K. & Demets, D. (1982). An aid to data monitoring in long-term clinical trials, Controlled Clinical Trials 3, 311–323. 35. Heart Special Project Committee (1988). Organization, review, and administration of cooperative studies (Greenberg Report): A report from the Heart Special Project Committee to the National Advisory Heart Council, May 1967, Controlled Clinical Trials 9, 137–148.

CLINICAL TRIALS, EARLY CANCER AND HEART DISEASE 36. Kannel, W. B., Dawber, T. R., Kagan, A., Nevotskie, N. & Stokes, J. (1961). Factors of risk in the development of coronary heart disease—six year followup experience: the Framingham Study, Annals of Internal Medicine 55, 33–50. 37. Lan, K. K. G., Simon, R. & Halperin, M. (1982). Stochastically curtailed tests in long-term clinical trials, Communications in Statistics—Stochastic Models 1, 207–219. 38. Lasagna, L. (1955). The controlled clinical trial: theory and practice, Journal of Chronic Diseases 1, 353–358. 39. Mantel, N. (1966). Evaluation of survival data and two new rank order statistics arising in its consideration, Cancer Chemotherapy Reports 50, 163–170. 40. Mantel, N. & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease, Journal of the National Cancer Institute 22, 719–748. 41. Medical Research Council (1948). Streptomycin treatment of pulmonary tuberculosis, British Medical Journal 2, 769–783. 42. Peto, R. & Peto, J. (1972). Asymptotically efficient rank invariant test procedures (with discussion), Journal of the Royal Statistical Society, Series A 135, 185–206. 43. Schneiderman, M. A. (1961). Statistical problems in the screening search for anti-cancer drugs by the National Cancer Institute of the United States, in Quantitative Methods in Pharmacology. North-Holland, Amsterdam. 44. Schneiderman, M. A. & Armitage, P. (1962). A family of closed sequential procedures, Biometrika 49, 41–56. 45. Schneiderman, M. A. & Armitage, P. (1962). Closed sequential t-tests, Biometrika 49, 359–366. 46. Simon, R. (1989). Optimal two stage designs for Phase II trials, Controlled Clinical Trials 10, 1–10. 47. Technical Group and Committee on Lipoproteins and Atherosclerosis (1956). Evaluation of serum lipoproteins and cholesterol measurements as predictors of clinical complications of atherosclerosis, Circulation 14, 691–742. 48. The National Program of Cancer Chemotherapy Research (1960). Cancer Chemotherapy Reports 1, 5–34. 49. Truett, J., Cornfield, J. & Kannel, W. B. (1967). A multivariate analysis of the risk factors of coronary heart disease in Framingham, Journal of Chronic Diseases 20, 511–524.

9

50. Tucker, W. B. (1954). Experiences with controls in the study of the chemotherapy of tuberculosis, Transactions of the 13th Veterans Administration Conference on the Chemotherapy of Tuberculosis, Vol. 15. 51. Wu, M., Wittes, J. T., Zucker, D. & Kusek, J. eds (1988). Proceedings of the Workshop on Methods for Longitudinal Data Analysis in Epidemiological and Clinical Studies, Statistics in Medicine 7, 1–361. 52. Zubrod, C. G. (1982). Clinical trials in cancer patients: an introduction, Controlled Clinical Trials 3, 185–187. 53. Zubrod, C. G., Schneiderman, M., Frei, E., III, Brindley, C., Gold, G. L., Shnider, B., Oviedo, R., Gorman, J., Jones, R., Jr, Jonsson, U., Colsky, J., Chalmers, T., Ferguson, B., Dederick, M., Holland, J., Selawry, O., Regelson, W., Lasagna, L. & Owens, A. H., Jr (1960). Appraisal of methods for the study of chemotherapy of cancer in man: Comparative therapeutic trial of nitrogen mustard and thiophosphoramide, Journal of Chronic Diseases 11, 7–33.

CASE STUDIES: OVER-THE-COUNTER DRUGS

challenges. These questions must probe a consumer’s understanding of OTC label messages without cueing particular responses. Open-ended questions may introduce the least bias but may be difficult to objectively score. Some of these issues will be illustrated using data from the label development program supporting the prescription-to-OTC switch of omeprazole (Prilosec) as presented to the FDA’s Nonprescription Drugs Advisory Committee (7). An early version of the OTC Prilosec label included a warning that the consumer should ‘‘Ask a doctor before use if you . . . are taking . . . phenytoin (seizure medicine)’’ (7) because of concerns about a potential drug–drug interaction. In a label comprehension study, participants were provided the proposed label and were asked: ‘‘You suffer from seizures and are taking a medicine called Phenytoin to help control your seizures. You also routinely suffer from heartburn several times per week. You have just heard about this new product, Prilosec 1 for the prevention and relief of heartburn. If you were the person described in this situation and you wanted to use Prilosec 1 to prevent or treat your heartburn, what would you do now?’’ (7). This question, which was asked of 504 participants, includes a scenario designed to assess comprehension of the warning against concomitant use. To focus on this issue, the scenario provides a clinical indication for the use of Prilosec. Approximately 90% of participants responded that they would check with their doctor or would not use the Prilosec. Both of these answers are acceptable in that they are consistent with the intent of the warning. A low-literacy subset of the participants performed in a manner similar to the total study population. These results are encouraging and suggest that the label would prevent concomitant use of phenytoin and omeprazole based on effective communication of this specific message. However, the same participants were asked a similar question but with Prozac, a drug not mentioned on the label, as a possible concomitant medication. Over half the participants again indicated that they would not use the Prilosec or would consult with their

ERIC P. BRASS Harbor-UCLA Center for Clinical Pharmacology Torrance, California

Switching a drug from prescription to overthe-counter (OTC) status is based on evidence that consumers can use the drug safely and effectively in the absence of a healthcare professional (1). The OTC drug label is the key source of information for consumers when deciding whether to use a specific OTC drug and how to self-manage their condition when using the drug. Manufacturers must provide data supporting the claim that consumers will use the drug properly in the unsupervised OTC setting. Specifically, manufacturers submit clinical research data to regulatory authorities demonstrating that consumers can understand the OTC drug label and will heed the instructions contained in the label. Label comprehension studies evaluate the ability of consumers to understand key communication objectives as presented in the proposed OTC label (2, 3). Self-selection and actual use studies test the ability of consumers to decide if the OTC drug is appropriate for their use based on their individual health history and whether they can selfmanage the treatment course. The design, conduct, and interpretation of these studies pose unique challenges, some of which are illustrated in our case examples. Although some label development studies have been published (4–6), many examples are available from the deliberations of Food and Drug Administration (FDA) Advisory Committee meetings. Importantly, each OTC clinical program must be individually designed to meet the specific issues associated with the specific OTC candidate. 1

LABEL COMPREHENSION STUDIES

Construction of questions or instruments to assess label comprehension poses particular

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

CASE STUDIES: OVER-THE-COUNTER DRUGS

doctor first. This conservative response by consumers is reassuring from a public health perspective, but it suggests that the excellent response to the phenytoin scenario was less a measure of comprehension than of a more universal tendency to give a safe response in the testing scenario. The communication of the concomitant medication warnings for Prilosec was further studied in a group of individuals who had label contraindications for Prilosec use. These people were provided the label and asked, ‘‘If you were a heartburn sufferer and you wanted to relieve heartburn, would it be OK for you to use Prilosec 1 or not?’’ (7). Note that, in contrast to the phenytoin scenario, this question is very focused and applies to the individual participant. Over 40% of participants with a label contraindication responded that it would be OK to use, in contrast to the label’s intent. Attempts to use open-ended questions to elicit an understanding of why the incorrect responders felt that they could use the drug yielded answers ranged from ‘‘Don’t know’’ to ‘‘Warning labels don’t apply to me’’ (7). The Prilosec example illustrates the challenge in motivating consumers to heed clearly communicated label warnings. Comprehension of the core message is necessary but not sufficient for an OTC label to ensure proper drug use by consumers. Thus, selfselection and actual use studies are required to assess consumer behaviors. The poor intent-to-heed rate in the Prilosec case would represent a barrier to OTC approval if the phenytoin–omeprazole interaction would result in substantial risk to consumers. Moreover, it illustrates how label comprehension study questions must be designed to truly assess label communication as opposed to general consumer insights into expected responses (for example, ‘‘I’d ask my doctor’’). Ultimately, Prilsoec was approved for OTC marketing. The final label has a more limited concomitant medication warning excluding phenytoin and focusing on drugs where a more clinically relevant drug–drug interaction might occur.

2 ACTUAL USE STUDIES Most proposed OTC drugs require a unique set of self-selection and overall self-management behaviors by consumers. Thus, the design of each actual use trial must incorporate assessments designed to evaluate those aspects most relevant to the specific OTC drug being studied. 2.1 Orlistat Orlistat is a drug that was approved for prescription marketing in the United States for obesity management, including weight loss and weight maintenance. It works as an inhibitor of lipases in the gastrointestinal tract, and thus its proper use requires that it be taken with meals in patients compliant with a low-fat diet (8). Additionally, as it may inhibit absorption of other compounds from the intestine, including fat-soluble vitamins, it is recommended that orlistat users also take a multivitamin at least 2 hours before or after meals. Thus, when orlistat was proposed for a switch to OTC status, the self-selection and actual use trials were an important aspect of the FDA’s evaluation (9). The actual use study for orlistat was conducted at 18 pharmacies with a wide geographic distribution (9). Study participants were recruited through in-store and newspaper advertisements. Interested consumers were shown the orlistat package and told: ‘‘Imagine you are in a store and this is a new over-the-counter medicine. You can take as much time as you need to look at the packaging. Let me know when you are finished.’’ They were then asked the selfselection question: ‘‘Do you think this medication is appropriate for you to use?’’ After answering the self-selection question, they were asked, ‘‘The cost of this medication is $45 for a bottle of 90 capsules. Would you like to purchase the medicine today?’’ Each bottle would provide up to 30 days of treatment, and participants were allowed to buy up to three bottles at a time. Consumers who purchased orlistat were followed by periodic phone contact, which used a structured question-based interview. Additionally, participants could return to the study pharmacy to purchase additional orlistat. They were followed for up to 90 days of treatment.

CASE STUDIES: OVER-THE-COUNTER DRUGS

3

Figure 1. Flow of research subjects in orlistat OTC actual use study. Of the 703 participants screened at sites, 237 were included in the core analysis of longitudinal use behaviors. However, other cohorts provided useful information on the effectiveness of other label elements. Data from Feibus (9).

In the orlistat actual use study, of 681 eligible participants who examined the package, 543 said the product was appropriate for their use, 339 wanted to purchase the product, 262 actually purchased the product, and 237 of the purchasers had evaluable behavioral data (Figure 1). This numerical characterization of the population highlights an important aspect of data analysis for actual use studies: which denominator should be used? There is no generalized answer as the population should be the most relevant to the question/hypothesis being examined. Most often this is either the self-selection population or the evaluable purchasers. However, there may be exceptions. For example, if the label’s ability to deter use by consumers with a contraindication is being tested, the denominator may be the number of individuals with the contraindication who evaluated the product, and the numerator those that self-selected or purchased the drug. This is more meaningful than simply expressing the percentage of purchasers with the contraindication, which may be misleadingly low if a small percentage of the evaluating individuals had the contraindication. As in any study, omission

of a portion of the study population from the analysis for any reason raises questions as to the robustness of the data set. Understanding how orlistat would be used in an unsupervised setting was a major objective of the actual use trial. Key behaviors studied included whether orlistat was used with meals (>95% used with meals), whether the maximal daily dose was exceeded (<3% exceeded the dose), whether consumers would use orlistat chronically as required to obtain benefit (46% of participants were still using orlistat after 90 days), and whether consumers were supplementing therapy with multivitamins properly (<50% were using multivitamin as per label). The latter two results raised important questions. The low adherence with chronic therapy is suboptimal, but adherence may be low in the prescription setting as well (10). The incorrect use of multivitamins led to recommendations that labeling for these directions be improved. Many consumer subsets of interest were underrepresented in the orlistat actual use study. For example, only two participants were taking cyclosporine, a drug known to

4

CASE STUDIES: OVER-THE-COUNTER DRUGS

interact with orlistat. Despite a label contraindication, one of the two cyclosporine users elected to purchase orlistat. Similarly, seven of 14 warfarin users elected to purchase orlistat contrary to label directions. These data led to label modifications and more intensive studies of cohorts with concomitant drug use to optimize heeding of the label directions. Additionally, only 4% of participants in the study were of low literacy, raising concerns about whether the results could be generalized. These results were of concern during advisory committee deliberations and led to recommendations for further testing and label refinements. Thus, the orlistat actual use study addressed a number of important issues germane to regulatory decision making, and guided further label development and research when the results identified nonoptimal consumer behaviors. 2.2 Lovastatin Statins have been considered for OTC use based on their proven efficacy in reducing cardiovascular morbidity and mortality, and on the large number of patients who are eligible for statin therapy based on guidelines but are not receiving therapy (11). However, use of a statin in the OTC setting requires consumers to comply with a complex selfmanagement paradigm that includes making proper self-selection based on knowledge of their serum cholesterol concentrations and cardiovascular risk factors, triaging to more intensive therapy based on their response to OTC statin therapy, avoiding drug interactions, and recognizing possible statin muscle toxicity. Aspects of OTC statin therapy have been studied in a number of actual use studies, including the Consumer Use Study of Over-the-Counter Lovastatin (CUSTOM) study (12). The CUSTOM study recruited to study sites over 3300 consumers interested in self-management of their cholesterol. These consumers were given the opportunity to purchase OTC lovastatin (20 mg) after reviewing the drug package, which included the proposed OTC label. This label contained information to guide self-selection and use of lovastatin. The label also contained warnings

about adverse events and instructions for self-management based on cholesterol concentrations after 8 weeks of OTC lovastatin treatment. Over 1000 consumers purchased and used lovastatin in the study. The study yielded valuable insights into how consumers might use a statin in the unsupervised OTC setting. The OTC label used in CUSTOM guided consumer self-selection based on their serum low-density lipoprotein (LDL) cholesterol concentration. Consumers could elect to purchase a cholesterol test at the study site, but this was not required. After their purchase decision, consumers were asked to report their LDL cholesterol concentration. Additionally, blood samples were taken for assay of cholesterol, but the results were not provided to the consumers unless they had obtained the test before purchase. When analyzed by broad ranges (<130 mg/dL, 130–170 mg/dL, >170 mg/dL), 76% of the 667 consumers who used OTC lovastatin and for whom data were available accurately reported their LDL cholesterol. Thirtynine consumers with LDL cholesterol levels above 170 mg/dL, who were thus likely to require more intensive statin treatment, selfreported their LDL cholesterol as less than 170 mg/dL (13). The target population for OTC lovastatin was consumers with a moderate 10-year cardiovascular risk of 5% to 20%. The ability of the surrogates used on the proposed OTC label to allow self-selection of this cohort was assessed by the CUSTOM study. Of note, 24% of the participants who self-selected to use lovastatin in CUSTOM had a 10-year cardiovascular risk of greater than 20% (Figure 2) (14). Thus, CUSTOM identified the potential for some consumers to be diverted from optimal, intensive statin therapy by using OTC statins. Labeling of OTC statins to minimize this diversion is thus a priority. Similarly, 29% of OTC lovastatin users were at low risk as assessed using conventional tools (see Figure 2). The CUSTOM study illustrates the ability of an actual use trial to assess complex self-selection paradigms and to identify areas requiring improvement before regulatory approval.

CASE STUDIES: OVER-THE-COUNTER DRUGS

5

Figure 2. Self-selection for OTC treatment and cardiovascular risk in the CUSTOM study. The proposed paradigm for OTC statin use requires consumers to risk-stratify based on surrogates provided on the label. The CUSTOM study assessed the effectiveness of this approach. Data from Brass et al. (14).

Overall, correct self-selection of OTC lovastatin required a large number of assessments by the consumer, including their LDL cholesterol level and their cardiac risk factors as well as age, concomitant medications, previous muscle symptoms, history of liver disease or pregnancy, cutoffs based on highdensity lipoprotein cholesterol and triglycerides concentration, and the presence or absence of concomitant diseases such as diabetes. When all these criteria were taken into account, only 10% of lovastatin users in the CUSTOM study met all label criteria (15). This illustrates an important aspect of interpreting actual use study results. Not all label directions and restrictions are equivalently important. A study analysis can benefit from having prespecified the most important label-heeding requirements and from defining the thresholds or performance standards that must be met to be consistent with unsupervised OTC use. For example, in the case of OTC lovastatin, the hypothetical decision of a 54-year-old woman with hypertension and a plasma LDL concentration of 160 mg/dL to use the drug and the decision of a 55year-old man currently taking gemfibrozil to use OTC lovastatin are both incorrect. The decision by the woman is unlikely to have an adverse health consequence, but that of the 55-year-old man is associated with a potentially clinically significant drug–drug interaction. Compliance with the concomitant medication warning thus merits a meaningful predefined standard of performance, and mild deviations from the age limits are substantially less important. In the case of CUSTOM, 10 of 48 consumers on gemfibrozil

who evaluated OTC lovastatin purchased and used the OTC statin (13). After they had started therapy with OTC lovastatin, the label instructed consumers to have their cholesterol rechecked. Those consumers with high LDL cholesterol concentrations on therapy, which might include those with very high levels when therapy was started, should seek guidance on their therapy from a health-care professional. This instruction is important to prevent undertreatment of higher risk consumers; heeding of this was evaluated by CUSTOM. The study design permitted participants to return to the study site for cholesterol testing as well as to purchase additional drug. Consumers could also obtain cholesterol testing independent of the study structure. Seventy percent of the patients for whom follow-up data were available reported at least one follow-up cholesterol test during the 26 week study (13). Of these participants, 75% made a self-triage decision consistent with the label directions (e.g., to continue therapy based on the result or discontinue therapy and talk to their doctor). This again illustrates the ability of an actual use study to assess complex consumer behaviors over time with minimal cueing. 3

CONCLUSIONS

Label comprehension and actual use studies provide data critical to developing OTC drugs that can be used safely and effectively in the OTC setting. Each study should contain design elements that permit robust assessment of the label elements most important to

6

CASE STUDIES: OVER-THE-COUNTER DRUGS

correct use of the OTC candidate. Statistical analysis of these studies should include prioritized endpoints with prespecified definition of acceptable consumer behavior rates. 3.0.1 Acknowledgment. The author is a consultant to GSK Consumer Healthcare, Johnson&Johnson-Merck, McNeil Consumer Pharmaceuticals, and Novartis Consumer Health. REFERENCES 1. E. P. Brass, Changing the status of drugs from prescription to over-the-counter availability. N Engl J Med. 2001; 345: 810–816. 2. E. P. Brass, ed. Clinical trials to support prescription to over-the-counter switches. In: R. D’Agostino, L. Sullivan, and J. Massaro (eds.), Wiley Encyclopedia of Clinical Trials, New York: Wiley, 2008. 3. E. Brass and M. Weintraub, Label development and the label comprehesion study for over-the-counter drugs. Clin Pharmacol Ther. 2003; 74: 406–412. 4. A. A. Ciociola, M. A. Sirgo, K. A. Pappa, J. A. McGuire, and K. Fung, A study of the nonprescription drug consumer’s understanding of the ranitidine product label and actual product usage patterns in the treatment of episodic heartburn. Am J Ther. 2001; 8: 387–398. 5. E. G. Raymond, S. M. Dalebout, and S. I. Camp, Comprehension of a prototype overthe-counter label for an emergency contraceptive pill product. Obstet Gynecol. 2002; 100: 342–349. 6. J. M. Melin, W. E. Struble, R. Tipping, T. C. Vassil, J. Reynolds, et al., The CUSTOM study: a consumer use study of OTC Mevacor. Am J Cardiol. 2004; 94: 1243–1248. 7. K. Lechter, DDMAC Review Omeprazole Magnesium Tablets Study 3358 Label Comprehension. August 16, 2000. Available from: http:// www.fda.gov/ohrms/dockets/ac/02/briefing/386 1B1 11 Label%20comprehension.pdf

8. S. Henness and C. M. Perry, Orlistat: a review of its use in the management of obesity. Drugs. 2006; 66: 1625–1656. 9. K. B. Feibus, Orlistat OTC, 60mg capsules GlaxoSmithKline (GSK) Consumer Healthcare New Drug Application 21-887 Actual Use Study NM17285. Slide presentation, 2006. Available from: http://www.fda.gov/ ohrms/dockets/ac/06/slides/2006-4201S1 07 FDA-Feibus files/frame.htm#slide0060.htm 10. M. Malone and S. A. Alger-Mayer, Pharmacist intervention enhances adherence to orlistat therapy. Ann Pharmacother. 2003; 37: 1598–1602. 11. A. M. Gotto, Jr., Over-the-counter statins and cardiovascular disease prevention: perspectives, challenges, and opportunities. Clin Pharmacol Ther. 2005; 78: 213–217. 12. J. M. Melin, W. E. Struble, R. W. Tipping, J. M. Reynolds, T. C. Vassil, et al., A consumer use study of over-the-counter lovastatin (CUSTOM). Am J Cardiol. 2004; 94: 1243–1248. 13. E. P. Brass, Consumer behavior in the setting of over-the-counter statin availability: lessons from the consumer use study of OTC Mevacor. Am J Cardiol. 2004; 94: 22F–29F. 14. E. P. Brass, S. E. Allen, and J. M. Melin, Potential impact on cardiovascular public health of over-the-counter statin availability. Am J Cardiol. 2006; 97: 851–856. 15. D. Shetty, Mevacor Daily 20mg Tablets Rx-toOTC Switch. PowerPoint presentation, 2005. Available from: http://www.fda.gov/ohrms/ dockets/ac/05/slides/2005-4086S1 06 FDAShetty.ppt#267,1,MevacorTM

CROSS-REFERENCES Clinical trials to support prescription to over-thecounter switches Confidence interval Statistical analysis plan Informed consent Power

CLUSTER RANDOMIZATION

Several recently published trials that are reviewed in Section 2 will be used to illustrate the key features of cluster randomization. The principles of experimental design, including the benefits of random assignment and the importance of replication, are discussed in Sections 3–5, whereas issues of sample size estimation are considered in Section 6. Methods of analysis at the cluster level and at the individual level are discussed in Sections 7 and 8, respectively, whereas designs involving repeated assessments are considered in Section 9. In Section 10, the authors provide recommendations for trial reporting, and in Section 11, they conclude the article by considering issues arising in meta-analyses that may include one or more cluster randomization trials. Readers interested in a more detailed discussion might wish to consult Donner and Klar (5) from which much of this article was abstracted.

NEIL KLAR PhD Division of Preventive Oncology Cancer Care Ontario Toronto, Ontario, Canada

ALLAN DONNER PhD The University of Western Ontario Department of Epidemiology and Biostatistics London, Ontario, Canada

1

INTRODUCTION

Randomized trials in which the unit of randomization is a community, worksite, school, or family are becoming increasingly common for the evaluation of lifestyle interventions for the prevention of disease. This form of treatment assignment is referred to as cluster randomization or group randomization. Reasons for adopting cluster randomization are diverse, but they include administrative convenience, a desire to reduce the effect of treatment contamination, and the need to avoid ethical issues that might otherwise arise. Data from cluster randomization trials are characterized by between-cluster variation, which is equivalent to saying that responses of cluster members tend to be correlated. Dependencies among cluster members typical of such designs must be considered when determining sample size and in the subsequent data analyses. Failure to adjust standard statistical methods for within-cluster dependencies will result in underpowered studies with spuriously elevated type I errors. These statistical features of cluster randomization were not brought to wide attention in the health research community until the now famous article by Cornfield (1). However the 1980s saw a dramatic increase in the development of methods for analyzing correlated outcome data in general (2) and methods for the design and analysis of cluster randomized trials in particular (3, 4). Books summarizing this research have also recently appeared (5, 6), and new statistical methods are in constant development.

2 EXAMPLES OF CLUSTER RANDOMIZATION TRIALS 1. A group of public health researchers in Montreal (7) conducted a household randomized trial to evaluate the risk of gastrointestinal disease caused by consumption of drinking water. Participating households were randomly assigned to receive an in-home water filtration unit or were assigned to a control group that used tap water. Households were the natural randomization unit in this trial for assessing an in-home water filtration unit. There were 299 households (1206 individuals) assigned to the filtered water group and 308 households (1201 individuals) assigned to the tap water group. The annual incidence of gastrointestinal illness was analyzed using an extension of Poisson regression that adjusted for the within-household correlation in the outcome variable. Based on these analyses, investigators concluded that approximately 35% of the reported gastrointestinal illnesses among control group subjects were preventable.

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

CLUSTER RANDOMIZATION

2. The National Cancer Institute of the United States funded the Community Intervention Trial for Smoking Cessation (COMMIT), which investigated whether a community-level, fouryear intervention would increase quit rates of cigarette smokers (8). Communities were selected as the natural experimental unit because investigators assumed that interventions offered at this level would reach the greatest number of smokers and possibly change the overall environment, thus making smoking less socially acceptable. Random digit dialing was used to identify approximately 550 heavy smokers and 550 light-to-moderate smokers in each community. Eleven matched-pairs of communities were enrolled in this study, with one community in each pair randomly assigned to the experimental intervention and the remaining community serving as a control. Matching factors included geographic location, community size, and general sociodemographic factors. Each community had some latitude in developing smoking cessation activities, which included mass media campaigns and programs offered by health-care providers or through worksites. These activities were designed to increase quit rates of heavy smokers, which in theory, should then also benefit lightto-moderate smokers whose tobacco use tends to be easier to change. The effect of the intervention was assessed at the community level by calculating the difference in community-specific quit rates for each pair. Hypothesis tests were then constructed by applying a permutation test to the 11 matched-pair difference scores, an analytic approach that accounts for both the between-community variability in smoking quit rates as well as for the matching. Further details concerning this cluster level method of analysis are provided in Section 7. Unfortunately, although the experimental intervention offered by COMMIT significantly increased smoking quit rates among light-to-moderate

smokers from about 28% to 31% (P = 0.004, one-sided), no similar effect was identified among the cohort of heavy smokers. 3. Antenatal care in the developing world has attempted to mirror care that is offered in developed countries even though not all antenatal care interventions are known to be effective. The World Health Organization (WHO) antenatal care randomized trial (9) compared a new model of antenatal care that emphasized health-care interventions known to be effective with the standard model of antenatal care. The primary hypothesis in this equivalence trial was that the new model of antenatal health care would not adversely effect the health of women. Participating clinics, recruited from Argentina, Cuba, Saudi Arabia, and Thailand, were randomly assigned to an intervention group or control group separately within each country. Clinics were selected as the optimal unit of allocation in this trial for reasons of administrative and logistic convenience. This decision also reduced the risk of experimental contamination that could have arisen had individual women been randomized. However, random assignment of larger units (e.g., communities) would have needlessly reduced the number of available clusters, thus compromising study power. Twenty-seven clinics (12568 women) were randomly assigned to the experimental arm, whereas 26 control group clinics (11958 women) received standard antenatal care. The primary analyses examining low birthweight (<2500 g) as the principal endpoint were based on (1) extensions of the Mantel–Haenszel test statistic and (2) extensions of logistic regression, both of which were adjusted for betweencluster variability and took into account the stratification by country. The resulting odds ratio comparing experimental to control group women with respect to low birthweight was estimated as equal to 1.06 (95% confidence interval 0.97–1.15). Based on

CLUSTER RANDOMIZATION

these results, and bolstered by similar findings from other study outcomes, the investigators concluded that the new antenatal care model does not adversely affect perinatal and maternal health.

3

PRINCIPLES OF EXPERIMENTAL DESIGN

The science of experimental design as initially put forward by R. A. Fisher (10) was based on the principles of random assignment, stratification, and replication. These principles may be illustrated by considering in more detail the cluster randomization trials described above. All three trials reviewed in Section 2 used random allocation in assigning clusters to an intervention group. They differed, however, in the experimental unit: Random assignment was by household in the Montreal trial (7), by clinic in the WHO Antenatal Care Trial (8), and by community in COMMIT (9). The advantages of random allocation for cluster randomization trials are essentially the same as for clinical trials randomizing individuals. These advantages include the assurance that selection bias has played no role in the assignment of clusters to different interventions; the balancing, in an average sense, of baseline characteristics in the different intervention groups; and formal justification for the application of statistical distribution theory to the analysis of results. A final compelling reason for randomized assignment is that the results are likely to have much more credibility in the scientific community, particularly if they are unexpected. Three designs are most frequently adopted in cluster randomization trials: 1. Completely randomized, involving no pre-stratification or matching of clusters according to baseline characteristics 2. Matched-pair, in which one of two clusters in a stratum is randomly assigned to each intervention 3. Stratified, involving the assignment of two or more clusters to at least some combination of stratum and intervention.

3

The completely randomized design is most appropriate for studies having large numbers of clusters, such as the Montreal trial, which enrolled over 600 households. Matching or stratification is often considered for community intervention trials such as COMMIT (8) in which the numbers of clusters that can be enrolled are usually limited by economic or practical considerations. The main attraction of this design is its potential to provide very tight and explicit balancing of important prognostic factors, thereby improving statistical power. The stratified design is an extension of the matched-pair design in which several clusters, rather than just one, are randomly assigned within strata to each intervention and control group. This design was selected for the WHO antenatal care trial (9). The stratified design has been used much less frequently than either the matched-pair or completely randomized design. However, for many studies, it would seem to represent a sensible compromise between these two designs in that it provides at least some baseline control on factors thought to be related to outcome, while easing the practical difficulties of finding appropriate pair-matches. 4 EXPERIMENTAL AND QUASI-EXPERIMENTAL DESIGNS Investigators may sometimes be hesitant to adopt a random allocation scheme because of ethical and/or practical concerns that arise. For example, the systematic allocation of geographically separated control and experimental clusters might be seen as necessary to alleviate concerns regarding experimental contamination. Nonrandomized designs may also seem easier to explain to officials, to gain broad public acceptance, and more generally, to allow the study to be carried out in a simpler fashion, without the resistance to randomization that is often seen. In this case, well-designed quasi-experimental comparisons (11) will be preferable to obtaining no information at all on the effectiveness of an intervention. At a minimum, it seems clear that non-experimental comparisons may generate hypotheses that can subsequently be tested in a more rigorous framework.

4

CLUSTER RANDOMIZATION

Several reviews of studies in the health sciences have made it clear that random assignment is in fact being increasingly adopted for the assessment of nontherapeutic interventions (5). Similar inroads have been made in the social sciences, where successfully completed randomized trials help investigators to recognize that random assignment may well be possible under circumstances in which only quasi-experimental designs had been previously considered. The availability of a limited number of clusters has been cited as a reason to avoid randomization, because it may leave considerable imbalance between intervention groups on important prognostic factors. This decision seems questionable as there is no assurance that quasi-experimental designs will necessarily create acceptably balanced treatment groups with a limited number of available clusters. Moreover, a matchedpairs or stratified design can usually provide acceptable levels of balance in this case. Concern has been raised by some researchers at the relatively small number of community intervention trials that have actually identified effective health promotion programs (12). It is worth noting, however, that the dearth of effective programs identified using cluster randomization is largely confined to evaluations of behavioral interventions. There has been considerably more success in evaluating medical interventions as in trials assessing the effect of vitamin A on reducing childhood morbidity and mortality (13). There is a long history of methodological research comparing the effectiveness of methods of treatment assignment in controlled clinical trials (14). For example, Chalmers et al. (15) reviewed 145 reports of controlled clinical trials examining treatments for acute myocardial infarction published between 1946 and 1981. Differences in casefatality rates were found in only about 9% of blinded randomized trials, 24% of unblinded randomized studies, and 58% of nonrandomized studies. These differences in the effects of treatment were interpreted by the authors as attributable to differences in patient participation (i.e., selection bias), which should give pause to investigators touting the benefits

of non-randomized comparisons as a substitute for cluster randomization. Murray (16) in addressing the ability of investigators to identify effective behavioral interventions stated that The challenge is to create trials that are (a) sufficiently rigorous to address these issues, (b) powerful enough to provide an answer to the research question of interest, and (c) inexpensive enough to be practical. The question is not whether to conduct group-randomized trials, but how to do them well.

5 THE EFFECT OF FAILING TO REPLICATE Some investigators have designed community intervention trials in which exactly one cluster has been assigned to the experimental group and one to the control group, either with or without the benefit of random assignment (5). Such trials invariably result in interpretational difficulties caused by the total confounding of two sources of variation: (1) the variation in response due to the effect of intervention, and (2) the natural variation that exists between two communities (clusters) even in the absence of an intervention effect. These sources of confounding can be only be disentangled by enrolling multiple clusters per intervention group. The natural between-household variation in infection rates may easily be estimated separately for subjects in the control and experimental arms of the completely randomized Montreal water filtration trial (7). However, this task is more complicated for COMMIT (8), because the pair-matched design adopted for this trial implies that there is only a single community in each combination of stratum and intervention. Inferences concerning the effect of intervention must therefore be constructed using the variation in treatment effect assessed across the 11 replicate pairs. Between cluster variation is also directly estimable in stratified trials such as the WHO antenatal care trial (9) using, for example, the replicate clusters available per intervention group within each of the four participating countries. The absence of replication, although not exclusive to community intervention trials, is

CLUSTER RANDOMIZATION

most commonly seen in trials that enroll clusters of relatively large size. Unfortunately, the large numbers of study subjects in such trials may mislead some investigators into believing valid inferences can be constructed by falsely assuming individuals within a community provide independent responses. A consequence of this naive approach is that the variance of the observed effect of intervention will invariably be underestimated. The degree of underestimation is a function of the variance inflation due to clustering (i.e., the design effect). In particular, the variance inflation factor is given by 1 + (m − 1)ρ, where m denotes average cluster size and ρ is a measure of intracluster correlation, interpretable as the standard Pearson correlation between any two responses in the same cluster. With the additional assumption that the intracluster correlation is non-negative, ρ may also be interpreted as the proportion of overall variation in response that can be accounted for by the between-cluster variation. As demonstrated in Section 6, even relatively small values of intracluster correlation, combined with large cluster sizes, can yield sizable degrees of variance inflation. Moreover, in the absence of cluster replication, positive effects of intervention artificially inflate estimates of between-cluster variation, thus invalidating the resulting estimates of ρ. More attention to the effects of clustering when determining the trial sample size might help to eliminate designs that lack replication. Even so, investigators will still need to consider whether statistical inferences concerning the effect of intervention will be interpretable if only two, or a few, replicate clusters are allocated to each intervention group. 6

SAMPLE SIZE ESTIMATION

A quantitatively justified sample size calculation is almost universally regarded as a fundamental design feature of a properly controlled clinical trial. Methodologic reviews of cluster randomization trials have consistently shown that only a small proportion of these studies have adopted a predetermined

5

sample size based on formal considerations of statistical power (5). There are several possible explanations for the difficulties investigators have faced in designing adequately powered studies. One obvious reason is that the required sample size formulas still tend to be relatively inaccessible, not being available, for example, in most standard texts or software packages. A second reason is that the proper use of these formulas requires some prior assessment of both the average cluster size and the intracluster correlation coefficient ρ. The average cluster size for a trial may at times be directly determined by the selected interventions. For example, households were the natural unit of randomization in the Montreal water filtration study (7), where the average cluster size at entry was approximately four. When relatively large clusters are randomized, on the other hand, subsamples of individual cluster members may be selected to reduce costs. For example, the primary endpoint in the community intervention trial COMMIT (8) was the quit rate of approximately 550 heavy smokers selected from each cluster. Difficulties in obtaining accurate estimates of intracluster correlation are slowly being addressed as more investigators begin publishing these values in their reports of trial results. Summary tables listing intracluster correlation coefficients and variance inflation factors from a wide range of cluster randomization trials and complex surveys are also beginning to appear (5). In practice, estimates of ρ are almost always positive and tend to be larger in smaller clusters. The estimated variance inflation factor may be directly applied to determine the required sample size for a completely randomized design. Let Zα/2 denote the two-sided critical value of the standard normal distribution corresponding to the error rate α, and Zβ denote the critical value corresponding to β. Then, assuming the difference in sample means for the experimental and control groups, Y 1 − Y 2 , can be regarded as approximately normally distributed, the number of subjects required per intervention group is

6

CLUSTER RANDOMIZATION

given by n=

(Zα/2 + Zβ )2 (2σ 2 )[1 + (m − 1)ρ] , (µ1 − µ2 )2

where µ1 − µ2 denotes the magnitude of the difference to be detected, σ 2 denotes the common but unknown variance for an individual subject’s outcome Y, and m denotes the average cluster size. Equivalently, the number of clusters required per group is given by k = n/m. Formulas needed to estimate sample size for pair-matched and stratified designs for a variety of study outcomes are provided by Donner and Klar (5). Regardless of the study design, it is useful to conduct sensitivity analyses exploring the effect on sample size by varying values of the intracluster correlation, the number of clusters per intervention group, and the subsample size. The benefits of such a sensitivity analysis are illustrated by a report describing methodological considerations in the design of the WHO antenatal care trial (9, 17). The values for ρ, suggested from data obtained from a pilot study, varied from 0 to 0.002, whereas cluster sizes were allowed to vary between 300 and 600 patients per clinic. Consequently, the degree of variance inflation due to clustering could be as great as 1 + (600 − 1)0.002 = 2.2. As is typical in such sensitivity analyses, power was seen to be much more sensitive to the number of clusters per intervention group than to the number of patients selected per clinic. Ultimately a total of 53 clinics were enrolled in this trial, with a minimum of 12 clinics per site and with each clinic enrolling approximately 450 women. It must be emphasized that the effect of clustering depends on the joint influence of both m and ρ. Failure to appreciate this point has led to the occasional suggestion in the epidemiological literature that clustering may be detected or ruled out on the basis of testing the estimated value of ρ for statistical significance, i.e., testing H0 : ρ = 0 versus HA : ρ > 0. The weakness of this approach is that observed values of ρ may be very small, particularly for data collected from the very large clusters typically recruited for community intervention trials. Therefore, the power of a test for detecting such values as

statistically significant tends to be unacceptably low (5). Yet small values of ρ, combined with large cluster sizes, can yield sizable values of the variance inflation factor, which can seriously disturb the validity of standard statistical procedures if unaccounted for in the analyses. Thus we would recommend that investigators inherently assume the existence of intracluster correlation, a well-documented phenomenon, rather than attempting to rule it out using statistical testing procedures. 7 CLUSTER LEVEL ANALYSES Many of the challenges of cluster randomization arise when inferences are intended to apply at the individual level, whereas randomization is at the cluster level. If inferences were intended to apply at the cluster level, implying that an analysis at the cluster level would be most appropriate, the study could be regarded, at least with respect to sample size estimation and data analysis, as a standard clinical trial. For example, one of the secondary aims of the Community Intervention Trial for Smoking Cessation (COMMIT) was to compare the level of tobacco control activities in the experimental and control communities after the study ended (18). The resulting analyses were then naturally conducted at the cluster (community) level. Analyses are inevitably more complicated when data are available from individual study subjects. In this case, the investigator must account for the lack of statistical independence among observations within a cluster. An obvious method of simplifying the problem is to collapse the data in each cluster, followed by the construction of a meaningful summary measure, such as an average, which then serves as the unit of analysis. Standard statistical methods can then be directly applied to the collapsed measures, which removes the problem of nonindependence because the subsequent significance tests and confidence intervals would be based on the variation among cluster summary values rather than on variation among individuals. An important special case arises in trials having a quantitative outcome variable when each cluster has a fixed number of subjects.

CLUSTER RANDOMIZATION

In this case, the test statistic obtained using the analysis of variance is algebraically identical to the test statistic obtained using a cluster level analysis (5). Thus, the suggestion that is sometimes made that a cluster level analysis intrinsically assumes ρ = 1 is misleading, because such an analysis can be efficiently conducted regardless of the value of ρ. It is important to note, however, that this equivalence between cluster level and individual level analyses, which holds exactly for quantitative outcome variables under balance, holds only approximately for other outcome variables (e.g., binary, time to event, count). A second implication of this algebraic identity is that the well-known ecological fallacy cannot arise in the case of cluster level intention-to-treat analyses, because the assigned intervention is shared by all cluster members. In practice, the number of subjects per cluster will tend to exhibit considerable variability, either by design or by subject attrition. Cluster level analyses that give equal weight to all clusters may therefore be inefficient. However, it is important to note that appropriately weighted cluster level analyses are asymptotically equivalent to individual level analyses. On the other hand, if there are only a small number of clusters per intervention group, the resulting imprecision in the estimated weights might even result in a loss of power relative to an unweighted analysis. In this case, it might therefore be preferable to consider exact statistical inferences constructed at the cluster level, as based on the randomization distribution for the selected experimental design (e.g., completely randomized, matched-pair, stratified). As noted in Section 2, COMMIT investigators (8) adopted this strategy, basing their primary analysis of tobacco quit rates on the permutation distribution of the difference in event rates within a pair-matched study design. Using a two-stage regression approach, investigators were also able to adjust for important baseline imbalances on known prognostic variables. 8

INDIVIDUAL LEVEL ANALYSES

Standard methods of analysis applied to individually randomized trials have all

7

been extended to allow for the effect of between-cluster sources of variation. These methods include extensions of contingency table methods (e.g., Pearson chi-square test, Mantel–Haenszel methods) and of two sample t-tests. More sophisticated extensions of multiple regression models have also been developed and are now available in standard statistical software. Attention in this section will be focused on methods for the analysis of binary outcome data, which arise more frequently in cluster randomization trials than continuous, count, or time-to-event data. The discussion will also be limited to data obtained from completely randomized and stratified designs. Methods of analysis for other study outcomes are considered in detail elsewhere (5), whereas some analytic challenges unique to pair-matched trials are debated by Donner and Klar (5) and by Feng et al. (19). Analyses of data from the WHO antenatal care trial (9) are now considered in which clinics were randomly assigned to experimental or control groups separately within each of the four participating sites (countries). An extension of the Mantel–Haenszel statistic adjusted for clustering was used to compare the risk of having a low birthweight outcome for women assigned to either the new model of antenatal care or a standard model. For clusters of fixed size m, this statistic is equal to the standard Mantel–Haenszel statistic divided by the variance inflation factor 1 + (m − 1)ρ, ˆ where ρˆ is the sample estimate of ρ. Thus, failure to account for between cluster variability (i.e., incorrectly assuming ρ = 0) will tend to falsely increase the type I error rate. A key advantage of this approach is that the resulting statistic simplifies to the standard Mantel–Haenszel test statistic. Similar advantages are shared by most other individual level test statistics. Additional analyses reported in this trial used an extension of logistic regression that allowed adjustment for other potential baseline predictors of low birth-weight including maternal education, maternal age, and nulliparity. These analyses allowed examination of the joint effects of individual-level and cluster-level predictors (i.e., intervention, strata).

8

CLUSTER RANDOMIZATION

Two frequently used extensions of logistic regression are the logistic-normal model and the generalized estimating equations (GEEs) extension of this procedure (5). The logisticnormal model assumes that the logit transform of the probability of having a low birthweight outcome follows a normal distribution across clusters. The resulting likelihood ratio tests will have maximum power for detecting effects of intervention as statistically significant when parametric assumptions such as these are satisfied. It may be difficult in practice to know whether the assumptions underlying the use of parametric models are reasonable. Therefore, attention is limited here to the GEE approach, which has the advantage of not requiring specification of a fully parametric distribution. Two distinct strategies are available to adjust for the effect of clustering using the GEE approach. The first can be said to be model-based, as it requires the specification of a working correlation matrix that describes the pattern of correlation between responses of cluster members. For cluster randomization trials, the simplest assumption to make is that responses of cluster members are equally correlated, i.e., to assume the correlation structure within clusters is exchangeable. The second strategy that may be used to adjust for the effect of clustering employs ‘‘robust variance estimators’’ that are constructed using betweencluster information. These estimators consistently estimate the true variance of estimated regression coefficients even if the working correlation matrix is misspecified. Moreover, provided there are a large number of clusters, inferences obtained using robust variance estimators will become equivalent to those obtained using the model-based strategy provided the working correlation matrix is correctly specified. The examples we have considered involve only a single level of clustering. More sophisticated multilevel methods of analysis are available (5, 20) that allow examination of effects at two more levels. For example, women participating in the WHO antenatal care trial might have been cared for by a specific physician within each clinic. Responses of women would then be clustered by physician nested within clinics, generating two

levels of clustering. This additional structure could then be used to explore differential intervention effects across physicians, e.g., to consider whether years of training was associated with relatively fewer women having low weight babies. Although these analyses may enrich our understanding of the trial results, they are almost always exploratory in nature. It is important to note that statistical inferences constructed using individual level analyses are approximate, with their validity only assured in the presence of a large number of clusters. This requirement essentially flows from the difficulty in accurately estimating between-cluster sources of variation. Thus the validity of statistical inferences constructed using individual level analyses may be in question should there be fewer than 20 clusters enrolled. If a small number of clusters are enrolled, it may only be possible to consider cluster level analyses by constructing statistical inferences based on the selected randomization distribution. 9 INCORPORATING REPEATED ASSESSMENTS Investigators are often interested in considering the longitudinal effects of intervention as part of a cluster randomized trial. The choice here is between a cohort design that tracks the same individuals over time and a repeated cross-sectional design that tracks the same clusters over time, but draws independent samples of individuals at each calendar point. In this section, how the study objectives should determine the choice between these designs and how the resulting decision affects data collection, data analysis, and the interpretation of study results are outlined. Cohort samples of subjects were included in each of the three trials presented in Section 2. For example, smoking quit rates were measured for subsamples of heavy and light-to-moderate smokers selected from each of the participating COMMIT communities (8). Cohort members were followed and contacted annually during the four-year intervention. The length of follow-up time was a consequence of the study objectives for which smoking quit rates were defined ‘‘ . . . as the

CLUSTER RANDOMIZATION

fraction of cohort members who had achieved and maintained cessation for at least six months at the end of the trial.’’ This outcome illustrates how cohort designs are best suited to measuring change within individual participants, implying that the unit of inference is most naturally directed at the individual level. A desirable design feature of COMMIT was that subjects were selected before random assignment with the avoidance of any concerns regarding possible selection bias. This strategy was not available for the WHO antenatal care trial (9), as women could only be identified after their initial clinic visit, which for most women occured after random assignment. Selection bias is unlikely, however, because all new patients from participating clinics were enrolled in the trial and birthweight data from singleton births were available for 92% of women from each intervention group. A secondary objective of COMMIT was to determine whether the intervention would decrease the prevalence of adult cigarette smoking (8). This objective was achieved by conducting separate surveys before random assignment and after completion of the intervention. A principal attraction of such repeated cross-sectional surveys is that any concerns regarding the effects of possible attrition would be avoided. Of course, differential rates of participation in crosssectional surveys conducted after random assignment can still compromise validity, because willingness to participate may be a consequence of the assigned intervention. Nonetheless, random samples of respondents at each assessment point will be more representative of the target population than will be a fixed cohort of smokers. The final decision regarding the selection of a cohort or cross-sectional design should be based primarily on the study objectives and the associated unit of inference. However, it can still be informative to quantitatively evaluate the relative efficiency of the two designs (21). As repeated assessments are made on the same subjects, the cohort design tends to have greater power than a design involving repeated cross-sectional surveys. Note, however, that in practice, subject attrition may eliminate these potential gains in power.

9

The number and timing of assessments made after baseline should be determined by the anticipated temporal responses in each intervention group. For example, it might be reasonable to expect different linear trends over time across intervention groups in a community randomized trial of smoking cessation if the effects of intervention were expected to diffuse slowly through each community. Alternatively, the effects of intervention might diffuse rapidly but be transient, requiring a more careful determination of assessment times to ensure that important effects are not missed. The methods of analysis presented in Sections 7 and 8 assumed the presence of only a single assessment after random assignment. Extensions of these methods to cluster randomization trials having longitudinal outcome measures are beginning to appear (22, 23). 10

STUDY REPORTING

Reporting standards for randomized clinical trials have now been widely disseminated (24). Many principles that apply to trials randomizing individuals also apply to trials randomizing intact clusters. These principles include a carefully posed justification for the trial, a clear statement of the study objectives, a detailed description of the planned intervention, and an accurate accounting of all subjects randomized to the trial. Unambiguous inclusion–exclusion criteria must also be formulated, although perhaps separately for cluster level and individual level characteristics. There are, however, some unique aspects of cluster randomization trials that require special attention at the reporting stage. The focus here is on some of the most important of these aspects. The decreased statistical efficiency of cluster randomization relative to individual randomization can be substantial, depending on the sizes of the clusters randomized and the degree of intracluster correlation. Thus, unless it is obvious that there is no alternative, the reasons for randomizing clusters rather than individuals should be clearly stated. This information, accompanied by a clear description of the units randomized, can

10

CLUSTER RANDOMIZATION

help a reader decide whether the loss of precision due to cluster randomization is in fact justified. Having decided to randomize clusters, investigators may still have considerable latitude in their choice of unit. As different levels of statistical efficiency are associated with different cluster sizes, it would seem important to select the unit of randomization on a carefully considered basis. An unambiguous definition of the unit of randomization is also required. For example, a statement that ‘‘neighbourhoods’’ were randomized is clearly incomplete without a detailed description of this term in the context of the planned trial. The clusters that participate in a trial may not be representative of the target population of clusters. Some indication of this lack of representativeness may be obtained by listing the number of clusters that met the eligibility criteria for the trial, but which declined to participate, along with a description of their characteristics. A continuing difficulty with reports of cluster randomization trials is that justification for the sample size is all too often omitted. Investigators should clearly describe how the sample size for their trial was determined, with particular attention given to how clustering effects were adjusted. This description should be in the context of the experimental design selected (e.g., completely randomized, matched-pair, stratified). It would also be beneficial to the research community if empirical estimates of ρ were routinely published (with an indication of whether the reported values have been adjusted for the effect of baseline covariates). It should be further specified what provisions, if any, were made in the sample size calculations to account for potential loss to follow-up. As the factors leading to the loss to follow-up of individual members of a cluster may be very different from those leading to the loss of an entire cluster, both sets of factors must be considered here. A large variety of methods, based on very different sets of assumptions, have been used to analyze data arising from cluster randomization trials. For example, possible choices for the analysis of binary outcomes include adjusted chi-square statistics, the method of

(GEE), and logistic-normal regression models. These methods are not as familiar as the standard procedures commonly used to analyze clinical trial data, partly because the methodology for analyzing cluster randomization trials is in a state of rapid development, with virtually no standardization and a proliferation of associated software. Therefore, it is incumbent on authors to provide a clear statement of the statistical methods used, and accompanied, where it is not obvious, by an explanation of how the analysis adjusts for the effect of clustering. The software used to implement these analyses should also be reported. 11 META-ANALYSIS Meta-analyses involving the synthesis of evidence from cluster randomization trials raise methodologic issues beyond those raised by meta-analyses of individually randomized trials. Two of the more challenging of these issues are (1) the increased likelihood of study heterogeneity, and (2) difficulties in estimating design effects and selecting an optimal method of analysis (25). These issues are illustrated in a metaanalysis examining the effect of vitamin A supplementation on child mortality (13). This investigation considered trials of hospitalized children with measles as well as communitybased trials of healthy children. Individual children were assigned to intervention in the four hospital-based trials, whereas allocation was by geographic area, village, or household in the eight community-based trials. One community-based trial included only one geographic area per intervention group, each of which enrolled approximately 3000 children. On the other hand, there was an average of about two children from each cluster when allocation was by household. Thus, an important source of heterogeneity arose from the nature and size of the randomization units allocated in the different trials. This problem was dealt with by performing the meta-analysis separately for the individually randomized and cluster randomized trials. It is straightforward to summarize results across trials when each study provides a common measure for the estimated effect

CLUSTER RANDOMIZATION

of intervention (such as an odds ratio, for example) and a corresponding variance estimate that appropriately accounts for the clustering. Unfortunately, the information necessary for its application in practice is rarely available to meta-analysts. One consequence of this difficulty is that investigators are sometimes forced to adopt ad-hoc strategies when relying on published trial reports that fail to provide estimates of the variance inflation factor. For example, in the meta-analysis described above, only four of the eight community-based trials reported that they accounted for clustering effects. The authors argued that increasing the variance of the summary odds ratio estimator computed over all eight trials by an arbitrary 30% was reasonable because the design effects ranged from 1.10 to 1.40 in those studies that did adjust for clustering effects. Even when each trial provides an estimate of the design effect, several different approaches could be used for conducting a meta-analysis. For example, a procedure commonly adopted for combining the results of individually randomized clinical trials with a binary outcome variable is the wellknown Mantel–Haenszel test. The adjusted Mantel–Haenszel test (5) may be used to combine results of cluster randomized trials. Other possible approaches are discussed by Donner et al. (26). REFERENCES 1. J. Cornfield, Randomization by group: a formal analysis. Amer. J. Epidemiol. 1978; 108: 100–102. 2. M. Ashby, J. M. Neuhaus, W. W. Hauck et al., An annotated bibliography of methods for analyzing correlated categorical data. Stat. Med. 1992; 11: 67–99. 3. A. Donner, N. Birkett, and C. Buck, Randomization by cluster: sample size requirements and analysis. Amer. J. Epidemiol. 1981; 114: 906–914. 4. R. F. Gillum, P. T. Williams, and E. Sondik, Some consideration for the planning of totalcommunity prevention trials: when is sample size adequate? J. Commun. Health 1980; 5: 270–278. 5. A. Donner and N. Klar, Design and Analysis of Cluster Randomization Trials in Health Research. London: Arnold, 2000.

11

6. D. M. Murray, Design and Analysis of GroupRandomized Trials. Oxford: Oxford University Press, 1998. 7. P. Payment, L. Richardson, J. Siemiatycki, R. Dewar, M. Edwardes, and E. Franco, A randomized trial to evaluate the risk of gastrointestinal disease due to consumption of drinking water meeting microbiological standards. Amer. J. Public Health 1991; 81: 703–708. 8. COMMIT Research Group, Community Intervention Trial for Smoking Cessation (COMMIT): I. Cohort results from a four-year community intervention. Amer. J. Public Health 1995; 85: 183–192. 9. J. Villar, H. Ba’aqeel, G. Piaggio, et al. for the WHO Antenatal Care Trial Research Group, WHO antenatal care randomised trial for the evaluation of a new model of routine antenatal care. Lancet 2001; 357: 1551–1564. 10. D. A. Preece, RA Fisher and experimental design: a review. Biometrics 1999; 46: 925–935. 11. W. R. Shadish, T. D. Cook, and D. T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin Company, 2002. 12. C. Merzel and J. D’Affitti, Reconsidering community-based health promotion: promise, performance, and potential. Amer. J. Public Health 2003; 93: 557–574. 13. W. W. Fawzi, T. C. Chalmers, M. G. Herrera, and F. Mosteller, Vitamin A supplementation and child mortality, a meta-analysis. JAMA 1993; 269: 898–903. 14. M. McKee, A. Britton, N. Black, K. McPherson, C. Sanderson, and C. Bain, Interpreting the evidence: choosing between randomised and non-randomised studies. Brit. Med. J. 1999; 319: 312–315. 15. T. C. Chalmers, P. Celano, H. S. Sacks, and H. Smith, Jr., Bias in treatment assignment in controlled clinical trials. N. Engl. J. Med. 1983; 309: 1358–1361. 16. D. M. Murray, Efficacy and effectiveness trials in health promotion and disease prevention: design and analysis of group-randomized trials. In: N. Schneiderman, M. A. Speers, J. M. Silva, H. Tomes, and J. H. Gentry (eds.), Integrating Behavioral and Social Sciences with Public Health. Washington, DC: American Psychology Association, 2001. 17. A. Donner, G. Piaggio, J. Villar et al. for the WHO Antenatal Care Trial Research Group, Methodological considerations in the design

12

CLUSTER RANDOMIZATION

of the WHO Antenatal care Randomised Controlled Trial. Paediatr. Perinatal Epidemiol. 1998; 12(Sup 2): 59–74. 18. B. Thompson, E. Lichtenstein, K. Corbett, L. Nettekoven, and Z. Feng, Durability of tobacco control efforts in the 22 Community Intervention Trial for Smoking Cessation (COMMIT) communities 2 years after the end of intervention. Health Educ. Res. 2000; 15: 353–366. 19. Z. Feng, P. Diehr, A. Peterson, and D. McLerran, Selected statistical issues in group randomized trials. Annu. Rev. Public Health 2001; 22: 167–187. 20. A. V. Diez-Roux, Multilevel analysis in public health research. Annu. Rev. Public Health 2000; 21: 171–192. 21. H. A. Feldman and S. M. McKinlay, Cohort vs. cross-sectional design in large field trials: precision, sample size, and a unifying model. Stat. Med. 1994; 13: 61–78. 22. A. I. Sashegyi, K. S. Brown, and P. J. Farrell, Application of a generalized random effects regression model for cluster-correlated longitudinal data to a school-based smoking prevention trial. Amer. J. Epidemiol. 2000; 152: 1192–1200. 23. D. M. Murray, P. J. Hannan, R. D. Wolfinger, W. L. Baker, and J. H. Dwyer, Analysis of data from group-randomized trials with repeat observations on the same groups. Stat. Med. 1998; 17: 1581–1600. 24. D. Moher, K. F. Schulz, and D. G. Altman for the CONSORT Group, The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 2001; 357: 1191–1194. 25. A. Donner and N. Klar, Issues in the metaanalysis of cluster randomized trials. Stat. Med. 2002; 21: 2971–2980. 26. A. Donner, G. Paiggio, and J. Villar, Statistical methods for the meta-analysis of cluster randomization trials. Statistic. Meth. Med. Res. 2001; 10: 325–338.

CODE OF FEDERAL REGULATIONS (CFR) The Code of Federal Regulations (CFR) is a codification of the general and permanent rules published by the U.S. Government Printing Office in the Federal Register by the Executive departments and agencies of the federal government. The CFR is divided into 50 titles, which represent broad areas subject to federal regulation. Each title is divided into chapters that usually bear the name of the issuing agency. Each chapter is subdivided even more into parts that cover specific regulatory areas. Large parts may be subdivided into subparts. All parts are organized in sections, and most citations to the CFR will be to the section level.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cdrh/devadvice/365.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

COHERENCE IN PHASE I CLINICAL TRIALS

in patients, as opposed to those in healthy volunteers. 1 COHERENCE: DEFINITIONS AND ORGANIZATION

YING KUEN (KEN) CHEUNG Columbia University New York

Phase I dose-finding designs dictate dose assignments during a trial in an outcomeadaptive manner, and escalate or de-escalate doses for future patients based on the previous observations. An escalation for the next patient is said to be incoherent if the most recent patient experiences a DLT. A dosefinding design is coherent in escalation if it does not induce incoherent escalation in all possible outcome sequences, that is, it allows escalation only after a non-DLT observation. For example, Fig. 1 shows an outcome sequence with an incoherent escalation, and thus by definition the design that generates this outcome sequence is not coherent in escalation. By the same token, a deescalation for the next patient is incoherent if the most recent patient has no sign of toxicity. A design is coherent in de-escalation if it does not induce any incoherent de-escalation. Figure 2 shows an example of incoherent de-escalation. Examples of coherent designs and the utility of coherence will be discussed under the headings ‘‘Coherent designs’’ and ‘‘Compatible initial design.’’ Because coherent escalation and deescalation are defined with respect to the outcome of the most recent patient, the concept of coherence is directly applicable to clinical trials with short-term toxicity so that the toxicity outcome from the most recent patient is readily available when the next patient is enrolled. In the clinical situations where the DLT is defined with respect to a nontrivial period of time, it becomes infeasible to wait for the outcome of the most recent patient before enrolling a new patient because it may cause repeated accrual suspensions and hence long trial duration. Two common ways to handle these situations are to enroll patients in groups and to monitor the trial in real-time. Therefore, extensions of the basic concept of coherence to these clinical settings are needed to enhance its

In the early phase clinical development of a new treatment, phase I trials are typically small studies that evaluate toxicity and determine a safe dosage range. Often, several doses of the treatment will be tested; in these situations, a more specific objective is to determine the maximum dose that does not exceed an acceptable level of toxicity, thus called the maximum tolerated dose (MTD). For ethical reasons, balanced randomization to each dose level is not feasible because it may unknowingly allocate many patients to excessively toxic doses. Rather, dose assignments in phase I trials are usually made in an outcome-adaptive manner in that previous outcomes in the trial form the basis of dose assignments for future subjects. Conventionally, the MTD is approached from lower doses according to the rule that escalates dose after a cohort of three subjects who show no treatment-related adverse or toxic event (Table 1). Although this conventional method has several limitations and shortcomings in terms of the statistical properties (1), it embraces a principle that appears to be clinically and ethically sound. When the most current subject suffers a treatment-related toxic event or dose-limiting toxicity (DLT), no escalation should take place for the next subject without testing the current or the lower doses in more subjects. This principle is called coherence in escalation (2). For phase I trials conducted in patients such as those in oncology, the flipside of the ethical concern is to avoid treating many patients at very low and hence inefficacious doses (3, 4). Analogously, coherence in de-escalation (2) stipulates that de-escalation should not take place when no DLT is observed in the current patient. This article reviews the coherence principles in phase I trials and some related concepts and applications. For brevity in discussion, we will focus on trials conducted

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

COHERENCE IN PHASE I CLINICAL TRIALS Table 1. Conventional Rules for Escalation and De-escalation from a Given Dose∗ # DLT /# Patients in Cohort 1 Cohort 2 0/3 1/3 1/3 ≥2/3

Action/conclusion

Not necessary 0/3 ≥1/3 Not necessary

Escalate dose Escalate dose or called MTD MTD has been exceeded; de-escalate dose MTD has been exceeded; de-escalate dose

∗ Up to two cohorts of three patients will be enrolled to a dose, and escalation will occur only when all patients in the most current cohort have no DLT.

5

Dose Level

4

3

2

1

1

3

5

7

9

11

13

15

Patient Number Figure 1. Dose assignments and outcomes of the first 15 patients generated according to a twostage CRM for a 25% target DLT rate. Each point represents a patient with a circle ‘‘o’’ indicating no DLT, and a cross ‘‘x’’ DLT. The initial design escalates doses after every three successive non-DLT observations; the model setup for the CRM is the same as that used in Reference (2). Escalation for patient 11 is incoherent after a DLT observation from patient 10. This escalation also violates compatibility because patient 11 would have received dose level 4 according to the initial design if patient 10 did not have DLT.

practical use. These extensions are to be discussed under the headings ‘‘group coherence’’ and ‘‘real-time coherence.’’ 2

COHERENT DESIGNS

The phase I design literature clearly indicates that it is almost instinctive to enforce coherence as a consequence of ethical considerations, even before the term ‘‘coherence’’ was coined in Reference 2. Phase I designs can be generally classified as algorithm-based and model-based (5). An algorithm-based design prespecifies the rules of escalation and

de-escalation before a trial starts. Examples include the conventional method (Table 1), the up-and-down designs (6), and the biased coin design (7). Coherence is enforced in the biased coin design by construction of the rules, and it is an intrinsic property of the method. The conventional method and the up-and-down designs are also coherent in escalation by construction; moreover, implementation with common sense will suffice coherence in de-escalation. For example, if the conventional method is used to facilitate dose assignments, and if the first two patients at a dose have DLT, then one should

COHERENCE IN PHASE I CLINICAL TRIALS

de-escalate dose for the next patient instead of rigidly treating a third patient at the same dose as in Fig. 2. A model-based design chooses the next dose based on a dose-toxicity curve, which is estimated repeatedly throughout the trial. As no fast and hard rule exists to determine escalation and de-escalation, there is no guarantee that a pure model-based design will adhere to coherence. Even if coherence holds, it is not as obvious as in an algorithm-based design, and it needs to be proved on a methodby-method basis. The continual reassessment method (CRM) and its modified versions (3, 8–12) are by far the most commonly used model-based design in phase I trials. Briefly, the original proposal of the CRM (3) defines the MTD as a dose that causes DLT in the patient population with a prespecified target rate, assumes a one-parameter model to describe the dose–toxicity relationship, treats the first patient at the prior MTD based on the model, and repeatedly estimates the model-based MTD after every patient so that the subsequent patient is treated at the updated MTD estimate. This original version of the CRM, regardless of the functional form of the model used, has been proved to be coherent (2). For practical purposes, several authors have suggested to start a trial conservatively at the lowest dose instead of the modelbased prior MTD (8–12). As a result of this deviation from the initial model-based dose assignment, the subsequent model-based dose assignments are not necessarily coherent in escalation (see Fig. 1). Being aware of the potential of incoherent escalation, all authors of the modified CRM routinely apply an additional restriction that the dose level for the next patient cannot be higher than the level for the most recent patient who has just had a DLT; see Reference 8 for example. In other words, coherence in escalation is enforced by restriction. Alternatively, Reference (2) suggests a simple way to achieve coherence of a modified CRM by calibration of the design parameters. An advantage of this latter calibration approach is that it ensures the initial dose escalation plan to be compatible with the model-based plan and the chosen target DLT rate; see the section ‘‘Compatible

3

initial design.’’ All the modified CRM referenced above are coherent in de-escalation. 3

COMPATIBLE INITIAL DESIGN

As mentioned earlier, a practical modification of the CRM is to start the trial at the lowest dose instead of the model-based prior MTD. As a result, an additional set of rules or an initial design is needed to govern escalation at the beginning of the trial before it turns to the model-based assignments. A two-stage CRM design is thus formed by an initial design and a CRM stage. In a two-stage design, the first observed DLT is often the trigger on which the trial will be switched to the CRM stage (11, 12). Therefore, the initial design in effect dictates dose assignments (escalations) in a trial when there is no DLT. As such, the initial design should represent the fastest escalation scheme that takes place when no DLT is observed throughout the entire trial. (Otherwise, it seems hard to justify the quickening of escalation on the observation of any DLT.) Motivated by this assertion, an initial design is said to be compatible with the CRM component in a two-stage design with respect to a target DLT rate if dose escalation according to the two-stage design is no faster after observation of a DLT than it would have been in the absence of DLT. As a general rule of thumb, incompatibility can be caused by (1) an initial design that escalates after a conservatively large number of non-DLT observations, and/or (2) a large target DLT rate. Capitalizing on this relationship between the initial design and the target DLT rate, one may calibrate an initial design so that it is compatible with the CRM component in the two-stage design with respect to the target rate. For example, one may start by considering the initial design that escalates after groups of moderate-sized (e.g., five patients) and by decreasing the group size until compatibility is satisfied. A sufficient but not necessary condition for compatibility of an initial design is to verify that the two-stage design thus obtained is coherent by calibration (2) instead of by restriction; see the discussion at the end of the previous section. The outcome sequence in Fig. 1 is in fact generated by a two-stage design for a 25%

4

COHERENCE IN PHASE I CLINICAL TRIALS

5

Dose Level

4 3 2 1

1

3

5

7 9 Patient Number

11

target DLT rate with an initial design that escalates dose after every three non-DLT observations. Once the first DLT is observed, the dose-toxicity curve would be updated after every single patient, and the next patient will be given the dose with DLT rate estimated to be closest to the target 25% according to the CRM. The dose-toxicity model setup for the CRM is the same as that used in Reference 2. The escalation for Patient 11 is incoherent and leads to the violation of compatibility. Thus, by the principles of coherence and compatibility, this initial design is overly conservative—this observation is contrary to the conventional yet unexamined intuition that groups-ofthree is reasonable for a 25% target rate. In particular, it can be verified that a groupsof-two initial design is compatible with the CRM setup in Fig. 1; see Reference 2. 4

GROUP COHERENCE

4.1 Clinical Settings In situations in which the DLT can only be evaluated after a nontrivial period of time (i.e., late toxicity) and patient enrollment is comparatively fast, the duration of a trial can be substantially reduced by making interim dose decisions after every small group of patients, as opposed to every single patient, is accrued. Thus, we will view each group of patients, instead of each individual patient, as an experimental unit for dose assignment purposes.

13

15

Figure 2. Dose assignments and outcomes generated according to the conventional method in Table 1. Each point represents a patient with a circle ‘‘o’’ indicating no DLT, and a cross ‘‘x’’ DLT. Deescalation for patient 13 is incoherent after a non-DLT observation from patient 12.

4.2 Definitions An escalation for the next group of patients is said to be group incoherent when the observed DLT rate in the most recent group is larger than or equal to the target DLT rate. Analogously, a de-escalation for the next group is group incoherent if the observed DLT rate in the most recent group is less than or equal to the target DLT rate. A dose-finding design is group coherent if it does not induce group incoherent escalation or de-escalation in any possible outcome sequences. Figure 3 provides an illustration of a group incoherent design. 4.3 Examples Examples of grouped accrual designs include the conventional method, Storer’s designs C and D (6), and grouped accrual CRM (10). Whether a design is group coherent depends on the target DLT rate. Take Storer’s design C as an example, which enrolls two patients at a time, it escalates if no DLT occurs, and it de-escalates if at least one DLT occurs. In other words, a de-escalation will occur when the observed DLT rate is 50%. Therefore, Storer’s design C is group incoherent in deescalation for a target DLT rate that is larger than 50%, although we note that such a high target rate is an unlikely choice in practice. Goodman et al. (10) propose grouped accrual CRM in attempt to reduce the trial duration imposed by the original CRM that enrolls one patient at a time. If the first group

COHERENCE IN PHASE I CLINICAL TRIALS

5

5 0.67

Figure 3. Dose assignments and outcomes of the first six patient groups (of size three) generated according to a grouped accrual CRM that starts at the lowest dose for a 25% target DLT rate. Each number indicates the observed DLT rate of the group. Escalation for Group 4 is group incoherent. The model setup for the CRM is the same as that used in Reference (2).

Dose Level

4 3

0.33

2

0.00

5

6

0.00

1 0.00

1

of patients is treated at the prior MTD without resorting to an initial design, the grouped accrual CRM is group coherent (2). However, if the trial starts at the lowest dose as suggested also in (10), then the grouped accrual CRM without additional restriction is not necessarily group coherent (Fig. 3). In this case, therefore, one may need to apply the ad hoc restriction of no escalation (de-escalation) if the observed DLT rate exceeds (falls below) the target DLT rate. Alternatively, one can achieve group coherence by calibrating the group size and the CRM parameters using the technique in Reference 2. 5

0.33

REAL-TIME COHERENCE

5.1 Clinical Settings In situations where a DLT may occur at a random time during the observation period, partial information from previously enrolled patients may be available as a new patient becomes eligible for the trial; thus, each individual patient should be treated as an experimental unit. For example, suppose that each patient is to be followed monthly for a maximum of six months for DLT evaluation. By the time of the arrival of a new patient, complete information is available from the currently enrolled patients who have already had a DLT. For those without any sign of DLT, we still have partial information because a patient without DLT for five months apparently conveys more

2

3 4 Group Number

safety information about the treatment than a patient without DLT for only one month. Ignoring such information is inefficient and could cause bias. In addition, incoherent moves may occur if we rigidly implement a grouped accrual design without considering the updated information between patient enrollments. Consider Fig. 2 for example: Suppose that patient 11 has already had a DLT by the time when patient 12 arrives, treating patient 12 at dose level 4 according to the conventional method leads to an incoherent de-escalation for patient 13. In this particular case, to avoid incoherence, deescalation should have taken place for patient 12. In general clinical settings where potentially updated information exists between patient enrollments, dose assignment decisions should be made continuously on real time during the trial. 5.2 Definitions For a phase I design defined on real time, coherence in de-escalation between any two time points stipulates that dose assignment at the later time point should not be lower than that at an earlier time point if no new DLT is observed in all enrolled patients between the two time points; see Fig. 4 for a graphical illustration. In contrast, it is difficult to provide a practical definition for coherence in escalation between any two time points. If a patient experiences a DLT at the later time point, then escalation from the earlier time

6

COHERENCE IN PHASE I CLINICAL TRIALS

Patient Number

8

6

X

4

2

0

2

4

6

8

10

12

Study time (months) Figure 4. Enrollment of the first eight patients to a phase I trial on the study timeline: Each line represents a patient with ‘‘•’’ indicating the entry time of the patient; a line ending with a cross ‘‘x’’ indicates the time of a DLT, and ending with a circle ‘‘o’’ indicates no DLT at the end of follow-up, which is six-month after study entry. According to coherence in de-escalation, patient 7 (entry at 4 months since study began) should receive a dose no less than that received by patients 1–6 because no DLT is observed during the period between the entries of these previous patients and four months. On the other hand, patient 8 (entry at 6 months) may receive a dose higher than that received by patient 7 without violating coherence in escalation despite that a DLT occurs between four and six months.

point may still be reasonable because other patients without DLT will also have longer follow-up at the later time point than at the earlier time point and serve as stronger evidence for the safety of the treatment at the later time. Consider the entry of patient 8 in Fig. 4. Although a DLT occurs between the entries of patient 7 (4 months) and patient 8 (6 months), the longer DLT-free follow-up of patients 1 through 6 at 6 months provides stronger evidence for the safety of the treatment than at 4 months. Therefore, it may not be unreasonable to escalate dose for patient 8 from that of patient 7. However, if the two time points are close enough, the patients who remain without DLT will contribute only little additional information to offset it because of the DLT observed at the later time point. Letting the two time points be arbitrarily close, we can then define real-time coherence in escalation as a condition that no escalation is allowed in such a short time interval; a rigorous, mathematical definition is given in Reference 2. We note

that this definition bears little practical relevance because the monitoring intervals are seldom short. However, the calculus involved is a useful, theoretical concept for a real-time dose-finding design to respect. We contend that a dose-finding design with desirable theoretical property such as consistency (13) and coherence is a necessary, yet by no means sufficient, condition for its general use. 5.3 Examples The time-to-event continual reassessment method (TITE-CRM) (14) is the first phase I design defined on real time. The main idea of the method is to incorporate into the CRM the partial information from incompletely followed patients by down-weighing the non-DLT observations according to the length of the follow-up via a weight function. If the weight function is chosen such that the weight given to each DLT-free patient is increasing as the trial progresses, the TITECRM will then be coherent. This monotonic increasing requirement is intuitive because

COHERENCE IN PHASE I CLINICAL TRIALS

as the follow-up period of a DLT-free patient becomes longer, his/her contribution to the likelihood (via his/her weight) should also increase. Readers interested in the technical details are referred to theorem 3 in Reference 2. Several recent developments in real-time phase I designs have occurred; see References 15 and 16 for example. Whether these methods are coherent have not yet been examined.

6

DISCUSSION

To summarize, this chapter reviews the concept and the applications of coherence under various phase I trial settings with (1) shortterm toxicities, (2) late toxicities, and (3) toxicities that occur at random in a nontrivial period of time. These principles are motivated by general ethical considerations rather than the operating characteristics of particular dose-finding designs. Therefore, any coherent designs should be evaluated with extensive simulations and numerical techniques (13) before being used in actual trials. This being said, calibrating coherent two-stage CRM avoids over-conservative initial escalation plan (see discussion in the section ‘‘Compatible initial design’’), and hence seems to adhere to motivating rationales of the CRM (3), which is to place fewer patients at the low doses. If one finds a compatible initial design to be too aggressive, it indicates that the specified target DLT rate may be too large, and a lower tolerance should be specified. Indeed, the concept of coherence provides an objective bridge between the dose escalation plan and the specified target.

REFERENCES 1. B. Storer and D. DeMets, Current phase I/II designs: are they adequate? J. Clin. Res. Drug Dev. 1987; 1: 121–130.

7

4. D. D. Rosa, J. Harris, and G. C. Jayson, The best guess approach to phase I trial design. J. Clin. Oncol. 2006; 24: 206–208. 5. Chevret, S (ed.), Statistical Methods for Dose-Finding Experiments. New York: John Wiley & Sons, 2006. 6. B. E. Storer, Design and analysis of phase I clinical trials. Biometrics 1989; 45: 925–937. 7. S. D. Durham, N. Flournoy, and W. F. Rosenberger, A random walk rule for phase I clinical trials. Biometrics 1997; 53: 745–760. 8. D. Faries, Practical modifications of the continual reassessment method for phase I cancer clinical trials. J. Biopharm. Stat. 1994; 4: 147–164. 9. E. L. Korn, D. Midthune, T. T. Chen, L. V. Rubinstein, M. C. Christian, and R. M. Simon, A comparison of two phase I trial designs. Stat. Med. 1994; 13: 1799–1806. 10. S. N. Goodman, M. L. Zahurak, and S. Piantadosi, Some practical improvements in the continual reassessment method for phase I studies. Stat. Med. 1995; 14: 1149–1161. 11. S. Møller, An extension of the continual reassessment methods using a preliminary up-and-down design in a dose finding study in cancer patients, in order to investigate a greater range of doses. Stat. Med. 1995; 14: 911–922. 12. J. O’Quigley and L. Z. Shen, Continual reassessment method: a likelihood approach. Biometrics 1996; 52: 673–684. 13. Y. K. Cheung, and R. Chappell, A simple technique to evaluate model sensitivity in the continual reassessment method. Biometrics 2002; 58: 671–674. 14. Y. K. Cheung and R. Chappell, Sequential designs for phase I clinical trials with late-onset toxicities. Biometrics 2000; 56: 1177–1182. 15. T. M. Braun, Z. Yuan, and P. F. Thall, Determining a maximum-tolerated schedule of a cyctotoxic agent. Biometrics 2005; 61: 335–343. 16. T. M. Braun, Generalizing the TITE-CRM to adapt for early- and late-onset toxicities. Stat. Med. 2006; 25: 2071–2083.

2. Y. K. Cheung, Coherence principles in dose-finding studies. Biometrika 2005; 92: 863–873.

CROSS-REFERENCES

3. J. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: a practical design for phase 1 clinical trials in cancer. Biometrics 1990; 46: 33–48.

Dose Escalation Design Phase I Trials

Continual Reassessment Method (CRM)

COHORT VS. REPEATED CROSS-SECTIONAL SURVEY DESIGNS

illustrated with relevant data. More detail is available elsewhere (9). Data from a large community-based program evaluation are used to motivate and illustrate the theory (10). A random digit dialing (RDD) survey was conducted at baseline (T 0 ) followed by a smaller RDD survey conducted 2 years later (T 1 ) in the same communities, yielding 5475 T 0 surveys and 3524 T 1 surveys with complete data. People in the T 0 cross-sectional samples were also followed at T 1 , and 3280 repeat interviews were obtained (40% attrition); this is referred to as the cohort sample. For the T 0 and T 1 cross-sectional samples, there was an overall 54% response rate, which is considered in the discussion section. Consider the Venn Diagram in Fig. 1. The two large circles refer to the community population at T 0 and at T 1 ; their intersection is the long-term residents who were in the community at both times; they are referred to here as ‘‘stayers.’’ This group will be considered the population of primary interest in the following analyses, although other choices might have been made (9, 11). The two smaller circles (A + B + C and D + E) represent the respondents to the two cross-sectional surveys taken at T 0 and T 1 , respectively. The smallest circle (C) represents the cohort sample, successfully surveyed at both T 0 and T 1 . People in segment A (outmigrants) were surveyed at T 0 but left the population before the T 1 survey; those in segment C (cohort sample) remained and were interviewed again at T 1 ; those in B remained but were not reinterviewed, because they either refused or could not be contacted. The T 1 cross-sectional survey collected information from segments D (stayers) and E (inmigrants). As B + C is a random sample of the stayers at T 0 , and D is a random sample of the stayers at T 1 , the survey estimates from these segments provide estimates for the stayers at the respective time points. Here ‘‘bias’’ is defined as the difference between the cohort (or cross-sectional) change estimate and the change observed among the stayers.

PAULA DIEHR University of Washington Department of Biostatistics Seattle, WA, USA

1

INTRODUCTION

A central goal of community-based health promotion programs is to reduce the prevalence of risky health-behaviors in the intervention communities. Surveys of community residents at two or more points in time are often required to obtain direct evidence on whether such a goal is met. These surveys might use a cohort design in which a panel of individuals is surveyed repeatedly or else a repeated cross-sectional design in which a fresh sample of individuals from each community is interviewed on each survey occasion, or occasionally both. The choice of survey design depends, of course, on the goals of the intervention under evaluation. A long-term intervention (changing laws, for example) would affect people in a community no matter how long they lived there, suggesting that two cross-sectional surveys would suffice, whereas an intervention that depended on exposing individuals to certain information or programs might be better evaluated in a cohort of residents. Cohort estimates of change are desirable because, in using each person as his or her own control, an estimate will generally have a smaller standard error than an estimate of change based on two independent samples. On the other hand, cohort estimates can be seriously biased if losses to follow-up or changes in the community over time make the cohort unrepresentative of the population of primary interest. The relative merits of the cohort and repeated cross-sectional sampling approaches have been described in several places, (1–8). The following discussion is based on a theoretical approach for choosing between the two survey designs that includes issues of cost and attrition, and it is

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

COHORT VS. REPEATED CROSS-SECTIONAL SURVEY DESIGNS

Population at t1

Population at t0

F

G

H

B A

D

C

E

A + B + C + D + F + G is the community at T0 B + C + D + E + G + H is the community at T1 B + C + D + G is the stayers A + B + C is the T0 cross-sectional sample D + E is the T1 cross-sectional sample C is the cohort sample Figure 1. The Venn diagram for smoking prevalence.

It is clear from Fig. 1 that the different surveys are reaching different populations. The T 0 cross-section surveys both stayers and outmigrants, the T 1 cross-section surveys stayers and inmigrants, and the cohort represents a subset of the stayers who were willing to be surveyed at both T 0 and T 1 . Clearly, different survey designs can yield different answers. If they differ, which is most appropriate? Table 1 gives sample information for each segment of the Venn diagram. The percent of people who smoke cigarettes (smoking prevalence) is used here as the health behavior of interest. Sample sizes, estimates of smoking prevalence at T 0 and T 1 , and the standard deviations of the estimates, changes in prevalence, and biases are shown in Table 1. For example, segment A at T 0 had a sample of 805 people whose smoking prevalence was 24.7%, with a standard deviation (standard error) of 1.5 percentage points. The change in prevalence for stayers was 23.4 − 23.9 = − 0.45 percentage points, a slight decrease in healthful behavior. The change in prevalence for the cohort was +2.29 percentage points, and the bias of the cohort estimate is thus 2.29 − ( − .45) = 2.74 percentage points. The cohort estimate of change has higher bias than the cross-sectional estimate, but a lower standard error. Note that segments A through E had different smoking

prevalences, and the cohort sample was more favorable than the others. 2 THEORY— BIAS AND PRECISION This section develops algebraic inequalities to show when a cohort survey is better than a cross-sectional survey, for a fixed survey budget, in terms of the mean squared error of its estimates. The choice of design in making optimal estimates of change is first considered for a single community and, second, for comparing changes in two communities, such as a treatment and control community. The goal is to measure change in the stayers, and that only two waves of surveys will be conducted, a baseline and a single follow-up. 2.1 Choice of Design for Estimating Change in One Community The goal of the surveys is to obtain an accurate estimate of the change in (for instance) prevalence of smoking for ‘‘stayers’’ in a particular community. Let a particular measure of personal health behavior be independently identically distributed with variance σ 2 at both times. If a cross-sectional sample of size N x is taken at T 0 and again at T 1 , then the variance of each estimate of smoking prevalence will be σ 2 /N x , and the variance of the estimated community change will be

COHORT VS. REPEATED CROSS-SECTIONAL SURVEY DESIGNS

3

Table 1. Smoking Prevalence for Different Segments and Groups % Current Smokers (s.d. of Estimate)* Group

N

T 0 Prevalence

A B C D E Stayer

805 1390 3280 2981 543 4670 2981 3280 5475 3524

24.7 (1.5) 27.1 (1.1) 21.9 (0.7)

Cohort X-sect  The

T 1 Prevalence

Change

Bias

19.6 (0.7) 23.9 (0.8) 25.4 (1.8)

23.4 (0.6) 21.9 (0.7) 23.6 (0.6)

23.9 (0.8) 19.6 (0.7)

− 0.45 (1.0) 2.29 (0.5)

2.74 (1.0)

24.1 (0.7)

− 0.50 (0.9)

− 0.05 (0.4)

standard deviation of an estimate is often referred to as a standard error.

2σ 2 /N x (assuming no overlap in the samples). If instead a cohort sample of size N c is taken at T 0 and followed over time, the variance of the estimated community change will be (1 − ρ)2σ 2 /N c , where ρ is the correlation between a cohort member’s smoking status at T 0 and T 1 . If N c = N x , the variance of the cohort estimator is smaller than the cross-sectional variance, especially if ρ is near 1. Two desirable characteristics of estimators are unbiasedness and small variance (similar to the concepts of validity and precision). An investigator might prefer a biased estimator if its variance was sufficiently small. In Table 1, the cohort estimate is more biased but has lower standard deviation than the cross-sectional estimate. If the sample size was much larger, the standard deviations for both methods would approach zero, and the cross-sectional estimate would be clearly better. If the sample size was much smaller, the cohort estimate might be preferable because of its smaller standard deviation. To formalize this choice, consider the mean squared error (MSE) of an estimator, which is the average squared distance of an estimator from the true mean. Suppose the estimator, X, has mean µx and variance σ 2 x , and that the true parameter value is . The expected (mean) squared distance of X from  is then E(X − )2 = E(X − µx + µx − )2 = E(X − µx )2 + E(µx − )2 = σ 2 x + (µx − )2 . The MSE is thus equal to the sum of the variance and the squared bias of X. Letting the bias of the cohort and cross-sectional estimates be Bc and Bx , respectively, the MSEs

are MSE(cross-sectional) = 2σ 2 /N x + B2 x and MSE(cohort) = (1 − ρ)2σ 2 /N c + B2 c . A cohort estimate has a lower mean squared error than a cross-sectional estimate if 2σ 2 /Nx + B2 x > (1 − ρ)2σ 2 /Nc + B2 c

(1)

For large values of N x and N c , the inequality depends only on the bias terms. The authors are interested in designs that have the same total cost, which includes the cost of nonresponse for the primary survey and implicitly assumes that the nonresponse rate = α. Let S be the total number of surveys that can be afforded. For a cross-sectional survey with two waves, N x = S/2. For a cohort survey, however, the effect of attrition must be included. To arrive at N c surveys in the final cohort sample, N c + W surveys had to be conducted at T 0 , where W is the number of T 0 surveys ‘‘wasted’’ because no follow-up interview was obtained (segments A and B). Let the attrition rate be α = W/(N c + W); then W = N c α/(1 − α). The total number of surveys in the cohort survey is N c + W at T 0 and N c at T 1 , so that S = 2N c + W = 2N c + N c α/(1 − α) = N c (2 − α)/(1 − α) and N c = S(1 − α)/(2 − α). Substituting these values for N x and N c into Equation (1), the cohort estimate has lower MSE if
 α (2 − α) ρ− > S(B2c − B2x ) (2) 2σ 2 (1 − α) (2 − α) This inequality can be used to determine situations in which a cohort design has lower MSE than a cross-sectional design. As the

4

COHORT VS. REPEATED CROSS-SECTIONAL SURVEY DESIGNS

cohort bias is usually larger than the crosssectional bias, the cross-sectional estimate will have lower MSE when S is large. Similarly, if ρ < α/(2 − α), then the left side of Equation (2) is negative, and the crosssectional design will be better. The equation can be solved for S*, the value of S for which the cohort and cross-sectional estimates have the same MSE. Notice that if (B2 c − B2 x ) is negative, dividing Equation (2) through by this term reverses the inequality; and if zero, the inequality is independent of S, and depends only on ρ and α. Continuing the smoking example, the correlation of smoking at T 0 and at T 1 is estimated as 0.80 and σ 2 = 1826.71. From Table 1, the attrition rate is 1 − 3280/5475 = 0.40. For 22 different health-related survey variables (including smoking), both the cohort and the cross section differed significantly from the stayers, with the stayers having less favorable results. The intertemporal correlation, ρ, ranged from 0.19 for being unemployed to 1.0 for race and gender (which do not change). For 19 of the 22 variables, the cohort bias was larger in absolute value than the cross-sectional bias, making the righthand side in Equations (2) and (3) positive. All variables but one had correlations larger than α/(2 − α) = 0.25, meaning that the lefthand term was usually positive. The values of S*, the total sample size at which the two MSEs are the same, ranged from 244 to infinity. A cross-sectional design may be best for variables with low S* for moderate and large numbers of surveys. A cohort estimate is better for the others unless very large samples can be afforded. 2.2 Comparison of Change in Two Communities Inequality (2) shows when a cohort is better than a cross-sectional design to estimate change in a single community. However, if the goal of the survey is to estimate the difference in change between two communities (usually a treatment versus a control community), it is possible that the biases in the treatment and control estimates are similar and will thus cancel out. A similar argument to that above shows that a cohort design estimate of the difference between the treatment

change and the control change has a lower mean squared error than a cross-sectional design estimate for the same fixed budget if 4σ 2


 α (2 − α) ρ− > S(βc2 − βx2 ) (3) (1 − α) (2 − α)

where β c = Bc,tx − Bc,ctrl (the difference in treatment and control cohort biases) and β x = Bx,tx − Bx,ctrl (the difference in treatment and control cross-sectional biases). Note that, even if Bc,tx and Bc,ctrl are large, β c may be small. Differences in costs and sample sizes can be incorporated into this equation. Of the 22 variables considered, the term [β 2 c − β 2 x ] was negative in only 5 instances, but S* was usually very large. That is, a cohort estimate has a lower mean squared error unless the samples are very large. If about 1000 surveys could be afforded, the cross-sectional estimate would be better for estimating the difference in changes for only 8 of the 22 variables. Equations (2) and (3) may be used to help plan community surveys when attrition is a factor, and other methods may be used when it is not (7, 8). The data used here for illustration had a high attrition rate, which should be favorable to repeated cross-sectional survey designs, but the cohort designs usually had lower MSE. Other articles cited here have different examples and somewhat different findings.

REFERENCES 1. D. C. Altman, A framework for evaluating community-based heart disease prevention programs. Soc. Sci. Med. 1986; 22: 479–487. 2. G. V. Glass, V. L. Willson, and J. M. Gottman, Design and Analysis of Time-Series Experiments; Boulder: Colorado Assoc. Univ., 1975. 3. S. Salvini, D. J. Hunter, L. Sampson, M. J. Stampfer, G. A. Colditz, et al., Food-based validation of dietary questionnaires: the effects of week-to-week variation in food consumption. Int. J. Epidemiol. 1989; 18: 858–867. 4. T. D. Koepsell, E. H. Wagner, A. C. Cheadle, et al., Selected methodological issues in evaluating community-based health promotion and disease prevention programs. Annu. Rev. Publ. Health. 1992; 13: 31–57.

COHORT VS. REPEATED CROSS-SECTIONAL SURVEY DESIGNS 5. J. T. Salonen, T. W. Kottke, D. R. Jacobs, and P. J. Hannan, Analysis of community-based studies-evaluation issues in the North Karelia Project and the Minnesota Heart Health Program. Int. J. Epidemiol. 1986; 15: 176–182. 6. L. S. Caplan, D. S. Lane, and R. Grimson, The use of cohort vs repeated cross-sectional sample survey data in monitoring changing breast cancer screening practices. Prev. Med. 1995; 24: 553–556. 7. S. M. McKinlay, Cost-efficient designs of cluster unit trials. Prev. Med. 1994; 23: 606–611. 8. H. A. Feldman and S. M. McKinlay, Cohort versus cross-sectional design in large field trials; precision, sample size, and a unifying model. Stat. Med. 1994; 13: 61–78. 9. P. Diehr, D. C. Martin, T. Koepsell, A. Cheadle, E. Wagner, and B. M. Psaty, Optimal survey design for community-intervention evaluations: cohort or cross-section? J. Clin. Epidemiol. 1995; 48: 1461–1472. 10. E. H. Wagner, T. D. Koepsell, C. Anderman et al., The evaluation of the Henry J Kaiser Family Foundation’s Community Health Promotion Grant Program: Design. J. Clin. Epidemiol. 1991; 44: 685–699. 11. M. H. Gail, D. P. Byar, T. F. Pechachek, and D. K. Corle, Aspects of statistical design for the Community Invervention Trial for Smoking Cessation (COMMIT). Controlled Clinical Trials. 1992; 13: 6–21.

5

COLLINEARITY

variable is, in turn, regarded as an outcome variable in a regression equation that includes the remaining p − 1 explanatory variables. Then, R2j represents the squared residual correlation obtained using explanatory variable j, j = 1, . . . , p, as the response. The VIF is then defined for each such regression as: 1 . VIFj = 1 − R2j

G. A. DARLINGTON Cancer Care Ontario, Toronto, Ontario, Canada

Collinearity (or ‘‘multicollinearity’’) refers to a high level of correlation within a set of explanatory variables. In a regression modeling situation, if explanatory variables are highly correlated, then regression coefficient estimates may become unstable and not provide accurate measures of the individual effects of the variables. The estimate of the precision of these coefficient estimates is also affected and therefore confidence intervals and hypothesis tests are, likewise, affected. For the estimation of regression coefficients, the columns of the design matrix must be linearly independent. At an extreme, if two explanatory variables are perfectly linearly associated (i.e. their correlation is equal to 1), then such collinearity is an example of linearly dependent columns in the design matrix, X. While two parameters require estimation (i.e. the regression coefficients for the two explanatory variables), information is not available in the design matrix to estimate both coefficients uniquely. The two individual effects cannot be distinguished as a result of this collinearity. While collinearity typically does not involve completely linearly related explanatory variables, high levels of correlation can still lead to difficulties in coefficient estimation. It should be noted that this issue pertains to the relationship among explanatory variables which, ultimately, affects the ability to investigate simultaneously the relationship between the response variable and the explanatory variables. Therefore, the identification of potential collinearity problems is usually addressed by examination of the relationships among explanatory variables. One simple technique for the identification of collinearity is presented in Kleinbaum et al. (1). The computation of the variance inflation factor (VIF) is suggested. If there are p explanatory variables, each explanatory

If there is a strong relationship between the explanatory variable j and the remaining p − 1 explanatory variables, then R2j is close to 1 and VIFj is large. It is suggested, in (1), that values of VIF greater than 10 indicate serious collinearity that will affect coefficient and precision estimation. Collinearity may also be indicated if coefficient estimates from fitting simple regression models of the response with each explanatory variable are substantially different from coefficient estimates from fitting a multiple regression model including all explanatory variables. Similarly, if the order in which certain terms are included in the model seriously affects the coefficient estimates for these terms, then collinearity is indicated. Of course, one of the primary purposes of multivariate regression models is to examine the role of explanatory variables having ‘‘adjusted’’ for other variables in the model so that such behavior is not necessarily a problem. However, serious collinearity problems may prohibit a multivariate model from being fitted at all. If two or more explanatory variables are highly correlated because they represent measurements of the same general phenomenon (e.g. highest attained level of education and current salary are both aspects of socioeconomic status), then collinearity can be addressed by choosing one variable thought to be the most relevant. This variable would then be included in any models and the remaining, so-called redundant, variables would be excluded. The identification of such redundant variables may be difficult, so, alternately, a new variable that combines information on the correlated variables can be derived. This aggregate variable would

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

COLLINEARITY

be included in models instead of all of the component variables. It is sometimes helpful, particularly when collinearity is created as a result of including polynomial terms (e.g. X and X2 are included in a model together) but also in general, to center the original explanatory variables. This is accomplished by computing new explanatory variables that are the original measurements with the means subtracted. Suppose there are n individuals and p explanatory variables measured on each individual, X ji , i = 1, . . . , n, j = 1, . . . , p. Then the new explanatory variables are Zji = X ji − X j . If a quadratic model is of interest, then one would include the terms Zji and Z2ji in the model. In (1), an example of the effectiveness of such an approach for correcting collinearity is presented.

When polynomial regression is being undertaken, then the further step of orthogonalization of the explanatory variables is also possible and frequently used in some settings. Orthogonalization of more general sets of explanatory variables is possible but not as widely used. REFERENCES 1. Kleinbaum, D. G., Kupper, L. L. & Muller, K. E. (1988). Applied Regression Analysis and Other Multivariate Methods, 2nd Ed. PWSKent, Boston.

COMBINATION THERAPY

as resistance rapidly emerged (2). The successive development of two- and three-drug regimens as more drugs became available, initially PAS (P) and isoniazid (H), led to effective treatments with regimens of three drugs for one to three months followed by two drugs for 12 to 18 months (SPH/PH) (3). The introduction of new drugs, particularly rifampicin (R) and pyrazinamide (Z), led to the development of more effective regimens, although it has proved impossible to reduce the duration below 6 months and maintain highly effective regimens (SHRZ/HR) (4). The need for new antituberculosis drugs is largely driven by the high incidence of resistance to one or more of these key drugs, but there would also be major benefits in combination regimens that would reduce the total duration to less than 6 months. A very similar process has led to the current standard therapies for HIV infection with trials demonstrating that, as they became available, two drugs were better than one (5) and three better than two (6). Now, although different combinations are compared in many trials, often only one of the drugs differs. For example, in the ACTG 384 trial, triple and quadruple regimens were compared, which all included two nucleoside analogue reverse transcriptase inhibitors (NRTIs) with either efavirenz (EFV), a non-nucleoside reverse transcriptase inhibitor (NNRTI), nelfinavir, a protease inhibitor (PI), or both drugs (7). This trial also explored, in a factorial design, two different nucleoside reverse transcriptase inhibitor (NRTI) combinations, didanosine (ddI) plus stavudine (d4 T) and ZDV plus lamivudine (3TC). As in tuberculosis, a need exists for new drugs and combinations for HIV infection as the current therapies have to be given for long periods and are associated with substantial failure rates, usually because of difficulties with adherence to therapy and the emergence of resistance. Once a combination has been shown to be effective and the optimal dosages of the drugs are clearly defined, advantages of combining the drugs in a single preparation exist, as this combination aids compliance by simplifying therapy. It also minimizes the risk of the

JANET DARBYSHIRE London, United Kingdom

1

DEFINITION

Combination therapy regimens most commonly combine two or more drugs that have additive or synergistic action to treat a disease more effectively, frequently infections or cancer, with the aim of producing more effective regimens and preventing the emergence of resistance of the micro-organism or the tumor to the drugs. However combinations of drugs are widely used to treat many other diseases such as hypertension, asthma, diabetes, and arthritis. Combinations of drugs and other modalities, such as surgery or radiotherapy, are also used in certain disease areas, such as cancer. Clinical trials may be designed to evaluate the combination therapy or to assess the effect of a new drug in addition to other drugs as part of a combination. In the first case, it is the combination that is being evaluated, and all the components may be different between the regimens or, indeed, a combination may be tested against a single drug (for example, the ICON 3 trial compared a combination of two drugs with either a regimen of 3 drugs or monotherapy) (1). Indeed, as the number of available drugs has increased for the treatment of diseases such as cancer, tuberculosis, or HIV infection, trials have successively evaluated combinations of two, three, or more drugs to achieve a more rapid or complete cure depending on the disease. The balance between increasing efficacy by increasing the number of drugs in a combination and the risk of increasing toxicity is key to the development of optimal regimens. The introduction of new drugs, especially with different modes of action, may lead to major changes in the therapy of a disease. Tuberculosis is a classic example of how the use of combination therapy changed over time. The first drug, streptomycin (S), was clearly highly effective, but only for a short time

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

COMBINATION THERAPY

development of resistance to a drug, in the case of antibiotics, as a result of the patients choosing to take only one of the drugs. However, disadvantages exist in terms of the lack of flexibility to relate dose to weight and the management of toxicity. Combinations of ethambutol and isoniazid, which are widely used for the treatment of tuberculosis, are available in a variety of formulations because of the dose-related toxicity of ethambutol and, therefore, the need to relate the dose to weight. Such combinations should be compared with the individual drugs to ensure that they are adequately absorbed and produce comparable pharmacokinetic profiles. If a trial is evaluating combination regimens that are available in combined preparations, a decision will have to be made as to whether the preparation should be used. If they are, and the comparator is not available as a combined preparation, it may overestimate the benefits of the combined preparation because it is more likely to be taken. However, if single drugs are given for both regimens, the potential benefits of the combined preparation cannot be adequately assessed. In a trial that is exploring experimental regimens, single preparations are more often used, but if the regimens are being assessed as they will be used in routine practice, then a case for using the combined preparation exists. The development of combination regimens is often built on evidence from laboratory studies of additive or synergistic activity of drugs and the need for combinations because no single drug is adequate to cure a disease in some or all patients (for example, cancer or tuberculosis) or to control it (for example, HIV infection, hypertension, or diabetes). In different diseases, the approach may be different according to availability of drugs or the disease course. In chronic diseases, such as hypertension, Type II diabetes, or epilepsy, the aim is to control the disease with the minimum therapy. New drugs may be added to a combination over time to achieve this result. Trials may be designed to compare aggressive therapy with this standard approach to assess the impact on long-term disease control. Trials of new drugs are likely to mimic this approach by adding a new drug or a standard drug compared with adding a placebo or,

alternatively, adding an existing drug, if this practice is standard. In some diseases, different aims from trials of combination therapies may exist. One approach is to try to improve the results of treatment by increasing the potency of the regimens, by adding more drugs to the initial regimen, ideally with a different mode of action. For example, the ACTG 384 trial compared a four-drug regimen with the two standard three-drug regimens for HIV infection (7). An alternative approach is to reduce the toxicity of the regimens by minimizing the exposure to drugs while maintaining control of the infection. Such an approach may be particularly important in diseases such as HIV infection where the drugs are likely to be given for long periods of time and have potentially serious long-term side effects. A number of ongoing trials are exploring different approaches, for example, comparing strict control of viral replication with less aggressive therapy based on immunological markers. Trials of combination therapies may be used to assess new drugs when they will only be given as part of such regimens. Two alternative approaches exist that may be appropriate in different circumstances. The first is to randomize to add the new drug or placebo to existing standard therapy (8). The second is to substitute the new drug for one of the drugs in a standard combination and to compare with the standard combination (9). The advantages of the former are that, theoretically, it is easier to demonstrate a difference from placebo, but the risk is that if the current therapy is highly effective, little if any benefit may exist from adding a new drug. The disadvantage of the second approach is that it is likely to be more difficult to demonstrate superiority or equivalence to an existing drug. Further, reluctance may exist to substitute a new drug for one that is known to be effective. In some areas, such as leukaemia or lymphoma where therapies are becoming more and more effective, and yet are still not uniformly successful, it is becoming increasingly difficult to assess new combination therapies as the improvements are likely to be small. The large trials needed to reliably assess such

COMBINATION THERAPY

small differences may not be feasible, especially for the types of disease that are more rare. The development of new drugs for diseases such as cancer and HIV brings new challenges, not the least of which is how best to evaluate the many potential combinations that can be selected by combining the new and old drugs. Novel trial designs are needed together with the development of better surrogate markers that can be used to select the best combinations to take forward into large trials to assess clinical benefits and risks. Two-stage designs, such as those reported by Royston and Parmar (10), are innovative approaches to this problem. Combination therapies are only needed if monotherapies are not potent enough. Ultimately, the aim is to provide a therapy that is effective, safe, and simple to take, and in many diseases, major advantages would exist in replacing regimens of three, four, or even more drugs by a monotherapy regimen. When the first trials of protease inhibitors demonstrated their high potency in HIV infection, some hope existed that they might be as effective as current two-drug combination regimens, but it soon became clear that they were not sufficiently potent to prevent the emergence of resistance on their own, although they had a major impact when added to the two-drug regimen. Combinations of drug and nondrug therapy may be used to treat diseases, such as cancer, where bone marrow or stem cell transplants require chemotherapy as part of the whole treatment. Similarly combinations of chemotherapy and radiotherapy are effective in some forms of tumor (referred to as chemoradiotherapy). Other multi-mode combinations, such as chemotherapy and immunotherapy, (with drugs or therapeutic vaccines) may be developed for infections. The approach to assessing all of these is similar to drug combinations but special issues often exist, such as the timing of the different interventions within the overall treatment plans to achieve maximum benefits and to minimize toxicity. These issues of timing may make the assessment of the regimens more complex, particularly if the timings differ between regimens.

3

REFERENCES 1. ICON Collaborators, Icon3: randomised trial comparing paclitaxel plus carboplatin against standard chemotherapyof either single agent carboplatin or CAP (Cyclophosphamide, doxirubicin, cisplatin) in women with ovarian cancer. Lancet 2002; 360: 505–515. 2. A five-year assessment of patients in a controlled trial of streptomycin in pulmonary tuberculosis. Quart. J. Med. 1954: 91: 347–366. 3. Medical Research Council, Long-term chemotherapy in the treatment of chronic pulmonary tuberculosis with cavitation. Tubercle 1962; 43: 201–267. 4. W. Fox, Whither short-course chemotherapy? Br. J. Dis. Chest 1981; 75: 331–357. 5. HIV Trialists’ Collaborative Group, Zidovudine, didanosine, and zalcitabine in the treatment of HIV infection: meta-analyses of the randomised evidence. Lancet 1999; 353: 2014–2025. 6. S. M. Hammer, K. E. Squires, M. D. Hughes, J. M. Grimes, L. M. Demeter, J. S. Cuyrrier et al., A controlled trial of two nucleoside analogues plus indinavir in persons with human immunodeficiency virus infection and CD4 cell counts of 200 per cubic millimeter or less. N. Engl. J. Med. 1997; 337(11): 725–733. 7. G. K. Robbins, V. De Gruttola, R. W. Shafer et al., Comparison of sequential three-drug regimens as initial therapy for HIV-1 infection. N. Engl. J. Med. 2003: 349; 2293–2363. 8. D. W. Cameron, M. Heath-Chiozzi, S. Danner, C. Cohen, S. Kravcik, C. Maurath et al., Randomised placebo controlled trial of ritonavir in advanced HIV-1 disease. Lancet 1998; 351(9102): 543–549. 9. S. Staszewski, J. Morales-Ramirez, K. T. Tashima, A. Rachlis, D. Skiest, J. Stanford, R. Stryker, P. Johnson, D. F. Labriola, D. Farina, D. J. Manion, and N. M. Ruiz, Efavirenz plus zidovudine and lamivudine, efavirenz plus indinavir, and indinavir plus zidovudine and lamivudine in the treatment of HIV1 infection in adults. NEJM 1999; 341(25): 1865–1873. 10. P. Royston, M. K. B. Parmar, and W. Qian, Novel designs for multi-arm clinical trials with survival outcomes with an application in ovarian cancer. Stat. Med. 2003; 22: 2239–2256.

COMMITTEE FOR MEDICINAL PRODUCT FOR HUMAN USE (CHMP)

in close cooperation with health-care professionals and the pharmaceutical companies themselves. The CHMP plays an important role in EU-wide pharmacovigilance by closely monitoring reports of potential safety concerns (Adverse Drug Reaction Reports, or ADRs) and, when necessary, by making recommendations to the European Commission regarding changes to a product’s marketing authorization or the product’s suspension/withdrawal from the market. In cases where there is an urgent requirement to modify the authorization of a medicinal product due to safety concerns, the CHMP can issue an Urgent Safety Restriction (USR) to inform health-care professionals about changes in how or under what circumstances the medication may be used. The CHMP publishes a European Public Assessment Report (EPAR) for every centrally authorized product that is granted a marketing authorization, which sets out the scientific grounds for the Committee’s opinion in favor of granting the authorization. A Summary of Product Characteristics (SPC) is also published with the labeling and packaging requirements for the product and details of the procedural steps taken during the assessment process. These EPARs are published on the EMEA’s website and are generally available in all official languages of the EU. Scientific assessment work conducted by the CHMP is subject to an internal peerreview system to safeguard the accuracy and validity of opinions reached by the Committee. The EMEA’s integrated qualitymanagement system ensures effective planning, operation, and control of the CHMP’s processes and records. Other important activities of the CHMP and its working parties include:

The Committee for Medicinal Products for Human Use (CHMP) is responsible for preparing the European Medicines Agency’s opinions on all questions concerning medicinal products for human use, in accordance with Regulation European Commission (EC) No 726/2004. The CHMP plays a vital role in the marketing procedures for medicines in the European Union (EU): • The CHMP is responsible for conduct-

ing the initial assessment of medicinal products for which a Community-wide marketing authorization is sought. The CHMP is also responsible for several postauthorization and maintenance activities, including the assessment of any modifications or extensions (variations) to the existing marketing authorization. • The CHMP arbitrates in cases of disagreement between member states over the marketing authorization of a particular medicinal product. The CHMP also acts in referral cases, initiated when there are concerns relating to the protection of public health or where other community interests are at stake. Assessments conducted by the CHMP are based on purely scientific criteria and determine whether the products concerned meet the necessary quality, safety, and efficacy requirements in accordance with EU legislation (particularly Directive 2001/83/EC). These processes ensure that once medicinal products reach the marketplace they have a positive risk–benefit balance in favor of the patients/users of the products. Subsequent monitoring of the safety of authorized products is conducted through the EU’s network of national medicines agencies,

• Assistance to companies in researching

and developing new medicines. • Preparation of scientific and regulatory

guidelines for the pharmaceuticals industry. • Cooperation with international partners on the harmonization of regulatory requirements for medicines.

This article was modified from the website of the European Medicines Agency (http://www.emea .europa.eu/htms/general/contacts/CHMP/CHMP. html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

COMMON TECHNICAL DOCUMENT (CTD)

text. Acronyms and abbreviations should be defined the first time they are used in each module. References should be cited in accordance with the current edition of the Uniform Requirements. The CTD should be organized into five modules: Module 1 is region specific; modules 2, 3, 4, and 5 are intended to be common for all regions. Conformance with the CTD guidances should help ensure that these four modules are provided in a format acceptable to the regulatory authorities (see the figure and overall outline on the following pages).

Through the International Conference on Harmonisation (ICH) process, considerable harmonization has been achieved among the three regions (Japan, Europe, and the United States) in the technical requirements for the registration of pharmaceuticals for human use. However, until now, no harmonization of the organization of a submission has existed. Each region has its own requirements for the organization of the technical reports in the submission and for the preparation of the summaries and tables. In Japan, the applicants must prepare the GAIYO, which organizes and presents a summary of the technical information. In Europe, expert reports and tabulated summaries are required, and written summaries are recommended. The U.S. Food and Drug Administration (FDA) has guidance regarding the format and content of the new drug application submission. To avoid generating and compiling different registration dossiers, this guidance describes a harmonized format for the Common Technical Document (CTD) that will be acceptable in all three regions. Throughout the CTD, the display of information should be unambiguous and transparent to facilitate the review of the basic data and to help a reviewer quickly become oriented to the application contents. Text and tables should be prepared by using margins that allow the document to be printed on both A4 paper (E.U. and Japan) and 8.5 x 11’’ paper (U.S.). The left-hand margin should be sufficiently large that information is not obscured through binding. Font sizes for text and tables should be of a style and size that are large enough to be easily legible, even after photocopying. Times New Roman, 12-point font is recommended for narrative

Module 1. Administrative Information and Prescribing Information This module should contain documents specific to each region; for example, application forms or the proposed label for use in the region. The content and format of this module can be specified by the relevant regulatory authorities. For information about this module see the guidance for industry, General Considerations for Submitting Marketing Applications According to the ICH/CTD Format. Module 2. Common Technical Document Summaries Module 2 should begin with a general introduction to the pharmaceutical, including its pharmacologic class, mode of action, and proposed clinical use. In general, the introduction should not exceed one page. Module 2 should contain seven sections in the following order: • CTD Table of Contents • CTD Introduction • Quality Overall Summary • Nonclinical Overview • Clinical Overview • Nonclinical Written and Tabulated

Summaries • Clinical Summary

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cber/gdlns/m4ctd.pdf) by Ralph D’Agostino and Sarah Karl.

Because Module 2 contains information from the Quality, Efficacy, and Safety sections of the CTD, the organization of the

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

COMMON TECHNICAL DOCUMENT (CTD)

individual Module 2 summaries is discussed in three separate documents: • M4Q: The CTD – Quality • M4 S: The CTD – Safety • M4E: The CTD – Efficacy

Module 3. Quality Information on Quality should be presented in the structured format described in the guidance M4Q. Module 4. Nonclinical Study Reports The Nonclinical Study Reports should be presented in the order described in the guidance M4 S. Module 5. Clinical Study Reports The human study reports and related information should be presented in the order described in the guidance M4E. The CTD should be organized according to the following general outline. Module 1: Administrative Information and Prescribing Information. 1.1 Table of Contents of the Submission Including Module 1 1.2 Documents Specific to Each Region (for example, application forms and prescribing information) Module 2: Common Technical Document Summaries. 2.1 CTD Table of Contents 2.2 CTD Introduction 2.3 Quality Overall Summary 2.4 Nonclinical Overview 2.5 Clinical Overview 2.6 Nonclinical Written and Tabulated Summary Pharmacology Pharmacokinetics Toxicology 2.7 Clinical Summary Biopharmaceutics and Associated Analytical Methods Clinical Pharmacology Studies

Clinical Efficacy Clinical Safety Synopses of Individual Studies Module 3: Quality. 3.1 Module 3 Table of Contents 3.2 Body of Data 3.3 Literature References Module 4: Nonclinical Study Reports. 4.1 Module 4 Table of Contents 4.2 Study Reports 4.3 Literature References Module 5: Clinical Study Reports. 5.1 Module 5 Table of Contents 5.2 Tabular Listing of All Clinical Studies 5.3 Clinical Study Reports 5.4 Literature References

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH IN ASIAN IMMIGRANT POPULATIONS

cervical cancer screening are as follows: 70% of age-eligible women should have received mammography in the preceding two years and 90% of women should have received a Pap test within the preceding three years (9). Data from the 2000 National Health Interview Survey (NHIS) show wide variations in breast and cervical cancer screening rates by race and ethnicity (Table 1). In addition, the ‘‘Pathways Project’’ surveyed five San Francisco Bay populations in 1994; nearly all the white (99%) and black (98%) respondents reported at least one Pap smear, compared with 76% of Latina, 67% of Chinese, and 42% of Vietnamese respondents; similar patterns were observed for mammography (10). As the NHIS is conducted in English, it excludes individuals with limited English proficiency (11). The ‘‘Breast and Cervical Cancer Intervention Study’’ found breast and cervical cancer screening rates were significantly lower among non-English speaking than English speaking Chinese and Latina women (12). These findings support the importance of cancer screening interventions targeting immigrants, particularly those that are less acculturated (12, 13). This article focuses on community-based breast and cervical cancer control research in Asian populations. First, the authors address principles of community participatory research and several important program evaluation issues: sampling, survey methods, recruitment and retention, translation, data quality, and control group ‘‘contamination.’’ Second, the authors summarize communitybased studies that aimed to increase mammography or Pap testing levels among Asian women. For this summary, the authors used the same approach as Legler et al. (14) in their meta-analysis of interventions to promote mammography among women with historically low mammography rates. Specifically, the authors only considered studies in which the researchers used an experimental or quasi-experimental design to evaluate the effectiveness of their intervention, and reported intervention outcomes based on actual receipt of breast or cervical cancer screening (14).

VICTORIA M. TAYLOR Fred Hutchinson Cancer Research Center Division of Public Health Sciences Seattle, Washington University of Washington Department of Health Services Seattle, Washington

T. GREGORY HISLOP Cancer Control Research British Columbia Cancer Agency Vancouver, British Columbia, Canada

YUTAKA YASUI Fred Hutchinson Cancer Research Center Division of Public Health Sciences Seattle, Washington

SHIN-PING TU and J. CAREY JACKSON University of Washington Department of Medicine Seattle, Washington Harborview Medical Center Seattle, Washington,

1

INTRODUCTION

The number of Asian Americans in the United States increased from a little over one million in 1970 to nearly seven million in 1990, and reached more than 10 million by 2000 (1–3). Two-thirds (66%) of Asian Americans are foreign born and the majority (56%) do not speak English ‘‘very well.’’ Further, North America’s Asian population is heterogeneous and includes individuals from East Asia (e.g., mainland China), South Asia (e.g., India), Southeast Asia (e.g., Vietnam), and Island Asia (e.g., the Philippines) (4). The Presidential Initiative on Race aims to eliminate disparities in six areas of health status, including cancer screening, over the next decade (5, 6). Current American Cancer Society screening guidelines specify that women aged 40 and older should receive a screening mammogram annually and women aged 18 and older should receive Papanicolaou (Pap) testing every one to three years, depending on their risk for disease (7, 8). The Healthy People 2010 objectives for breast and

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH Table 1. Breast and Cervical Cancer Screening Rates by Race and Ethnicity—United States, 2000

Race/Ethnicity

Mammogram within the past two yearsa %

Pap test within the past three yearsb %

72 68 63 52 57

82 84 77 77 67

White Black Latina American Indian or Alaska Native Asian or Pacific Islander a b

Women aged 40 years and older Women aged 18 years or older

2 COMMUNITY PARTICIPATORY RESEARCH The importance of active community participation in research projects involving racial/ethnic minority groups is increasingly recognized (15, 16). Community-based research principles, developed by the University of Washington School of Public Health and Community Medicine, specify the following: Community partners should be involved in the earliest stages of the project and have real influence on project direction; research processes and outcomes should benefit the community; members of the community should be part of the analysis and interpretation of data; productive partnerships between researchers and community members should be encouraged to last beyond the life of the project; and community members should be empowered to initiate their own research projects that address needs they identify themselves (17). The University of California at San Francisco’s ongoing study ‘‘Reaching Vietnamese American Women: A Community Model for Promoting Cervical Cancer Screening’’ provides one example of successful communitybased research. The study collaborates with a community coalition that includes representatives from community-based organizations and health agencies that provide services to Vietnamese Americans; it uses community forums to both develop and evaluate program components; and it employs lay health workers from the targeted community. However, this research group has noted that principles of community-based research,

developed in the United States as a product of American culture, cannot always be applied to immigrants. For example, the researchers had to strike a balance between respecting the dominant cultural pattern of deferring to authority figures (e.g., local Vietnamese physicians) and encouraging community members to contribute their views. Additionally, the researchers were not able to involve non-English speaking community members in data collection because their Institutional Review Board would not allow individuals to engage in data collection activities unless they received human subjects certification (and no human subjects training materials are available in Vietnamese) (18).

3 EVALUATION ISSUES 3.1 Sampling Simple random sampling can only be used for studies targeting Asian-American populations when the group of interest is highly concentrated within a defined geographic boundary (19). For example, the ‘‘Pathways Project’’ successfully identified Chinese households using random digit dialing within San Francisco telephone prefixes that were at least 35% Chinese (20). In less concentrated populations, the following approaches have variously been used: purchasing marketing company lists for individuals of a particular Asian ethnicity, acquiring lists from organizations that serve one particular community (e.g., the Chinese Community Health Plan in Northern California), and convenience sampling

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

(e.g., from Filipino and Korean church congregations and community organizations) (20–24). Certain Asian populations have characteristic surnames that can be applied to existing databases (e.g., telephone books) (25–29). However, the efficiency of this approach to sampling varies by Asian subgroup and is a function of population density (19, 27, 29). For example, it works well for Vietnamese who have a relatively small number of highly characteristic last names (it has been shown that 99% of Vietnamese households can be identified by using 37 surnames) (30). Surname lists work less well for other groups. It has been found to be an inefficient method of identifying Cambodian households, and it is doubtful that this approach would work for Filipinos who often have Spanish surnames (19, 31). Additionally, last name lists are not useful for identifying women from Asian subgroups with a high rate of interracial marriage (e.g., Japanese Americans) (19). Finally, for some Asian populations, a universal set of last names that can be used in research projects may not exist; rather, each locale and time period may require its own list of names. For example, some Chinese immigrant communities include a higher proportion of people with Mandarin (as opposed to Cantonese) last names than others (25). 3.2 Survey Methods Cancer control intervention studies often use surveys to identify individuals for randomized controlled trials or to evaluate intervention effectiveness. In-person interviews, conducted in the language of an interviewee’s choice, are believed to be the most effective method of enlisting cooperation from Asian immigrant groups, and can be cost-efficient when the sample is drawn from nondispersed communities (e.g., Vietnamese in the Tenderloin area of San Francisco); however, it is usually a cost-prohibitive approach for surveying dispersed Asian populations (e.g., Vietnamese in multiple Californian counties) (32, 33). Table 2 provides survey response rates from population-based breast and cervical cancer control projects, and shows that response rates vary by Asian subgroup and geographic area as well as method of survey administration (30, 31, 33–38).

3

3.3 Recruitment and Retention It can be difficult to obtain high response rates and low loss to follow-up in populations over-represented by immigrants for multiple reasons. Specifically, immigrants may not have working telephones, frequently have more than one job, may be concerned about safety and suspicious of strangers, often have multiple health and social problems, and tend to move frequently (19, 31, 39). Also, some Asian immigrants travel frequently between their original and adopted countries. A British research group identified South Asian women who were registered with general practitioners in England for a breast cancer intervention study, and found that 14% of the sample had recently moved while 12% were currently in Asia (40). Generally, recruitment and retention are enhanced by employing bilingual and bicultural staff members from the targeted community. Gender matching of research personnel and participants is also important, especially for members of traditional Asian cultures in which discussions of certain topics (e.g., cancer and gynecologic issues) are inappropriate with the opposite sex (41). 3.4 Translation As many Asian immigrants do not speak English, questionnaires have to be prepared in Asian languages, which is a somewhat cumbersome and time-consuming process that usually involves one of the following processes: translation, back-translation to ensure equivalence, and reconciliation of any discrepancies between the original and backtranslated English versions; or double forward translation into the relevant Asian language, and then review by a referee (41–44). Additionally, some Asian subgroups include individuals who speak multiple languages or dialects (e.g., Filipino American communities include Cebuano-, Ilaccano-, and Tagalog-speaking individuals) (45). Experience from the ‘‘Pathways Project’’ shows that Pap smear and mammogram, for example, cannot be translated into Chinese or Vietnamese. Therefore, the terms have to be stated in English and subsequently defined in the relevant Asian language (43, 44).

4

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

Table 2. Response Rates from Surveys of Asian Populations Geographic area Taylor, 1999 Hislop, 2003

Cambodian Chinese

Taylor, 2002 Wismer, 1998

Chinese Korean

Bird, 1998

Vietnamese

Jenkins, 1999

Vietnamese

Nguyen, 2002

Vietnamese

Taylor, Submitted

Vietnamese

Seattle, Washington Vancouver, British Columbia Seattle, Washington Alameda/Santa Clara counties, California San Francisco, California Sacramento, California Alameda/Santa Clara counties, California Los Angeles/Orange counties, California Santa Clara county, California Harris county, Texas Seattle, Washington

3.5 Data Quality Most community-based studies to evaluate breast and cervical cancer screening intervention programs have used self-reported survey data to evaluate intervention effectiveness (46). However, increasing evidence exists that the quality of survey data may differ by race and ethnicity (46, 47). The ‘‘Pathways Project’’ surveyed five San Francisco Bay populations in 1994. Agreement between the baseline survey and a callback survey (conducted in a 10% randomly selected subsample) for the question ‘‘Have you ever had a mammogram?’’ was higher among white than racial/ethnic minority women. Specifically, the test-retest reliabilities among Chinese, Vietnamese, Latina, black, and white women were 0.90 (95% CI—0.74, 0.97), 0.74 (95% CI—0.51, 0.88), 0.90 (95% CI—0.79, 0.95), 0.93 (0.72—0.99), and 1.00 (95% CI—0.86, 1.00), respectively. Following a baseline telephone survey in multi-ethnic Alameda County, California, investigators from the ‘‘Pathfinders Project’’ examined medical records to validate breast and cervical cancer self-reports. The proportions of mammograms and Pap smears that

Year(s) survey conducted

Survey method

Response rate (%)

1997-98 1999

In-person In-person

72 55

1999 1994

In-person Telephone

64 80

1996

In-person

79 74

1996

Telephone

45

42

2000

Telephone

63

2002

In-person

54 82

could be validated were significantly lower among racial/ethnic minority than white women. Specifically, mammograms were validated for 89% of white women, 72% of black women, 72% of Latina women, 67% of Chinese women, and 76% of Filipina women. The corresponding proportions for Pap testing were 85%, 66%, 66%, 68%, and 67%, respectively (46). Several researchers have concluded that Asian Americans have a greater tendency than whites to provide socially desirable responses to survey questions, and have recommended using other methods of outcome ascertainment (e.g., medical record review), when possible (43, 46). 3.6 Control Group ‘‘Contamination’’ Program evaluation in Asian immigrant communities can be compromised by dissemination of the intervention to a study’s control group (24, 38, 48). Many Asian immigrant communities are relatively small and self-contained with strong social as well as extended family networks, and information is often quickly disseminated throughout the community. Although these communication channels serve the community well, they can

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

compromise the methodological rigor needed for randomization protocols (48). 4

COMMUNITY-BASED STUDIES

4.1 Overview The authors identified nine studies that met inclusion criteria for review (i.e., the study design was experimental or quasiexperimental, and intervention outcomes were based on actual screening test receipt) (Table 3). These studies targeted Cambodian, Chinese, Filipina, Korean, South Asian, and Vietnamese women in Canada, the United States, and England. Overall, three of the studies randomized individual women to experimental or control status, two randomized groups of women, and four used a quasiexperimental (two-community) study design. 4.2 Individual Randomized Trials Taylor et al. (39) conducted a three-arm randomized controlled trial to evaluate cervical cancer screening interventions for Chinese American/Canadian women in Seattle, Washington, and Vancouver, British Columbia. Baseline survey respondents who under-utilized Pap testing were randomized to an outreach worker intervention (that included a home visit, use of a video and print materials, and logistic assistance accessing screening services), a direct mail intervention (that included a video and print materials), or control status. Outcome evaluation was based on results from a follow-up survey as well as medical record verification of selfreported Pap testing. Overall, 39% of the 129 women who received the outreach intervention, 25% of the 139 women who received the direct mail intervention, and 15% of the 134 controls reported Pap testing following randomization (outreach worker versus control P < 0.001, direct mail versus control P = 0.03, and outreach worker versus direct mail P = 0.02) (39). Investigators in Leicester, England, conducted a four-arm randomized controlled trial to evaluate the effects of health education on the uptake of Pap smears among women originally from the Indian subcontinent (Bangladesh, Pakistan, and India).

5

Nearly one-half (47%) of women who were shown a video during a home visit by an outreach worker adhered to screening recommendations, as did 37% of those who were visited and given a leaflet as well as a fact sheet. In contrast, only 5% of women who were not contacted and 11% of women who were sent print materials in the mail completed Pap testing (49). In another British study, researchers evaluated the effect of a home visit by an outreach worker on mammography participation by South Asian (Bangladeshi and Pakistani) women aged 50–64 years in Oldham. No difference existed in the proportion of intervention and control group women who subsequently responded to an invitation to Britain’s population-based mammography screening program (40). 4.3 Group Randomized Trials A Seattle research team conducted a grouprandomized controlled trial to evaluate a neighborhood-based outreach worker intervention to increase Pap testing among Cambodian refugees. Interventions were delivered by bicultural, bilingual Cambodian outreach workers; they included home visits and small group neighborhood meetings, use of a motivational video, and tailored logistic assistance accessing screening services. At baseline, 44% of the women in intervention neighborhoods and 51% of women in control neighborhoods reported Pap smear receipt in the past year. At follow-up, the proportions reporting a Pap test in the last 12months were 61% and 62% among intervention and control women, respectively. Increases in intervention group (17%) and control group (11%) cervical cancer screening rates were not significantly different (48). Maxwell et al. recently reported their results from a randomized controlled trial to increase breast and cervical cancer screening among Filipina American women in Los Angeles (24). Women aged 40 and older were recruited from community-based organizations as well as churches, and invited to attend a group session with a health educator. Groups were randomly assigned to receive a cancer screening module (intervention) or a physical activity module (control). Moderate increases in breast and cervical

6

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

cancer screening rates were observed in both groups (9 to 12 percentage points). However, among recent immigrants (women who had spent less than 10 years in the United States), mammography screening increased significantly more in the intervention arm than in the control arm (a 27 versus a 6 percentage point increase, P < 0.05). 4.4 Quasi-Experimental Studies Wismer et al. (38) have reported interim evaluation data from a community-based project that aimed to increase breast and cervical cancer screening rates among Korean Americans. Lay health workers were trained to provide workshops and print materials were distributed. After an 18-month intervention period, no significant changes occurred in Pap testing rates in either the intervention community (Alameda County, California) or the control community (Santa Clara County, California). Observed mammography increases in both the intervention county and the control county were equivalent. The researchers concluded that competing programs in Santa Clara County, diffusion of the intervention from Alameda County to neighboring Santa Clara County, and secular trends for mammography may all have contributed to their negative findings (38). The ‘‘Vietnamese Community Health Promotion Project’’ in San Francisco has evaluated several breast and cervical cancer screening interventions for VietnameseAmerican women (32–34, 50–52). One study evaluated an outreach intervention to promote receipt and screening interval maintenance of mammography and Pap smears. Indigenous lay health workers conducted a series of three small group educational sessions with Vietnamese women in the Tenderloin district of San Francisco while women in Sacramento, California, served as controls. Pre- and post- intervention surveys showed that the proportions of women reporting at least one mammogram increased from 54% to 69% in the experimental area (P = 0.006). In contrast, rates remained constant in the control community. Similar results were obtained for previous Pap testing; rates increased from 46% to 66% (P = 0.001) in San Francisco, but did not increase in Sacramento (34).

Another ‘‘Vietnamese Community Health Promotion Project’’ study evaluated a medialed campaign that included use of television, newspaper, and billboard advertising as well as the distribution of audio-visual and print educational materials. Post-intervention, no difference existed in recent mammography or Pap testing use among women in two northern California experimental counties and those in two southern California control counties. However, women in the intervention area were more likely to be planning to have mammograms and Pap tests in the future than women in the control area (33). Finally, the ‘‘Vietnamese Community Health Promotion Project’’ used a two-community study design to evaluate a multi-faceted community intervention to increase mammography participation in a northern California community. No intervention effect was demonstrated in this study (52). 5 CONCLUSION The Indigenous Model has been proposed as an effective framework for the delivery of health education programs targeting immigrant groups. This model specifies that successful programs should be delivered by individuals who are acceptable and accessible to the target population, in convenient locations, and in multiple ways (53–55). The three interventions that were successful in promoting cancer screening among women of Asian descent have all used these principles (34, 39, 49). For example, Taylor et al. recruited Chinese women from the targeted communities to serve as outreach workers, delivered the intervention in women’s homes, and provided education through discussion sessions with outreach workers as well as audio-visual and print materials (39). Rychetnik et al. (56) believe that criticisms of the randomized controlled trial in public health research are based on a consideration of classic trials in which the intervention is standardized and the individual is the unit of randomization. They proposed the routine use of cluster trials, with groups as the unit of randomization and analysis, to evaluate complex public health interventions (56). Two studies of breast and cervical

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

cancer interventions in Asian communities have used this approach with negative findings. In both cases, the investigators reported increases in screening levels among both the intervention and control groups, and proposed that control group contamination may have occurred (24, 48). As cancer control intervention programs often involve multi-faceted, community driven strategies, randomized controlled trials cannot always accommodate such programs (56). Consequently, investigators have used quasi-experimental designs with intervention and control communities (33, 34, 38, 52). Bird et al. (34) successfully used this study design to evaluate their breast and cervical cancer control intervention targeting Vietnamese women. A marked increase in breast and cancer screening rates among women in the experimental community with no change among women in the control community (together with the absence of any other promotional activities in the experimental community) provide compelling evidence for an intervention effect (34). However, other researchers have had difficulties evaluating breast and cervical cancer intervention programs, using a two-community design, because of unanticipated promotional activities in their control communities (38, 52, 57). This article addressed selected issues in the evaluation of public health interventions for Asian-American women. However, the authors did not address other important issues that are highly relevant to program development and delivery. For example, the importance of using qualitative methods during the development of interventions for immigrant groups and applying a culturally appropriate conceptual framework are not discussed (13, 41, 58, 59, 60, 61). In 1994, the Centers for Disease Control and Prevention published its report Chronic Disease in Minority Populations. The authors of the section about Asian Americans made the following recommendations: Special data collection efforts should focus on each Asian subgroup, attention should be given to low use of preventive measures, and intervention approaches should be culturally tailored to Asian communities (62). Almost a decade later, only a few studies have evaluated

7

breast and cervical cancer control interventions in Asian communities. However, these studies highlight the methodologic challenges in conducting evaluative public health research, particularly in racial/ethnic minority communities. 6

ACKNOWLEDGEMENTS

This work was supported by cooperative agreement #86322 and grant #82326 from the National Cancer Institute. REFERENCES 1. J. S. Lin-Fu, Asian and Pacific Islanders: an overview of demographic characteristics and health care issues. Asian Amer. Pacific Islander J. Health 1993; 1: 21–36. 2. U.S. Census Bureau, The Asian Population: 2000. Washington, DC: U.S. Department of Commerce, 2000. 3. U.S. Census Bureau, A Profile of the Nation’s Foreign-born Population from Asia (2000 update). Washington, DC: U.S. Department of Commerce, 2002. 4. U.S. Department of Commerce, We the Asian Americans. Washington, DC: U.S. Department of Commerce, 1993. 5. Department of Health and Human Services, Racial and Ethnic Disparities in Health. Washington, DC: Department of Health and Human Services, 1998. 6. D. Satcher, Eliminating racial and ethnic disparities in health: the role of the ten leading health indicators. J. Natl. Med. Assoc. 2000; 92: 315–318. 7. American Cancer Society, Cancer Prevention and Early Detection: Facts and Figures. Atlanta, GA: American Cancer Society, 2003. 8. D. Saslow et al., American Cancer Society guideline for the early detection of cervical neoplasia and cancer. CA Cancer J. Clinicians 2002; 52: 342–362. 9. Department of Health and Human Services, Healthy People 2010. Washington, DC: U.S. Government Printing Office, 2000. 10. R. A. Hiatt et al., Pathways to early cancer detection in the multiethnic population of the San Francisco Bay Area. Health Educ. Quart. 1996; 23: 10–27. 11. M. Kagawa-Singer and N. Pourat, Asian American and Pacific Islander breast and cervical carcinoma screening rates and Healthy

8

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH People 2000 objectives. Cancer 2000; 89: 696–705. R. A. Hiatt et al., Community-based cancer screening for underserved women: design and baseline findings from the Breast and Cervical Cancer Intervention Study. Prevent. Med. 2001; 33: 190–203. R. A. Hiatt and R. J. Pasick, Unsolved problems in early breast cancer detection: focus on the underserved. Breast Cancer Res. Treat. 1996; 40: 37–51. J. Legler et al., The effectiveness of interventions to promote mammography among women with historically lower rates of screening. Cancer Epidemiol. Biomark. Prevent. 2002; 11: 59–71. B. A. Israel et al., Review of community-based research: assessing partnership approaches to public health. Annu. Rev. Public Health 1998; 19: 173–202. P. M. Lantz et al., Can communities and academia work together on public health research? Evaluation results from a community-based participatory research partnership in Detroit. J. Urban Health 2001; 78: 495–507. University of Washington School of Public Health and Community Medicine. (2003). Community-based research principles. (online). Available: http://sphcm.washington. edu/research/community.htm. T. K. Lam et al., Encouraging VietnameseAmerican women to obtain Pap tests through lay health worker outreach and media education. J. Gen. Intern. Med. 2003; 18: 516–524. S. H. Yu and W. T. Lui, Methodologic issues. In: N. W. S. Zane et al., (eds.), Methodologic Issues. Thousand Oaks, CA: Sage Publications, 1994. M. Lee, F. Lee, and F. Stewart, Pathways to early breast and cervical cancer detection for Chinese American women. Health Educ. Quart. 1996; 23: 76–88. A. E. Maxwell, R. Bastani, and U. S. Warda, Breast cancer screening and related attitudes among Filipino-American women. Cancer Epidemiol. Biomark. Prevent. 1997; 6: 719–726. A. E. Maxwell, R. Bastani, and U. S. Warda, Mammography utilization and related attitudes among Korean-American women. Women Health 1998; 27: 89–107. A. E. Maxwell, R. Bastani, and U. S. Warda, Demographic predictors of cancer screening among Filipino and Korean immigrants in the United States. Amer. J. Prevent. Med. 2000; 18: 62–68.

24. A. E. Maxwell, R. Bastani, P. Vida, and U. S. Warda, Results of a randomized trial to increase breast and cervical cancer screening among Filipino American women. Prevent. Med. 2003; 37: 102–109. 25. B. C. K. Choi et al., Use of surnames to identify individuals of Chinese ancestry. Amer. J. Epidemiol. 1993; 138: 723–734. 26. B. K. Hage et al., Telephone directory listings of presumptive Chinese surnames: an appropriate sampling frame for a dispersed population with characteristic surnames. Epidemiology 1990; 1: 405–408. 27. D. S. Lauderdale and B. Kestenbaum, Asian American ethnic identification by surname. Population Res. Policy Rev. 2000; 19: 283–300. 28. A. Nicoll, K. Bassett, and S. J. Ulijaszek, What’s in a name? Accuracy of using surnames and forenames in ascribing Asian ethnic identity in English populations. J. Epidemiol. Community Health 1986; 40: 364–368. 29. E. Y. Tjam, How to find Chinese research participants: use of a phonologically based surname search method. Can. J. Public Health 2001; 92: 138–142. 30. T. Nguyen et al., Predictors of cervical Pap smear screening awareness, intention, and receipt among Vietnamese-American women. Amer. J. Prevent. Med. 2002; 23: 207–214. 31. V. M. Taylor et al., Cervical cancer screening among Cambodian-American women. Cancer Epidemiol. Biomark. Prevent. 1999; 8: 541–546. 32. S. J. McPhee et al., Pathways to early cancer detection for Vietnamese women: Suc Khoe La Vang! (Health is gold!). Health Educ. Quart. 1996; 23: 60–75. 33. C. N. Jenkins et al., Effect of a media-led education campaign on breast and cervical cancer screening among Vietnamese-American women. Prevent. Med. 1999; 28: 395–406. 34. J. A. Bird et al., Opening pathways to cancer screening for Vietnamese-American women: lay health workers hold a key. Prevent. Med. 1998; 27: 821–829. 35. T. G. Hislop et al., Facilitators and barriers to cervical cancer screening among Chinese Canadian women. Can. J. Public Health 2003; 94: 68–73. 36. V. M. Taylor et al., Pap testing adherence among Vietnamese American women. Cancer Epidemiol. Biomark. Prevent. 2004; 13: 613–619.

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH 37. V. M. Taylor et al., Cervical cancer screening among Chinese Americans. Cancer Detect. Prevent. 2002; 26: 139–145. 38. B. A. Wismer et al., Interim assessment of a community intervention to improve breast and cervical cancer screening among Korean American women. J. Public Health Manag. Pract. 2001; 7: 61–70. 39. V. M. Taylor et al., A randomized controlled trial of interventions to promote cervical cancer screening among Chinese women in North America. J. Natl. Cancer Inst. 2002; 94: 670–677. 40. T. Hoare et al., Can the uptake of breast screening by Asian women be increased? A randomized controlled trial of a linkworker intervention. J. Public Health Med. 1994; 16: 179–185. 41. M. Kagawa-Singer, Improving the validity and generalizability of studies with underserved US populations expanding the research paradigm. Ann. Epidemiol. 2000; 10: S92–S103. 42. J. Eyton and G. Neuwirth, Cross-cultural validity: ethnocentrism in health studies with special reference to the Vietnamese. Social Sci. Med. 1984; 18: 447–453. 43. R. J. Pasick et al., Problems and progress in translation of health survey questions: the Pathways experience. Health Educ. Quart. 1996; 23: S28–S40. 44. S. P. Tu et al., Translation challenges of crosscultural research and program development. Asian Amer. Pacif. Islander J. Health 2003; 10: 58–66. 45. M. R. McBride et al., Factors associated with cervical cancer screening among Filipino women in California. Asian Amer. Pacif. Islander J. Health 1998; 6: 358–367. 46. S. J. McPhee et al., Validation of recall of breast and cervical cancer screening by women in an ethnically diverse population. Prevent. Med. 2002; 35: 463–473. 47. R. J. Pasick et al., Quality of data in multiethnic health surveys. Public Health Reports 2001; 116: 223–243. 48. V. M. Taylor et al., Evaluation of an outreach intervention to promote cervical cancer screening among Cambodian American women. Cancer Detect. Prevent. 2002; 26: 320–327. 49. B. R. McAvoy and R. Raza, Can health education increase uptake of cervical smear testing among Asian women? Brit. J. Med. 1991; 302: 833–386.

9

50. S. J. McPhee, Promoting breast and cervical cancer screening among Vietnamese American women: two interventions. Asian Amer. Pacif. Islander J. Health 1998; 6: 344–350. 51. S. J. McPhee and T. T. Nguyen, Cancer, cancer risk factors, and community-based cancer control trials in Vietnamese Americans. Asian Amer. Pacif. Islander J. Health 2000; 8: 18–31. 52. T. Nguyen et al., Promoting early detection of breast cancer among Vietnamese-American women. Results of a controlled trial. Cancer 2001; 91: 267–273. 53. M. S. Chen, Jr. et al., Implementation of the indigenous model for health education programming among Asian minorities: beyond theory and into practice. J. Health Educ. 1992; 23: 400–403. 54. M. S. Chen, Jr., Cardiovascular health among Asian Americans/Pacific Islanders: an examination of health status and intervention approaches. Amer. J. Health Promot. 1993; 7: 199–207. 55. M. S. Chen, Jr. et al., An evaluation of heart health education for Southeast Asians. Amer. J. Health Promot. 1994; 10: 205–208. 56. L. Rychetnik et al., Criteria for evaluating evidence on public health interventions. J. Epidemiol. Community Health 2002; 56: 119–127. 57. L. Suarez et al., Why a peer intervention program for Mexican American women failed to modify the secular trend. Amer. J. Prevent. Med. 1997; 13: 411–417. 58. S. J. Curry and K. M. Emmons, Theoretical models for predicting and improving compliance with breast cancer screening. Ann. Behav. Med. 1994; 16: 302–316. 59. F. A. Hubbell et al., From ethnography to intervention: developing a breast cancer control program for Latinas. J. Natl. Cancer Inst. Monographs 1995; 109–115. 60. J. C. Jackson et al., Development of a cervical cancer control intervention program for Cambodian American women. J. Community Health 2000; 25: 359–375. 61. J. C. Jackson et al., Development of a cervical cancer control intervention for Chinese immigrants. J. Immigrant Health 2002; 4: 147–157. 62. B. I. Truman, J. S. Wing, and N. L. Keenan, Asians and Pacific Islanders. In: D. Satcher et al., (eds.), Chronic Disease in Minority Populations. Atlanta, GA: Centers for Disease Control, 1994.

10

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

Table 3. Community-based Breast and Cervical Cancer Intervention Studies Breast and/or cervical cancer screening

Group

Taylor, 2002

Cervical

Taylor, 2002

Cervical

First author, year of publication

Assignment unit

Intervention strategies

Evaluation method(s)

Cambodian

Small area neighborhood

Survey 12 months after randomization Review of medical records

No effect

Chinese

Individual

Home visit by outreach worker Group education session Use of video Logistic assistance Group 1: Home visit by outreach worker Use of video and print materials Logistic assistance

Survey six months after randomization Review of medical records

Both interventions effective Outreach worker intervention more effective than direct mailing intervention

Survey 12 months after randomization

No overall effect for mammography or Pap testing Effective for mammography in recent immigrant sub-group No effect

Maxwell, 2003

Both

Filipina

Small group

Wismer, 2002

Both

Korean

Community

McAvoy, 1991

Cervical

South Asian

Individual

Hoare, 1994

Breast

South Asian

Individual

Group 2: Direct mailing of video and print materials Group education session Use of print materials

Workshops delivered by lay health workers Distribution of print materials Group 1: Home visit by outreach worker Use of video

Group 2: Home visit by outreach worker Use of print materials Group 3: Direct mailing of print materials Home visit by outreach worker

Pre- and postintervention cross-sectional surveys

Main finding(s)

Review of screening program computerized records four months after randomization

Home visits were effective

Review of screening program computerized records

No effect

COMMUNITY-BASED BREAST AND CERVICAL CANCER CONTROL RESEARCH

11

Table 3. (continued) Breast and/or cervical cancer screening

Group

Assignment unit

Intervention strategies

Evaluation method(s)

Bird, 1998

Both

Vietnamese

Community

Pre- and postintervention cross-sectional surveys

Effective for mammography and Pap testing

Jenkins, 1999

Both

Vietnamese

Community

Series of three group education sessions delivered by lay health workers Distribution of print materials Media campaign Distribution of audio-visual and print materials

Pre-and postintervention cross-sectional surveys

Nguyen, 2001

Breast

Vietnamese

Community

No effect on mammography or Pap testing behavior Increased mammography and Pap testing intentions No effect

First author, year of publication

Media campaign Group education sessions Distribution of audio-visual and print materials

Pre- and postintervention cross-sectional surveys

Main finding(s)

COMPLIANCE AND SURVIVAL ANALYSIS

assignment have no impact on outcome, is consistently tested irrespective of compliance levels. Under the alternative, we expect, however, different (smaller) intent-to-treat effects than the prescribed regime would create when it materializes. This happens as the treatment group becomes a mix of varying (lower) degrees of exposure (2,8). Estimation of the causal effect of actual dose timing becomes challenging, when observed exposure patterns are no longer randomized. (Un)measured patient characteristics and earlier experience may determine exposure levels that become confounded with the natural treatment-free hazard of the patient. The association between compliance which induces treatment exposure levels, and treatment-free hazards is often called a selection effect in line with missing data terminology. An ‘‘as-treated’’ analysis, such as a PH analysis, with the currently received treatment as a time-dependent covariate, compares hazards between differently treated groups at a given time and thus estimates a mix of selection and causal effects (11). An ‘‘on-treatment’’ analysis censors patients as soon as they go off the assigned treatment and thus generates informative censoring for the same reason. Structural accelerated failure time (SAFT) models and structural PH models have been designed to avoid these biases. We explain their key features and potential through a simple example first.

ELS GOETGHEBEUR Ghent University, Ghent, Belgium

1

COMPLIANCE: CAUSE AND EFFECT

Today, new treatments must prove their worth in comparative (double blind) randomized clinical trials, the gold standard design for causal inference. With noninformatively right-censored survival outcomes, a typical robust intention-to-treat analysis compares groups as randomized using the popular (weighted) logrank test. Accompanying Kaplan–Meier, curves describe nonparametrically how survival chances differ between arms. A one-parameter summary of the contrast follows from a semiparametric Cox proportional hazards (PH) model Accelerated Failure-Time Model (6). In general, and especially with long-term treatments, patients tend to deviate from their prescribed treatment regime. Varying patterns of observed exposure relative to the assigned are called ‘‘compliance (levels)’’ and recognized as a likely source of variation in treatment effect. Because deviations from prescribed regimes occur naturally in clinical practice, it is wise to learn about them within the trial context rather than restrict the study population to perfect compliers, an atypical and sometimes small and unobtainable subset of the future patient horizon (12). Treatments that are stopped or switched or are less dramatic lapses in dosing happen in response to a given assignment. Different exposure patterns between treatment arms therefore point to (perceived) differences following alternative assignments. Studying compliance patterns as an outcome can yield valuable insights (15). Of course, actual treatment regimes may also influence primary outcomes. From the intent-to-treat perspective, underdosing causes reduced power and a requirement for larger samples. Fortunately, the strong null hypothesis, where treatment and its

2

ALL-OR-NOTHING COMPLIANCE

In randomized studies that evaluate a oneshot treatment, such as surgery (7), vaccination (4), or an invitation to undergo screening (1), all-or-nothing compliance with experimental assignment arises naturally. Let Ri = 1(0) indicate whether individual i, with baseline covariates Xi , gets randomized to the experimental (control) arm. When experimental treatment is not available outside the treatment arm the control arm remains

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

COMPLIANCE AND SURVIVAL ANALYSIS

uncontaminated by experimental exposure and its outcomes can serve as reference outcomes for causal inference. Let T 0i denote such a potential survival time for individual i following a control trajectory free of experimental exposure. Let T 1i and Ci respectively be survival time and compliance following a possible treatment assignment. Ri operates independently of the vector (T 0i , T 1i , Ci , Xi ) but determines, which components are observed. Observed survival time and exposure are simply denoted T i , Ei for all. With an uncontaminated control arm, Ei = Ci Ri . One goal of causal inference is to estimate how the contrast between potential survival times T 1i and T 0i varies over the subpopulations determined by different Ci levels and their induced level of experimental exposure on the treatment arm, Ei . The sharp null hypothesis assumes it makes no difference what arm one is assigned to and hence: d|X T0i = i T1i ,

(1)

d|X where = i indicates equality in distribution conditional on Xi . The most obvious violation of (1) occurs when (some) patients on the experimental arm become exposed and exposure alters survival chances. This is called a direct causal effect of exposure (10). When an assignment influences survival through mechanisms of action operating independently from exposure levels, we have an indirect effect. Below, we consider a cancer clinical trial (7), where the experimental intervention consists of implanting an arterial device during surgical resection of metastases. A planned implant could lead to an operation scheduled earlier in the day and timing may create its own set of prognostic circumstances. In addition, the news that a planned implant did not happen could be depressing to the patient and diminish survival chances beyond what would have happened on the control arm. Both mechanisms can lead to an indirect (clinical) effect of exposure assignment. Double blind studies are carefully designed to avoid indirect {d|C =0, X } effects, so they satisfy: T0i i = i T1i and hence, P(T 1i > t—Ci = 0, Ri = 1, Xi ) = P(T 0i > t|Ci = 0, Ri = 0, Xi ), for all t. The contrast

between P(T 1i > t|Ci = e, Ri = 1, Xi ) and P(T 0i > t|Ci = e, Ri = 0, Xi ) then represents the causal effect of exposure level e. In general, however, this combines direct and indirect effects of assignment in the population with compliance level e. In what follows, we ignore Xi for simplicity, but stronger inference can be drawn when assumptions condition on Xi . To estimate P(T 0i > t|Ci = 1, Ri = 1), one can solve P(T 0i > t|Ci = 1|Ri = 1)P(Ci = 1, Ri = 1) + P(T 0i > t|Ci = 0, Ri = 1)P(Ci = 0|Ri = 1) = (P(T 0i > t|Ri = 1) =)P(T 0i > t|Ri = 0) after substituting empirical means or (Kaplan–Meier) estimates for the other unknown terms. Isotonic regression can turn the pointwise estimates in a monotone survival curve. To evaluate the treatment effect among the exposed, one comˆ 0i
t|Ci = ˆ 1i
t|Ci = 1, Ri = 1) with P(T pares P(T 1, Ri = 1). The selective nature of exposure is seen by contrasting treatment-free survival probabilities for the exposed and nonexˆ 0i
t|Ci = 1, Ri = 1) posed subpopulations: P(T ˆ 0i
t|Ci = 0, Ri = 1). and P(T 3 MORE GENERAL EXPOSURE PATTERNS A structural model parameterizes the shift in distribution from observed survival time T i to a reference time, T ei following a specific exposure regime e, in terms of observed (possibly time-dependent) exposures Ei and covariates Xi . One can thus perform parameter-specific (Ei , Xi )-dependent back transformations of observed survival times (or distributions). The parameter value that solves estimating equations demanding equality of estimated T ei distributions (conditional on baseline covariates) between arms is our point estimate. The procedure is illustrated in Figure 1 d|R

for the SAFT model Ti exp{−β0 Ei } = i T0i in our trial, where Ei indicates an actual implant of the arterial device. For timedependent implants Ei (t), we could have used
T d|R the SAFT model 0 i exp(−β0 Ei (u))du = i T0i . For technical points concerning the specific treatment of censored data, we refer the reader to (5,12). The left-hand panel shows ITT Kaplan–Meier curves in the standard and intervention arm. In the right-hand panel, the survival curve for the standard

COMPLIANCE AND SURVIVAL ANALYSIS

3

Figure 1. Estimation of structural parameters

arm is compared with KM-curves following the transformations T i exp{−βEi } with β = −1.5 and β = −0.36 on the intervention arm. Reducing treated failure times by the factor exp(−1.5) overcompensates for the observed harmful treatment effect as survival chances on the intervention arm are now higher than on the standard arm. This is confirmed by the logrank chi-squared value of 9.326, plotted in the middle panel. The survival curve corresponding to the point ˆ = 70%) is conestimate βˆ = −0.36 (exp(β) vincingly close to the observed survival in the standard arm. The middle panel reveals chisquared values for a range of hypothesized structural parameter values. Those that do not lead to significantly different curves at the 5% level form the 95% confidence interval [−1.07, 0.39] for β 0 . 4 OTHER STRUCTURAL MODELING OPTIONS One can propose many different maps of the observed into the treatment-specific survival distributions. This may happen on a PH scale

(7) or involve time-dependent measures of effect in the SAFT setting (12). Estimation methods, which rely on the instrument of randomization, protect the α –level just like the intent-to-treat test, but shift the point estimate away from a diluted average. To achieve this, they rely on the postulated structural model, which can sometimes be rejected by the data, but generally not confirmed owing to a lack of observed degrees of freedom. Special care is thus required when interpreting these models and their results. Some diagnostic procedures have been proposed and forms of sensitivity analyses (13,14). To explicitly account for measured timedependent confounders, structural nested failure-time models can be used as an alternative, or marginal structural models for T ei as in (3). The estimation process then relies on the assumption of ‘‘no residual confounding’’, ignores the instrument Ri , and loses its robust protection of the α –level. Structural modeling of failure time distributions has opened a world of practical and theoretical developments for the analysis of compliance and survival time. The field of

4

COMPLIANCE AND SURVIVAL ANALYSIS

research is very much alive today. Recent work (9), for instance, proposes to estimate optimal treatment regimes from compliance data. Our brief account can give but a flavor of this wealth.

5

ACKNOWLEDGMENTS

We thank Tom Loeys for drawing the figure.

REFERENCES 1. Baker, S. G. (1999). Analysis of Survival data from a randomized trial with all-or-nothing compliance: estimating the cost-effectiveness of a cancer screening program, Journal of the American Statistical Association 94, 929–934. 2. Frangakis, C. E. & Rubin, D. B. (1999). Addressing complications of intention-totreat analysis in the combined presence of all-or-none treatment-noncompliance and subsequent missing outcomes, Biometrika 80, 365–379. 3. Hernan, M. A., Brumback, B. & Robins, J. M. (2001). Marginal structural models to estimate the joint causal effect of nonrandomized treatments, Journal of the American Statistical Association 96, 440–448. 4. Hirano, K., Imbens, G., Rubin, D. & Zhou, X. H. (2000). Assessing the effect of an influenza vaccine in an encouragement design, Biostatistics 1, 69–88. 5. Joffe, M. M. (2001). Administrative and artificial censoring in censored regression models, Statistics in Medicine 20, 2287–2304. 6. Kalbfleisch, J. D. & Prentice, R. L. (2002). The Statistical Analysis of Failure Time Data, 2nd Ed. Wiley, New Jersey, Hoboken. 7. Loeys, T. & Goetghebeur, E. (2003). A causal proportional hazards estimator for the effect of treatment actually received in a randomized trial with all-or-nothing compliance, Biometrics 59, 100–105. 8. Mark, S. D. & Robins, J. M. (1993). A method for the analysis of randomized trials with compliance information: an application to the multiple risk factor intervention trial, Controlled Clinical Trials 14, 79–97. 9. Murphy, S. (2003). Optimal dynamic treatment regimes, Journal of the Royal Statistical Society, Series B 65, 331–355.

10. Pearl, J. (2001). Causal inference in the health sciences: a conceptual introduction, Health Services and Outcomes Research Methodology 2, 189–220. 11. Robins, J. M. & Greenland, S. (1994). Adjusting for differential rates of PCP prophylaxis in high-versus low-dose azt treatment arms in an aids randomized trial, Journal of the American Statistical Association 89, 737–749. 12. Robins, J. M. & Tsiatis, A. A. (1991). Correcting for non-compliance in randomized trials using rank preserving structural failure time models, Communications in Statistics, A 20, 2609–2631. 13. Scharfstein, D., Robins, J. M., Eddings, W. & Rotnitzky, A. (2001). Inference in randomized studies with informative censoring and discrete time-to-event endpoints, Biometrics 57, 404–413. 14. White, I. & Goetghebeur, E. (1998). Clinical trials comparing two treatment arm policies: which aspects of the treatment policies make a difference? Statistics in Medicine 17, 319–340. 15. White, I. & Pocock, S. (1996). Statistical reporting of clinical trials with individual changes from allocated treatment, Statistics in Medicine 15, 249–262.

CROSS-REFERENCES Noncompliance, Adjustment for Survival Analysis, Overview

COMPOSITE ENDPOINTS IN CLINICAL TRIALS

of type I error). Thus, the study is judged to have achieved statistically significant evidence of efficacy and to be a positive trial only if the test for the primary endpoint achieves the prespecified level of significance. The approach of relying on a single primary endpoint enables the investigators to control the overall probability of falsely declaring the experimental therapy to be efficacious, and it assists in the interpretation of analyses of the secondary endpoints, which will be helpful in delineating the nature of any treatment effect when the primary endpoint is significant. If the primary endpoint is not statistically significant, analyses of secondary endpoints are not used to definitively establish efficacy, but they may provide insights about treatment effects as well as information that may be helpful in designing future clinical trials. In performing secondary analyses, one must be concerned about the problem of multiple testing—that one or more secondary endpoints might reach statistical significance by chance. Because the overall error rate for the trial is controlled by specifying the primary endpoint, the usual approach to minimizing the problems with multiple testing is to limit the group of secondary endpoints to a relatively small number of the most important endpoints. Others are classified as tertiary endpoints. Because specification of the primary endpoint is critically important in the design of a clinical trial, it deserves serious consideration, a review of the relevant literature, and perhaps even preliminary studies. Five desirable properties of a primary endpoint have been proposed (1): it should be (1) clinically relevant, (2) accurate, (3) reproducible, (4) objective, and (5) quantitative. In practice, specification of a primary endpoint will also need to consider the nature of the disease and the manifestations under study as well as the availability of patients, investigators, and resources. The choice of the components of composite endpoint also should be related to the primary question of interest (2). The use of a composite endpoint, defined a priori, that comprises a variety of patient characteristics is often better suited to accom-

PETER C. O’BRIEN Division of Biostatistics Mayo Clinic College of Medicine Rochester, Minnesota

BARBARA C. TILLEY Department of Biostatistics, Bioinformatics, and Epidemiology Medical University of South Carolina Charleston, South Carolina

PETER J. DYCK Peripheral Nerve Lab Mayo Clinic College of Medicine Rochester, Minnesota

Composite endpoints have become increasingly useful as a measure of a patient’s disease status. They have been found useful in both clinical practice and medical research. Here, we shall examine the usefulness of composite endpoints as a primary endpoint in clinical trials. The basic rationale for their use is discussed and illustrated with examples, followed by a variety of procedures for forming composites including how they should be interpreted. In the section on global assessment variables, alternate approaches to conducting a primary analysis with multiple endpoints are considered and these methods are compared with the use of composite endpoints. 1 THE RATIONALE FOR COMPOSITE ENDPOINTS We consider the outcome measures used in the typical phase III clinical trial for efficacy, for simplicity assuming that an experimental therapy is being compared with a placebo. Most commonly, a protocol for such a study specifies a single primary endpoint (see the discussion of primary endpoints), various secondary endpoints (see the discussion of secondary endpoints), and often additional tertiary endpoints. The goal of categorizing study endpoints in this manner is to control the type I error rate (see the discussion

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

COMPOSITE ENDPOINTS IN CLINICAL TRIALS

plishing the objectives. In particular, by providing a more comprehensive assessment of the patient, it may be able to achieve a much higher level of clinical relevance. These considerations are illustrated with examples, including a detailed example involving clinical trials of drugs to treat diabetic neuropathy. 2 FORMULATION OF COMPOSITE ENDPOINTS Next, some useful methods for defining a composite endpoint for use as a primary endpoint in a clinical trial are considered. 2.1 Existing Scoring Systems Ideally, a suitable comprehensive composite endpoint has already been identified and extensively studied, such as the composite scores for diabetic sensory polyneuropathy. These types of composites seek to cumulate information about qualitatively different patient characteristics, all of which are directly relevant to efficacy. A commonly used type of composite endpoint consists of scores assigned to each of numerous individual questions, which are then summated to obtain the composite score. Many such scoring systems have been developed, validated, and widely used in clinical applications as well as in clinical trials. Their usefulness may have been studied by a group of experts in the field, and the appropriate use and interpretation may have been delineated by a consensus statement that will indicate what constitutes a meaningful effect size, among other things. The availability of such an existing composite endpoint that adequately measures patient status for the purposes of the specific clinical trial under consideration may be viewed as the ideal. If such a scoring system is available, a statistical approach for defining a composite endpoint is not needed. 2.2 Methods Based on Rankings In many situations, no well accepted scoring systems for combining variables are available. It may also be considered more meaningful to weight each of the endpoints equally

and without regard to their units of measure. A rank sum scoring approach (see the discussion of global assessment variables) may be useful in these situations. Specifically, one ranks all the subjects from best to worst separately for each variable. One then computes the sum of the ranks received for each subject to obtain rank sum scores measuring the relative performance of each subject. The relative rankings obtained are helpful in assessing the clinical relevance of the treatment effect and can be used to estimate the probability that the experiment will produce a better result than placebo (again, see the discussion of global assessment variables). 2.3 Methods for Endpoints That Are All Dichotomous or Censored In many trials, all the individual endpoints are censored. Cardiology trials, for example, are often interested in the time to any of several cardiac related events. An approach is to consider the time to the first event as the composite endpoint. Similarly, if all the variables are binary, a composite binary endpoint can be defined as the occurrence of any of the individual events. 3 EXAMPLES 3.1 Clinical Trials in Diabetes We can illustrate the use of composite versus individual measurement with the condition diabetic sensory polyneuropathy (DSPN) (3), which we have studied in detail in a crosssectional and longitudinal study (the Rochester Diabetic Neuropathy Study) and in several controlled clinical trials. Intuitively, we observe that DSPN is the sum total of a patient’s neuropathic symptoms; neurologic signs of weakness, sensory loss (different modalities), and autonomic nerve dysfunctions as well as overall impairments and disability. To track all of these diverse manifestations and appropriately assign a relative worth to each component is not presently feasible. Use of every measure might also be too intrusive, time consuming, and expensive. The problem is even more complex because a variety of composite scores are available (e.g., attributes of nerve conduction, neurologic

COMPOSITE ENDPOINTS IN CLINICAL TRIALS

signs, sensation tests, and so on). Without going into great detail, we found that no one attribute of nerve conduction or single clinical sign adequately represented the condition DSPN or was a suitable marker for its presence or severity. For controlled trials focusing on the development of DSPN, it is possible to use a single criterion such as that chosen for conduct of the Diabetes Control and Complications Trial study (4), which compared rigorous versus conventional glycemic control. The criterion for DSPN was: ≥2 abnormalities from among these three: (1) decreased or absent ankle reflexes, (2) decreased or absent vibration sensation of great toes, or (3) symptoms of polyneuropathy not attributable to other neuropathies. Although this score performed well for this purpose, it provided information only about frequency and not about severity of DSPN. For many purposes, knowledge about severity of DSPN would provide additional important information. We have developed several composite and continuous measures of severity
of DSPN. One such composite score is 5 NC nds. In this score, five attributes of nerve conduction (NC) of leg—chosen because they are distal nerves, representing different functional groups—are each expressed as a normal deviate (nd), from percentiles corrected for age, sex, height, and weight based on a study of healthy subjects and with all abnormalities expressed in the upper tail of the normal distribution, which are summed, divided by the number of measurable attributes, and multiplied by 5. We have found that this summated score can be used to provide an excellent measure not only of the presence but also the severity of polyneuropathy because it tracks severity of DSPN without a floor or ceiling effect, is sensitive at defined levels of specificity, is very reproducible, and correlates reasonably with clinical neurologic deficit. It has the advantage of being objective (patients cannot will the result). This measure has also been shown to have good monotonicity and is highly suitable for controlled trials. It is also our impression that
5 NC nds might have a low degree of inter–medical center variability when used by certified electromyographers whose training and choice of equipment would likely

3

result in a high degree of reproducibility of results. However, for judging severity of DSPN, other primary outcome measures might be used. For example, if the emphasis is to be on the effect on neurologic signs (i.e., leg weakness, reflex loss, or sensation loss), a summed score of neurologic signs (e.g., neuropathy impairment score of lower limb), or
summated quantitative sensation tests ( QST nds) might be used. Scales that judge the severity of a patient’s symptoms are needed when the drug to be tested has its putative action on the relief of symptoms (e.g., pain). Thus, in conditions such as headache, trigeminal or other neuralgias, fatigue states, anxiety, and the like, if composite clinical endpoints are chosen, these outcomes should include a measure of the frequency, duration, and severity of symptoms. In contrast to use of impairment measures in which an external physician or scientific observer makes observations, the patient’s judgment of quality of life, impairment for acts of daily living, or disability may also need to be considered. Scales of acts of daily living and quality of life are used to represent the patient’s perception of the impact of his or her illness on performance of life tasks and quality of life and whether an intervention improves the dysfunction or disability (5). Scoring of a single or a battery of motor functions or acts of daily living may also be used to assess overall severity of polyneuropathy. This approach has been extensively explored in assessment of therapy in immune polyneuropathies (6).

3.2 Clinical Trials in Cardiovascular Disease In cardiovascular disease (CVD) and other diseases where treatment is expected to affect both morbidity and mortality, it is common to choose an outcome representing failure of any one of a set of poor outcomes occur. For example, in assessing the benefits of aspirin for CVD prevention, outcomes such as vascular death, myocardial infarction, or major stroke; or death, myocardial infarction, or stroke; or death or reinfarction have been used as composite outcomes (7).

4

COMPOSITE ENDPOINTS IN CLINICAL TRIALS

3.3 Clinical Trials in Rheumatoid Arthritis The American College of Rheumatology criteria (ACR) for treatment success have been developed as a composite outcome. The ACR criteria were based on achieving success on at least some of the outcomes of interest. The composite outcome has been validated in a variety of settings and has been modified as background treatment has improved (8). 3.4 Other Comments In general, one can categorize the situations where multiple endpoints are needed to measure patient status in two ways: (1) each endpoint measures a qualitatively different patient attribute versus providing alternate measures of the same attribute, and (2) each endpoint considered by itself is sufficiently important that it could be used as a primary endpoint (but is not as informative and meaningful as the multiple measures taken together) versus a situation where one or more of the endpoints taken alone would not be sufficiently meaningful to provide a primary endpoint by itself but nonetheless adds important information in arriving at an overall assessment of the patient. Although these distinctions are helpful, they may be difficult to make in practice. For example, one might consider electromyographic measures of nerve conduction and measures of sensation to be assessing different attributes. However, for both endpoints, it is possible to distinguish whether the function of small or large nerve fibers is being assessed. Therefore, it could be argued that if both endpoints are evaluating small fibers then they are measuring the same attribute. The distinctions mentioned above also would be important in a disease that has qualitatively different manifestations. For example, Hunter syndrome affects primarily lung function in some patients and primarily ambulation in others, so focusing on only one of these attributes with a single primary endpoint would not provide a clinically comprehensive and meaningful assessment. Although the primary goal of a composite endpoint is to obtain the most clinically meaningful measure of patient status with which to measure efficacy, a more comprehensive measure will typically also have the

desirable statistical property of increasing the power of the trial. Combining multiple endpoints to provide a more comprehensive assessment of the patient may also be helpful in evaluating treatment safety. 4 INTERPRETING COMPOSITE ENDPOINTS The appropriate interpretation of a composite endpoint depends on how it was formed and its intended use. The most straightforward situation is the case of existing scoring systems that have been validated and widely used in clinical practice. In this case, there is a clear understanding of the clinical relevance of the magnitude of group differences observed in the trial. If the composite consists of a summation of many items, reporting of trial results often focuses only on the overall score and does not examine the individual components. This is unfortunate because one would expect that important additional information may be gleaned from determining whether the treatment effects were seen in only a few of the individual variables (in which case, the nature of any clustering may be important) or whether the effects were displayed consistently across all variables. One way to approach the interpretation of the composite score is to focus on inspection of the individual components. For example, with relatively few individual variables, it may be possible to prespecify what constitutes a clinically meaningful effect size for each endpoint. Individual statistical tests accompanying each endpoint may further assist in interpretation. Because the overall error rate has been controlled by the test for the composite endpoint, adjustment for these individual tests is not needed. Even if testing of individual outcomes is not conducted, at a minimum the results for each of the components of the composite outcome should be reported. As Wong (9) has commented, a review article on aspirin and CVD (7) that reports on only the composite outcome and myocardial infarction could mask the relationships between aspirin and stroke; if the individual outcomes as well as the composite outcome had been reported, this concern would have been addressed.

COMPOSITE ENDPOINTS IN CLINICAL TRIALS

Strategies for making decisions about clinical relevance with composite endpoints depend on the context. For example, if the overall test is significant and at least one of the endpoints reaches a clinically meaningful threshold, should the corresponding statistical test associated with that endpoint also be required to reach statistical significance? In our view, this depends on the circumstances. Alternatively, the overall test for the composite endpoint might reach statistical significance (demonstrating that the experimental treatment has a real beneficial effect), but none of the individual endpoints may reach the threshold for clinical importance or statistical significance. This could be likely to happen when the composite outcome is defined as the occurrence of any one of several events. Should the magnitude of the treatment effect be considered not clinically meaningful, or might the nature of the effect be diffuse but sufficiently consistent across endpoints that it cumulates to something clinically meaningful? Again, this depends on the circumstances, and clinical judgment is most important for arriving at a conclusion. It may be particularly important to examine the treatment effect on individual endpoints if the composite is the time to first event or the occurrence of any event because the type of events that occur soonest or most frequently may have an outsize effect on the composite. 5

CONCLUSIONS

Composite endpoints are valuable in clinical trials of efficacy, primarily because they provide a comprehensive and clinically meaningful assessment of the patient. The efficient use of multiple sources of information may also result in an increase in power over a single outcome. REFERENCES 1. P. C. O’Brien, Commentary to: A clinical trial endpoint based on subjective rankings, by Follmann D, Wittes J and Cutler JA. Stat Med. 1992; 11: 447–449. 2. N. Freemantle, M. Calvert, J. Wood, J. Eastaugh, and C. Griffen, Composite outcomes

5

in randomized trials: greater precision but with greater uncertainty? JAMA. 2003: 289: 2554–2559. 3. DCCT Research Group. The Diabetes Control and Complications Trial (DCCT): design and methodologic considerations for the feasibility phase. Diabetes. 1986; 35: 530–545. 4. DCCT Research Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med. 1993; 329: 977–986. 5. P. J. Dyck and P. C. O’Brien, Polyneuropathy dysfunction scores. J Neurol Neurosurg Psychiatry. 2006; 77: 899–900. 6. Graham RC, Hughes RAC. A modified peripheral neuropathy scale: the Overall Neuropathy Limitations Scale. J Neurol Neurosurg Psychiatry. 2006; 77: 973–976. 7. C. L. Campbell, S. Smyth, G. Montalescot, and S. Steinhubl, Aspirin dose for the prevention of cardiovascular disease: a systematic review. JAMA. 2007. 297; 2018–2024. 8. D. T. Felson, J. J. Anderson, M. L. Lange, G. Wells, and M. P. LaValley, Should improvement in rheumatoid arthritis clinical trials be defined as fifty percent or seventy percent improvement in core set measures, rather than twenty percent? Arthritis Rheum. 1998; 41: 1564–1570. 9. G. K. Wong, Letter to the editor. JAMA. 2007; 298: 625.

COMPUTER-ASSISTED DATA COLLECTION

appearing for in-person data collection as well. Before then, research data from study participants was typically captured on paper forms. Key differences exist between computerbased data collection (or EDC) and traditional paper-based data collection. In a traditional data-collection setting, an interviewer reads question items and response options to the study participant and then captures the responses on paper. Alternatively, participants independently complete paper forms or self-administered questionnaires. Unlike a paper-based form, an EDC system can dynamically tailor question wording on the basis of previously entered responses, implement complex routing logic so that only applicable questions are asked of respondents, and incorporate real-time data edits to ensure that responses are within valid ranges and are consistent with prior responses. Moreover, the computer system can enforce response to questionnaire items (using a ‘‘don’t know’’ or ‘‘refuse to answer’’ response when necessary) to minimize or eliminate missing data. Also unlike paper data, data collected through EDC or computer-assisted interviewing can rapidly be made available to a data management team for quality control and preliminary results. Under typical computerassisted, in-person data collection, data are collected with use of a laptop or desktop computer and then retrieved from the computer as often as desired (generally by secure electronic transmissions initiated from each computer). By contrast, data captured on paper forms are less immediately available because the responses must be subjected to a data-entry and verification process. If the data are collected at multiple locations, there may be the further delay of shipping paper forms to a central data entry location or the added overhead of setting up multiple data entry sites. The advantages of EDC, or computer-assisted interviewing, over traditional penciland-paper methods can be summarized as follows: Reduction in time from last patient visit to database release

TIM J. GABEL , SUSAN D. PEDRAZZANI and ORIN A. DAY Survey and Computing Sciences RTI International Research Triangle Park, North Carolina

Computer-assisted data collection, or electronic data capture (EDC), has been used for more than two decades in the government survey and market research industries but has only in recent years begun gaining broad acceptance in the clinical research setting, which relies heavily on the use of paper-based case report forms (CRFs). In many clinical studies, participants typically use paper forms to provide self-reported data for the study. These forms are then converted to electronic format using data-entry or clinical data management systems suitable for EDC. An ever-increasing number of clinical studies are deploying computerassisted or EDC systems, choosing them over paper-and-pencil data collection as information on the advantages becomes more widely known. Driving this conversion from paper to computer-assisted data collection is the need to collect quality data in a more accurate, efficient, and cost-effective manner. Audio computer-assisted self-interviewing (A-CASI) and telephone audio computerassisted self-interviewing (T-ACASI) are just two of the numerous methods of computerassisted data collection that can be used in clinical trials. A-CASI and T-ACASI may be particularly applicable to the collection of high-quality patient-reported data, especially in Phase IV and postmarketing surveillance studies. 1 DESCRIPTION OF COMPUTER-ASSISTED INTERVIEWING Computer-assisted interviewing for telephone-based data collection has been in use since at least the early 1970s (1). With the introduction of personal computers in the 1980s, computer-assisted methods began

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

COMPUTER-ASSISTED DATA COLLECTION

Reduction in number of required queries (fewer data clarification forms) Faster and easier query resolution Reduction in site monitoring costs (improved and easier monitoring) Reduction in project management costs Efficiency gains from reuse of forms (development of standards library) Ability to perform interim or ad hoc analysis earlier Faster access to data, enabling better-informed decision making Elimination of paper handling Potential for more secure data with use of passwords, data encryption, and limitation of access to database Potential for more complicated questionnaire routing Ability to specify and computerize enforcement of logic checks and range checks Elimination of or reduction in missing data items Despite the advantages of computerassisted interviewing methods over paperbased data collection, however, often the necessary involvement of a live interviewer carries disadvantages that a self-administered instrument avoids. In 1990 Miller et al. (2) summarized some of the methodological challenges in conducting surveys concerned with acquired immunodeficiency syndrome (AIDS). Observing that participation and response were impacted by participants’ concerns about confidentiality, Miller and colleagues reported that ‘‘because of the sensitive and highly personal nature of these questions, virtually all of the surveys made some provision to permit respondents to reveal the details of their sexual behavior without undue embarrassment’’ (2). A-CASI and T-ACASI currently answer this kind of need for private response while retaining the advantages of EDC. 2 AUDIO COMPUTER-ASSISTED SELF-INTERVIEWING In 1994, O’Reilly et al. (3) described the concept of A-CASI as a new technology for

collecting data. With the use of A-CASI, instead of being read questions by an interviewer and responding verbally while the interviewer enters the responses into the computer, the respondent listens in privacy to digitally recorded audio questions delivered to headphones through the sound card of the computer. He or she then enters responses independently, using the computer keyboard or a touch-screen. In this way, an A-CASI instrument temporarily removes the interviewer from the process; in fact, an A-CASI instrument can offer modes in which questions are not even displayed on the screen so that interviews can be conducted in a completely confidential manner in an otherwise less than private setting. Building on the research by O’Reilly and colleagues, Turner et al. (4) reported on the outcomes of an A-CASI experiment embedded in the 1995 National Survey of Adolescent Males. The embedded A-CASI experiment demonstrated statistically significant differences in the reporting of ‘‘sensitive’’ behavior (in this case, male–male sexual contact within a national probability sample of males aged 15 to 19 years) when A-CASI methods were used instead of interviewer-administered survey questions. Since that time, A-CASI has gained widespread acceptance in governmentsponsored social science research. In more recent years, the A-CASI approach has been extended to telephone-based data collection efforts. Health-related studies collecting sensitive data have been implemented using Telephone A-CASI, or T-ACASI. This approach combines interactive voice-response technologies with interviewer-administered questions (5). With T-ACASI, the respondent listens to digitally recorded questions over the telephone and enters his or her response by using a touch-tone telephone keypad. Again, without a clinician or interviewer present to ask the questions or to enter the responses, the study participant can feel more comfortable answering sensitive questions in a confidential setting (6).

COMPUTER-ASSISTED DATA COLLECTION

3 COMPUTER-ASSISTED METHODS IN CLINICAL TRIALS These computer-assisted data collection methods apply especially well to the clinical research arena. The more prevalent EDC becomes in clinical trials, the more likely these methods will gain acceptance in the clinical community. However, careful consideration and planning regarding A-CASI and T-ACASI approaches are crucial for effective implementation and study outcomes. 3.1 Questionnaire Design Most clinical researchers recognize the importance of devoting adequate time and resources to the development of the questionnaire or clinical forms, but this is especially true for studies incorporating A-CASI or T-ACASI methods. Because these approaches involve self-administration by study participants who interact with a computer, all questionnaire items must be clear and understandable to the participants. We recommend that a forms design team develop and test the questions with the target population in mind, giving especially careful consideration to those items that will be self-administered. To design and test self-administered questions for requisite clarity, the design team must first assess study participant characteristics such as age, literacy, computer experience or access to a touch-tone telephone, culture, language, and disease or condition severity. Consideration of these characteristics will help determine, at the outset, whether A-CASI or T-ACASI can be a successful mode for interviewing a given population. For example, the population of older persons may require a large-point font in which to view an onscreen A-CASI application, a mentally challenged population may require the digitally recorded text to be administered at a much slower pace than is customary for other populations, or it may be necessary to record the text in multiple languages. The key is to build an A-CASI or T-ACASI application that the target population will understand and easily use, so that required data will truly inform the hypotheses. A clinical study that has successfully addressed the specific characteristics and

3

needs of a target population can fully benefit from one of the major advantages of using A-CASI and T-ACASI: computerized standardization of questionnaire administration. This standardization permits the collection of better-quality data than is possible with a live interviewer because the prerecorded questions are administered as audio files in exactly the same way to every respondent who completes the questionnaire. The voice, tone, inflection, speed, and text are identical for each respondent, no matter how many times a question is repeated for each respondent. Unlike clinician- or intervieweradministered questionnaires, A-CASI or T-ACASI prevents alteration of question inflection or of specific word or phrase emphasis that might compromise standardization. Because no interviewer or clinician is involved in the self-administered phase of the interview process, computerized questionnaires must be designed accordingly. One design feature common to most computerassisted questionnaires is provision for a ‘‘don’t know’’ response if the respondent does not understand the question or simply has no response. Typically, at the beginning of the self-administered portion of the questionnaire, some standard conventions are provided to the study participant. For example, the respondent may be told to press a specific key or button for a ‘‘don’t know’’ response, and a different key or button to hear the question repeated. Help screens or help text can also be built into the questionnaire to clarify a specific question or series of response categories. In keeping with the goal of maximizing study response participation when a live interviewer is absent, the design team for an A-CASI questionnaire may do well to consider whether the study participant will provide input by computer keyboard or by touch-screen monitor. Most adults and children in today’s society have become adept at using touch-screen systems such as airport kiosks, automatic teller machines, and grocery store checkout machines. There is some evidence to suggest that, when given a choice, clinical study participants prefer responding by touch-screen computer rather than by keyboard. For example, among a

4

COMPUTER-ASSISTED DATA COLLECTION

convenience sample of 108 patients at a Baltimore, Maryland, sexually transmitted disease (STD) clinic, nearly 70% indicated that using a touch-screen A-CASI application was easier than responding by computer keyboard (7). 3.2 Edit Checks Computer-assisted questionnaire edit checks can preempt the need for laborious data editing and cleaning on the back end. In clinical research, errors and anomalies occur despite careful study design and implementation of quality assurance and quality control strategies. To minimize the impact of such anomalies and errors on the study results, data editing and cleaning typically occur as part of the data management process once the data are collected. Data editing is the process of identifying ambiguous or erroneous data and diagnosing the source of the problem, and data cleaning is the process of changing the erroneous data shown to be incorrect. Typical clinical trial data editing procedures include a ‘‘query’’ process, whereby data anomalies are resolved by referring back to a CRF, and in some instances by interacting with staff at a clinical site. With A-CASI and T-ACASI, there is no CRF. In fact, there is no paper record at all for clarifying data issues. The patient-reported source data are initialized in electronic format, and real-time edits must be incorporated into the computerized application to ensure that high-quality data are captured. Data edits, such as logic and range checks, are routinely programmed into the questionnaire to ensure that data values are within legitimate ranges and to help ensure consistency across items. Although the ability to implement edit checks with the use of A-CASI or T-ACASI is an advantage over paper-and-pencil data collection, the questionnaire designer must still consider how often to use edit checks, when to use them, and whether to deploy ‘‘soft’’ edit checks, ‘‘hard’’ edit checks, or both during administration of the questionnaire. Use of edit checks involves balancing the need for the highest quality data possible with the added burden and potential frustration they pose for the respondent. With a triggered hard edit check, for example, a

message will appear—usually in a pop-up box on the same screen—requiring a change in a patient’s response. By contrast, the soft edit check, if triggered, will simply recommend that the respondent make a change in response but also will permit the response to remain unchanged and will allow the respondent to continue with the questionnaire. Among options for edit checks, one type of hard edit requires that the respondent actually enter a value into an item before moving on to the next question. This safeguard is particularly recommended for ACASI and T-ACASI because the clinician, being removed from the self-administered interviewing phase, cannot help prevent missing data. Good candidates for hard edit checks are questionnaire items that aim to collect data for critical or key study endpoints. Hard edits ensure that questions collecting key study hypotheses data will have little or no missing data, and that the data will be within expected and acceptable ranges. Soft edit candidates are items for which a logical range of values can be established but outlier values may be feasible. In such instances, the range edit check can be triggered when out-of-range values are entered, preventing data errors due to miskeyed values but allowing the respondent to reconfirm or reenter the out-of-range value and continue completing the computerized form. One way to establish optimal use of edit checks is to pretest computerized forms. Interviewing a small number of participants who have the same characteristics as the target population provides valuable feedback about where and when a questionnaire requires edit checks. For example, if more than half the pretest participants enter a particular question response outside the range of values the team was expecting, adding an edit check to that question may be wise. If the pretest indicates that many questionnaire items require an edit check, the team may want to consider prioritizing the questions and implementing edit checks for the more critical data items, and clarifying the other questions or adding help screens. Pretesting also provides an opportunity to judge the overall usability (e.g., clarity of question wording, placement of entry fields on the

COMPUTER-ASSISTED DATA COLLECTION

screen) of the computerized system by study participants. In developing any type of edit checks, the study team should consider how best to convey each ‘‘error’’ or notification message with minimal burden to the respondent. Ideally, an ‘‘error’’ or notification message is evident to the respondent and clearly specifies what the respondent is to do. For example, an error message regarding mis-keyed birth month might read, ‘‘Month of Birth must be between 1 and 12. Please reenter.’’ A well-designed error message is brief and clear; the subsequent instructions are simple and relatively easy to act upon. 3.3 Quality Assurance Although not the focus of this article, a brief discussion is essential to address clinical studies that because of sponsor requirements must follow specified regulations—such as studies that ultimately submit their data to the U.S. Food and Drug Administration (FDA) or other regulatory agencies. The conduct of these studies, including the development and documentation surrounding the computer application, must follow all relevant federal regulations and guidelines. One example of such a regulation is ‘‘Electronic Records; Electronic Signatures,’’ 21 C.F.R. pt. 11 (2003). Audit trails are a key component of such regulations and should be included in the computerized application. The FDA defines an audit trail as a ‘‘chronological record of system activities that is sufficient to enable the reconstruction, reviews, and examination of the sequence of environments and activities surrounding or leading to each event in the path of a transaction from its inception to output of final results’’ (8). Although most commercial survey systems were not originally designed to be used for clinical research, many of them (e.g., Blaise from Statistics Netherlands, www.blaise.com) include electronic audit trail features. The key is to know the requirements of the sponsor and any relevant governing agencies and to implement a system that meets all the requirements at study inception. Implementation includes creating and maintaining the system and the testing documentation according to those same requirements.

5

4 CLINICAL CASE STUDIES USING COMPUTER-ASSISTED METHODS In this section, selected case studies illustrate how A-CASI and T-ACASI have been used in clinical studies. Although the research described includes both federally and privately sponsored studies and covers a wide array of topics, it all shares one common thread: the need to collect sensitive or personal information in a private setting. Each of the four studies programmed its instrument to incorporate real-time data edits, with consistency checks across numerous questions. At the end of data collection, each study extracted a data file from the system to allow for further edits of the data. Because of the inclusion of the edit and consistency checks, and the ability to review interim data files throughout data collection, data managers performed very limited data cleaning before beginning data analyses. 4.1 Case 1 A seroprevalence study of herpes simplex virus-2 was conducted at 36 primary care physician (PCP) offices in suburban areas surrounding six cities in the United States in 2002 (9). Adults, aged 18 to 59 years, were asked to participate during a routine visit to the PCP office. After providing consent, patients were asked to complete a 15minute, computer-assisted self-interview and to provide a blood sample. The questionnaire included demographic questions as well as a series of questions designed to elicit sexual attitudes, behaviors, and symptom history relating to genital herpes. The A-CASI technology was selected to provide patients a confidential method to report sensitive information. The questionnaire was completed and a blood sample provided by 5433 patients. 4.2 Case 2 The survey of Male Attitudes Regarding Sexual Health (MARSH) was conducted in 2001 to determine the age- and race-specific prevalence of mild, moderate, and severe erectile dysfunction (ED) in African American, Caucasian, and Hispanic males aged 40 years or older (10). The survey was a nationally administered random-digit-dial, list-assisted

6

COMPUTER-ASSISTED DATA COLLECTION

representative telephone interview. A trained telephone interviewer administered the first part of the questionnaire, which included the screening, demographics, and general healthrelated questions, and then transferred the respondent to the T-ACASI section. The respondent then answered the remaining questions with the keypad on his touch-tone telephone. The T-ACASI method provided a private setting in which the more sensitive questions were asked (e.g., questions about sexual activity, sexual gratification, the number of sexual partners, ED treatment, and the discussion of erectile problems). The survey was conducted in both English and Spanish. Interviews were completed by 2173 respondents.

interventions aimed at reducing the number of infants there who are at increased risk of dying during their first year of life. One of the numerous protocols implemented under this initiative and supported by the National Institute on Alcohol Abuse and Alcoholism was the Prevention and Fetal Alcohol Effects Study, which used A-CASI technology to allow for a more private environment when disadvantaged, pregnant women were asked about their alcohol consumption. The study results demonstrated that using computerassisted technology to screen for alcohol use in disadvantaged pregnant populations is feasible and acceptable to the respondents (13).

4.3 Case 3

REFERENCES

The National Institute of Mental Health– sponsored Collaborative HIV/STD Prevention Trial began in 1999 to examine the efficacy of a community-level intervention to reduce HIV/STD incidence and high-risk behaviors in China, India, Peru, Russia, and Zimbabwe. The intervention sought to modify social norms at the community level to effect mass changes in HIV/STD risk behaviors. In each of the five countries, community popular opinion leaders were engaged as behaviorchange agents within their community of friends, neighbors, and coworkers. As part of this work, several pilot studies were conducted with use of A-CASI technology and developed in the respective languages of each country (11). The study selected this technology to test the feasibility of its use in developing countries and to allow for added privacy when sensitive and personal information is collected (e.g., risk behaviors). The study concluded that A-CASI appears to be feasible in these settings (12). 4.4 Case 4 The National Institute of Child Health and Human Development’s Initiative to Reduce Infant Mortality in Minority Populations in the District of Columbia began in 1993 to develop projects designed to better understand the factors that influence the high rate of infant mortality and morbidity in Washington, DC, and to design and evaluate

1. J. C. Fink, CATI’s first decade: the Chilton experience. Sociol Methods Res. 1983; 12: 153–168. 2. H. G. Miller, C. F. Turner, and L. E. Moses, eds. Methodological issues in AIDS surveys. In: H. G. Miller, C. F. Turner, and L. E. Moses (eds.), AIDS: The Second Decade. Washington, DC: National Academy Press, 1990, Chapter 6. 3. J. M. O’Reilly, M. L. Hubbard, J. T. Lessler, P. P. Biemer, and C. F. Turner, Audio and video computer-assisted self-interviewing: preliminary tests of new technologies for data collection. J Off Stat. 1994; 10: 197–214. 4. C. F. Turner,. L. Ku, F. L. Sonenstein, and J. H. Pleck, Impact of ACASI on reporting of male-male sexual contacts: preliminary results from the 1995 National Survey of Adolescent Males In: R. Warnecke (Ed.), Health Survey Research Methods. DHHS Pub. No. (PHS) 96-1013. Hyattsville, MD: National Center for Health Statistics, 1996, pp. 171–176. 5. P. C. Cooley, H. G. Miller, J. N. Gribble, and C. F. Turner, Automating telephone surveys: using T-ACASI to obtain data on sensitive topics. Comput Human Behav. 2000; 16: 1–11. 6. J. N. Gribble, H. G. Miller, J. A. Catania, L. Pollack, and C. F. Turner, The impact of T-ACASI interviewing on reported drug use among men who have sex with men. Subst Use Misuse. 2000; 35: 63–84. 7. P. C. Cooley, S. M. Rogers, C. F. Turner, A. A. Al-Tayyib, G. Willis, and L. Ganapathi, Using touch screen audio-CASI to obtain data on

COMPUTER-ASSISTED DATA COLLECTION sensitive topics. Comput Human Behav. 2001; 17: 285–293. 8. Office of Regulatory Affairs, U.S. Food and Drug Administration. Glossary of Computerized System and Software Development Terminology. Available at: http://www.fda.gov/ ora/inspect ref/igs/gloss.html 9. P. Leone, D. T. Fleming, A. Gilsenan, L. Li, and S. Justus, Seroprevalence of herpes simplex virus-2 in suburban primary care offices in the United States. Sex Transm Dis. 2004; 31: 311–316. 10. E. O. Laumann, S. West, D. Glasse, C. Carson, R. Rosen, and J. H. Kang, Prevalence and correlates of erectile dysfunction by race and ethnicity among men aged 40 or older in the United States: from the male attitudes regarding sexual health survey. J Sex Med. 2007; 4: 57–65.

7

11. L. C. Strader, Developing of a multi-lingual survey instrument in A-CASI. Paper presented at the American Public Health Association Annual Meeting; October 24, 2001, Atlanta, GA. Abstract 28287. 12. The NIMH Collaborative HIV/STD Prevention Trial Group. The feasibility of audio computer-assisted self-interviewing in international settings. AIDS. 2007; 21(Suppl 2): S49–58. 13. J. Thornberry, B. Bhaskar, C. J. Krulewitch, B. Weslet, M. L. Hubbard, et al., Audio computerized self-report interview in prenatal clinics: audio computer-assisted self-interview with touch screen to detect alcohol consumption in pregnant women: application of a new technology to an old problem. Comput Inform Nurs. 2002; 20: 46–52.

CONDITIONAL POWER

2

To introduce the concept with statistical rigor and clarity, consider the statistical framework of testing a normal mean into which many clinical trials can be formulated (2,3). We have H 0 : m £ 0 versus the alternative H1 : m > 0, where m denotes the treatment effect parameter and (for example) is the mean elevation of diastolic blood pressure above 90 mm Hg. Let m be the maximum sample size of the RNST and the current data be Sn . Then the conditional power is defined as

MING T. TAN Division of Biostatistics, University of Maryland School of Medicine and Greenebaum Cancer Center, College Park, Maryland

1

CONDITIONAL POWER

INTRODUCTION

For ethical, economical, and scientific reasons, clinical trials may be terminated early for a pronounced treatment benefit or the lack thereof. In addition to group sequential tests and type I error-spending functions, another approach is to address the question more directly by asking whether the accumulated evidence is sufficiently convincing for efficacy. Or, if evidence is lacking, one should ask whether the trial should continue to the planned end and whether the conventional (reference) nonsequential test (RNST) should be employed. Conditional power is one way to quantify this evidence. It is simply the usual statistical power of RNST conditional on the current data. Thus, it is the conditional probability that the RNST will reject the null hypothesis on completion of the trial given the data currently available at a given parameter value of the alternative hypothesis. Therefore, if the conditional power is too low or exceedingly high, then the trial may be terminated early for futility or for efficacy, respectively. The early stopping procedure derived based on conditional power is referred to as stochastic curtailing by which an ongoing trial is curtailed as soon as the trend based on current data becomes highly convincing (1). Therefore, conditional power serves two closely related purposes: (1) as a measure of trend reversal (e.g., futility index) and (2) as an interim analysis procedure (e.g., formally a group sequential method). However, this method can be readily communicated to nonstatisticians because it simply answers the question whether the evidence in the interim data is sufficient for making an early decision on treatment efficacy or the lack thereof given current data in reference to what the RNST concludes.

Pn (µ) = P(the RNST will reject H0 at m|Sn ). Furthermore, let X i be the observed elevation of diastolic blood pressure above 90 mm Hg of the ith subject. Thus, X i ∼ (µ, σ 2 ), i = 1, L, n, L, m. Then the current data are summarized by the sufficient statistic, the n
X i . Then Sn ∼ N(nµ, partial sum Sn = i=1

nσ 2 ), n = 1,2, L, m, where it is assumed that σ = 28.3 mm Hg based on preliminary data. We are interested in detecting a clinically important difference of 10 mm Hg at a significance level of α (for example, 0.025) with power 1 − β (for example, 0.90). The fixed sample design requires m = 86 subjects, and the null hypothesis would be√rejected if Zm > Zα = 1.96, where Zm = Sm / mσ and zα is the lower α-percentile of the standard normal curve, √ or equivalently Sm > s0 , where s0 = zα σ m = 554.68. Therefore, the conditional power is evaluated under the following conditional distribution:   Sm |µ, Sn ∼ N (m − n)µ + Sn , (m − n)σ 2 (1) and thus it is given by Pn (µ) = Pµ (RNST rejects H0 |Sn ) √ = Pµ (Sm > zα σ m|Sn ) √   Zα σ m − Sn − (m − n)µ √ . =1− σ m−n A similar derivation for two-sided tests can be found in Reference 3. Because the conditional power depends on the unknown true

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

CONDITIONAL POWER

0.2

Power 0.4 0.6

0.8

1.0

2

0.0

Usual Power xbar =8 xbar =2 0

2

4

6

8

10

12

mu

treatment effect parameter µ, different hypothetical values of µ have to be given to evaluate the conditional power. A common practice is to consider the three values: the null value, the value under the alternative hypothesis that the study is designed to detect, and the current estimate. One such current value is the average of the null and the alternative. If in the midcourse (n = 43), the average elevation of blood pressure (xbar in Fig. 1) is 8 mm Hg (i.e., Sn = 43*8 = 344) and the true treatment effect µ is 10 mm Hg, then the conditional power is 92%, which implies there is a good chance the null hypothesis will be rejected at the planned end of the trial. On the other hand, if the average elevation is 2, then the conditional power is only 49%. The conditional power has been used as a futility index. If the conditional power at the alternative hypothesis is too low (for example, less than 0.20), then the trial is not likely to reach statistical significance, which provides an argument for early termination of the trial for futility. Figure 1 gives the usual power function of the test as well as the conditional power that corresponds to the two average elevations of blood pressures (xbar values) at midcourse (n = 43) of the trial. Figure 2 gives the stochastic curtailing boundaries based on the conditional power. The difficulty is which µ to choose because it may be hard to anticipate a future trend. A stochastic curtailing procedure can be derived

14

Figure 1. Illustration of the conditional power.

using the conditional power of the RNST given Sn and some plausible values of the treatment effects. Then, we can derive a formal sequential test with upper boundary an and lower boundary bn where we reject H 0 the first time Zn ≥ an or accept H 0 the first time Zn ≤ bn . If the conditional power at µ = 0 is greater than γ 0 (for example, 0.80), then H 0 is rejected, and if the conditional power at µ = 10 is less than 1 − γ 1 (for example, 0.20), then H1 is accepted. Then, the sequential boundaries are √ an = zα  √ m/n + z1−γ0 (m − n)/n and bn = zα m/n −  √ z1−γ1 (m − n)/n − µ(m − n)/(σ n). It can be shown (1) that the derived curtailing procedure has a type I error no greater than α/γ 0 (0.0625) and type II error no greater than β/γ 1 (0.25). Figure 2 gives the boundaries of the two stochastic curtailing procedures based on conditional power with γ 0 = γ 1 = γ = 0.80 and 0.98. The extreme early conservatism of stochastic curtailing is apparent. It is now well known and widely utilized in the monitoring of clinical trials that the test statistic in most of the common phase III clinical trials can be formulated into the general Brownian motion framework (4). In other words, the test statistic can be rescaled √ into z-statistic Bt = Zn t (0 ≤ t ≤ 1) that follows approximately a Brownian motion with drift parameter µ. Thus µ may represent the pre–post change, or a log odds ratio or a log hazard ratio The primary goal is to test H0 :

20

CONDITIONAL POWER

3

Figure 2. Stopping boundaries of different stochastic curtailing procedures (DP denotes discordance probability).

−20

−10

0

Zn

10

gamma = 0.8 gamma = 0.98 Max DP = 0.02

m £ 0 against the alternative H1 : m > 0. Then, the conditional distribution of B1 given Bt is again normal with mean Bt + (1−t)µ and variance 1−t.Therefore, the conditional power in the general Brownian motion formulation is pt (µ) = Pµ (RNST rejects H0 |Bt ) = Pµ (B1 > Zα |Bt )   zα − Bt (1 − t)µ . =1− √ 1−t For a two-sided test, the conditional power is given by pt (µ) = Pµ (|S1 | > zα/2 |Bt )   zα/2 − Bt − (1 − t)µ = 1− √ 1−t   −zα/2 − Bt − (1 − t)µ . √ + 1−t Several authors (3–5) have documented in detail on how to formulate common clinical trials with various types of endpoints into the Brownian motion framework. For example, the sequentially computed log-rank statistic is normally distributed asymptotically with an independent increment structure (6,7). The conditional power is given in Reference 8 for comparing two proportions, in Reference 9 for censored survival time for log-rank or

0

20

40 n

60

80

weighted log-rank statistics, in Reference 10 for longitudinal studies, and in Reference 11 for models with covariates. In addition, several authors have used conditional power as an aid to extend an ongoing clinical trial to beyond the originally planned end for survival outcome (12) and in Brownian motion (13). More recently, the discordance probability is also extended and derived under the general Brownian motion framework (14,15) 3 WEIGHT-AVERAGED CONDITIONAL POWER Another way to avoid explicit choices of the unknown parameter is to use the weighted average of the conditional power with weights given by the posterior distribution of the unknown parameter µ given currently available data. Let the prior distribution of µ be π (µ) and its posterior be π (µ|Sn ). Then, the weight-averaged conditional power (also known as predictive power) for the one-sided hypothesis testing is given by  Pn = pn (µ)π (µ|Sn )dµ. If the improper as well as noninformative prior π (µ) = 1 is chosen, then the posterior of µ|Sn is normal with mean Sn /n and variance σ 2 /n. Then from Equation (1), the marginal distribution of Sm |Sn is again normal with

4

CONDITIONAL POWER

mean (m/n)Sn and variance σ 2 ((m − n)m/n). The predictive power is thus simply: √ Pn = P(Sm > zα σ m|Sn )  √ zα σ m − (m/n)Sn =1−  . σ (m − n)(m/n) Several authors have used the predictive power approach (16–18). Similar to conditional power, if Pn ≥ γ 0 , we consider rejecting the null, and if Pn ≤ 1 − γ 1 , we consider accepting the null. This criterion results in the following interim with  √ analysis procedure boundaries an = zα n/m + z1−γ0 (m − n)/m  √ and bn = zα n/m − z1−γ1 (m − n)/m. Unfortunately, no simple relationship relates the type I and II errors of the procedure with the predictive power. However, more informative usage of the predictive power may be through an informative prior. The data monitoring committee can make full use of the predictive power to explore the consequences of various prior believes on the unknown treatment effect parameter. 4 CONDITIONAL POWER OF A DIFFERENT KIND: DISCORDANCE PROBABILITY Based on the same principle of stopping a trial early as soon as the trend becomes inevitable, it is revealing to consider the conditional likelihood of the interim data given the reference test statistic at the planned end of the trial (m):  Sn |Sm ∼ N

 n n(m − n) 2 Sm , σ . m m

The distinct advantage of using this conditional probability is that it does not depend on the unknown parameter µ because conditioning is made on Sm , which is a sufficient statistic for µ. Using this conditional likelihood of the test statistic calculated at an interim time, we can derive a different kind of stochastic curtailing based on discordance probability defined as the probability the sequential test does not agree with the RNST in terms of accepting or rejecting the null hypothesis should the trial continue to the planned end (m). At a

given interim time point n, let an be the upper (rejection) boundary (i.e., if Sn ≥ an , reject the null hypothesis). Then Pµ (Sn ≥ an |Sm ≤ s0 ) is the probability that the decision to reject H 0 at n with Sn ≥ an is discordant with the decision to accept H0 at m when Sm ≤ s0 . Because for any µ, Pµ (Sn ≥ an |Sm ≤ s0 ) ≤ P(Sn ≥ an |Sm = s0 ). Then we can use P(Sn ≥ an |Sm = s0 ) to derive a sequential boundary. If this probability is smaller than ξ , then we stop the test and reject the null hypothesis; and, similarly, if P(Sn ≤ bn |Sm = s0 ) < ξ , then we stop the test and do not reject the null hypothesis. If we choose ξ (say, 0.05), that is, the same cut-off point for each n (n = 1, L, m), then we have P(Sn ≥ an |Sm = s0 )  an − ns0 /m =1−  ≤ ξ. n(m − n)/mσ √ Solving n = zα n/m  this equation we have a√ +  z1−ξ (m − n)/m and bn = zα n/m − z1−ξ (m − n)/m. It is worth noting again that the boundaries are derived using marginal probability ξ for each n(n = 1, L, m). Marginally, the stopping boundaries are the same as those from predictive power with noninformative prior. A more accurate statement may be by a global discordance probability defined as the probability that the sequential test on interim data does not agree with the acceptance/rejection conclusion of the RNST at the planned end (19). Xiong (19) derived the elegant sequential conditional probability ratio test (SCPRT) via a conditional likelihood ratio approach and obtained the boundaries of the same form. Most importantly, he derived the intricate relationship among the type I, II errors, the discordance probability, and he provided efficient algorithms to compute them. It shows that the sequential boundary can be derived such as it has virtually the same type I and II errors as the RNST and the probability that the rejection or acceptance of the null hypothesis based on interim data might be reversed is less than a given level ρ 0 (for example, 0.02) should the trial continues to the planned end. With instantaneous computation of the type I and

CONDITIONAL POWER

II errors and various discordance probabilities, a sharper monitoring boundaries can be derived (14,19–22). It is noted that similar boundaries themselves are also derived for Bernoulli series using the same parameter free approach in the context of reducing computation in a simulation study designed to evaluate the error rates of a bootstrap test (23). Figure 2 also gives the boundaries of the stochastic curtailing procedure based on the discordance probability with a maximum discordance probability (denoted DP in Fig. 2) less than 0.02. In contrast to the extreme early conservatism of stochastic curtailing based on conditional power, the three boundaries become closer as the trial approaches its end. Interestingly, in the last quarter of the information fraction of the trial, the curtailing procedure (with γ = 0.98) almost coincides with that of the SCPRT, whereas the boundary with γ = 0.80 becomes slightly tighter than that of the SCPRT, which results in an increase in discordance probability relative to that of the RFSST and reflecting the conservatism in the SCPRT. A more detailed comparison of the two curtailing approaches and the SCPRT with common group sequential procedures such as the O’Brien-Fleming, Pocock, and Haybittle-Peto procedures is given in References 21 and 22. 5

ANALYSIS OF A RANDOMIZED TRIAL

The Beta-Blocker Heart Attack Trial was a randomized double-blind trial to compare propranolol (n = 1916) with placebo (n = 1921) in patients who had recent myocardial infarction sponsored by the National Institute of Health. Patients were accrued from June 1978 to June 1980 with a 2-year follow-up period resulting in a 4-year maximum duration. The trial was terminated early for a pronounced treatment benefit. Aspects on the interim monitoring and early stopping of this trial have been summarized (24–26). The minimum difference of clinical importance to be detected is 0.26 in log hazard ratio derived based on projected 3-year mortality rates of 0.1746 for the placebo group and 0.1375 for the treatment group adjusting for compliance. Roughly 628 deaths are

5

required for a fixed sample size test to detect such a difference at a significance level of 5% with 90% power. Seven interim analyses that correspond to the times the Policy and Data Monitoring Board met were planned. The trial was stopped 9 months early at the sixth interim analysis with 318 deaths (183 in the placebo arm and 125 in the treatment arm) with the standardized z-statistic valued at 2.82, and the O’Brien-Fleming boundary was crossed. The conditional power can be evaluated for various expected deaths. For example, a linear interpolation of the life table based the current survival data suggests 80 additional deaths in the ensuing 9 months. Therefore, the information time at the sixth analysis is √ 318/(318 + 80) = 0.80, then Bt = 2.82 0.80 = 2.52. The conditional power p0.80 (0) is 0.89. If an additional 90 deaths are expected, then the conditional power p0.78 (0) is 0.87. Both suggest a rather high conditional power for a treatment effect. Assuming an additional 90 deaths, the SCPRT curtailing based on discordance probability can be derived (14), which gives a maximum discordance probability of 0.001. This finding implies that there is only 0.1% chance that the conclusion might be reversed had the trial continued to the planned end (14). If the 628 total deaths in the original design are used, and if an SCPRT is to be used as stated in the protocol, then a maximum discordant probability is 1%, which implies only a slight chance (1%) that the decision based on the SCPRT procedure in the protocol may be reversed had the trial continued to the planned end (21). Therefore, it is highly unlikely that the early stopping decision for efficacy would be reversed had the trial continued to the planned end by all three procedures. However, the SCPRT based curtailing provides a sharper stopping boundary for trend reversal as expected. 6 CONDITIONAL POWER: PROS AND CONS To put things in perspective, the conditional power approach attempts to assess whether evidence for efficacy or the lack of it based on the interim data is consistent with that

6

CONDITIONAL POWER

at the planned end of the trial by projecting forward. Thus, it substantially alleviates the major inconsistency in all other group sequential tests where different sequential procedures applied to the same data yield different answers. This inconsistency with the nonsequential test sets up a communication barrier in practice where we can claim a significant treatment effect via the nonsequential test, but we cannot to do so via the sequential test based on the same data set or we can claim significance with one sequential method but cannot do so with another. For example, in a clinical trial that compares two treatments at the 5% significance level where five interim analyses were planned, the nominal level at the fifth analysis for the Petcock procedure is 0.016 whereas the nominal level at the fifth interim analysis for the O’Brien and Fleming is 0.041. If the trial has a nominal P-value of 0.045 at the fifth analysis, then according to either of the group sequential designs, the treatment effect would not be significant, whereas investigators with the same data just carrying out a fixed sample size test would claim a significant difference. However, if the nominal P-value is 0.03, then the treatment effect is significant according to the O’Brien-Fleming procedure but not according to the Pocock procedure. The advantage of the conditional power approach for trial monitoring is its flexibility. It can be used for unplanned analysis and even analyses whose timing depends on previous data. For example, it allows inferences from over-running or under-running (namely, more data come in after the sequential boundary is crossed, or the trial is stopped before the stopping boundary is reached). Conditional power can be used to aid the decision for early termination of a clinical trial to complement the use of other methods or when other methods are not applicable. However, such flexibility comes with a price: a potentially more conservative type I and type II error bounds (α/γ 0 and β/γ 1 ) that one can report. The SCPRT based approach removes the unnecessary conservatism of the conditional power and can retain virtually the same type I and II errors with a negligible discordance probability by accounting for how the data pattern (sample path) is trended (traversed). The use of the SCPRT

especially in making decision in early stages has been explored by Freidlin et al. (27) and for one-sided test by Moser and George (28). The greatest advantage of predictive power is that it allows us to explore consequences of various prior believes about the unknown treatment effect parameter. Finally, conditional power has also been used to derive tests adaptive to the data in the first stage of the trial (see, Reference 13). More recently, the related reverse stochastic curtailing and the discordance probability are used to derive group tests adaptive to updated estimates of the nuisance parameter (14).

REFERENCES 1. K. Lan, R. Simon, and M. Halperin, Stochastically curtailed tests in long-term clinical trials. Sequent. Anal. 1982; 1: 207–219. 2. J. Whitehead, A unified therory for sequential clinical trials. Stat. Med. 1999: 2271–2286. 3. C. Jennison and B. W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. New York: Chapman & Hall/CRC, 2000. 4. K. K. Lan and D. M. Zucker, Sequential monitoring of clinical trials: the role of information and Brownian motion. Stat. Med. 1993; 12: 753–765. 5. J. Whitehead, Sequential methods based on the boundaries approach for the clinical comparison of survival times. Stat. Med. 1994; 13: 1357–1368. 6. M. H. Gail, D. L. DeMets, and E. V. Slud, Simulation studies on increments of the twosample logrank score test for survival time data, with application to group sequential boundaries. In:: J. Crowley and R. A. Johnson, (eds.), Survival Analysis. Hayward, CA: Instititute of Mathematical Statistics, 1982, pp. 287–301. 7. A. A. Tsiatis, Repeated significance testing for a general class of statistics used in censored survival analysis. J. Am. Stat. Assoc. 1982; 77: 855–861. 8. M. Halperin, K. K. Lan, J. H. Ware, N. J. Johnson, and D. L. DeMets, An aid to data monitoring in long-term clinical trials. Control. Clin. Trials 1982; 3: 311–323. 9. D. Y. Lin, Q. Yao, Z. Ying, A general theory on stochastic curtailment for censored survival data. J. Am. Stat. Assoc. 1999; 94: 510–521.

CONDITIONAL POWER 10. M. Halperin, K. K. Lan, E. C. Wright, and M. A. Foulkes, Stochastic curtailing for comparison of slopes in longitudinal studies. Control. Clin. Trials 1987; 8: 315–326. 11. C. Jennison and B. W. Turnbull, Groupsequential analysis incorporating covariate information. J. Am. Stat. Assoc. 1997; 92: 1330–1341. 12. P. K. Andersen, Conditional power calculations as an aid in the decision whether to continue a clinical trial. Control. Clin. Trials 1986; 8: 67–74. 13. M. A. Proschan and S. A. Hunsberger, Designed extension of studies based on conditional power. Biometrics 1995; 51: 1315–1324. 14. X. Xiong, M. Tan, and J. Boyett, Sequential conditional probability ratio tests for normalized test statistic on information time. Biometrics 2003; 59: 624–631. 15. X. Xiong, M. Tan, and J. Boyett, A sequential procedure for monitoring clinical trials against historical controls. Stat. Med. 2007; 26: 1497–1511. 16. J. Herson, Predictive probability early termination plans for phase iI clinical trials. Biometrics 1979; 35:775–783. 17. S. C. Choi, P. J. Smith, and D. P. Becker, Early decision in clinical trials when the treatment differences are small. Experience of a controlled trial in head trauma. Control. Clin. Trials 1985; 6: 280–288. 18. D. J. Spiegelhalter, L. S. Freedman, and P. R. Blackburn, Monitoring clinical trials: Conditional or predictive power? Control. Clin. Trials 1986; 7: 8–17. 19. X. Xiong, A class of sequential conditional probability ratio tests. J. Am. Stat. Assoc. 1995; 90: 1463–1473. 20. X. Xiong, M. Tan, M. H. Kutner, Computational methods for evaluating sequential tests and post-test estimation via the sufficiency principle. Statist. Sin. 2002; 12: 1027–1041. 21. M. Tan, X. Xiong, M. H. Kutner, Clinical trial designs based on sequential conditional probability ratio tests and reverse stochastic curtailing. Biometrics 1998; 54: 682–695. 22. M. Tan and X. Xiong, Continuous and group sequential conditional probability ratio tests for phase II clinical trials. Stat. Med. 1996; 15: 2037–2051. 23. C. Jennison, Bootstrap tests and confidence intervals for a hazard ratio when the number of observed failures is small, with applications to group sequential survival studies. In: C. Page and R. LePage (eds.), Computing Science and Statistics: Twenty-second Symposium on

7

the Interface. Berlin: Springer-Verlag, 1992, pp. 89–97. 24. D. L. DeMets and K. K. Lan, Interim analysis: the alpha spending function approach. Stat. Med. 1994; 13: 1341–1352; discussion 1353–1346. 25. D. L. DeMets, R. Hardy, L. M. Friedman, and K. K. Lan, Statistical aspects of early termination in the beta-blocker heart attack trial. Control. Clin. Trials 1984; 5: 362–372. 26. K. K. Lan and D. L. DeMets, Changing frequency of interim analysis in sequential monitoring. Biometrics 1989; 45: 1018–1020. 27. B. Freidlin, E. L. Korn, and S. L. George, Data monitoring committees and interim monitoring guidelines. Control. Clin. Trials 1999; 20: 395–407. 28. B. K. Moser and S. L. George, A general formulation for a one-sided group sequential design. Clin. Trials 2005; 2: 519.

CROSS-REFERENCES Group Sequential Designs Interim Analysis

CONFIDENCE INTERVAL

sample value Q

JENO˝ REICZIGEL ´ University Szent Istvan Faculty of Veterinary Science Budapest, Hungary

“likely range” for the true value L

(95%)

U

Figure 1. Point estimate θˆ and confidence interval (L,U) for a parameter θ (prevalence, mean, difference, relative risk, etc.). θˆ represents the value experienced in the sample, whereas (L, U) defines a ‘‘reasonable range’’ or ‘‘likely range’’ for the unknown true (population) value θ

A confidence interval (CI) or interval estimate for an unknown parameter θ is a ‘‘from-to’’ range calculated from the sample so that it contains θ with a high—usually 90%, 95%, or 99%—probability (Fig. 1). As opposed to a point estimate θˆ , which is a single number calculated from the sample, the interval (L,U) gives a more expressive and easy-tounderstand picture about the uncertainty of the estimate. L and U are also called confidence limits for the parameter θ . Some examples of how CIs are usually communicated in papers are as follows:

the sample is meaningful at all, a CI is also meaningful. For curves like a regression line or a ROC curve, interval estimation results in a so-called confidence band (Fig. 2). Technically, a CI, as being a ‘‘fromto’’ range calculated from the sample, consists of two functions of the sample values L = L(x1 , x2 , . . . , xn ) and U = U(x1 , x2 , . . . , xn ), the lower and upper confidence limit, which satisfy P(L < θ < U) = 1 − α, where 1 − α is called coverage probability or level of confidence, and α is called error probability. (These probabilities may also depend on the true value of θ ; see below for details.) For symmetry, CIs are usually constructed so that they have an error probability of α/2 at each side, that is, P(θ < L) = P(θ > U) = α/2 (Fig. 3a), which are called symmetric or equaltailed CIs. In the asymmetric case, the sum of the error probabilities is α, but they do not need to be equal (Fig. 3b). One-sided CIs are intervals of the form (−∞, U) or (L, ∞) with the property P(θ < U) = 1 − α or P(θ > L) = 1 − α (Fig. 3c,d). It is easy to verify that given two 1 − α level one-sided CIs (−∞, U) and (L, ∞) for a certain parameter, then (L, U) forms a 1 − 2α level symmetric CI for the same parameter. For example, two 95% one-sided intervals define a 90% two-sided one. Similarly, if (L, U) is a 1 − α level equal-tailed CI, then (−∞, U) as well as (L, ∞) are 1 − α/2 level one-sided CIs for the same parameter. That is, a 95% level twosided symmetric interval can be converted into two 97.5% one-sided ones. In the frequentist (as opposite to Bayesian) view, the parameter θ is assumed to be a fixed

• ‘‘The estimated prevalence rate of the

disease in the study population was 8.3% (95% CI 5.8% to 11.7%).’’ • ‘‘Blood cholesterol fell by 0.76 mmol/l in group A vs 0.32 mmol/l in group B (difference 0.44 mmol/l, 95% CI 0.16 to 0.73).’’ • ‘‘Acute infections occurred in 25 of the patients on active treatment and in 36 of the patients on placebo (relative risk 0.69, 95% CI 0.55 to 0.87).’’ • ‘‘Treament failure was eight times more likely in the placebo than in the antibiotic group (13.1% vs 1.6%, odds ratio 8.2, 95% CI 1.9 to 34.7).’’ CIs can be given for unknown population parameters like the mean, variance, median, proportion, or for model parameters like a regression coefficient, odds ratio, correlation coefficient, or for functions of the abovementioned parameters like, for example, the variance/mean ratio, the difference of two proportions or two means, and so on (1). In clinical studies, CIs are typically given for the mean or median treatment effect, proportion of cure, difference in these outcome measures comparing two treatments, and so on. Practically for anything for which an estimate from

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

CONFIDENCE INTERVAL

(a)

Figure 2. Confidence band for a regression line (a) and for a ROC curve (b)

(b)

value, whereas L and U, as they depend on the sample, are random variables. Therefore, the statement P(L < θ < U) = 1 − α can only be interpreted in relation to drawing samples repeatedly and recalculating L and U each time. A small α corresponds to a low proportion of failures in repeated application of the method (i.e., it indicates that the method is good enough and makes mistakes as seldom as with a rate of α). However, considering one particular sample and the CI calculated from that sample, say when L = 12.1 and U = 13.5, the probability P(12.1 < θ < 13.5) is meaningless. Clearly, it is either true of false, given that also θ is a fixed value. To emphasize this fact, some authors suggest that in reporting the results one should avoid saying ‘‘the unknown parameter lies between 12.1 and 13.5 with 95% probability’’; one should rather say ‘‘with 95% confidence’’ or the CI should be reported as shown in the examples above. In the Bayesian model (in which even θ is regarded as a random variable), the corresponding interval estimates are the credible interval (i.e., the range of parameter values with the highest posterior probability) and Fisher’s fiducial interval. However, the calculation as well as the interpretation of these notions are different from each other, and quite different from that of a frequentist CI. If n unknown parameters to estimate exist (i.e., a parameter vector θ ∈ Rn ), one may want to construct a simultaneous confidence region (or confidence set) S ⊆ Rn with the property P(θ ∈ S) = 1 − α, rather than to give a CI for each parameter separately. Such questions come up most naturally in multiparameter models, for example in regression models. A basic example of a two-dimensional confidence region develops when one has

2.5%

95%

2.5%

(a) 2%

95%

3%

(b) 95%

5% (c)

5%

95% (d)

Figure 3. Symmetric (a), asymmetric (b), and onesided (c, d) confidence intervals

variance confidence region s 2obs

for (m, s 2)

mobs

mean

Figure 4. Joint confidence region (gray shaded area) for the mean and variance of a normal vari2 denote the sample mean and able (µobs and σobs variance)

a sample from a normal distribution with unknown µ and σ 2 , and one would like to construct a joint confidence set for the parameter pair (µ, σ 2 ), rather than two separate confidence intervals for µ and σ 2 (2). Such a confidence set (Fig. 4) can be used to derive CIs for functions of the parameters, for example, for the variance/mean ratio.

CONFIDENCE INTERVAL

In nonparametric inference, as no ‘‘parameters of the distribution’’ exist (in this case, neither the mean nor the variance are regarded as parameters of the distribution in the strict statistical sense), functionals of the distribution play the role of the above ‘‘unknown parameters.’’ For the nonstatistician user, however, it does not make any difference. CIs are closely related to statistical tests. Given a (1 − α)-level two-sided CI for a parameter θ , it is easy to convert it to a test for H0 : θ = θ0 , H1 : θ = θ0 , simply rejecting H 0 if θ 0 is not contained in the CI, that is, taking the CI as the acceptance region of the test (for a one-tailed test, a one-sided CI is needed). The resulted test has a Type I error rate of α. It works in the opposite direction as well, that is, tests can serve as the basis for CI construction (see the test inversion method below). 1 THE ROLE OF CONFIDENCE INTERVALS IN CLINICAL TRIALS In the last few decades, there has been a tendency toward wider use of CIs in reporting the results from clinical trials as well as from other studies (3–7). The main advantage of reporting CIs instead of (or even better, together with) P-values lies in better quantification of the effect size (the confidence limits are easy to compare with the clinically relevant effect size) and in direct indication of the uncertainty of the results (the wider the CI, the greater the uncertainty). Confidence intervals are particularly important in equivalence trials. To verify the equivalence of a new experimental treatment to the standard one, it must be shown that the difference between the treatment effects Ex and Es remains below a prespecified limit T defining the range of equivalence. Here, one can take either absolute or relative difference [i.e., equivalence can be defined either as −T < Ex − Es < T (absolute difference) or as 1 − T < Ex /Es < 1 + T (relative difference)]. For example, using relative difference with a range of equivalence of ±10% can be formally written as 0.90 < Ex /Es < 1.10. Then, a common method of analysis is to construct a CI for the difference Ex − Es or Ex /Es based

−T

0

3

T

(a) −T

0 (b)

T

−T

0 (c)

T

0

T

−T

(d) Figure 5. Possible locations of a confidence interval for the difference of treatment effects in an equivalence trial (if the true difference lies between the limits −T and T, treatments are regarded as equivalent)

on the observed sample, and to accept treatment equivalence if this CI is fully contained in the interval (−T, T) or (1 − T, 1 + T). The rationale behind this approach is that one would like to know whether the unknown true difference (rather than the difference between the particular observed samples) lies in the range of equivalence. According to this method, the interpretation of the CIs in Fig. 5 is as follows. a. A good evidence for equivalence. b. The new treatment is less effective than the standard one. c. The new treatment is noninferior (i.e., it may be either more effective or equivalent). d. The trial failed to have enough power to allow any definite conclusion concerning the relation between treatment effects. A larger trial is needed. Confidence intervals are also helpful in the interpretation of the results of superiority trials. Let D denote the smallest difference between the experimental and the standard treatment that is still clinically relevant. Then, the CI for the difference between treatment effects (Ex − Es ) (absolute difference) allows drawing the following conclusions (Fig. 6).

4

CONFIDENCE INTERVAL

many cases (e.g., because of the discreteness of the distribution, see Fig. 7), global measures were introduced to evaluate the performance of the various methods. The tra(b) ditional confidence coefficient, also called minimum coverage, is defined as the minimum (or infimum), whereas the mean coverage (c) (8) is the average of the coverage probability over the whole parameter range (e.g., for the binomial parameter, over the [0,1] (d) range). An important difference exists in the interpretation of these measures. The confidence coefficient can be interpreted as a lower (e) bound of the coverage rate, guaranteed by the procedure. Mean coverage, however, being 0 D based on averaging over the whole parameter (f) set, does not have any meaningful interpretation in relation with a single problem, in Figure 6. Possible locations of a confidence interwhich the parameter is assumed to be a cerval for the difference of treatment effects in a tain fixed value. (Note that, in the Bayesian superiority trial (a difference greater than D is model, averaging could be naturally made regarded as clinically relevant) with respect to the prior distribution of the parameter.) A CI constructing method is said to be cona. Experimental treatment is definitely worse. servative if its minimum coverage is never b. Experimental treatment is either worse less than the nominal and anticonservaor the difference is clinically irrelevant. tive otherwise. Although conservatism is a c. No clear conclusion: A larger trial is desirable property as compared with antineeded. conservatism, it is unsatisfactory if a method d. Experimental treatment is better but is too conservative— say, if at the nomthe difference is clinically irrelevant. inal level of 95%, the method produces a CI with actual coverage of 98%—because it e. Experimental treatment is better: The results in unnecessarily wide intervals. A difference may or may not be clinically way to control conservatism is to determine relevant. the actual coverage and to adjust it if necf. Experimental treatment is definitely betessary. Reiczigel (9) describes a computer ter: The difference is clinically relevant. intensive level-adjustment procedure varying the nominal level iteratively until the confidence coefficient gets close enough to the 2 PROPERTIES AND EVALUATION OF desired level. The procedure can be combined PERFORMANCE with any reasonable CI construction method, and it can also be applied for adjusting the The coverage probability of a CI for a parammean coverage. eter is defined as the probability that the The notion ‘‘exactness’’ of a CI is associinterval contains the parameter. However, ated with various meanings in the literature. this probability may depend on the true value In one sense, exactness means that the CI of the parameter as well as on some nuiis based on the exact probability distribusance parameters. A good method produces a tion of the given test statistic. In another CI with coverage probabilities possibly close sense, exactness means the same as conserto the desired (nominal) level irrespective of vatism above (i.e., that the coverage probabilthe values of the parameters: Ideally, the ity reaches at least the nominal level for any coverage probability should always be equal parameter value). The third sense is that the to the nominal. Because it is impossible in coverage is strictly equal to the nominal level (a)

CONFIDENCE INTERVAL

5

Coverage 1.00 0.95 0.90 0.85 Figure 7. Actual coverage of the exact’’ 95% Clopper-Pearson CI for the binomial parameter P. Although minimum coverage is 95%, for most parameter values, the coverage probability is high above 95%

0.80

for all parameter values. Computer simulations by several investigators show that exact CIs meant in the first two senses may perform rather poorly (8, 10), which is because a method may be too conservative even if its minimum coverage is equal to the nominal one, because for some parameter values the coverage probability may well exceed its minimum (as it is the case in Fig. 7). Therefore, it is a little misleading to designate the property that the minimum coverage is equal to the nominal as exactness. In fact, a further necessary condition exists for a method to perform well: Its coverage probability should not show much fluctuation. The so-called asymptotic or ‘‘large-sample’’ CIs are based on an approximate probability distribution of the test statistic, and therefore only have approximately the stated confidence level and only for large samples. Whereas an exact CI (meant in the third sense above) is valid for any sample size, an asymptotic CI is valid only for a sufficiently large sample size—and it depends on the procedure as well as on the true value of the parameter to determine what ‘‘sufficiently large’’ means. Such approximations are most often made by the normal distribution, based on the Central Limit Theorem (see below).

0

0.2

0.4

0.6

0.8

1 true p

when estimation is made by the maximum likelihood method), an asymptotic CI can be obtained as the point estimate plus and minus the asymptotic standard error of the estimate multiplied by a critical value from the normal distribution. This is demonstrated by two examples: 1. Let x and s denote the mean and standard deviation of a variable, estimated from a sample of n. Then, a 95% asymptotic CI for the true √ mean can be obtained as x ± 1.96 s/ n. 2. Let ln(OR) denote the logarithm of the odds ratio estimated from a two-bytwo table, that is, ln(OR) = ln(ad) − ln(bc) where a, b, c, d denote the table cell counts. It can be proven that the asymptotic standard error of ln(OR) is
s = 1/a + 1/b + 1/c + 1/d. From this equation, a 95% asymptotic CI for the true ln(OR) is obtained as ln(OR) ± 1.96 s. The actual coverage of such a CI approaches the nominal if the sample size tends to infinity. Recommendations exist (for each case separately!) for what sample size is the approximation acceptable. 3.2 Test Inversion

3 METHODS FOR CONSTRUCTING CONFIDENCE INTERVALS 3.1 Normal Approximation In case of consistency and asymptotic normality of a point estimator (in most cases,

Test inversion can be applied if a hypothesis test exists for the parameter θ that one wants to estimate. Suppose one wants to invert a test of H0 : θ = θ0 based on a test statistic t, to get a 95% CI for θ . The basic idea is that a 95% confidence set should consist of

6

CONFIDENCE INTERVAL

test statistic t

t2 tobs t1

L

Q0

U

true parameter Q

all such θ 0 values for which H0 : θ = θ0 is not rejected at the 95% level. That is, 95% acceptance regions (t1 , t2 ) of H0 : θ = θ0 should be determined for all possible θ 0 , and the smallest and largest θ 0 for which the acceptance region contains the observed tobs should be taken (Fig. 8). Note that, for some tests, the endpoints of the acceptance regions do not increase monotonically with θ . In such cases, the set of those θ 0 values for which H0 : θ = θ0 is not rejected at a given level may not form a proper interval, but it may contain ‘‘holes.’’ Of course, taking the smallest and largest θ 0 always results in a proper interval. Properties of the CI are easy to derive from the properties of the acceptance regions. For example, an exact test (i.e., one having exact acceptance regions for all θ 0 ) results in an exact CI; if the acceptance regions of one test are contained in those of another test for all θ 0 , then even the CIs are contained in the intervals obtained from the other test, and so on. CIs obtained by inverting a likelihood ratio test, as well as those based on the socalled empirical likelihood, are reported to have good properties (11). 3.3 Bootstrap Resampling Bootstrap CIs are applied either in the nonparametric case (i.e., if one has no parametric model for the population or process that generated the observed sample) or if the sampling distribution of an estimator is difficult or impossible to determine analytically (e.g., for the correlation coefficient). When bootstrapping from the sample, the observed sample itself is used as an estimate

Figure 8. 95% acceptance region (t1 ,t2 ) of the test H0 : θ = θ0 (thick vertical line) and confidence interval (L,U) belonging to the observed value tobs of the test statistic (thick horizontal line). L and U are obtained as the smallest and largest value of θ, for which the 95% acceptance region contains tobs

of the underlying population. A large number of samples are taken from the original (observed) sample by sampling with replacement, and these bootstrap samples are used to obtain an approximation of the sampling distribution of the estimate—to determine bias, standard error, tail probabilities, and so on—or to construct bootstrap CIs (12). Although sometimes this determination can be made analytically, typically it is carried out by computer simulation. A variety of methods exist for constructing bootstrap CIs, and many improvements to these methods also exist, so one must consult the latest books and articles before implementing a method for a specific problem. REFERENCES 1. M. J. Gardner and D. G. Altman, Statistics with Confidence: Confidence Intervals and Statistical Guidelines. London: BMJ Books, 1989. 2. B. C. Arnold and M. Shavelle, Joint confidence sets for the mean and variance of a normal distribution. Amer. Stat. 1998; 52: 133–140. 3. Uniform Requirements for Manuscripts Submitted to Biomedical Journals. JAMA 1993; 269: 2282–2286. 4. Uniform Requirements for Manuscripts Submitted to Biomedical Journals. (2003). (online). Available: http://www.icmje.org. 5. M. J. Gardner and D. G. Altman, Confidence intervals rather than p-values: estimation rather than hypothesis testing. BMJ 1986; 292: 746–750. 6. L. R´ozsa, J. Reiczigel, and G. Majoros, Quantifying parasites in samples of hosts. J. Parasitol. 2000; 86: 228–232.

CONFIDENCE INTERVAL 7. J. A. C. Sterne and G. D. Smith, Sifting the evidence: what’s wrong with significance tests? BMJ 2001; 322: 226–231. 8. R. G. Newcombe, Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat. Med. 1998; 17: 857–872. 9. J. Reiczigel, Confidence intervals for the binomial parameter: some new considerations. Stat. Med. 2003; 22: 611–621. 10. A. Agresti and B. A. Coull, Approximate is better than ‘‘exact’’ for interval estimation of binomial proportions. Amer. Stat. 1998; 52: 119–126. 11. A. B. Owen, Empirical Likelihood. London: Chapman and Hall/CRC, 2001. 12. B. Efron and R. Tibshirani, An Introduction to the Bootstrap. New York: Chapman & Hall, 1993.

7

CONFIDENCE INTERVALS AND REGIONS

idea is not entirely misleading. An example should help to explain the technical meaning.

Confidence intervals are used for interval estimation. Whether an interval estimate is required depends upon the reason for the statistical analysis. Consider the analysis of measurements of the compressive strength of test cylinders made from a batch of concrete. If we were concerned with whether the mean strength of the batch exceeds some particular value, our problem would be one of hypothesis testing∗ . Our conclusion might be to accept or to reject the hypothesis, perhaps with an associated degree of confidence. If a simple indication of the strength likely to be achieved under the particular conditions of test is required, the observed mean strength might be quoted as an estimate of the true mean strength. This is called point estimation∗ . Interval estimation is the quoting of bounds between which it is likely (in some sense) that the real mean strength lies. This is appropriate when it is desired to give some indication of the accuracy with which the parameter is estimated. A large number of statistical problems may be included in the classes of hypothesis testing, point estimation, or interval estimation. It must be pointed out that there are several schools of thought concerning statistical inference. To quote confidence intervals is the interval estimation method advocated by the most widely accepted of these schools, variously referred to as the Neyman-Pearson∗ , Neyman-Pearson-Wald, frequentist, or classical school. There are other ways of obtaining interval estimates and we will refer to them later. (See also BAYESIAN INFERENCE, FIDUCIAL INFERENCE, LIKELIHOOD, STRUCTURAL INFERENCE.)

Example 1. Suppose that some quantity is measured using a standard testing procedure. Suppose that the quantity has a welldefined true value µ, but that the measurement is subject to a normally distributed error that has known variance σ 2 . Let X denote the random variable that is the result of a single measurement and let x be a particular value for X. Now X is normally distributed with mean µ and variance σ 2 . Using the properties of the normal distribution we can make probability statements about X; e.g., Pr[µ − 1.96σ
X
µ + 1.96σ ] = 0.95. (1) We could rewrite this as Pr[X − 1.96σ
µ
X + 1.96σ ] = 0.95 (2) or Pr[µ ∈ (X − 1.96σ , X + 1.96σ )] = 0.95. (3) Although µ may appear to be the subject of statements (2) and (3), the probability distribution referred to is that of X, as was more obvious in statement (1). If X is observed to be x, we say that we have 95% confidence that x − 1.96σ
µ
x + 1.96σ or say that (x − 1.96σ , x + 1.96σ ) is a 95% confidence interval for µ. No probability statement is made about the proposition x − 1.96σ
µ
x + 1.96σ

(4)

involving the observed value, x, since neither x nor µ has a probability distribution. The proposition (4) will be either true or false, but we do not know which. If confidence intervals with confidence coefficient p were computed on a large number of occasions, then, in the long run, the fraction p of these confidence intervals would contain the true parameter value. (This is provided that the occasions are independent and that there is no selection of cases.)

BASIC IDEA OF A CONFIDENCE INTERVAL The term ‘‘confidence interval’’ has an intuitive meaning as well as a technical meaning. It is natural to expect it to mean ‘‘an interval in which one may be confident that a parameter lies.’’ Its precise technical meaning differs substantially from this (see Jones [13], Cox [7], and Dempster [9]) but the intuitive

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CONFIDENCE INTERVALS AND REGIONS

CONFIDENCE INTERVALS BASED ON A SINGLE STATISTIC Many confidence intervals can be discussed in terms of a one-dimensional parameter θ and a one-dimensional statistic T(X), which depends upon a vector of observations X. A more general formulation will be given below under the heading ‘‘Confidence Regions.’’ Provided that T(X) is a continuous random variable, given probabilities α1 and α2 , it is possible to find T1 (θ ) and T2 (θ ) such that Pr[T(X)
T1 (θ )|θ ] = α1

This confidence interval may be described as a central confidence interval because α1 = α2 (= 0.025). Noncentral confidence intervals are seldom quoted except when we are primarily concerned about large values for the parameter or about small values. In such cases it is common to quote only a single confidence limit. In Example 2 Pr[T/θ < 2.73] = 0.05 or, equivalently, Pr[θ > 0.366T] = 0.05.

(5)

Thus (0, 0.366T) is a confidence interval for θ = σ 2 at confidence level 0.95.

and Pr[T(X)  T2 (θ )|θ ] = α2 .

(6)

DISCRETE DISTRIBUTIONS

In other words, T1 (θ ) and T2 (θ ) are as shown in Fig. 1. Another diagram that can be used to illustrate the functions T1 (θ ) and T2 (θ ) is Fig. 2. For every particular value of θ the probability that T lies between T1 (θ ) and T2 (θ ) is 1 − α1 − α2 . The region between the curves T = T1 (θ ) and T = T2 (θ ) is referred to as a confidence belt. In terms of Fig. 2, the basic idea of confidence intervals is to express confidence 1 − α1 − α2 that the point (θ , T) lies in the confidence belt after T has been observed. If T1 and T2 are well-behaved functions, they will have inverse functions θ2 and θ1 , as shown in the figure, and the three propositions ‘‘T1 (θ )
T(X)
T2 (θ ),’’ ‘‘θ lies in the confidence belt,’’ and ‘‘θ1 (T)
θ
θ2 (T)’’ will be equivalent. Thus (θ1 (T), θ2 (T)) is a (1 − α1 − α2 ) confidence interval for θ .

When the statistic T is a discrete∗ random variable it is generally not possible to find functions T1 and T2 such that (5) and (6) hold precisely. Instead, we ask that Pr[T(X)
T1 (θ )|θ ] be as large as possible but not greater than α1 and that Pr[T(X)  T2 (θ )|θ ] be as large as possible but not greater than α2 . The functions T1 and T2 define a confidence belt which generally has a staircase-shaped perimeter. Keeping [14, p. 98] and Kendall and Stuart [15, p. 105] give examples.

Example 2. Consider eight observations from a normal distribution with known mean µ and unknown variance σ 2 . Take θ = σ 2 , X = (X1 , . . . , X8 ), and T(X) = 8 i=1 (Xi − µ)2 .

= (1 − p)12 + 12p(1 − p)11

From the fact that T/θ has a χ 2 distribution with 8 degrees of freedom, we know that Pr[T/θ
2.18] = 0.025 and that Pr[T/θ  17.53] = 0.025. Thus we take T1 (θ ) = 2.18θ , T2 (θ ) = 17.53θ and calculate θ1 (T) = 0.057T, θ2 (T) = 0.46T. The interval (0.057T, 0.46T) may be quoted as a 95% confidence interval for σ 2 .

Example 3. Consider the problem of finding a 90% confidence interval for the probability, p, of success on each trial in a sequence of independent trials, if two successes are observed in 12 trials. Some calculation yields that Pr[number of successes
2|p]

+ 66p2 (1 − p)10 = 0.05 if p = 0.438 < 0.05 if p > 0.438 and Pr[number of successes  2|p] = 1 − (1 − p)12 − 12p(1 − p)11 = 0.05 if p = 0.03046 < 0.05 if p < 0.03046.

CONFIDENCE INTERVALS AND REGIONS

3

Figure 1. Illustration of the meanings of T1 and T2 for fixed θ.

Figure 2. Confidence limits for θ based on the statistic T.

Thus the required 90% confidence interval is (0.03046, 0.0348). (Although this method of construction does not make the probability of including the true value of p to be equal to 90%, it does ensure that this probability is not less than 90%.) NUISANCE PARAMETERS∗ AND SIMILAR REGIONS∗ Under some circumstances it is easy to find confidence intervals despite the presence of a nuisance parameter. Consider the following example. Example 4. Suppose that X1 , X2 , . . . , Xn are normally distributed with mean µ and variance σ 2 , both unknown. Let X and s denote the sample mean and √ sample standard deviation. Now (X − µ) n/s has a t-distribution with n − 1 degrees of freedom no matter what the value of σ 2 . Therefore, letting t denote the 1 − 12 α quantile∗ of that t-distribution. √ √ Pr[µ − ts/ n
X
µ + ts/ n] = 1 − α;

or, equivalently, √ √ Pr[X − ts/ n
µ
X + ts/ n] = 1 − α. √ √ The interval (X − ts/ n, X + ts/ n) is a confidence level for µ at confidence level 1 − α. The parameter σ 2 is described as a nuisance parameter because we are not interested in estimating it, but it does affect the probability distribution of the observations. The regions of the sample space of the form (X − as, X + as) are described as similar regions because the probability of each of them is independent of the parameters. Confidence regions are generally based on similar regions when they exist. However, they often do not exist. CONFIDENCE REGIONS Confidence regions are a generalization of confidence intervals in which the confidence set is not necessarily an interval. Let θ be a (possibly multidimensional) parameter and let  denote the set of possible values for

4

CONFIDENCE INTERVALS AND REGIONS

θ . Let X denote a random variable, generally vector-valued. A function I that gives a subset of  for a value x of X is said to be a confidence set estimator or a confidence region for θ with confidence coefficient p if Pr[θ ∈ I(X)] = p.

(7)

For any such confidence region, to reject the hypothesis θ = θ0 whenever θ0 is not in I(X) is a Neyman—Pearson hypothesis test which has probability 1 − p of wrongly rejecting the hypothesis θ = θ0 . Choosing between Possible Confidence Regions There may be many functions I such that Pr[θ ∈ I(X)|θ ] = p for every θ . How should we choose which to use? Within the formulation where confidence intervals are based on a single statistic T, the problem is essentially that of choosing a statistic on which to base the confidence intervals. Perhaps confidence intervals based on the sample median∗ would be better in some ways than confidence intervals based on the sample mean. A number of criteria have been advanced to help decide between alternative confidence regions. We discuss some of them briefly. Standard texts on theoretical statistics may be consulted for further details. Confidence intervals should be based on sufficient statistics (see SUFFICIENT STATISTICS) and should be found conditional on the value of ancillary statistics (see ANCILLARY STATISTICS —I). A confidence region I is said to be unbiased if Pr[θ1 ∈ I(X)|θ2 ]
p for all θ1 , θ2 ∈ . This means that wrong values for the parameter are not more likely to be included in the region I(X) than the correct values. The region I is said to be shorter, more accurate, or more selective than the region J if Pr[θ1 ∈ I(X)|θ2 ]
Pr[θ1 ∈ J(x)|θ2 ] for all θ1 , θ2 ∈ . Intuitively, this means that incorrect values for θ are more likely to be

in J than in I. More selective regions correspond to more powerful tests of hypotheses and unbiased regions correspond to unbiased tests when parametric hypotheses are rejected whenever the parameter does not lie in the confidence region. The term ‘‘more selective’’ is preferred to ‘‘shorter’’ (which stems from Neyman [19]) to avoid confusion with the expected length of confidence intervals. For complex problems it may be difficult or impossible to apply some of these and other criteria. Sometimes it may only be possible to show that a particular confidence region is optimal in some sense within a particular class of regions, such as those invariant in some way. Different criteria sometimes suggest different regions. There is no completely general way of deciding which confidence interval to use. CRITICISMS OF THE THEORY OF CONFIDENCE INTERVALS There have been many arguments about the foundations of statistical inference, and there will probably be many more. Three (not independent) criticisms of the theory of confidence intervals are mentioned below. Note that they are criticisms of the frequentist school of thought, not merely of confidence intervals, which are the interval estimation technique used by that school. Likelihood Principle∗ The likelihood principle states that the ‘‘force’’ of an experiment should depend only upon the likelihood function, which is the probability density for the results obtained as a function of the unknown parameters. Many people find this principle compelling. Pratt [20] presents a persuasive defense of it in an entertaining way. Confidence interval theory violates the likelihood principle essentially because confidence intervals are concerned with the entire sample space. Coherence∗ It has been shown in several ways (e.g., Savage [23]), using various simple coherence conditions, that inference must be Bayesian if it is to be coherent. This means that every

CONFIDENCE INTERVALS AND REGIONS

Neyman confidence interval procedure that is not equivalent to a Bayesian procedure violates at least one of each set of coherence properties. Conditional Properties For a confidence region I such that Pr[θ ∈ I(X)] = α

for all θ ,

if there is a subset C of the sample space and a positive number  such that either Pr[θ ∈ I(X)|X ∈ C]
α − 

Conditional properties of confidence intervals for practical problems are seldom as poor as for this example and those of Robinson [22]. Note particularly that the complements of relevant subsets are not necessarily relevant. However, such examples do illustrate the point made by Dempster [9], Hacking [11], and others that confidence coefficients are a good measure of uncertainty before the data have been seen, but may not be afterward. LINKS WITH BAYESIAN INFERENCE∗

for all θ

or Pr[θ ∈ I(X)|X ∈ C]  α + 

5

for all θ ,

then the set C is a relevant subset. The idea stems from Fisher’s use of the term ‘‘recognizable subset’’ [10] and was formalized by Buehler [6]. Some people argue that the existence of a relevant subset implies that the confidence coefficient α is not an appropriate measure of confidence that θ ∈ I(x) when it happens that x belongs to the relevant subset. Consider the following quite artificial example, in which there are only two possible parameter values and four values for a random variable that is observed only once. Example 5. Suppose that when θ = θ1 , Pr[X = 1] = 0.9, Pr[X = 2] = 0.01, Pr[X = 3] = 0.05, and Pr[X = 4] = 0.04, whereas when θ = θ2 , Pr[X = 1] = 0.02, Pr[X = 2] = 0.9, Pr[X = 3] = 0.03, and Pr[X = 4] = 0.05. The region
{θ1 } if X = 1 or X = 3 I(X) = . {θ2 } if X = 2 or X = 4 is a confidence region for θ with confidence coefficient 0.95. However, Pr[θ ∈ I(X)|X ∈ {1, 2}, θ ]  90/92 for both θ values

(8)

and Pr[θ ∈ I(X)|X ∈ {3, 4}, θ ]
5/8 for both θ values.

(9)

Thus both {1, 2} and {3, 4} are relevant subsets.

Bayesian confidence regions are derived by taking a prior distribution, usually considered to represent subjective belief about unknown parameters, modifying it using observed data and Bayes’ theorem∗ to obtain a posterior distribution, and quoting a region of the parameter space which has the required probability according to the posterior distribution∗ (see BAYESIAN INFERENCE). Bayesian procedures satisfy most coherence principles, satisfy the likelihood principle, and have good conditional properties. However, their conclusions depend upon the arbitrarily or subjectively chosen prior distribution, not merely upon the data, and this is widely considered to be undesirable. A clear distinction must be made between proper and improper Bayesian procedures. Proper Bayesian procedures are those based on prior distributions which are proper (i.e., are probability distributions) and which use bounded loss and utility functions should loss or utility functions be required. Other Bayesian procedures are called improper and sometimes lack some of the desirable properties of proper Bayesian procedures. However, they are often used because they are more tractable mathematically. The bases of the frequentist and Bayesian schools of thought are quite different. However, many statistical procedures that are widely used in practice are both confidence interval procedures and improper Bayesian procedures. (see Bartholomew [1], Jeffreys [12], de Groot [8], and Lindley [17]). Of direct interest to people using the confidence intervals that may also be derived as improper Bayesian interval estimates is that the alternative derivation is often sufficient to ensure that these confidence intervals have

6

CONFIDENCE INTERVALS AND REGIONS

most of the desirable properties of proper Bayesian procedures. An exception is that there are relevant subsets for the usual confidence intervals based on the t-distribution∗ for the unknown mean of a normal distribution when the variance is also unknown (see Brown [4]. RELATIONSHIP TO FIDUCIAL INFERENCE∗ Fiducial inference generally proceeds by finding pivotal variables∗ , functions of both random variables and parameters which have a distribution that is independent of all parameters, and assuming that those pivotal variables have the same distribution after the random variables have been observed. Given the observed values of the random variables, the distribution of the pivotal variables implies a distribution for the parameters, called a fiducial distribution. To the extent that fiducial inference and confidence intervals both involve asserting faith, after seeing the data, in statements for which probabilities could be quoted before seeing the data they are similar theories. Bartlett [2] has argued that resolving the difference between these two theories is less important than resolving the difference between the pair of them and Bayesian methods. The clearest point of disagreement between them is that they support different solutions to the Behrens—Fisher problem∗ .

Example 6. Suppose that we wish to discriminate between two simple hypotheses, H0 : X has a standard normal distribution and H1 : X is distributed normally with mean 3 and unit variance, on the basis of a single observation. A standard Neyman—Pearson procedure would be to accept H0 (or fail to reject H1 ) is X
1.5 and to accept H1 if X > 1.5, and note that the probability of being correct is 0.933 as the measure of conclusiveness. That the same conclusiveness is expressed when X = 1.6 and when X = 3.6 seems unsatisfactory to Kiefer. Kiefer’s idea is to partition the sample space and to evaluate the conclusiveness of a statistical procedure conditionally for each subset of the partition. Here the sample space might be partitioned into three sets: (−∞, 0] ∪ (3, ∞), (0, 1] ∪ (2, 3], and (1, 2]. Conditionally on X being in the various sets, the probabilities of the decision to accept H0 or H1 being correct are 0.9973, 0.951, and 0.676. These could be considered to indicate ‘‘quite conclusive,’’ ‘‘reasonably conclusive,’’ and ‘‘slight’’ evidence, respectively. The article and discussion of Kiefer [16] refers to most other work relevant to conditional confidence regions. Most research has addressed the problem of which partitions of the sample space to use. Until the theory is further developed, it is difficult to see whether it will escape from the known weaknesses of Neyman—Pearson inference.

CONDITIONAL CONFIDENCE REGIONS

CONFIDENCE INTERVALS IN PRACTICAL STATISTICS

Brownie and Kiefer [5] consider that one of the weaknesses of Neyman—Pearson∗ methodology is that classical procedures generally do not give a measure of conclusiveness, which depends upon the data observed. Most other schools of thought do vary their measure of conclusiveness with the data. Kiefer [16] has developed a theory of conditional confidence which extends Neyman—Pearson methodology to allow both a data-dependent measure of conclusiveness and a frequency interpretation of this measure. The basic idea is most easily explained by an example of testing between two hypotheses. (See also CONDITIONAL INFERENCE.)

Confidence intervals are widely used in practice, although not as widely supported by people interested in the foundations of statistics. One reason for this dominance is that the most readily available statistical computer programs are based on the methods of the Neyman—Pearson school. Another reason is that many common confidence intervals (those based on normal, t, and binomial distributions) may also be derived as improper Bayesian procedures and do not suffer from most of the possible weaknesses of confidence intervals. These common procedures have some robustness∗ with respect to the vagaries of inference theory and may therefore be used without worrying very much

CONFIDENCE INTERVALS AND REGIONS

about the theory behind a particular derivation of them. Furthermore, it is fairly safe to use the intuitive notion of confidence rather than the restricted technical notion in such cases. When interpreting confidence intervals for several comparable parameters it should be noted that for two confidence intervals to overlap does not imply that the confidence interval for the difference between the two parameters would include the point zero. Also note that comparing more than two parameters at a time requires special theory (see MULTIPLE COMPARISONS —I). REFERENCES 1. Bartholomew, D. J. (1965). Biometrika, 52, 19–35. 2. Bartlett, M. S. (1965). J. Amer. Statist. Ass., 60, 395–409. 3. Birnbaum, A. (1962). J. Amer. Statist. Ass., 57, 269–326. (Very difficult to read.) 4. Brown, L. D. (1967). Ann. Math. Statist., 38, 1068–1071. 5. Brownie, C. and Kiefer, J. (1977). Commun. Statist. A—Theory and Methods, 6, 691–751. 6. Buehler, R. J. (1959). Ann. Math. Statist., 30, 845–863. (Fundamental reference on conditional properties of statistical procedures.) 7. Cox, D. R. (1958). Ann. Math. Statist., 29, 357–372. 8. de Groot, M. H. (1973). J. Amer. Statist. Ass., 68, 966–969. 9. Dempster, A. P. (1964). J. Amer. Statist. Ass., 59, 56–66. 10. Fisher, R. A. (1956). Statistical Methods and Scientific Inference, Oliver and Boyd, Edinburgh. (See p. 32.) 11. Hacking, I. (1965). Logic of Statistical Inference. Cambridge University Press, Cambridge. 12. Jeffreys, H. (1940). Annals of Eugenics, 10, 48–51. 13. Jones, H. L. (1958). J. Amer. Statist. Ass., 53, 482–490. 14. Keeping, E. S. (1962). Introduction to Statistical Inference. Van D. Nostrand, Princeton, N. J. 15. Kendall, M. G. and Stuart, A. (1961). The Advanced Theory of Statistics, Vol. 2: Inference and Relationship. Charles Griffin, London.

7

16. Kiefer, J. (1977). J. Amer. Statist. Ass., 72, 789–827. 17. Lindley, D. V. (1965). Introduction to Probability and Statistics from a Bayesian Viewpoint, Part 2, Inference. Cambridge University Press, Cambridge, England. 18. Neyman, J. (1934). J. R. Statist. Soc. A, 97, 558–606. (Especially Note I, p. 589, and discussion by Fisher R. A. p. 614. Mainly of historical interest.) 19. Neyman, J. (1937). Philos. Trans. R. Soc. Lond. A, 236, 333–380. (Fundamental reference on confidence intervals. These papers by Neyman are reproduced in Neyman, J. A Selection of Early Statistical Papers of J. Neyman, Cambridge University Press, Cambridge, 1967.) 20. Pratt, J. W. (1962). J. Amer. Statist. Ass., 57, 314–316. 21. Rao, C. R. (1965). Linear Statistical Inference and Its Applications. Wiley, New York. 22. Robinson, G. K. (1975). Biometrika, 62, 155–161. (Contrived, but reasonably simple examples.) 23. Savage, L. J. (1954). The Foundations of Statistics. Wiley, New York. (Argues that Bayesian inference is the only coherent theory.) 24. Savage, L. J. (1962). The foundations of Statistical Inference (a Discussion). Methuen, London. See also BAYESIAN INFERENCE; CONDITIONAL INFERENCE; FIDUCIAL INFERENCE; and INFERENCE, STATISTICAL.

G. K. ROBINSON

CONFIRMATORY TRIALS

when such trials are run, strict adherence to protocols and standard operating procedures is mandatory. Reporting of confirmatory trials requires discussion of any unavoidable changes and their impact on the study (1). Confirmatory trials typically address only a limited number of questions, and fundamental components of the analysis and justification for the design of confirmatory trials are set out in the protocol (1).

RAFE JOSEPH MICHAEL DONAHUE Vanderbilt University Medical Center, Nashville, Tennessee

1 DEFINITION AND DESCRIPTION 1.1 What They Are Confirmatory clinical trials, as defined in ICH guideline E9 (1), are ‘‘adequately controlled trials in which the hypotheses are stated in advance and evaluated.’’ ICH guideline E8 (2) partitions clinical studies according to objectives; ‘‘Therapeutic Confirmatory’’ studies are one class, along with ‘‘Human Pharmacology,’’ ‘‘Therapeutic Exploratory,’’ and ‘‘Therapeutic Use.’’ These classes correspond to a partition of the drug development process into what are typically viewed as four phases: Phase I (human pharamcology), Phase II (therapeutic exploratory), Phase III (therapeutic confirmatory), and Phase IV (therapeutic use). As such, confirmatory clinical trials are most often considered Phase III trials. These Phase III therapeutic confirmatory trials have objectives that are typically demonstration or confirmation of efficacy, establishment of a safety profile, construction of an adequate basis for assessing the benefit/risk relationship in order to support licensing, or establishment of a dose-response relationship. ICH E9 continues: ‘‘As a rule, confirmatory trials are necessary to provide firm evidence of efficacy or safety. In such trials the key hypothesis of interest follows directly from the trial’s primary objective, is always predefined, and is the hypothesis that is subsequently tested when the trial is complete. In a confirmatory trial it is equally important to estimate with due precision the size of the effects attributable to the treatment of interest and to relate these effects to their clinical significance.’’ This prespecification of the hypothesis that will be used to examine the primary objective is the key component of confirmatory trials. Since confirmatory trials are intended to provide firm evidence in support of claims,

1.2 What They Are Not Confirmatory trials are not exploratory trials. Although exploratory trials have clear and precise objectives like all clinical trials, these objectives may not lead to simple tests of predefined hypotheses. Such trials cannot, by themselves, provide sufficient evidence of efficacy; yet they may contribute to the full body of evidence (1). 2

CONSEQUENCES

In January 2005, the innovative pharmaceutical industry, represented worldwide by the European Federation of Pharmaceutical Industries and Associations (EFPIA), the International Federation of Pharmaceutical Manufacturers and Associations (IFPMA), the Japanese Pharmaceutical Manufacturers Association (JPMA), and the Pharmaceutical Research and Manufacturers of America (PhRMA), committed itself to increasing the transparency of their members’ clinical trials (3). One outcome of this commitment was the establishment of a registry of the results from confirmatory clinical trials. An industrywide registry, www.clinicalstudyresults.org, houses the results from ‘‘all clinical trials, other than exploratory trials’’ (3). Another registry of clinical trials, www .clinicaltrials.gov, contains lists of ongoing confirmatory and exploratory clinical trials. 3

ISSUES AND CONTROVERSIES

D’Agostino and Massaro (4) address issues involved in the design, conduct, and analysis of confirmatory clinical trials. Among the

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CONFIRMATORY TRIALS

issues discussed in detail are study objectives, target populations, sample population, efficacy variables, control groups, study design, comparisons, sample size, trial monitoring, data analysis sets, unit of analysis, missing data, safety, subsets, and clinical significance. All of these issues need to be addressed to develop successfully a confirmatory clinical trial. Parmar et al. (5) address issues around whether a specific, unique, confirmatory trial is necessary. They present a Bayesian statistical framework in which differences are assessed. Through this, they provide a method to assess the need for a confirmatory trial. Their examples revolve around non-smallcell lung cancer. Their argument centers on the belief that a major reason in performing confirmatory randomized clinical trials is a prior skepticism over whether the new treatment is likely to be clinically worthwhile. They argue that it might be wiser to accept the treatment into practice than to wait perhaps years to accrue patients and carry out a confirmatory trial. An editorial by Berry (6) discusses the pros and cons of the Parmer et al. approach. He concludes that confirmatory trials are certainly important from a scientific perspective, but whether they are ethical is a different matter. He points out the fact that earlier trials may have been carried out in different settings and that a confirmatory trial might likely show a different treatment effect—usually a smaller one—than was observed in the exploratory trials, due to regression to the mean. This regression to the mean bias makes it difficult to assess the magnitude of the treatment benefit as observed in a single confirmatory trial. REFERENCES 1. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH). ICH Guideline E9: General considerations for clinical trials. ICH web site. Available: www.ich.org, Accessed 2006.12.31. 2. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH). ICH

Guideline E8: Statistical Principles for Clinical Trials. ICH web site. Available: www.ich.org, Accessed 2006.12.31. 3. Joint Position on the Disclosure of Clinical Trial Information via Clinical Trial Registries and Databases. European Federation of Pharmaceutical Industries and Associations (EFPIA) web site. Available: www.efpia.org, Accessed 2006.12.31. 4. R. B. D’Agostino Sr, and J. M. Massaro, New devlopments in medical clinical trials. J Dent Res. 2004; 83 Spec No C: C18-24. 5. M. K. B. Parmar, R. S. Ungerleider, R. Simon, Assessing whether to perform a confirmatory randomized clinical trial. J. Natl. Cancer Inst. 1996; 88: 1645-1651. 6. D. A. Berry, When is a confirmatory randomized clinical trial needed? J. Natl. Cancer Inst. 1996; 88, No. 22: 1606-1607.

CROSS-REFERENCES Phase III Trials Pivotal Trial

CONFOUNDING

the agents whose properties we wish to study (emphasis added).

SANDER GREENLAND University of California, Los Angeles, CA, USA

It should be noted that, in Mill’s time, the word ‘‘experiment’’ referred to an observation in which some circumstances were under the control of the observer, as it still is used in ordinary English, rather than to the notion of a comparative trial. Nonetheless, Mill’s requirement suggests that a comparison is to be made between the outcome of his experiment (which is, essentially, an uncontrolled trial) and what we would expect the outcome to be if the agents we wish to study had been absent. If the outcomes is not as one would expect in the absence of the study agents, then his requirement ensures that the unexpected outcome was not brought about by extraneous circumstances. If, however, those circumstances do bring about the unexpected outcome, and that outcome is mistakenly attributed to effects of the study agents, then the mistake is one of confounding (or confusion) of the extraneous effects with the agent effects. Much of the modern literature follows the same informal conceptualization given by Mill. Terminology is now more specific, with ‘‘treatment’’ used to refer to an agent administered by the investigator and ‘‘exposure’’ often used to denote an unmanipulated agent. The chief development beyond Mill is that the expectation for the outcome in the absence of the study exposure is now almost always explicitly derived from observation of a control group that is untreated or unexposed. For example, Clayton & Hills (2) state of observational studies,

The word confounding has been used to refer to at least three distinct concepts. In the oldest usage, confounding is a bias in estimating causal effects (see Causation). This bias is sometimes informally described as a mixing of effects of extraneous factors (called confounders) with the effect of interest. This usage predominates in nonexperimental research, especially in epidemiology and sociology. In a second and more recent usage, confounding is a synonym for noncollapsibility, although this usage is often limited to situations in which the parameter of interest is a causal effect. In a third usage, originating in the experimental-design literature, confounding refers to inseparability of main effects and interactions under a particular design. The term aliasing is also sometimes used to refer to the latter concept; this usage is common in the analysis of variance literature. The three concepts are closely related and are not always distinguished from one another. In particular, the concepts of confounding as a bias in effect estimation and as noncollapsibility are often treated as identical, although there are many examples in which the two concepts diverge (8,9,14); one is given below. 1 CONFOUNDING AS A BIAS IN EFFECT ESTIMATION 1.1 Confounding

. . . there is always the possibility that an important influence on the outcome . . . differs systematically between the comparison [exposed and unexposed] groups. It is then possible [that] part of the apparent effect of exposure is due to these differences, [in which case] the comparison of the exposure groups is said to be confounded (emphasis in the original).

A classic discussion of confounding in which explicit reference is made to ‘‘confounded effects’’ is Mill [15, Chapter X] (although in Chapter III Mill lays out the primary issues and acknowledges Francis Bacon as a forerunner in dealing with them). There, he lists a requirement for an experiment intended to determine causal relations:

In fact, confounding is also possible in randomized experiments, owing to systematic improprieties in treatment allocation, administration, and compliance. A further

. . . none of the circumstances [of the experiment] that we do know shall have effects susceptible of being confounded with those of

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CONFOUNDING

and somewhat controversial point is that confounding (as per Mill’s original definition) can also occur in perfect randomized trials due to random differences between comparison groups (6,8). Various mathematical formalizations of confounding have been proposed. Perhaps the one closest to Mill’s concept is based on a formal counterfactual model for causal effects. Suppose our objective is to determine the effect of applying a treatment or exposure x1 on a parameter µ of population A, relative to applying treatment or exposure x0 . For example, A could be a cohort of breast-cancer patients, treatment x1 could be a new hormone therapy, x0 could be a placebo therapy, and the parameter µ could be the 5-year survival probability. The population A is sometimes called the target population or index population; the treatment x1 is sometimes called the index treatment; and the treatment x0 is sometimes called the control or reference treatment (which is often a standard or placebo treatment). The counterfactual model assumes that µ will equal µA1 if x1 is applied, µA0 if x0 is applied; the causal effect of x1 relative to x0 is defined as the change from µA0 to µA1 , which might be measured as µA1 − µA0 or µA1 /µA0 . If A is observed under treatment x1 , then µ will equal µA1 , which is observable or estimable, but µA0 will be unobservable. Suppose, however, we expect µA0 to equal µB0 , where µB0 is the value of the outcome µ observed or estimated for a population B that was administered treatment x0 . The latter population is sometimes called the control or reference population. Confounding is said to be present if in fact µA0
= µB0 , for then there must be some difference between populations A and B (other than treatment) that is affecting µ. If confounding is present, a naive (crude) association measure obtained by substituting µB0 for µA0 in an effect measure will not equal the effect measure, and the association measure is said to be confounded. For example, if µB0
= µA0 , then µA1 − µB0 , which measures the association of treatments with outcomes across the populations, is confounded for µA1 − µA0 , which measures the effect of treatment x1 on population A. Thus, saying a measure of association such as

µA1 − µB0 is confounded for a measure of effect such as µA1 − µA0 is synonymous with saying the two measures are not equal. The preceding formalization of confounding gradually emerged through attempts to separate effect measures into a component due to the effect of interest and a component due to extraneous effects (1,4,10,12,13). These decompositions will be discussed below. One noteworthy aspect of the above formalization is that confounding depends on the outcome parameter. For example, suppose populations A and B have a different 5-year survival probability µ under placebo treatment x0 ; that is, suppose µB0
= µA0 , so that µA1 − µB0 is confounded for the actual effect µA1 − µA0 of treatment on 5-year survival. It is then still possible that 10-year survival, ν, under the placebo would be identical in both populations; that is, ν A0 could still equal ν B0 , so that ν A1 − ν B0 is not confounded for the actual effect of treatment on 10-year survival. (We should generally expect no confounding for 200-year survival, since no treatment is likely to raise the 200-year survival probability of human patients above zero.) A second noteworthy point is that confounding depends on the target population of inference. The preceding example, with A as the target, had different 5-year survivals µA0 and µB0 for A and B under placebo therapy, and hence µA1 − µB0 was confounded for the effect µA1 − µA0 of treatment on population A. A lawyer or ethicist may also be interested in what effect the treatment x1 would have had on population B. Writing µB1 for the (unobserved) outcome of B under treatment x1 , this effect on B may be measured by µB1 − µB0 . Substituting µA1 for the unobserved µB1 yields µA1 − µB0 . This measure of association is confounded for µB1 − µB0 (the effect of treatment x1 on 5-year survival in population B) if and only if µA1
= µB1 . Thus, the same measure of association, µA1 − µB0 , may be confounded for the effect of treatment on neither, one, or both of populations A and B. 1.2 Confounders A third noteworthy aspect of the counterfactual formalization of confounding is that it invokes no explicit differences (imbalances)

CONFOUNDING

between populations A and B with respect to circumstances or covariates that might influence µ (8). Clearly, if µA0 and µB0 differ, then A and B must differ with respect to factors that influence µ. This observation has led some authors to define confounding as the presence of such covariate differences between the compared populations. Nonetheless, confounding is only a consequence of these covariate differences. In fact, A and B may differ profoundly with respect to covariates that influence µ, and yet confounding may be absent. In other words, a covariate difference between A and B is a necessary but not sufficient condition for confounding. This point will be illustrated below. Suppose now that populations A and B differ with respect to certain covariates, and that these differences have led to confounding of an association measure for the effect measure of interest. The responsible covariates are then termed confounders of the association measure. In the above example, with µA1 − µB0 confounded for the effect µA1 − µA0 , the factors responsible for the confounding (i.e. the factors that led to µA0
= µB0 ) are the confounders. It can be deduced that a variable cannot be a confounder unless it can affect the outcome parameter µ within treatment groups and it is distributed differently among the compared populations (e.g. see Yule (23), who however uses terms such as ‘‘fictitious association’’ rather than confounding). These two necessary conditions are sometimes offered together as a definition of a confounder. Nonetheless, counterexamples show that the two conditions are not sufficient for a variable with more than two levels to be a confounder as defined above; one such counterexample is given in the next section. 1.3 Prevention of Confounding Perhaps the most obvious way to avoid confounding in estimating µA1 − µA0 is to obtain a reference population B for which µB0 is known to equal µA0 . Among epidemiologists, such a population is sometimes said to be comparable to or exchangeable with A with respect to the outcome under the reference treatment. In practice, such a population may be difficult or impossible to find. Thus, an

3

investigator may attempt to construct such a population, or to construct exchangeable index and reference populations. These constructions may be viewed as design-based methods for the control of confounding. Perhaps no approach is more effective for preventing confounding by a known factor than restriction. For example, gender imbalances cannot confound a study restricted to women. However, there are several drawbacks: restriction on enough factors can reduce the number of available subjects to unacceptably low levels, and may greatly reduce the generalizability of results as well. Matching the treatment populations on confounders overcomes these drawbacks and, if successful, can be as effective as restriction. For example, gender imbalances cannot confound a study in which the compared groups have identical proportions of women. Unfortunately, differential losses to observation may undo the initial covariate balances produced by matching. Neither restriction nor matching prevents (although it may diminish) imbalances on unrestricted, unmatched, or unmeasured covariates. In contrast, randomization offers a means of dealing with confounding by covariates not accounted for by the design. It must be emphasized, however, that this solution is only probabilistic and subject to severe constraints in practice. Randomization is not always feasible, and (as mentioned earlier) many practical problems, such as differential loss and noncompliance, can lead to confounding in comparisons of the groups actually receiving treatments x1 and x0 . One somewhat controversial solution to noncompliance problems is intention-totreat analysis, which defines the comparison groups A and B by treatment assigned rather than treatment received. Confounding may, however, affect even intentionto-treat analyses. For example, the assignments may not always be random, as when blinding is insufficient to prevent the treatment providers from protocol violations. And, purely by bad luck, randomization may itself produce allocations with severe covariate imbalances between the groups (and consequent confounding), especially if the study size is small (6,8,19). Block randomization can help ensure that random imbalances on

4

CONFOUNDING

the blocking factors will not occur, but it does not guarantee balance of unblocked factors. 1.4 Adjustment for Confounding Design-based methods are often infeasible or insufficient to prevent confounding. Thus there has been an enormous amount of work devoted to analytic adjustments for confounding. With a few exceptions, these methods are based on observed covariate distributions in the compared populations. Such methods can successfully control confounding only to the extent that enough confounders are adequately measured. Then, too, many methods employ parametric models at some stage, and their success may thus depend on the faithfulness of the model to reality. These issues cannot be covered in depth here, but a few basic points are worth noting. The simplest and most widely trusted methods of adjustment begin with stratification on confounders. A covariate cannot be responsible for confounding within internally homogeneous strata of the covariate. For example, gender imbalances cannot confound observations within a stratum composed solely of women. More generally, comparisons within strata cannot be confounded by a covariate that is constant (homogeneous) within strata. This is so regardless of whether the covariate was used to define the strata. Generalizing this observation to a regression context, we find that any covariate with a residual variance of zero conditional on the regressors cannot confound regression estimates of effect (assuming that the regression model is correct). A broader and more useful observation is that any covariate that is unassociated with treatment conditional on the regressors cannot confound the effect estimates; this insight leads directly to adjustments using a propensity score. Some controversy has existed about adjustment for covariates in randomized trials. Although Fisher asserted that randomized comparisons were unbiased, he also pointed out that they could be confounded in the sense used here (e.g. see Fisher [6, p. 49]). Fisher’s use of the word ‘‘unbiased’’ was unconditional on allocation, and therefore of little guidance for analysis of a given trial. The ancillarity

of the allocation naturally leads to conditioning on the observed distribution of any pretreatment covariate that can influence the outcome parameter. Conditional on this distribution, the unadjusted treatment–effect estimate will be biased if the covariate is associated with treatment; this conditional bias can be removed by adjustment for the confounders (8,18). Note that the adjusted estimate is also unconditionally unbiased, and thus is a reasonable alternative to the unadjusted estimate even without conditioning. 1.5 Measures of Confounding The parameter estimated by a direct unadjusted comparison of cohorts A and B is µA1 − µA0 . A number of authors have measured the bias (confounding) of the unadjusted comparison by (10,12) (µA1 − µB0 ) − (µA1 − µA0 ) = µA0 − µB0 . When the outcome parameters, µ, are risks (probabilities), epidemiologists use instead the analogous ratio µA0 µA1 /µB0 = µA1 /µA0 µB0 as a measure of bias (1,4,14); µA0 /µB0 is sometimes called the confounding risk ratio. The latter term is somewhat confusing because it is sometimes misunderstood to refer to the effect of a particular confounder on risk. This is not so, although the ratio does reflect the net effect of the differences in the confounder distributions of A and B. 1.6 Residual Confounding Suppose now that adjustment for confounding is done by subdividing the total study population (A + B) into K strata indexed by k. Let µA1k be the parameter of interest in stratum k of populations A and B under treatment x0 . The effect of treatment x1 relative to x0 in stratum k may be defined as µA1k − µA0k or µA1k /µA0k . The confounding that remains in stratum K is called the residual confounding in the stratum, and is measured by µA0k − µB0k or µA1k /µB0k .

CONFOUNDING

Like effects, stratum-specific residual confounding may be summarized across the strata in a number of ways, for example by standardization methods or by other weighted-averaging methods. As an illustration, suppose we are given a standard distribution p1 , . . . , pK for the stratum index k. In ratio terms, the standardized effect of x1 vs. x0 on A under this distribution is
pk µA1k k

RAA =


, pk µA0k

5

pose population k is given treatment xk , even though it could have been given some other treatment. The absolute effect of x1 vs. x2 on µ in population 1 is µ1 (x1 ) − µ1 (x2 ) = (x1 − x2 )β. Substitution of µ2 (x2 ), the value of µ in population 2 under treatment x2 , for µ1 (x2 ) yields µ1 (x1 ) − µ2 (x2 ) = α1 − α2 + (x1 − x2 )β, which is biased by the amount

k

whereas the standardized ratio comparing A with B is
pk µA1k k

RAB =


. pk µB0k

k

The overall residual confounding in RAB is thus
pk µA0k RAB k =
, RAA pk µB0k k

which may be recognized as the standardized ratio comparing A and B when both are given treatment x0 , using p1 , . . . , pK as the standard distribution. 1.7 Regression Formulations For simplicity, the above presentation has focused on comparing two populations and two treatments. The basic concepts extend immediately to the consideration of multiple populations and treatments. Paired comparisons may be represented using the above formalization without modification. Parametric models for these comparisons then provide a connection to more familiar regression models. As an illustration, suppose population differences and treatment effects follow the model µk (x) = αk + xβ, where the treatment level x may range over a continuum, and k indexes populations. Sup-

µ1 (x2 ) − µ2 (x2 ) = α1 − α2 . Thus, under this model no confounding will occur if the intercepts α k equal a constant α across populations, so that µk (x) = α + βx. When constant intercepts cannot be assumed and nothing else is known about the intercept magnitudes, it may be possible to represent our uncertainty about α k via the following mixed-effects model: µk (x) = α + xβ + k . Here, α k has been decomposed into α +  k , where  k has mean zero, and the confounding in µ1 (x1 ) − µ2 (x2 ) has become an unobserved random variable,  1 −  2 . Correlation of population membership k with xk leads to a correlation of  k with xk , which in turn leads to bias in estimating β. This bias may be attributed to or interpreted as confounding for β in the regression analysis. Confounders are now covariates that causally ‘‘explain’’ the correlation between  k and xk . In particular, confounders normally reduce the correlation of xk and  k when entered in the model. The converse is false, however: a variable that reduces the correlation of xk and  k when entered need not be a confounder; it may, for example, be a variable affected by both the treatment and the exposure. 2 CONFOUNDING AND NONCOLLAPSIBILITY Much of the statistics literature does not distinguish between the concept of confounding as described above and the concept

6

CONFOUNDING

of noncollapsibility. Nonetheless, the two concepts are distinct: for certain outcome parameters, confounding may occur with or without noncollapsibility and noncollapsibility may occur with or without confounding (8,9,14,17,20,22). Mathematically identical conclusions have been reached by other authors, albeit with different terminology in which noncollapsibility corresponds to ‘‘bias’’ and confounding corresponds to covariate imbalance (7,11). As an example of no collapsibility with no confounding, consider the response distributions under treatments x1 and x0 given in Table 1 for a hypothetical index population A, and the response distribution under treatment x0 given in Table 2 for a hypothetical reference population B. If we take the odds of response as the outcome parameter µ, we get 1460 = 2.70 µA1 = 540 and µA0 = µB0 =

1000 = 1.00. 1000

Table 1. Distribution of Responses for Population A, within Strata of Z and Ignoring Z, under Treatments x1 and x0 Number of Responses Under Subpopulation

x1

x0

Subpopulation Size

Z=1 Z=2 Z=3 Totals

200 900 360 1460

100 600 300 1000

400 1200 400 2000

Table 2. Distribution of Responses for Population B, within Strata of Z and Ignoring Z, under Treatment x0

Subpopulation

Number of Responses Under x0

Subpopulation Size

Z=1 Z=2 Z=3 Totals

200 200 600 1000

800 400 800 2000

There is thus no confounding of the odds ratio: µA1 /µA0 = µA1 /µB0 = 2.70/1.00 = 2.70. Nonetheless, the covariate Z is associated with response and is distributed differently in A and B. Furthermore, the odds ratio is not collapsible: within levels of Z, the odds ratios comparing A under treatment x1 with either A or B under x0 are (200/200)/(200/600) = (900/300)/(200/200) = (360/40)/(600/200) = 3.00, a bit higher than the odds ratio of 2.70 obtained when Z is ignored. The preceding example illustrates a peculiar property of the odds ratio as an effect measure: treatment x1 (relative to x0 ) elevates the odds of response by 170% in population A, yet within each stratum of Z it raises the odds by 200%. When Z is associated with response conditional on treatment but unconditionally unassociated with treatment, the stratum-specific effects on odds ratios will be further from the null than the overall effect if the latter is not null (7). This phenomenon is often interpreted as a ‘‘bias’’ in the overall odds ratio, but in fact there is no bias if one does not interpret the overall effect as an estimate of the stratum-specific effects. The example also shows that, when µ is the odds, the ‘‘confounding odds ratio’’ (µA1 /µB0 )/(µA1 /µA0 ) = µA0 /µB0 may be 1 even when the odds ratio is not collapsible over the confounders. Conversely, we may have µA0 /µB0
= 1 even when the odds ratio is collapsible. More generally, the ratio of crude and stratum-specific odds ratios does not equal µA0 /µB0 except in some special cases. When the odds are low, however, the odds will be close to the corresponding risks, and so the two ratios will approximate one another. The phenomenon illustrated in the example corresponds to the differences between cluster-specific and population-averaged (marginal) effects in nonlinear mixedeffects regression (16). Specifically, the clusters of correlated outcomes correspond to the strata, the cluster effects correspond to covariate effects, the cluster-specific treatment effects correspond to the stratumspecific log odds ratios, and the populationaveraged treatment effect corresponds to the crude log odds ratio. Results of Gail (7) imply that if the effect measure is the difference or ratio of response proportions, then the above phenomenon–

CONFOUNDING

noncollapsibility over Z without confounding by Z–cannot occur, nor can confounding by Z occur without noncollapsibility over Z. More generally, when the effect measure is an expectation over population units, confounding by Z and noncollapsibility over Z are algebraically equivalent. This equivalence may explain why the two concepts are often not distinguished. 3 CONFOUNDING IN EXPERIMENTAL DESIGN Like the bias definition, the third usage of confounding stems from the notion of mixing of effects. However, the effects that are mixed are main (block) effects and interactions (or different interactions) in a linear model, rather than effects in the nonparametric sense of a counterfactual model. This definition of confounding differs even more markedly from other definitions in that it refers to an intentional design feature of certain experimental studies, rather than a bias. The topic of confounded designs is extensive; some classic references are Fisher (6), Cochran & Cox (3), Cox (5), and Scheff´e (21). Confounding can serve to improve efficiency in estimation of certain contrasts and can reduce the number of treatment groups that must be considered. The price paid for these benefits is a loss of identifiability of certain parameters, as reflected by aliasing of those parameters. As a simple example, consider a situation in which we wish to estimate three effects in a single experiment: that of treatments x1 vs. x0 , y1 vs. y0 , and z1 vs. z0 . For example, in a smoking cessation trial these treatments may represent active and placebo versions of the nicotine patch, nicotine gum, and buspirone. With no restrictions on number or size of groups, a fully crossed design would be reasonable. By allocating subjects to each of the 23 = 8 possible treatment combinations, one could estimate all three main effects, all three two-way interactions, and the threeway interaction of the treatments. Suppose, however, that we were restricted to use of only four treatment groups (e.g. because of cost or complexity considerations).

7

A naive approach would be to use groups of equal size, assigning one group to placebos only (x0 , y0 , z0 ) and the remaining three groups to one active treatment each: (x1 , y0 , z0 ), (x0 , y1 , z0 ), and (x0 , y0 , z1 ). Unfortunately, with a fixed number N of subjects available, this design would provide only N/4 subjects under each active treatment. As an alternative, consider the design with four groups of equal size with treatments (x0 , y0 , z0 ), (x1 , y1 , z0 ), (x1 , y0 , z1 ), and (x0 , y1 , z1 ). This fractional factorial design would provide N/2 subjects under each active treatment, at the cost of confounding main effects and interactions. For example, no linear combination of group means containing the main effect of x1 vs. x0 would be free of interactions. If one could assume that all interactions were negligible, however, this design could provide considerably more precise estimates of the main effects than the naive four-group design. To see these points, consider the following linear model: µXYZ = α + β1 X + β2 Y + β3 Z + γ1 XY + γ2 XZ + γ3 YZ + δXYZ, where X, Y, and Z equal 1 for x1 , y1 , and z1 , and 0 for x0 , y0 , and z0 , respectively. The group means, in the fractional factorial design are then µ000 = α, µ110 = α + β1 + β2 + γ1 , µ101 = α + β1 + β3 + γ2 , µ011 = α + β2 + β3 + γ3 . Treating the means as observed and the coefficients as unknown, the above system is underidentified. In particular, there is no solution for any main effect β j in terms of the means µijk . Nonetheless, assuming all γ j = 0 yields immediate solutions for all the β j . Additionally assuming a variance of σ 2 for each estimated group mean yields that the main-effect estimates under this design would have variances of σ 2 , as opposed to 2σ 2 for the main-effect estimates from the naive four-group design of the same size. For

8

CONFOUNDING

example, under the confounded fractional factorial design (assuming no interactions) βˆ1 =

µˆ 110 + µˆ 101 − µˆ 000 − µˆ 011 , 2

so var(βˆ1 ) = 4σ 2 /4 = σ 2 , whereas under the naive design, βˆ = µˆ 100 − µˆ 000 so var(βˆ1 ) = 2σ 2 . Of course, the precision advantage of the confounded design is purchased by the assumption of no interaction, which is not needed by the naive design. REFERENCES 1. Bross, I. D. J. (1967). Pertinency of an extraneous variable, Journal of Chronic Diseases 20, 487–495. 2. Clayton, D. & Hills, M. (1993). Statistical Models in Epidemiology. Oxford University Press, New York. 3. Cochran, W. G. & Cox, G. M. (1957). Experimental Designs, 2nd Ed. Wiley, New York. 4. Cornfield, J., Haenszel, W., Hammond, W. C., Lilienfeld, A. M., Shimkin, M. B. & Wynder, E. L. (1959). Smoking and lung cancer: recent evidence and a discussion of some questions, Journal of the National Cancer Institute 22, 173–203. 5. Cox, D. R. (1958). The Planning of Experiments. Wiley, New York.

10. Groves, E. R. & Ogburn, W. F. (1928). American Marriage and Family Relationships. Henry Holt & Company, New York, pp. 160–164. 11. Hauck, W. W., Neuhaus, J. M., Kalbfleisch, J. D. & Anderson, S. (1991). A consequence of omitted covariates when estimating odds ratios, Journal of Clinical Epidemiology 44, 77–81. 12. Kitagawa, E. M. (1955). Components of a difference between two rates, Journal of the American Statistical Association 50, 1168–1194. 13. Miettinen, O. S. (1972). Components of the crude risk ratio, American Journal of Epidemiology 96, 168–172. 14. Miettinen, O. S. & Cook, E. F. (1981). Confounding: essence and detection, American Journal of Epidemiology 114, 593–603. 15. Mill, J. S. (1843). A System of Logic, Ratiocinative and Inductive. Reprinted by Longmans, Green & Company, London, 1956. 16. Neuhaus, J. M., Kalbfleisch, J. D. & Hauck, W. W. (1991). A comparison of cluster-specific and population-averaged approaches for analyzing correlated binary data, International Statistical Review 59, 25–35. 17. Pearl, J. (2000). Causality. Cambridge University Press, New York, Ch. 6. 18. Robins, J. M. & Morgenstern, H. (1987). The mathematical foundations of confounding in epidemiology, Computers and Mathematics with Applications 14, 869–916. 19. Rothman, K. J. (1977). Epidemiologic methods in clinical trials, Cancer 39, 1771–1775.

6. Fisher, R. A. (1935). The Design of Experiments. Oliver & Boyd, Edinburgh. 7. Gail, M. H. (1986). Adjusting for covariates that have the same distribution in exposed and unexposed cohorts, in Modern Statistical Methods in Chronic Disease Epidemiology, S. H. Moolgavkar & R. L. Prentice, eds. Wiley, New York. 8. Greenland, S. & Robins, J. M. (1986). Identifiability, exchangeability, and epidemiological confounding, International Journal of Epidemiology 15, 413–419.

20. Rothman, K. J. & Greenland, S. (1998). Modern Epidemiology, 2nd ed. Lippincott, Philadelphia, Ch. 4. 21. Scheff´e, H. A. (1959). The Analysis of Variance. Wiley, New York. 22. Wickramaratne, P. & Holford, T. (1987). Confounding in epidemiologic studies: the adequacy of the control groups as a measure of confounding, Biometrics 43, 751–765.

9. Greenland, S., Robins, J. M. & Pearl, J. (1999). Confounding and collapsibility in causal inference. Statistical Science 14, 29–46.

23. Yule, G. U. (1903). Notes on the theory of association of attributes in statistics, Biometrika 2, 121–134.

CONSORT A report of a randomized controlled trial (RCT) should convey to the reader, in a transparent manner, why the study was undertaken, and how it was conducted and analyzed. To assess the strengths and limitations of an RCT, the reader needs and deserves to know the quality of its methodology. Despite several decades of educational efforts, RCTs still are not being reported adequately [2, 5, 10]. The Consolidated Standards of Reporting Trials (CONSORT) statement, published in the Journal of the American Medical Association in 1996 [1], was developed to try to help rectify this problem. The CONSORT statement was developed by an international group of clinical trialists,

statisticians, epidemiologists and biomedical editors. The CONSORT statement is one result of previous efforts made by two independent groups, the Standards of Reporting Trials (SORT) group [9] and the Asilomar Working Group on Recommendations for Reporting of Clinical Trials in the Biomedical Literature [11]. The CONSORT statement consists of two components, a 21-item checklist (Table 1) and a flow diagram (Figure 1). The checklist has six major headings that pertain to the contents of the report of a trial, namely Title, Abstract, Introduction, Methods, Results and Discussion. Within these major headings there are subheadings that pertain to specific items that should be included in any clinical trial manuscript. These items constitute the key pieces of information necessary for authors to address when reporting

Registered or Eligible Patients (n =...)

Not Randomized (n =...) Reasons (n =...)

Randomization

Received Standard Intervention as Allocated (n =...)

Received Standard Intervention as Allocated (n =...)

Did not Receive Standard Intervention as Allocated (n =...)

Did not Receive Standard Intervention as Allocated (n =...)

Followed UP (n =...)

Followed UP (n =...)

Timing of Primary and Secondary Outcomes

Timing of Primary and Secondary Outcomes

Withdrawn (n =...) Intervention ineffective (n =...) Lost to Follow-up (n =...) Other (n =...)

Completed Trial (n =...)

Withdrawn (n =...) Intervention ineffective (n =...) Lost to Follow-up (n =...) Other (n =...)

Completed Trial (n =...)

Figure 1 CONSORT flowchart. Reproduced with permission from the Journal of the American Medical Association, 1996, Volume 276, 637–665. Copyrighted (1996), American Medical Association Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

2

CONSORT

Table 1

CONSORT checklist

Heading

Subheading

Title Abstract Introduction

Methods

Protocol

Assignment

Masking (Blinding)

Results

Comment

Participant Flow and Follow-up Analysis

Descriptor Identify the study as a randomized trial. Use a structured format. Identify the study as a randomized trial. Use a structured format. State prospectively defined hypothesis, clinical objectives, and planned subgroup or covariate analyses Describe 1. Planned study population, together with inclusion/exclusion criteria. 2. Planned interventions and their timing. 3. Primary and secondary outcome measure(s) and the minimum important difference(s), and indicate how the target sample size was projected. 4. Rationale and methods for statistical analyses, detailing main comparative analyses and whether they were completed on an intention-to-treat basis. 5. Prospectively defined stopping rules (if warranted) Describe 1. Unit of randomization (e.g. individual, cluster, geographic). 2. Method used to generate the allocation schedule. 3. Method of allocation concealment and timing of assignment. 4. Method to separate the generator from the executor of assignment. Describe mechanism (e.g. capsules, tablets); similarity of treatment characteristics (e.g. appearance, taste); allocation schedule control (location of code during trial and when broken); and evidence for successful blinding among participants, person doing intervention, outcome assessors, and data analysts. Provide a trial profile (Figure 1) summarizing participant flow, numbers and timing of randomization assignment, interventions, and measurements for each randomized group. State estimated effect of intervention on primary and secondary outcome measures, including a point estimate and measure of precision (confidence interval). State results in absolute numbers when feasible (e.g. 10/20, not 50%). Present summary data and appropriate descriptive and inferential statistics in sufficient detail to permit alternative analyses and replication. Describe prognostic variables by treatment group and any attempt to adjust for them. Describe protocol deviations from the study as planned, together with the reasons. State specific interpretation of study findings, including sources of bias and imprecision (internal validity) and discussion of external validity, including appropriate quantitative measures when possible. State general interpretation of the data in light of the totality of the available evidence.

Was it reported? Page no.?

CONSORT the results of an RCT. Their inclusion is based on evidence, whenever possible. For example, authors are asked to report on the methods they used to achieve allocation concealment, possible in every randomized trial. There is growing evidence that inadequately concealed trials, compared with adequately concealed ones, exaggerate the estimates of intervention benefit by 30%–40%, on average [7, 8]. Additional benefits of the checklist (and flow diagram) include facilitating editors, peer reviewers and journal readers to evaluate the internal and external validity of a clinical trial report. The flow diagram pertains to the process of winnowing down the number of participants from those eligible or screened for a trial to those who ultimately completed the trial and were included in the analysis. The flow diagram pertains particularly to a two-group, parallel design, as stated in the CONSORT statement. Other checklists and flow diagrams have been developed for reporting cluster randomized trials [4] and other designs (see http://www.consortstatement.org). The flow diagram, in particular, requests relevant information regarding participants in each of the intervention and control groups who did not receive the regimen for the group to which they were randomized, those who during the course of the trial were discontinued, withdrew, became lost to follow-up, and those who have incomplete information for some other reason. There is emerging evidence to suggest that the quality of reporting of RCTs, based on the use of the CONSORT statement, compared with not using it, is higher on several dimensions, such as reduced reporting of unclear allocation concealment [6]. Similarly, use of the flow diagram was associated with better overall reporting of RCTs [3]. The CONSORT statement (checklist and flow diagram) is available on the CONSORT website (www.consort-statement.org). This site includes information on the growing number of health care journals and biomedical editorial groups, such as the International Council of Medical Journal Editors (ICMJE), who support the use of the CONSORT statement for reporting RCTs. At this writing the CONSORT statement is undergoing revision. Present plans call for the revised Statement to appear in Spring 2001 along with an

3

extensive explanatory and elaboration document to overcome some of the shortcomings of the original statement, both of which will be available on the above website.

References [1]

Begg, C., Cho, M., Eastwood, S., Horton, R., Moher, D., Olkin, I., Pitkin, R., Rennie, D., Schulz, K.F., Simel, D. & Stroup, D.F. (1996). Improving the quality of reporting of randomized controlled trials: the CONSORT Statement, Journal of the American Medical Association 276, 637–639. [2] Dickinson, K., Bunn, F., Wentz, R., Edwards, P. & Roberts, I. (2000). Size and quality of randomized controlled trials in head injury: review of published studies, British Medical Journal 320, 1308–1311. [3] Egger, M., J¨uni, P., Bartlett, C. for the CONSORT Group (2001). The value of CONSORT flow charts in reports of randomized controlled trials: bibliographic study, Journal of the American Medical Association, in press. [4] Elbourne, D.R. & Campbell, M.K. (2001). Extending the CONSORT statement to cluster randomized trials: for discussion, Statistics in Medicine 20, 489–496. [5] Hotopf, M., Lewis, G. & Normand, C. (1997). Putting trials on trial – the costs and consequences of small trials in depression: a systematic review of methodology, Journal of Epidemiology and Community Health 51, 354–358. [6] Moher, D., Jones, A., Lepage, L. for the CONSORT Group (2001). Does the CONSORT statement improve the quality of reports of randomized trials: a comparative before and after evaluation?, Journal of the American Medical Association, in press. [7] Moher, D., Pham, B., Jones, A., Cook, D.J., Jadad, A.R., Moher, M. & Tugwell, P. (1998). Does the quality of reports of randomized trials affect estimates of intervention efficacy reported in meta-analyses?, Lancet 352, 609–613. [8] Schulz, K.F., Chalmers, I., Hayes, R.J. & Altman, D.G. (1995). Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials, Journal of the American Medical Association 273, 408–412. [9] The Standards of Reporting Trials Group (1994). A proposal for structured reporting of randomized controlled trials, Journal of the American Medical Association 272, 1926–1931. Correction: Journal of the American Medical Association 273, 776. [10] Thornley, B. & Adams, C.E. (1998). Content and quality of 2000 controlled trials in schizophrenia over 50 years, British Medical Journal 317, 1181–1184. [11] Working Group on Recommendations for Reporting of Clinical Trials in the Biomedical Literature (1994). Call for comments on a proposal to improve reporting

4

CONSORT of clinical trials in the biomedical literature: a position paper, Annals of Internal Medicine 121, 894–895.

(See also QUORUM) DAVID MOHER

CONTRACT RESEARCH ORGANIZATION (CRO) A Contract Research Organization (CRO) is a person or an organization (commercial, academic, or other) contracted by the sponsor to perform one or more of the trial-related duties and functions of the sponsor. A sponsor may transfer any or all of the trial-related duties and functions of the sponsor to a CRO, but the ultimate responsibility for the quality and integrity of the trial data always resides with the sponsor. The CRO should implement quality assurance and quality control. Any trial-related duty and function that is transferred to and assumed by a CRO should be specified in writing. Any trial-related duties and functions not specifically transferred to and assumed by a CRO are retained by the sponsor. All references to a sponsor in this guideline also apply to a CRO to the extent that a CRO has assumed the trial-related duties and functions of a sponsor.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

CONTROL GROUPS

Informal observational methods have been used to identify treatments throughout the history of medicine. Control groups were, in essence, previously treated patients assessed according to general clinical impression. The humoralistic philosophy of medicine— treatment to restore the balance of blood, phlegm, black bile, and yellow bile in the sick—espoused by Hippocrates and codified by Galen (130 AD) remained largely unchallenged and untested until the Renaissance (4). Unfortunately, the replacement theories were not much better, nor were the treatments or experimental methods. Lack of adequately controlled trials might not have been so important except that so many treatments were actively harmful. Numerical methods were introduced in the early 1800s, leading to important epidemiologic observations such as Snow’s discovery that cholera is a waterborne infectious disease (5). However, the methods were not as useful in assessing treatment effectiveness because if they were applied at all, they were applied using unreliable control groups. For example, a diphtheria antiserum was introduced in Europe in 1894 to 1895, and death rates due to diphtheria declined—but the decline had started before 1894 and rose again to pre-1894 rates by 1924 (6). The mix of responsible bacteria changed over time, making the contribution of treatment to the initial decline uncertain. An early exception to use of control groups was the seawater-controlled 6 arm, 12 patient scurvy study undertaken by James Lind in 1753, which has been noted to be ‘‘the first deliberately planned controlled experiment ever undertaken on human subjects’’ (7). This success lacked a completely happy ending, however, in that it took decades before the results were reliably applied and accepted (8, 9). Although principles of controlled medical experiments were expressed as early as 1866 (2), the modern randomized controlled trial is a relatively recent development. The first properly randomized controlled treatment trial was a study of streptomycin in tuberculosis. In 2 years, the trial convincingly demonstrated that streptomycin plus

STEPHANIE GREEN Clinical Biostatistics Pfizer, Inc. New London, Connecticut

A control group for a clinical trial is a group of uniformly treated patients selected to be compared with a group receiving a test (new or different) treatment. For the comparison to be a valid assessment of differences in outcome due to test treatment, the two groups must be as similar as possible with the exception of treatment. Various types of control groups are employed, and the choice is critical to the interpretation of a trial (1). The type chosen for a particular trial will depend on many factors such as availability of resources, subjectivity of the primary outcome, availability of effective treatments, current extent of knowledge concerning treatment, severity of disease, and ethical considerations. Randomized trials comparing a control group with an experimental treatment group are the most convincing and reliable method for minimizing bias and demonstrating effectiveness of new treatments. Depending on standard of care, control groups for randomized trials may be patients randomized to no treatment, a placebo (i.e., an inactive agent or a procedure with the appearance of the new treatment), or standard active therapy with or without a placebo. Because of practical and ethical constraints, not all clinical questions can be addressed with a randomized trial. Nonrandomized controlled trials can have specific comparator groups or be controlled in the sense that carefully chosen historical information from the literature is used for reference. Proper use of control groups has been a significant advance in medicine, with controlled clinical trials now the mainstay of clinical research without which ‘‘the doctor walks at random and becomes the sport of illusion’’ (2, 3). 1

HISTORY

It can be argued that the history of clinical research is the history of control groups.

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

CONTROL GROUPS

bed rest was superior to the control treatment of bed rest alone, in stark contrast to centuries of unanswered questions on previous tuberculosis treatments (10). Sir Austin Bradford Hill, the statistical champion for this study, is credited as being the most instrumental individual in fostering use of the randomized controlled clinical trial (11). Since then, use of controlled trials has resulted in numerous treatment breakthroughs, such as the beautifully executed Salk vaccine trial for prevention of polio (12). Perhaps of equal or greater importance, controlled trials have also proven the lack of effectiveness of purported breakthroughs, such as the Cardiac Arrhythmia Suppression Trial (CAST) demonstrating that encainide and flecainide increased the death rate in patients with recent myocardial infarction instead of decreasing it, despite the well-documented suppression of ventricular arrhythmia by these agents (13). Use of proper control groups continues to be a critical principle in clinical research methods. 2

ETHICS

Ethical considerations with respect to experiments using control groups and experimental treatments center on the tension between care of individual patients and the need to study treatment effectiveness. The welfare of the individual patient should not be compromised by inclusion in a clinical research trial. On the other hand, the welfare of patients in general depends on identification of new effective treatments and discarding ineffective treatments. We often do not know whether the welfare of the individual is being compromised or enhanced, thus treatment without knowledge also raises ethical issues—consider the CAST study example of unknowingly doing harm instead of good. International guidance for ethics in medical research such as the Declaration of Helsinki (14) emphasizes protection of health and rights of patients. The Helsinki declaration acknowledges the physicians’ responsibilities to individual subjects: ‘‘considerations related to the well-being of the human subject should take precedence over the interests of science and society’’ and ‘‘it is the duty

of the physician in medical research to protect the life, health, privacy, and dignity of the human subject.’’ It also acknowledges responsibilities to patients as a whole: ‘‘it is the duty of the physician to safeguard the health of the people,’’ ‘‘medical progress is based on research which ultimately must rest in part on experimentation involving human subjects,’’ and ‘‘the benefits, risks, burdens and effectiveness of a new method should be tested against those of the best current prophylactic, diagnostic, and therapeutic methods.’’ The U.S. Belmont report (15) identified three major ethical principles: respect for persons (requiring informed and voluntary consent), beneficence (requiring that benefits outweigh risks plus careful study monitoring), and justice (requiring equitable selection of subjects and protection of vulnerable subjects). The main principle related to use of a control group is beneficence. Items from the Declaration of Helsinki that help inform decisions related to use of a control group include a requirement for careful assessment of risk–benefit, agreement that risks have been assessed and can be adequately managed, agreement that the importance of the objective outweighs the inherent risks to individuals, and a reasonable likelihood that populations in which the research is carried out stand to benefit from the results. Key considerations in the type of control group to be used are severity of disease and availability of effective treatments. If the disease is severe and there is an effective treatment, for instance, a no-treatment or placebo-alone arm on a randomized trial will not be an appropriate control as the risks to the individual are too high. The Declaration of Helsinki explicitly states that use of placebos or no-treatment are not precluded although ‘‘extreme care’’ must be taken in the decision to employ these. The current wording on this point is controversial, and the implications are perhaps too restrictive (16). Another key to ethical use of a control group is equipoise, the acknowledgment that usefulness of the new treatment compared with the control treatment is not yet known. Individual investigators may hold strong opinions and decline to participate in a trial,

CONTROL GROUPS

but the general sense in the scientific community should be one of uncertainty. In the CAST study, for example, there was considerable controversy in doing a randomized controlled trial as many investigators incorrectly believed that the striking arrhythmia evidence made a control arm unethical. Fortunately, the level of uncertainty in the scientific community allowed the trial to proceed, thereby sparing future patients a harmful treatment. An interesting illustration of the issues in using placebo-treated control groups was a double-blind transplantation trial in patients with Parkinson’s disease that used the standard treatment plus sham surgery as the control arm (17). The sham surgery included general anesthesia, a scalp incision, a partial burr hole, antibiotics and cyclosporine, and positron emission tomography (PET) studies, all of which entailed some risk to the patient. Because outcomes in Parkinson’s disease are subjective, use of a placebo-treated control group was the best way to assess the effectiveness of new therapy. The question was important, there was no adequate alternative therapy, the new treatment held promise but was uncertain, and future patients would benefit. Thus, the suitability of placebo use centered on whether the risk–benefit ratio for patients treated with placebo was acceptable. The potential benefits were the contribution to science, the no-cost standard medical treatment provided as part of the study, and later transplant at no cost if the treatment was found to be beneficial. As well, these patients were spared the additional risks of the transplant if it was found not to be beneficial. The risks included the possibility of injury or death due to the sham procedure and the inconvenience and discomfort of an extensive procedure with no possible benefit. As presented in a New England Journal of Medicine sounding board (18, 19), the case illustrates how assessment of levels of risk and benefit is not always clear cut, and how reasonable people may disagree on whether the ratio, and therefore the trial, is acceptable.

3

3 TYPES OF CONTROL GROUPS: HISTORICAL CONTROLS A comparison of control versus experimental groups will indicate whether outcomes are different in the experimental group, but without randomization causality cannot be assumed. Any difference in the groups (or lack of difference) may be due to factors other than treatment. Any particular choice of control group, no matter how carefully chosen, will be systematically different from the experimental group in many (often unknown or unmeasurable) ways due to the systematic reasons patients are chosen for treatment. Many factors contribute to a choice of treatment, and many of these factors are related to outcome; to the extent that the experimental group has better or worse prognosis than the control group, the comparison will be biased. Biases can make the test treatment appear either more or less effective it actually is. Because of potential toxicities, for instance, investigators may be inclined to include only patients who are relatively healthy; in other circumstances, they may choose only patients who are too compromised for other options. For the first case, an ineffective treatment might appear to be an improvement over historical treatment. For the second, an effective treatment might appear ineffective. If a historically controlled trial is to be done, the patient population for the control group should be as similar as possible to the population for the experimental group. This includes similar general health status, similar general medical care, and use of the same diagnostic and screening procedures. Also, the primary outcome should be defined and assessed the same way in both groups and be objective so that results from the experimental group have the same interpretation as for results from the historical control group. Results of such trials must always be interpreted cautiously because of the potential for bias. 3.1 Historical Control from the Medical Literature Historically controlled trials are usually single-arm trials without specific control groups. Rather, these trials are controlled

4

CONTROL GROUPS

in the sense that statistical hypotheses are based on estimates from the literature: instead of being compared with a specific set of patients, patients on the new treatment are assessed against a fixed value. For instance, if the success rate of the standard treatment in the literature is consistently 20%, a test may be done to ascertain whether the percentage of patients with success on the new treatment is significantly greater than 20%. Or if a particular time to event distribution is hypothesized, a 1-sample logrank test (20) might be used to test superiority. Such trials are conducted for reasons such as a severely limited patient population or a need for preliminary assessment before planning a large definitive trial. This approach works best if historical estimates are well characterized and are stable over time with low variability. This may be the case for uniformly nonresponsive disease with no effective treatment, resulting in low variability in patient outcomes. There should also be no recent treatment improvements, changes in staging definitions, improvements in diagnostic procedures, or changes in the way primary outcome is assessed so that results will not be confounded by temporal changes. Of course, definitions, treatments, and methods typically do not remain stable over time, and uniformity is not common, so historical estimates for the same treatment in ostensibly the same patient population may vary substantially. It is often difficult to ascertain which estimates may be appropriate for assessing a new treatment. Considering the high potential for bias of unknown magnitude or direction, in most circumstances a single-arm trial will not provide a definitive answer to the question of whether the new treatment is an improvement over the standard. In the past, the probability of success for various types of cancer was uniformly dismal, so this setting provides an example of feasible use of this approach. With recent treatment and diagnostic advances, old assumptions no longer hold, so the approach is becoming less informative. Other approaches to controlling early studies of new regimens in

cancer are becoming more common as variability in treatment outcome increases and as previously stable historical rates improve. 3.2 Specific Historical Control Groups Trials with control groups consisting of a specific set of past patients are sometimes designed. For this type of trial, individual patient data from the control group are used for comparison with patients on the test treatment. Such trials may be justified when the probability of success and expected benefit of therapy are both considered to be so high that ethical considerations dictate a current control arm would not be in the best interests of patients. A limited patient population may also be a reason to design a historically controlled trial. A carefully chosen specific control group potentially will be better than using an estimate from the literature but will still be subject to selection biases. Known prognostic factors should be available for use in the analysis of this type of trial, and only large differences after suitable adjustment for prognostic factors should be accepted as evidence of improvement due to experimental therapy. An example illustrating how difficult it is to select a suitable specific historical control is provided by a sequence of four Southwest Oncology Group studies in multiple myeloma. The trials had the same eligibility criteria, and largely the same institutions participating on the trials. The overall survival estimates by study were remarkably similar across all four trials. There were no significant improvements demonstrated for any of the test treatments. This stability over time would suggest that use of a prior control arm for assessing the next treatment might be reasonable. However, when the identically treated control arms for each these four trials were compared, the results appeared different, and the test of differences was suggestive (P = 0.07) of heterogeneity among the arms (21). Comparison of a new regimen with the worst of these control arms could have resulted in a spurious positive result. If comparability cannot be counted on in ideal cases such as this, then it is unlikely to apply in less ideal cases. One should assume that there will be systematic differences between the

CONTROL GROUPS

chosen control group and the test treatment no matter how carefully they are chosen. The myeloma trials also provide an example of how poorly chosen historical control groups can mislead. A specific comparison of a high-dose therapy regimen with one of the trial arms indicated a striking benefit due to the experimental regimen. It is tempting to conclude in such cases that not all of the difference could be due to bias. However, patients must be in better health to receive a high-dose regimen than to receive the standard treatment. When the comparison was restricted to younger patients with good renal function, a much smaller difference remained (21). A randomized trial was deemed necessary, and it did not demonstrate a benefit due to high-dose therapy, despite the initial promising comparison (22). It should be noted that design considerations are different when a retrospective control group is used instead of a prospective control. In this case, sample size considerations are a function of the results of the control group, and they may also account for covariate adjustment (23–25). 4 TYPES OF CONTROL GROUPS: RANDOMIZED CONTROLS A control versus experimental comparison allows an assessment of whether outcomes are different in the experimental group. Randomization allows the possibility of concluding that the difference is actually caused by the test treatment, due to the elimination of biases in treatment assignment. Although randomization is necessary, it may not be sufficient for attributing differences to the test treatment. A poorly conducted randomized trial may still result in substantial bias. Some potential sources of bias in the comparison of groups are structural and can be avoided relatively easily. For instance, outcome assessment schedules should be the same for both the control group and the test group. If, for instance, time to failure is assessed less frequently in one group, then results will be biased in favor of this group. Methods of assessment should also be the same. If the method of assessment for failure in one group is more sensitive, then the

5

results will be biased against this group. Criteria for inclusion in the analysis should also be the same. If information is available in one group but not the other, then this information should not be used to exclude patients from analysis. For example, disease information in a surgical test group might identify patients unlikely to benefit from treatment, but these cannot be excluded from analysis because similarly unsuitable patients from the control group will not be excluded. Other sources of bias are not so easily eliminated. To the extent these differ according to treatment group, the treatment comparison is compromised. For instance, patients in the control group may be less compliant with trial requirements than those in the test group or may drop out of the trial all together, potentially resulting in worse than expected outcome. Or if the test treatment includes over-the-counter agents, patients in the control group may treat themselves with the agents, potentially resulting in a better than expected outcome. Subjective outcomes are particularly problematic because assessments are easily influenced by knowledge of the treatment assignment. Investigators may overestimate improvement in the test group in anticipation or hope of benefit from a promising new treatment, or adverse events may be more likely to be attributed to treatment in the experimental group compared with the control group. Patients are also subject to the wellknown placebo effect. A proportion of patients will report improvement in subjective disease symptoms or experience of treatment side effects whether or not the treatment is active (26). For example, trials of venlafaxine in sexual dysfunction (27), hot flashes (28), panic disorder (29), generalized anxiety disorder (30), and migraine pain (31) all noted improvements with placebo, sometimes less than the effect of venlafaxine, sometimes not. Or consider an antiemetic trial of placebo versus prochlorperazine versus tetrahydrocannabinol (THC, the active marijuana component). In this trial, sedation side effects of treatment were reported in 46% of placebo patients, and ‘‘highs’’ were reported in 12% of prochlorperazine patients (32), presumably due in part to anticipation of being on

6

CONTROL GROUPS

THC. In a study for which the placebo effect occurs mainly in the experimental group, an improvement in the experimental group may be observed even when the test treatment is inactive. 4.1 Untreated Control Group, Randomized Trial The category of untreated controls includes control groups for which patients receive the same palliative care or routine monitoring as patients in the experimental group, but the control group receives no additional treatment while the experimental group receives the test treatment. A trial with this type of control group has potential for all of the biases previously discussed. Problems may be particularly acute in this type of trial because of the no-treatment control. For instance, patients may have little motivation to return to the clinic for routine visits, so outcomes may not be carefully collected. As well, the placebo effect will occur only in the experimental group. 4.2 Standard Treatment Control Group, Add-On Randomized Trial When not treating is considered unethical or unfeasible, one option may be to treat both the control group and the experimental group with standard treatment and to add the test treatment in the experimental group. Bias issues are similar to those for an untreated control group. Use of this type of control group allows for assessment of improvement over standard treatment due to the test treatment. However, conclusions cannot be made concerning the usefulness of the test treatment alone. Because of potential treatment synergy or inhibition, improvement over the standard treatment does not prove singleagent activity, and lack of improvement over the standard treatment does not disprove single-agent activity. 4.3 Placebo-Treated Control Group, Randomized Trial The placebo-treated control category includes control groups for which patients receive the same palliative treatment or routine monitoring as the experimental group plus a placebo

while the experimental group receives the test treatment. Patients are given identical appearing treatments in each treatment arm to mask knowledge of the assigned treatment. Blinded placebo-controlled trials are done to reduce the potential for biases related to knowledge of the treatment assignment. For patients, compliance is enhanced, and supplemental treatments, while not necessarily eliminated, should at least be balanced across the groups. For investigators, the objectivity of outcome assessments is improved. In addition, use of a placebo will control for the placebo effect. Both groups will experience the placebo effect, so differences between the groups will be due to the active treatment. Sometimes only patients are blinded (single blind), but preferably both patients and investigators will be blinded (double blind) to avoid bias from both sources. Although it would seem best always to have a placebo-treated control group, it may not always be practical or ethical. Blinded placebo-controlled trials are resource intensive. Significant time and money are needed for manufacturing of the placebo; for labeling, shipping, and tracking the coded placebos and active agents; for setting up mechanisms for distribution with pharmacies; and for arranging for emergency unblinding. If the outcomes are objective and other problems associated with knowledge of the treatment assignment are anticipated to be minor, it may be concluded unnecessary to blind. In other cases, it may be impossible to blind, such as when a treatment has a distinctive side effect that cannot be reproduced in a placebo. In yet other cases, the placebo treatment may entail too much risk to the patient. Sham surgeries are particularly controversial, as already noted. 4.4 Placebo-Treated Control Group, Add-On Randomized Trial For an add-on randomized trial, patients in the control group can receive standard treatment and a placebo that appears identical to the test treatment while patients in the experimental group receive the standard plus the test treatment. The issues are the same as for a placebo-treated control group. Again, use of this type of control group allows for

CONTROL GROUPS

assessment of improvement over the standard treatment due to the test treatment, but it would not address the usefulness of the test treatment alone. 4.5 Active Control Group When no-treatment or placebo-only are not appropriate and when adding the test treatment to the standard is not of interest, then an active control trial may be done. The control group in this type of trial receives standard treatment while the experimental group receives the test treatment. The test treatment in this case may be a single agent, a combination of therapies, different ways of administering treatment, new schedules, or other variations that cannot be described simply as standard plus new. Double placebos are sometimes used to mask treatment assignment for these trials. Risks and benefits of forgoing proven useful treatment must be assessed carefully in this setting. Such trials are appropriate when standard treatment is useful but not fully effective and a delay in receiving standard treatment is acceptable. The aim of an active control trial may be to show superiority of the test treatment or to show equivalence. A superiority trial will allow for assessment of the difference between control and the test treatment. However, unlike an add-on trial, if the test treatment is a combination of therapies, the cause of any difference will likely not be attributable to a particular component of the combination. Equivalence trials aim to show that the test treatment is as good as the standard. ‘‘As good as’’ is difficult to establish. Equivalence trials need both active control groups and historical information because results of the standard versus test comparison must not only demonstrate that the test treatment is similar to control, but also that it would have been superior to placebo had the trial been placebo controlled. Design considerations for such trials are complex, and they are the subject of ongoing research and discussion (33–35). Lack of accurate estimates of the benefit of active control compared with placebo is a common challenge in this setting. As for other historical control settings, the assumption of no change over time is particularly troublesome.

7

4.6 Multiple Control Groups In some studies, more than one control group may be used. When appropriate, using both a placebo-only control group and an active control group can be useful for addressing both the absolute effect of a new treatment and the effect relative to the standard treatment. It may also provide information on why a trial fails to show usefulness of a new treatment. If the standard group also fails to show improvement compared with the placebo, the failure may be due to the trial rather than to lack of efficacy of the new treatment. 5

CONCLUSION

Choice of the control group in a clinical trial is critical to the success of the trial. A poor choice may result in biases that severely compromise interpretation of results. A blinded randomized trial with a placebo-treated control group will provide the most definitive evidence concerning the usefulness of a new treatment. If it is not possible to conduct this type of trial, the best control group under the circumstances should be used. The trial should be randomized if feasible. For any choice, every effort should be made to ensure the control group is as similar as possible to the experimental group. Reliability of conclusions about new treatments depends on it! REFERENCES 1. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E10 Choice of Control Group and Related Issues in Clinical Trials. Current Step 4 version, July 20, 2000. Available at: http://www.ich.org/LOB/media/ MEDIA486.pdf 2. J. P. Boissel, Impact of randomized clinical trial on medical practices. Control Clin Trials. 1989; 10: 120S–134S. 3. C. L. Bernard, Introduction a` l’Etude de la M´edecine Exp´erimentale. First published in 1866. London: Garnier-Flammarion, 1966. 4. D. DeMoulin, A Short History of Breast Cancer. Dorderct, Germany: Klewer, 1989.

8

CONTROL GROUPS 5. D. Freedman, From association to causation: some remarks on the history of statistics. Stat Sci. 1999; 14: 243–258.

21. S. Green, J. Benedetti, and J. Crowley, Clinical Trials in Oncology. Boca Raton, FL: Chapman and Hall/CRC Press, 2003.

6. H.O. Lancaster, Quantitative Methods in Biological and Medical Sciences. New York: Springer-Verlag, 1994.

22. B. Barlogie, R. Kyle, K. Anderson, P. Greipp, H. Lazarus, et al., Standard chemotherapy compared with high-dose chemoradiotherapy for multiple myeloma: final results of phase III US Intergroup Trial S9321. J Clin Oncol. 2006; 24: 929–936. 23. R. Makuch and R. Simon, Sample size consideration for non-randomized comparative studies. J Chronic Dis. 1980; 33: 175–181.

7. C. Stuart and D. Guthrie, eds., Lind’s Treatise on Scurvy. Edinburgh: University Press, 1953. 8. G. Cook, Scurvy in the British Mercantile Marine in the 19th century, and the contribution of the Seamen’s Hospital Society. Postgrad Med J. 2004; 80: 224–229. 9. D. Thomas, Sailors, scurvy and science. J Roy Soc Med. 1997; 90: 50–54. 10. W. Silverman, Doctoring: From art to engineering. Control Clin Trials. 1992; 13: 97–99. 11. W. Silverman and I. Chalmers, Sir Austin Bradford Hill: an appreciation. Control Clin Trials. 1991; 10: 1–10. 12. J. Smith, Patenting the Sun: Polio and the Salk Vaccine Trial. 1990. New York: William and Morrow, 1990. 13. D. Echt, P. Liebson, L. Mitchell, R. Peters, D. Obias-Manno, et al., and the CAST investigators. Mortality and morbidity in patients receiving encainide, flecainide or placebo: the Cardiac Arrhythmia Suppression Trial. N Engl J Med. 1991; 324: 781–788. 14. World Medical Association General Assembly. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. J Int Bioethique. 2004; 15: 124–129. 15. Protection of human subjects; Belmont Report: notice of report for public comment. Fed Regist. 1979; 44: 23191–23197. 16. S. Piantadosi, Clinical Trials: A Methodologic Approach, 2nd ed. New York: Wiley, 2005. 17. C. W. Olanow, C. Goetz, J. Kordower, A. J. Stoessl, V. Sossi, et al., A double-blind controlled trial of bilateral fetal nigral transplantation in Parkinson’s disease. Ann Neurol. 2003; 54: 403–414. 18. T. Freeman, D. Vawtner, P. Leaverton, J. Godbold, R. Hauser, et al., Use of placebo surgery in controlled trials of a cellular based therapy for Parkinson’s disease. N Engl J Med. 1999; 341: 988–992. 19. R. Macklin, The ethical problems with sham surgery in clinical research. N Engl J Med. 1999; 341: 992–996. 20. R. Woolson, Rank tests and a one-sample logrank test for comparing observed survival data to a standard population. Biometrics. 1981; 37: 687–696.

24. D. Dixon and R. Simon, Sample size consideration for studies comparing survival curves using historical controls. J Clin Epidemiol. 1988; 41: 1209–1213. 25. J. O’Malley, S. L. Normand, and R. Kuntz, Sample size calculation for a historically controlled clinical trial with adjustment for covariates. J Biopharm Stat. 2002; 12: 227–247. 26. A. Shapiro and K. Shapiro, The Powerful Placebo: From Ancient Priest to Modern Physician. Baltimore: Johns Hopkins University Press, 1997. 27. S. Kilic, H. Ergin, and Y. Baydinc, Venlafaxine extended release for the treatment of patients with premature ejaculation: a pilot, single-blind, placebo-controlled, fixeddose crossover study on short-term administration of an antidepressant drug. Int J Androl. 2005; 28: 47–52. 28. M. Evans, E. Pritts, E. Vittinghoff, K. McClish, K. Morgan, and R. Jaffe, Management of postmenopausal hot flushes with venlafaxine hydrochloride: a randomized, controlled trial. Obstet Gynecol. 2005; 105: 161–166. 29. M. Pollack, U. Lepola, H. Koponen, N. Simon, J. Worthington, et al., A double-blind study of the efficacy of venlafaxine extended-release, paroxetine, and placebo in the treatment of panic disorder. Depress Anxiety. 2006; 2007; 24: 1–14. 30. A. Gelenberg, R. B. Lydiard, R. Rudolph, L. Aguiar, J. T. Haskins, and E. Salinas, Efficacy of venlafaxine extended-release capsules in nondepresssed outpatients with generalized anxiety disorder: a 6-month randomized controlled trial. JAMA. 2000; 283: 3082–3088. 31. S. Ozyalcin, G. K. Talu, E. Kiziltan, B. Yucel, M. Ertas, and R. Disci, The efficacy and safety of venlafaxine in the prophylaxis of migraine. Headache. 2005; 45: 144–152. 32. S. Frytak, C. Moertel, J. O’Fallon, J. Rubin, E. Cregan, et al., Delta-9-Tetrahydrocannabinol

CONTROL GROUPS as an antiemetic for patients receiving cancer chemotherapy. Ann Int Med. 1979; 91: 825–830. 33. R. D’Augustino, J. Massaro, and L. Sullivan, Non-inferiority trials: design concepts and issues—the encounters of academic consultants in statistics. Stat Med. 2003; 22: 169–186. 34. S. Durrleman and P. Chaikin, The use of putative placebo in active control trials: two applications in a regulatory setting. Stat Med. 2003; 22: 941–952. 35. Y. Tsong, S. J. Wang, H. M. Hung, and L. Cui, Statistical issues on objective, design and analysis of non-inferiority activecontrolled clinical trial. J Biopharm Stat. 2003; 13: 29–41.

CROSS-REFERENCES Adequate and well-controlled trial Bellmont Report Historical control Placebo-controlled trial Active-controlled trial

9

COOPERATIVE NORTH SCANDINAVIAN ENALAPRIL SURVIVAL STUDY

2

CONSENSUS was a randomized, doubleblind, placebo-controlled, parallel-group trial (2). To be eligible, patients had to have the most severe form of heart failure, New York Heart Association (NYHA) class IV, which means that the symptoms of heart failure were present at rest. Patients were receiving optimal treatment for heart failure at the start of the trial, including digitalis and diuretics, and continued to receive these treatments during the trial. In addition, patients were randomized to receive either enalapril or placebo. The starting dose was 5 mg twice a day, and could be titrated up to 20 mg twice a day, depending on clinical response. Note that early in the trial the occurrence of symptomatic hypotension in some patients led the investigators to reduce the starting dose to 2.5 mg daily for some high-risk patients. The primary endpoint of the trial was death by any cause within 6 months of randomization. Secondary endpoints included 12-month mortality and mortality during the entire trial period. The sample size was calculated to be 400 patients, 200 per treatment group, based on the assumption that the 6month mortality rate would be 40% in the placebo group, and would be reduced to 24% by enalapril. This sample size provides 90% power at a two-sided significance level of 5%. Differences in mortality between treatment groups were to be analyzed using life-table methods, and the analysis was to be by intention-to-treat; that is, the survival information for each patient from the date of randomization to the date of death or study termination was to be included in the analysis. Although the trial was to be monitored by an ethical review committee, there was no formal rule governing the committee’s decisions. As its name suggests, CONSENSUS took place in three countries in the north of Scandinavia: Finland, Norway, and Sweden (see also Multinational [Global] Trial). The CONSENSUS Trial Study group consisted of the set of investigators in these three countries, a steering committee chaired by John Kjekshus, an ethical review committee (some-

STEVEN M. SNAPINN Biostatistics, Amgen, Inc. Thousand Oaks, California

Heart failure, also known as congestive heart failure, is a condition that occurs when the heart is unable to pump sufficient blood to meet the body’s oxygen demand. Signs and symptoms include dyspnea, or shortness of breath, particularly when lying down; edema, or the buildup of fluid; and cough. It is a very serious condition and, in its most severe form, is associated with a high mortality rate. Heart failure can be caused by a number of factors such as hypertension, ischemic heart disease, and cardiomyopathy. As of the mid-1980s, the standard treatment for heart failure usually included digitalis and diuretics. There was also growing evidence that direct-acting vasodilator therapy would be beneficial. However, a metaanalysis by Furberg and Yusuf (1) found little evidence for improved survival with this therapy, and instead suggested that angiotensinconverting enzyme (ACE) inhibition (see also the article on ACE inhibitors) held the most promise. This relatively new class of drugs, including enalapril, captopril, and lisinopril, was known to be effective in the treatment of hypertension and had been associated with symptomatic improvement in patients with heart failure. However, the effects of these agents on survival was unknown. 1

STUDY DESIGN

OBJECTIVES

The primary objective of the Cooperative North Scandinavian Enalapril Survival Study, also known as CONSENSUS, was to study the effect on mortality of enalapril compared with placebo, when added to conventional therapy, in patients with severe heart failure (2). Other objectives included evaluating the safety of enalapril, its effect on symptoms of heart failure, and its effects on neurohormones known to be associated with mortality in these patients.

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

COOPERATIVE NORTH SCANDINAVIAN ENALAPRIL SURVIVAL STUDY

times referred to in other trials as a data and safety monitoring board or a data monitoring committee) chaired by Jacobus Lubsen, and an administrative office run by the study’s sponsor, Merck Research Laboratories. 3

RESULTS

3.1 Interim Analysis Results CONSENSUS terminated ahead of schedule based on a recommendation from the ethical review committee (2). Their opinion was that the results favoring enalapril were so strong that continuation of the trial would be of limited scientific interest and could not be justified from an ethics perspective. An editorial in the Lancet (3) commented that, while correct, the decision to stop CONSENSUS was highly unusual, and that it is desirable for trials to run their entire course. This is especially true given that CONSENSUS had no formal stopping rule. Lubsen (4) provided a detailed review of the ethical review committee’s deliberations. Table (1) contains the interim mortality results reviewed by the committee. The members were informed by telephone of the June 27, 1986, results and agreed to a meeting to discuss them. At the meeting, held on September 14, 1986, the sponsor provided a more current update: the 6-month mortality rates were now 24 and 49% (P = 0.0002). Despite this large difference, the committee felt that they needed additional information from the sponsor on patient characteristics. This required a large data entry effort, and the information was not available until December 7, 1986. At this meeting, the 6-month mortality rates were 27 and 48%

(P = 0.001), and a review of the baseline characteristics showed balance between groups and a consistent treatment effect among important subgroups. Therefore, the committee decided to recommend termination of the trial, and they authorized the committee chair to inform the steering committee. This took place on December 13, 1986; the steering committee accepted this recommendation, set December 14, 1986, as the study termination date, and informed the clinical centers. 3.2 Final Analysis Results The final results of CONSENSUS were reported in 1987 (2) and 1988 (5). A total of 253 patients were randomized, 127 to enalapril and 128 to placebo. The mean age was approximately 70 years, approximately 70% of the patients were men, and most patients had an etiology of ischemic heart disease. The median duration of heart failure before enrollment was approximately 48 months. The final mortality results are summarized in Table (2) and displayed in Figure (1), which shows the Kaplan-Meier curves for overall mortality (see also the article on Kaplan-Meier plots). Overall, 68 placebo patients (54%) had died, compared with 50 enalapril patients (39%, P = 0.003 using lifetable methods). The benefit of enalapril was restricted to progression of heart failure: 44 placebo patients and 22 enalapril patients died of this cause, and 24 placebo patients and 28 enalapril patients died of other causes (primarily sudden cardiac death). Patients treated with enalapril also experienced symptomatic benefit. In the enalapril group, 54 surviving patients had an endof-study NYHA class of between I and III,

Table 1. Interim Mortality Results Enalapril Status date January 1, 1986 April 1, 1986 May 1, 1986 June 27, 1986 September 14, 1986 December 7, 1986

Placebo

No. of deaths

No. randomized

No. of deaths

No. randomized

8 16 20 26 28 44

52 77 84 100 101 124

15 24 31 44 52 66

51 72 78 93 99 120

COOPERATIVE NORTH SCANDINAVIAN ENALAPRIL SURVIVAL STUDY

3

Table 2. Final Mortality Results

Mortality within 6 months Mortality within 1 year Total mortality

Placebo (n = 126) No. %

Enalapril (n = 127) No. %

55 66 68

33 46 50

44 52 54

Reduction in relative risk risk (%)

P-value (life-table analysis)

40 31 27

0.002 0.001 0.003

26 36 39

Cumulative Probability of Death

0.8 0.7 0.6 0.5 0.4 0.3 0.2 Placebo Enalapril

0.1

0

1

2

3

4

5

6 7 Months

8

9

10

11

12

Placebo N = 126 102

78

63

59

53

47

42

34

30

24

18

17

Enalapril N = 127 111

98

88

82

79

73

64

59

49

42

31

26

Figure 1. Kaplan-Meier curves comparing total mortality for enalapril and placebo.

including 16 patients with class I or II; in the placebo group, there were only 27 surviving patients with an end-of-study NYHA class of between I and III and only two patients in class I or II. Enalapril was very welltolerated; the major side effect associated with enalapril was hypotension. 3.3 Neurohormone Analysis CONSENSUS provided a wealth of information on neurohormones. Blood samples were obtained from most randomized patients at the time of randomization and after 6 weeks of treatment. The first set of publications focused on the following neurohormones: angiotensin-converting enzyme, angiotensin II, aldosterone, noradrenaline, adrenaline, dopamine, and atrial natriuretic factor (6, 7). When looking at the placebo group, there were strong and statistically significant

associations between several of these neurohormones and mortality: angiotensin II (P < 0.05), aldosterone (P = 0.003), noradrenaline (P < 0.001), adrenaline (P = 0.001), and atrial natriuretic factor (P = 0.003). However, similar associations were not seen in the enalapril group. In addition, the mortality benefit of enalapril appeared stronger among patients with values of these neurohormones above the median. These results suggested that the effect of enalapril on mortality is related to hormonal activation, and to activation of the renin-angiotensin system in particular. Subsequent publications focused on the novel neurohormones N-terminal proatrial natriuretic factor ANF(1-98) and atrial natriuretic peptides ANP(1-98) and ANP(99-126) (8, 9). It was concluded that the magnitude of changes in these neurohormones provide important information on prognosis and therapeutic effects of enalapril.

4

COOPERATIVE NORTH SCANDINAVIAN ENALAPRIL SURVIVAL STUDY

3.4 Long-Term Follow-Up When CONSENSUS terminated in December of 1986, all surviving patients were removed from blinded study medication but were informed of the reason for terminating the trial and given the option of taking openlabel enalapril. In addition, patients continued to be followed for their survival status. The follow-up mortality data were analyzed in two ways. First, the follow-up data were analyzed as if the surviving patients were randomized (using the original randomization schedule) into a new study beginning on the date the blinded portion of the trial terminated, December 15, 1986. Second, the data from the blinded portion of the trial were combined with the follow-up data, as if the trial had never stopped. The first report of additional follow-up came 8.5 months after the end of blinded therapy (10). There were 58 surviving patients from the placebo group of whom 18 (31%) died during the follow-up period, and 77 surviving patients from the enalapril group of whom 16 (21%) died during the follow-up period. The next report included 2-year follow-up information (11). By the end of this period, 26 patients from the original placebo group were still alive, compared with 38 patients from the original enalapril group. The final follow-up publication was based on a 10-year follow-up period (12). By the end of the 10-year follow-up period, only five patients were still alive, all of whom had been in the enalapril group during the blinded portion of the trial. This analysis showed that, despite the fact that enalapril was made available to all surviving patients at the termination of the trial, the benefit accrued to patients randomized to enalapril persisted for at least 3.5 years after termination. 4

failure and the adoption of ACE inhibitors as a standard treatment for these patients. CONSENSUS was the first heart failure study to show dramatic benefits from ACE inhibition (14, 15). However, CONSENSUS was conducted in a population with severe heart failure and did not answer the question of whether this treatment was beneficial in patients with milder forms of the disease. Subsequent to CONSENSUS, two major trials evaluated the effects of enalapril in patients with NYHA class II–III heart failure. The second Veterans Administration Cooperative Vasodilator-Heart Failure Study (VHeFT-II) compared enalapril with a vasodilator regimen in 804 men. Over a 5years of follow-up period, mortality was consistently lower in the enalapril group, but not significantly so (P = 0.08). The Studies of Left Ventricular Dysfunction (SOLVD) Treatment trial compared enalapril and placebo with respect to total mortality in 2569 patients. Cumulative 4-year all-cause mortality among patients randomized to enalapril was 16% lower than in the placebo group (P = 0.004). As a result of these trials, ACE inhibition is now standard treatment for NYHA class II–IV heart failure. Note that there was a later trial with a similar name: Cooperative New Scandinavian Enalapril Survival Study (CONSENSUS II). Although this trial did involve the same experimental treatment, enalapril, it was conducted in patients with an acute myocardial infarction rather than in patients with heart failure. In addition, the letter ‘‘N’’ in the acronym refers to ‘‘New’’ rather than ‘‘North,’’ a reference to the fact that CONSENSUS II included the three countries that were part of CONSENSUS plus Denmark. Therefore, to avoid confusion with CONSENSUS II, CONSENSUS is now sometimes referred to as CONSENSUS I.

CONCLUSIONS

CONSENSUS was a milestone for treatment of patients with heart failure (13) and a landmark trial in the history of cardiovascular clinical research (see also the article on disease trials for cardiovascular diseases). The highly significant results and the magnitude of the clinical benefit led to the approval of enalapril for the treatment of severe heart

REFERENCES 1. C. D. Furberg and S. Yusuf, Effect of drug therapy on survival in chronic congestive heart failure. Am J Cardiol. 1988; 62: 41A–45A. 2. CONSENSUS Trial Study Group. Effects of enalapril on mortality in severe congestive heart failure: results of the Cooperative North Scandinavian Enalapril Survival Study

COOPERATIVE NORTH SCANDINAVIAN ENALAPRIL SURVIVAL STUDY (CONSENSUS). N Engl J Med. 1987; 316: 1429–1435. 3. Lancet [editorial]. Consensus on heart failure management? Lancet. 1987; 330: 311–312. 4. J. Lubsen, for the CONSENSUS Ethical Review Committee. Appendix: monitoring methods, considerations, and statement of the Cooperative North Scandinavian Enalapril Survival Study (CONSENSUS) Ethical Review Committee. Am J Cardiol. 1988; 62: 73A–74A. 5. K. Swedberg and J. Kjekshus, Effects of enalapril on mortality in severe congestive heart failure: results of the Cooperative North Scandinavian Enalapril Survival Study (CONSENSUS). Am J Cardiol. 1988; 62: 60A–66A. 6. K. Swedberg, P. Eneroth, J. Kjekshus, and S. Snapinn, for the CONSENSUS Trial Study Group. Effects of enalapril and neuroendocrine activation on prognosis in Severe Congestive Heart Failure (follow-up of the CONSENSUS Trial). Am J Cardiol. 1990; 66: 40D–45D. 7. K. Swedberg, P. Eneroth, J. Kjekshus, L. Wilhelmsen, for the CONSENSUS Trial Study Group. Hormones regulating cardiovascular function in patients with severe congestive heart failure and their relation to mortality. Circulation. 1990; 82: 1730–1736. 8. C. Hall, J. Kjekshus, P. Eneroth, and S. Snapinn, The plasma concentration of Nterminal proatrial natriuretic factor ANF(198) is related to prognosis in severe heart failure. Clin Cardiol. 1994; 17: 191–195. 9. S. V. Eriksson, K. Caidahl, C. Hall, P. Eneroth, J. Kjekshus, et al. Atrial natriuretic petptide ANP(1-98) and ANP(99-126) in patients with severe chronic congestive heart failure: relation to echocardiographic measurements. A subgroup analysis from the Cooperative North Scandinavian Enalapril Survival Study (CONSENSUS). J Cardiac Failure. 1995; 1: 109–116.

5

10. K. Swedberg and J. Kjekshus. Effect of enalapril on mortality in congestive heart failure: follow-up survival data from the CONSENSUS Trial. Drugs. 1990; 39(Suppl 4): 49–52. 11. J. Kjekshus, K. Swedberg, and S. Snapinn, for the CONSENSUS Trial Group. Effects of enalapril on long-term mortality in severe congestive heart failure. Am J Cardiol. 1992; 69: 103–107. 12. K. Swedberg, J. Kjekshus, and S. Snapinn, for the CONSENSUS Investigators. Long-term survival in severe heart failure in patients treated with enalapril: ten year follow-up of CONSENSUS I. Eur Heart J 1999; 20: 136–139. 13. G. A. J. Riegger, Lessons from recent randomized controlled trials for the management of congestive heart failure. Am J Cardiol. 1993; 71: 38E–40E. 14. W. B. Hood, Role of converting enzyme inhibitors in the treatment of heart failure. J Am Coll Cardiol. 1993; 22(Suppl A): 154A–157A. 15. J. B. Young, Angiotensin-converting enzyme inhibitors in heart failure: new strategies justified by recent clinical trials. Int J Cardiol. 1994; 43: 151–163. 16. P. Sleight, Angiotensin II and trials of cardiovascular outcomes. Am J Cardiol. 2002; 89(Suppl): 11A–17A.

CROSS-REFERENCES Multinational (global) trial Kaplan-Meier plot Data and safety monitoring board Angiotensin-converting enzyme (ACE) inhibitors Disease trials for cardiovascular diseases

Cooperative Studies Program, US Department of Veterans Affairs The Department of Veterans Affairs (VA) is in a unique position in the US, and perhaps the world, in conducting multicenter clinical trials. This is due to several factors: (1) its network of 172 medical centers geographically dispersed throughout the country, under one administrative system; (2) a dedicated group of talented physicians and other health professionals serving at these medical centers; (3) a loyal and compliant patient population of nearly four million veterans; (4) a system of experienced coordinating centers that provide biostatistical, data processing, pharmacy and administrative support; and (5) a research service that recognizes the uniqueness and importance of the program and strongly supports its mission. The VA has conducted multicenter clinical trials for more than half a century, beginning with its first trial, which was organized in 1945 to evaluate the safety and efficacy of chemotherapeutic agents for tuberculosis. This article describes the history of the program, its organization and operating procedures, some of its noteworthy trials, and current challenges and opportunities.

History of the Cooperative Studies Program (CSP) The first cooperative clinical trial conducted by the VA was a joint study with the US Armed Forces to evaluate the safety and efficacy of chemotherapeutic agents for tuberculosis. Drs John B. Barnwell and Arthur M. Walker initiated a clinical trial to evaluate various drugs in the treatment of tuberculosis, including the antibiotic streptomycin [3, 48]. The challenge of caring for 10 000 veterans suffering from the disease following World War II was the impetus for the study. Not only did the results revolutionize the treatment of tuberculosis, they also led to the development of an innovative method for testing the effectiveness of new therapies – the cooperative clinical trial. A VA Program for conducting cooperative studies in psychiatry was started in 1955 and supported by a newly developed Central Neuropsychiatric

Research Laboratory at the Perry Point, Maryland VA Medical Center (VAMC). This Program emphasized the design and conduct of randomized trials for the treatment of chronic schizophrenia. Trials were completed evaluating the efficacy of prefrontal lobotomy [2], chlorpromazine and promazine [8], phenothiazine derivatives [10], other psychotropic drugs [9, 31], the reduction or discontinuation of medication [6], the combination of medication and group psychotherapy [20], brief hospitalization and aftercare [7], the need for long-term use of antiparkinsonian drugs [30], and intermittent pharmacotherapy [43]. Noteworthy VA cooperative clinical trials in other disease areas were started in the late 1950s and 1960s. A VA cooperative study group on hypertension was started in the 1950s (and still exists today). This group was the first to show that antihypertensive drug therapy reduces the long-term morbidity and mortality in patients with severe [54] and moderate [55] elevations of blood pressure. Other areas researched by the early VA cooperative studies included: use of long-term anticoagulants after myocardial infarction; lipid lowering drugs to prevent myocardial and cerebral infarction; treatment of gastric ulcer disease; efficacy of gamma globulin in posttransfusion hepatitis; analgesics to reduce postoperative pain; surgical treatment of coronary artery disease; the effect of portal caval shunt in esophageal varices; and the effects of radical prostatectomy, estrogens, and orchiectomy in the treatment of prostate cancer. In 1962, the VA developed a concept, novel in Federal Government medical research programs at that time, of providing its investigators access to techniques and specialized help and information essential to their research. Four regional research support centers were established: the Eastern Research Support Center at the West Haven, CT VAMC; the Midwest Research Support Center at the Hines, IL VAMC; the Southern Research Support Center at the Little Rock, AR VAMC; and the Western Research Support Center at the Sepulveda, CA VAMC (see Data Management and Coordination). Individual investigators were assisted in such areas as research design, statistical methods, data management, computer programming, and biomedical engineering. The early VA cooperative studies were coordinated by VA Central Office staff in Washington, DC, by these regional research support

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

2

Cooperative Studies Program, US Department of Veterans Affairs

centers, and by contracts with university coordinating centers. The program was led by Mr Lawrence Shaw. Beginning in 1972, a special emphasis was placed on the CSP in the VA’s Medical Research Service and its budget was quadrupled over the next decade. Under the leadership of James Hagans, MD, PhD, the program’s current organization and structure were developed and codified in the Cooperative Studies Program Guidelines [1]. This included the establishment of four statistical/data processing/management coordinating centers and a research pharmacy coordinating center solely dedicated to conducting cooperative studies; central human rights committees attached to each of the statistical coordinating centers; a standing central evaluation committee for the review of all new proposals for VA cooperative studies and all ongoing studies every three years; and clearly defined procedures for the planning, implementation, conduct, and closeout of all VA cooperative studies. The Central Neuropsychiatric Research Laboratory at the Perry Point, MD VAMC; the Eastern Research Support Center at the West Haven, CT VAMC; the Midwest Research Support Center at the Hines, IL VAMC; and a new center at the Palo Alto, CA VAMC were established as the four new VA Cooperative Studies Program Coordinating Center (CSPCCs). The Cooperative Studies Program Clinical Research Pharmacy Coordinating Center (CSPCRPCC) was established at the Washington, DC VAMC, but later relocated to the Albuquerque, NM VAMC in 1977. Daniel Deykin, MD was the first person to head simultaneously the VA research programs both in Health Services Research and Development and the CSP, from 1985 to 1996. He took advantage of this opportunity to promote the development of a series of multicenter clinical trials in the organization and delivery of health services. These trials represented unique challenges in design and conduct. Some of these trials have recently been completed and are in the process of being published [25, 44, 57]. In 1996, John Feussner, MD, MPH was appointed as the VA’s Chief Research & Development Officer, and simultaneously assumed leadership of the CSP. Up until the time Dr Feussner was appointed, the VA Research Service was composed of three major research programs – Medical Research (of which the CSP was a part), Rehabilitation Research & Development, and Health Services Research & Development.

Dr Feussner moved the CSP out of the VA Medical Research Service and elevated the Program to an equal level with the three other major VA research programs. New emphases brought to the Program by Dr Feussner include: initiation of a strategic planning process; more integration and interdependence of the coordinating centers; institution of good clinical practices and standard operating procedures at the coordinating centers; pharmaceutical manufacturing; experimentation to improve the process of informed consent [32]; educational programs in clinical research for VA investigators; partnering with industry, National Institutes of Health (NIH), and international clinical trials groups; the development of three new Epidemiology Research and Information Centers at the VAMCs in Seattle, WA, Boston, MA, and Durham, NC [5]; and Intranet and Internet communications. The strategic planning process initiated in 1997 defined the vision, mission, and specific goals for the Program (Table 1).

Organization and Functioning of the CSP This section describes how a VA cooperative study evolves and the support provided by the VACSP. Table 1

Vision/mission/goals of the VACSP

Vision • The CSP is a premier research program conducting multicenter studies with world-wide impact on health care Mission • To advance the health and care of veterans through education, training, and collaborative research studies that produce innovative and effective solutions to national healthcare problems Goals • To enhance the proficiency of CSP staff and CSP partners (chairpersons, participating investigators) in the conduct of multicenter trials • To enhance the consistency of management support for the CSP • To increase the flow of new research ideas for cooperative studies • To increase the application of research products into clinical practice • To enhance the interdependence of the CSP coordinating centers • To improve the capabilities of dissemination of research findings

Cooperative Studies Program, US Department of Veterans Affairs

3

Investigator Submits Idea to CSP Headquarters Office

Chief, CSP Obtains Reviews from 4-5 Experts on study Idea and Makes Decision

Disapprove

About 70% of Study Ideas are Disapproved

Approve Chief, CSP Assigns Study to One of the 4 CSPCCs and to CSPCRPCC if Drug / Device Study

First Planning Meeting is Held Chief, CSP Must Decide if Satisfactory Progress is Made

No

Study Terminated

Yes Major Changes Changes Made and Resubmitted

Draft Protocol Developed and Second Planning Meeting Held

Changes Requested

HRC Reviews Protocol at Second Meeting

Disapprove

Study Terminated

Approved Minor Changes

Final Protocol Prepared and Submitted to CSEC

Disapproved but Resubmittal Encouraged

CSEC Review

Disapprove

Study Terminated About 40% of protocols are Disapproved

Approve

Figure 1

Development of a VA cooperative study

These aspects of the Program have been reported previously [24, 27, 28].

Planning Request Initiation of a planning request through the evaluation phase is outlined in Figure 1. The VA Research Program, including the CSP, involves strictly intramural research. To receive VA research funding, the investigator must be at least five-eighths time VA. One of the strengths of the CSP is that most of its studies are investigator-initiated. The research questions come from investigators throughout the VA health care system who are on the front lines in providing health care for veterans.

To start the process, the investigator submits to VA Headquarters a 5–10 page planning request outlining the background of the problem, the hypothesis, a brief overview of the design, and anticipated size of the study. The planning request is given a CSP number to aid in tracking the study through its evolutionary phases. The planning request is sent to four or five independent experts in the field who initially judge the importance and feasibility of the study. If this review is sufficiently positive, the study is put into planning and assigned to one of the CSPCCs (and the CSPCRPCC if it involves drugs or devices) for development of the full proposal. This process has evolved to satisfy two important needs. First, the CSP recognizes that the ability to

4

Cooperative Studies Program, US Department of Veterans Affairs

Once the study is approved for planning, the resources of the CSPCCs are applied to the development of the full proposal. Within the coordinating centers, the study is assigned to a specific biostatistician and clinical research pharmacist. These individuals work with the principal proponent in nominating a planning committee which is reviewed and approved by the CSPCC and CSP Directors. The planning committee generally consists of the principal proponent, study biostatistician, study clinical research pharmacist, CSPCC Director, two or three potential participating site investigators, and outside consultants, as needed. The planning committee is funded for two planning meetings. At the first meeting, the basic design features of the study are agreed upon (hypothesis, patient population, treatment groups, primary and secondary endpoints, pharmacologic and drug handling issues, baseline and follow-up data collection and frequency, treatment effect size, sample size, number of sites, duration of study, publication plan, and budget). The full proposal is then written, and at the second planning meeting a draft of the protocol is fine-tuned. Development of a full proposal generally requires six to nine months.

CSPCC. This committee is comprised of scientists and laypeople from the community who review proposals for all new VA cooperative studies and all ongoing studies annually. The committee serves as a central Institutional Review Board (IRB) for studies assigned to the CSPCC and considers such aspects of the proposal as risks versus benefits to the patients, patient management, burdens placed on the patients from participation in the study, community equipoise with regard to the treatments being compared, and the informed consent procedures. This committee has absolute authority over approval or disapproval of the study. Only the HRC has the authority to change its own decisions. The composition of the committee follows VA regulations and is consistent with Food and Drug Administration (FDA) guidelines. Minimum membership of the HRC includes a VA chairperson, a practicing physician from the community, a nonphysician scientist, a veteran representative, a member of a recognized minority group, a clergyman or ethicist, and an attorney. If the HRC approves the study, then the proposal is submitted to VA Headquarters for scientific review and a funding decision. The proposal is sent to four or five experts in the field and a biostatistician for written reviews. All cooperative study proposals are reviewed by a standing scientific review committee, called the Cooperative Studies Evaluation Committee (CSEC). This committee is composed of senior physician scientists and biostatisticians who have had extensive experience in cooperative studies and clinical trials. The CSEC meets in the spring and fall of each year. The principal proponent and study biostatistician present the study to the CSEC in person (reverse site visit), defend their proposal and answer questions from the CSEC members. The CSEC then goes into executive session, and decides to recommend approval or disapproval of the study and, for approvals, gives a scientific priority score, ranging from 10 to 50 with 10 being the best score. The final funding decision is made by the CSP Director. The advantages of this review process are that the study investigators have the opportunity to interact personally with the review body to answer their criticisms and concerns, and the final decision is known immediately following the review and executive session.

Evaluation Phase

Implementation of the Trial

The completed proposal is first reviewed by the Human Rights Committee (HRC) attached to the

Implementation and conduct of a VA cooperative trial are outlined in Figure 2. Once the CSP Director

come up with a good idea needing rigorous test does not necessarily carry with it the ability to pull together all the expertise necessary to plan a clinical trial. This was especially true in the early days, when “trialists” were few and far between, and clinical researchers seldom had training in modern statistical trials design. So it is important to provide access to this expertise early in the planning process. However, such aid is expensive and scarce, so it is important not to waste it on ideas that do not show promise. Thus, the second need is for an initial concept review. This has proved to be a very efficient allocation method; about 70% of all initial proposals are not approved to go on to planning, and of the surviving 30%, about two-thirds complete the planning process. Of those that are successfully planned, about three-quarters are approved and funded. Thus, the method helps to avoid the problem of insufficiently developed protocols, while conserving the scarce resources of planning.

Planning Phase

Cooperative Studies Program, US Department of Veterans Affairs

5

CSEC Review Approved Scientific Priority Score In Fundable Range

N0

Study Terminated

Yes Study is Funded

Organizational Phase: Forms Approval and Printed; Executive Committee and DMB Meet; Organization Meeting, etc.

Patient Recruitment & Follow-up

Study Group & Executive Committee Meet Yearly

DMB & HRC Meet Yearly; Both Committees Must Approve Continuation

Disapprove

Study Terminated

Approve Disapprove CSEC Mid-Year Or Every 3-Year Review

Study Terminated

Approve No Study Completed Yes Closeout Procedures Performed

Manuscripts Prepared

Figure 2

Conduct of a VA cooperative study

approves funding, the implementation phase of the cooperative study begins. All activities in this process are closely coordinated by the CSPCC, CSPCRPCC and the Study Chairperson’s Office. The necessity for carefully controlled medical treatment and data collection procedures for the successful conduct of multicenter clinical trials is well recognized. Because of its administrative structure, the VA provides an environment that is uniquely suited to this type of research. Each participating

facility is funded by one control point for the entire period of the study, and the VAMC system provides a structure in which a relatively high degree of medical, scientific, and administrative control can be exercised. This same degree of control is often more difficult to obtain in studies that involve participating sites from different administrative systems [27]. The CSPCC recommends funding levels and monitors the performance of the individual medical centers. This information is reviewed regularly by the

6

Cooperative Studies Program, US Department of Veterans Affairs

study biostatistician, center director, the Executive Committee, and at least annually by the Data and Safety Monitoring Board. This integrated monitoring of scientific, biostatistical, and administrative aspects by the CSPCC provides a comprehensive approach to the management of multicenter clinical trials, in contrast to other clinical trials biostatistics groups that are responsible only for the analytical and data processing aspects and exercise no administrative control [27]. The Research and Development Committee and the IRB of each participating medical center must review and approve a cooperative study before it can be implemented at that facility. They are able to make modifications to the prototype consent form approved by the CSPCC HRC, but all local modifications must be reviewed and approved by the CSPCC. Included in the implementation component of a cooperative study is the establishment of the Executive Committee and Data and Safety Monitoring Board (DSMB) who share responsibility for the conduct and monitoring of the cooperative study in the ongoing phase. The Executive Committee, which often includes several members of the original Planning Committee, consists of the study chairperson who heads the committee, the study biostatistician, the clinical research pharmacist, two or three participating investigators, and one or two consultants. This committee is responsible for the detailed operational aspects of the study during its ongoing phase and ensures adherence to the study protocol, including aspects relating to patient recruitment, treatment, laboratories, data collection and submission, biostatistical analysis, data processing, subprotocols and reporting. The Executive Committee sometimes recommends probation or termination of sites whose performance is poor. The DSMB consists of five to eight individuals who have not been involved in the planning and development of the proposal, and includes one or two biostatisticians and two or more subject-matter experts in the field(s) of the cooperative study. This committee is charged with the responsibility of monitoring and determining the course of the ongoing study and considers such aspects as patient accrual; performance of the participating sites, CSPCC, and chairperson’s office; and safety and efficacy data. Perhaps a unique feature of the CSP is that the CSPCC HRC also reviews each ongoing study annually and receives the same data reports as presented

to the DSMB. The study chairperson, participating site investigators, and other members of the Executive Committee are masked to the outcome data during the course of the study. Only the DSMB, HRC, CSPCC and CSPCRPCC see the outcome data during the conduct of the study. The fourth body involved in the conduct of the cooperative study is the Study Group, which consists of all participating investigators, the study chairperson (co-chairpersons), biostatistician, clinical research pharmacist, and consultants. This body meets once annually to consider the progress of the study and to resolve problems that may arise at the participating centers. Within the CSPCC, the biostatistician heads a team of administrative, programming and data management personnel that provides regular monitoring of the study. This team develops an operations manual (see Clinical Trials Protocols), in conjunction with the chairperson’s office, to train study personnel in the day-to-day conduct of the trial. They also develop a computer data management system to edit, clean, and manage the study data. Automated query reports are generated by the computer system and sent to the participating sites for data checking and cleaning. Statistical progress reports are published by the CSPCC and distributed to the Study Group, Executive Committee, and DSMB at scheduled meetings. An initial kickoff meeting is held before the study starts to train site personnel in the conduct of the study. Annual meetings are held thereafter to refresh training and discuss issues in the conduct of the study. Frequent conference calls of the study committees are also used to facilitate communication and training. Another unique aspect of the CSP is the CRPCC, which operationalizes the pharmaceutical aspects of the clinical trials (Table 2). In the planning stages of the study the clinical research pharmacist designs the drug or device treatment and handling protocol and works with the study chairperson and pharmaceutical and device industries to purchase or obtain donations of clinical supplies for the study. The CRPCC coordinates the development of appropriate drug dosage formulations and the manufacture of study drugs or devices. In the event that drug donations are not possible, the CRPCC has the expertise and capability to provide the in-house manufacture of active drugs and matching placebo. Drugs for all cooperative studies

Cooperative Studies Program, US Department of Veterans Affairs Table 2 Unique functions and roles of the CSP pharmacy coordinating center • • • • • • • • • • •

Design of a drug or device handling protocol for each study involving drugs or devices Preparation and submission of INDAs or IDEs Obtaining donations or purchase of clinical supplies for study Coordination of appropriate drug dosage formulations and manufacture of study drugs or devices Quality control testing of drugs Development of blinding methods Storage, packaging and shipment of clinical supplies to pharmacies at the participating sites Computerized drug inventory and information system to track and replenish supplies at site pharmacies Monitoring adverse medical events and reporting to appropriate authorities Monitoring, auditing and education services to ensure sites are in compliance with GCP Preparation of final drug/device accountability reports

must pass the testing of the CRPCC’s quality control testing laboratory. The CRPCC also assesses study product blinding methods. At the CRPCC, study medications are stored in an electronically controlled and secured environment. The CRPCC customizes labels and packages all study medications, which are centrally distributed to the pharmacies at the participating sites. In doing so, the CRPCC provides a computerized drug inventory and information system for complete accountability of clinical supplies. This includes automated study supply tracking and replenishment systems for maintaining adequate study supplies at participating sites as well as automated telephone randomization and drug assignment systems. The clinical research pharmacist is then able to direct and monitor the study prescribing and dispensing activities as well as to monitor the compliance with the study protocol treatments at the participating sites. At the end of the study the CRPCC directs the retrieval and disposition of unused clinical supplies and prepares a final drug/device accountability report. The clinical research pharmacist also works closely with the study chairperson and the manufacturers to prepare, submit, and maintain Investigational New Drug Application (INDAs) or Investigational Device Exemption (IDEs), which

7

includes preparing and submitting annual and special reports to the FDA. Along with this responsibility, the clinical research pharmacist coordinates the monitoring and reporting of all adverse medical events to study management, FDA and associated manufacturers. Recently the CRPCC established a central Good Clinical Practices (GCP) Assurance Program. The Program provides monitoring, auditing, and educational services for all VA cooperative studies to ensure that the participating sites are in GCP compliance. If needed, the Program is capable of providing full GCP monitoring for studies under regulatory (FDA) scrutiny.

Final Analysis and Publication Phase Upon completion of patient intake and follow-up, the study enters the final analysis and publication phase. If the Executive Committee, the CSPCC, and the study biostatistician have performed their tasks well, this phase should be quite straightforward. It requires an updating of all study files and the processing and analysis of the complete data set. The interim statistical analyses that were run during the ongoing phase of the study are now executed on the complete data. In addition, some analyses may point to additional questions that would be of interest; however, it is anticipated that the majority of final analyses and interpretation of results will occur within 6 to 12 months after study termination. All publications emanating from the cooperative study must be approved by the Executive Committee. Although the responsibility of the DSMB terminates at the end of data collection, the Board is at times requested to review manuscripts and give advice prior to submission for publication [27]. Usually each trial generates a number of manuscripts. The Executive Committee establishes priorities for statistical analyses and manuscript development and appoints writing committees composed of members of the Executive Committee and Study Group for each paper. Authorship of the main paper(s) usually consists of the chairperson, study biostatistician, study clinical research pharmacist, members of the Executive Committee, and, in some cases, the participating site investigators. Secondary papers are often written by other members of the Executive Committee and site investigators.

8

Cooperative Studies Program, US Department of Veterans Affairs

The CSPCC serves as the final data repository for the study. The study database, protocol, operations manuals, forms, study correspondence and interim and final statistical progress reports are archived at the CSPCC.

Role of the Biostatistician and Pharmacist in the CSP One of the unique features of the CSP is that the biostatistician at the CSPCC plays a major organizational, fiscal, and administrative role, in addition to the usual technical role. In recent times, as the administration of studies has become more complex, the biostatistician may be assisted by a study coordinator but, as in the past, the greater part of the burden of management falls on the biostatistician. In contrast to the pharmaceutical industry and to many Contract Research Organization (CROs), the biostatistician is responsible for monitoring site adherence to protocol, recruitment, and many other aspects of the study conduct. In addition, the study pharmacist plays a key role in monitoring adverse effects, maintaining supplies of the study drug, regulatory reporting, and the like. In a sense, the study team is deployed to support the investigators, but has independent authority and responsibility as well. One of the strengths of this approach to study management is that it is possible to guarantee some degree of uniformity in the conduct of the studies, independent of the varying managerial skills and style of the study chairs. The biostatistician and pharmacist, together with the coordinating centers of which they are a part, provide institutional memory and continuity. Their central position on the study teams reinforces the key idea that the studies mounted by the VACSP are the joint responsibility of the program and the investigators. Such an intramural program can only succeed on a limited budget if issues of cost and complexity are kept to the forefront during the planning process. A consequence that is easily observed is that the typical CSP trial is a lean, focused attack on a single important clinical question, rather than a broad-based research project with many interwoven strands of investigation. In contrast to the much larger NIH clinical trials efforts, which are organized along disease lines, the CSP biostatisticians and CSPCCs are generalists, doing studies in all areas of medicine with

relevance to the VA. Along the way, some centers have developed some special experience in certain areas, but there has never been a “heart” center or a “cancer” center. Because the CSP has such a broad medical purview, but a relatively low volume of studies, it has not made economic sense to specialize. The scarce resource of statistical and data management expertise has needed to be allocated efficiently to support the proposals that were emerging from the field. Since VA resources have followed the strength of the proposals rather than disease areas, the CSPCCs have not specialized to any large degree. While there are undoubted advantages to specialization, as shown by the contributions made by the National Cancer Institute (NCI) (see Cooperative Cancer Trials) and the National Heart, Lung, and Blood Institute (NHLBI) (see Cooperative Heart Disease Trials) statisticians to the statistical science of their disease areas, there are some advantages to generalizing. In particular, it has been possible to transplant methods and lessons learned from well-studied areas such as cancer and heart disease, to other areas such as psychopharmacology, device research, health services research, and trials of surgical procedures. The absence of disease-area “stovepiping” has facilitated a high general level of sophistication in the conduct of trials, with techniques travelling readily across borders. This cross-pollination has also been facilitated by the structure of the VACSP scientific peer review. The standing committee that reviews and recommends studies for funding mixes disciplines with common expertise in multisite studies. Ad hoc reviewers provide the crucial discipline-specific input to the committee, but the same committee may review a heart failure trial in the morning and a psychopharmacology trial in the afternoon. The result is a high degree of uniformity in the standards for the research across disease areas, and this has been an enduring strength of the program.

Ongoing and Completed Cooperative Studies (1972–2000) One hundred and fifty-one VA cooperative studies were completed or are currently ongoing in the period 1972–2000. Table 3 presents the health care areas

Cooperative Studies Program, US Department of Veterans Affairs Table 3 Health care areas of ongoing and completed VA cooperative studies (1972–2000) Health care area Cardiology/cardiac surgery Hypertension Gastrointestinal Substance abuse Mental health Infectious diseases Cancer Dental General surgery/anesthesia Cerebrovascular Peripheral vascular Military service effects Ambulatory care Epilepsy Genitourinary Diabetes Renal Sleep Pulmonary Hematology Hearing One each in seven areasa Total

Number of studies

Percent of studies

24 15 14 11 10 9 8 6 6 5 5 4 4 4 4 3 3 3 2 2 2 7 151

15.9 10.0 9.3 7.3 6.6 6.0 5.3 4.0 4.0 3.3 3.3 2.6 2.6 2.6 2.6 2.0 2.0 2.0 1.3 1.3 1.3 4.7 100.0

a Analgesics, arthritis, geriatrics, hospital-based home care, laboratory quality control, computerized neuropsychological testing, ophthalmology

of these studies. These areas are generally reflective of the major health problems of the US veteran population, consisting mainly of middle-aged and senior adult males. Studies in cardiology and cardiac surgery represent 15.9% of the 151 studies, followed by hypertension (10.0%), gastrointestinal diseases (9.3%), substance abuse (7.3%), mental health (6.6%), infectious diseases (6.0%), and cancer (5.3%). There are a few notable disease areas that are prevalent in the VA population and yet might be considered underrepresented in the CSP. These include diabetes (2.0%), renal diseases (2.0%), pulmonary diseases (1.3%), hearing diseases (1.3%), arthritis (0.7%), and ophthalmologic diseases (0.7%). Because the CSP mainly relies on investigatorinitiated studies, the conclusion might be drawn that these subspecialties have underutilized the Program. Although studies on effects of military service represent only 2.6% of the 151 studies, studies listed in other categories have investigated treatments for

9

service-connected illnesses (e.g. posttraumatic stress disorder studies are categorized under mental health, and the substance abuse studies could be considered consequences of military service). Table 4 briefly summarizes some of the noteworthy VA cooperative clinical trials that were completed in the 1980s and 1990s. Many of these trials resulted in advances in clinical medicine that could immediately be applied to improve the health care of US veterans and the US population in general.

Current Challenges and Opportunities Although the VACSP has had numerous past successes, it faces many challenges and opportunities in the future. These include: (1) changes in the VA health care system and their effects on research; (2) nationwide concerns about violations of patients’ rights in research; (3) increasing the efficiency and interdependence among the coordinating centers and standardizing procedures; (4) ensuring the adequacy of flow of research ideas and training of investigators; and (5) partnering with industry, other federal agencies, nonprofit organizations, and international clinical trial groups to enhance the capacity of the Program.

Changes in the VA Health Care System The VA health care system has been undergoing substantial changes that could adversely affect research. In 1996, the VA reorganized into 22 geographically defined Veterans Integrated Service Networks (VISNs). Much of the central authority, decisionmaking, and budgeting once performed in VA Headquarters in Washington, DC, has been delegated to the 22 VISN offices. Within the VISNs, administrative and health care services and in some cases entire VAMCs are being consolidated. The largest component of the VA patient population, the World War II veterans, is rapidly declining. Health care personnel in some VISNs are experiencing reductions in force, with the result that the remaining personnel have limited time to devote to research. These factors may already be adversely affecting the Program’s ability to meet recruitment goals in ongoing trials [23].

Concerns About Patients’ Rights in Research The nature of the veteran population treated at VA hospitals raises some special issues in human rights

10

Cooperative Studies Program, US Department of Veterans Affairs Table 4 • • • • • • • • • • • • • • • • • • • • • • • • • • •

Noteworthy VA cooperative studies

80% of strokes in patients with atrial fibrillation can be prevented with low-dose warfarin [15] Carotid endarterectomy is effective in preventing strokes in symptomatic and transient ischemic attacks in asymptomatic patients [26, 39] Aggressive treatment of moderate hypertension works well in elderly patients [19, 37] Age and racial groupings can be used to optimize selection of first line drugs in hypertension [38] Coronary artery bypass surgery prevents mortality in patients with left main disease and in high-risk patients without left main disease [42, 49] Low dose aspirin reduces heart attacks and death in 50% of patients with unstable angina [34] Vasodilators and angiotensin converting enzyme inhibitors prevent deaths in patients with congestive heart failure [12, 13] Low dose aspirin started 6 hours after coronary artery bypass surgery and continued for one year prevents the occlusion of the bypass grafts [17, 18] Mechanical artificial aortic heart valves prolong survival more than bioprosthetic aortic heart valves [22] A conservative, ischemia-guided strategy is safe and effective for management of patients with non-Q-wave myocardial infarction [4] Digoxin does not reduce mortality but does reduce hospitalizations in patients with congestive heart failure [50] The rate of coronary events (myocardial infarction or death) in men with coronary heart disease can be reduced by 22% with Gemfibrozil therapy, which increases high density lipoprotein cholesterol and lowers triglyceride levels [45] Progression of human immunodeficiency virus (HIV) infection to full blown acquired immune deficiency syndrome (AIDS) can be delayed with the drug zidovudine [21] Steroid therapy does not improve survival of patients with septic shock [52] Patients with advanced laryngeal cancer can be treated with larynx-sparing chemotherapy and radiation compared with standard surgical removal of the larynx and have equivalent long-term survival [14] The drug Terazosin is more effective than Finasteride in relieving the symptoms of benign prostatic hyperplasia [33]. Transurethral resection of the prostate is an effective operation, but Watchful Waiting can be effective in many patients [56] An implantable insulin pump is more effective than multiple daily insulin injections in reducing hypoglycemic side-effects, and enhancing quality of life in adult-onset Type II diabetes mellitus [46] Multi-channel are superior to single-channel cochlear implants in restoring hearing to patients with profound hearing loss [11] Sclerotherapy is an effective treatment for esophageal varices in patients who have had prior bleeds but not in patients without prior bleeds [51] Antireflux surgery is more effective than medical therapy in patients with complicated gastroesophageal reflux disease [47] Severely malnourished patients benefit from pre-operative total parenteral nutrition but mildly malnourished patients do not [53] Clozapine is a cost-effective treatment for patients with refractory schizophrenia who have high hospital use [44] Erythropoietin administered subcutaneously compared with intravenously can significantly reduce the costs of hemodialysis [29] Use of intrapleural tetracycline reduces recurrence rate by 39% in patients with spontaneous pneumothorax [35] Rapid access to high quality primary care for patients with severe chronic illnesses greatly improves patient satisfaction with care but may lead to an increase in hospital readmissions [57] Levomethadyl acetate (LAAM) is a safe and efficacious drug to use for heroin addiction. Studies were used to gain FDA approval for LAAM as treatment for heroin addiction [16, 36] Systemic corticosteroids improve clinical outcomes up to three months in patients with chronic obstructive pulmonary disease [40]

Cooperative Studies Program, US Department of Veterans Affairs protections (see Medical Ethics and Statistics). The VA treats about four million veterans, who tend to be less well off than the average veteran (or the average citizen). They are often more severely ill than non-VA patients with the same age and diagnosis, and often have multiple co-morbidities. They are on average more dependent on the VA for their health care than the typical non-VA patient is on his or her usual health care provider. Against this background we note the extraordinary willingness of the veteran patient to engage in research, trusting the clinical researcher to an astonishing degree. Such trust demands an extraordinary level of protection in response. The CSP has instituted a unique framework of human subjects’ protections, going beyond the usual procedures that other federal sponsors and drug companies require. This begins in the planning stage, when each proposal must undergo a rigorous review by the HRC attached to the coordinating center. It typically meets for several hours over a single protocol, reviewing it in fine detail. The protocol cannot go forward without their independent approval. The CSP also requires the usual individual site IRB approval, and other reviews that are mandated at the local site, before a study can start at a site. The ongoing IRB reviews at the local sites (annually, or more often, as stipulated in the initial review) are monitored by the CSP staff. As has become standard in multisite trials, each CSP study has its own independent DSMB that meets at least annually to review the progress of the study. The unique CSP innovation to this process is the joint review by the HRC and DSMB. Thus, after every DSMB meeting, the two groups meet to review and recommend, with the same basis of information on study progress. The CSP has found that the HRC is able to hear the recommendation of the DSMB, which is typically heavily weighted with subjectmatter expertise, and interpret it in the light of the other perspectives they bring. The CSP believes that this has been a successful experiment in resolving the knotty issue of how to obtain full and informed ongoing review of studies where investigators are kept blind, and site-level information must be far less informative than the big picture presented to the DSMB. We believe that such joint reviews add considerably to the level of protection of human subjects.

11

In addition, members of the central HRCs conduct three site visits per year during which patients are interviewed about their participation in the trials. Thus, the Program as a whole conducts 12 such visits per year. The Albuquerque auditing group periodically site visits VAMCs participating in cooperative studies and performs audits to ensure that the sites are complying with GCP guidelines. The CSPCCs also receive copies of consent forms from all patients in all of the trials as further evidence of proper consent procedures. The VA recently established its own office to oversee the protection of patients’ rights in VA research, performing functions similar to those of the Office of Protection from Research Risks (OPRR) of the Department of Health and Human Services. IRBs at VAMCs currently are required to be accredited by an external, non-VA entity. In addition to these standard procedures, followed in all studies, the CSP has recognized two other areas of human subjects’ protection in which it can make a contribution. The Enhancing the Quality of Informed Consent (EQUIC) program [32] is designed to institutionalize the process of testing innovations in methods for obtaining informed consent. It piggybacks tests of new methods on ongoing CSP studies, and provides a centralized assessment of the quality of informed consent encounters (by remote telephone interview of patients). In the spirit of EQUIC, a substudy is being conducted in one ongoing VA cooperative study to evaluate the utility of an informed consent document developed by a focus group of subjects eligible for the trial [41]. The second topic that the CSP has engaged are the ethical, legal, and social implications of genetics research, specifically of deoxyribonucleic acid (DNA) banking with linked clinical (phenotype) data. The CSP has begun a project to provide uniform methods for obtaining and banking such samples. Steps to ensure human subjects’ protection in VA cooperative studies are listed in Table 5.

Efficiency and Interdependence of the CSPCCs The VACSP recently contracted with an outside vendor to help develop standard operating procedure (SOPs) for the CSPCCs. Twenty-two SOPs were developed in the areas of administration, planning and implementing clinical trials, data management,

12

Cooperative Studies Program, US Department of Veterans Affairs

Table 5 Steps to ensure human subjects’ protection in the VACSP • • • • • • • • • • • • • •

Investigator’s integrity Development of proposal through collaboration between investigators and CSPCCs HRC review of proposal initially Site Monitoring and Review Team (SMART) audit of consent form contents CSEC review of proposal Initial review of proposal by participating site R&D and IRB Annual central reviews of trial by DSMB and HRC Annual reviews of study by local R&D committee and IRB SMART audit of participating sites HRC site visits and interviews of study patients Receipt of copies of patient consent forms by CSPCC, local research offices, and local pharmacies Implementation of SOPs and good clinical practices Compliance with all FDA and VA regulations Innovative studies on improving informed consent

study closeout, and study oversight (Table 6). By standardizing among and within the coordinating centers certain procedures that are performed in every study, we will achieve an even higher level of support to all studies more efficiently than previously done. The SOPs will also enable the CSPCCs to be in better compliance with GCP principles and International Conference on Harmonization (ICH) guidelines. Since 1996, the Directors of the Program and centers have been meeting as a group semiannually to identify current and future challenges and opportunities, and to develop annual strategic plans to respond to these challenges and opportunities. This has enhanced the development of mutual projects which the centers can work on together to further the goals of the organization as a whole, such as the development of a Clinical Research Methods Course, a one-year sabbatical program for clinical investigators to enhance their training and skills, and SOPs for the central HRCs.

Ensuring the Adequacy of Flow of Ideas and Training of Investigators In recent years, the CSP has developed several educational opportunities to help train VA investigators

Table 6

Recently adopted SOPs for the VACSP

Administration • Preparing, issuing and updating SOPs • Training and training documentation Planning and implementation of clinical trials • Developing, approving and amending protocols • Study/training meetings • Preparing and approving operations manuals • Study initiation • Developing and approving case report forms • Creating and validating data entry screens and programs • Preparing, documenting and validating data checking programs • Preparing, documenting and validating statistical programs • Developing and conducting statistical analyses Handling data from clinical trials • Randomization, blinding and unblinding • Central monitoring • Case report form flow and tracking • Data entry and verification • Data cleaning • Reporting adverse events Study closeout • Archiving study documentation • Study closeout Study oversight • Assuring site R&D and IRB approvals • DSMB • HRC

in clinical research and to encourage utilization of the Program to answer important clinical questions. These include a five-day course in clinical trials and sabbatical and career development programs focused on clinical research methodology. The five-day course is taught once each year and involves 10 faculty members (two from each of the five coordinating centers) and 60 VA investigators selected from applications from throughout the country. The course consists of 15 lecture/discussion sessions on various aspects of designing a clinical trial, interspersed with breakout sessions during which the students are divided into five planning committees to design a clinical trial. On the last day of the course, the student groups take turns in presenting their clinical trials and receiving critiques from the audience. The course has been taught twice and has received excellent feedback from the students.

Cooperative Studies Program, US Department of Veterans Affairs

13

The CSP Career Development Program provides protected time to clinician–investigators for a period of concentrated clinical research activity. The objective is to build capacity in a wide geographic distribution for the Department of Veterans Affairs to conduct clinical research in acute-care hospitals, long-term care facilities, or outpatient settings. The Program is designed to foster the research careers of clinician–scientists who are not yet fully independent but who seek to become independent clinical researchers. The award provides three years of salary and some supplemental research support, and the awardees are expected to work at least part of the time at one of the five CSPCCs or three Epidemiology Research and Information Center (ERICs). In 1999 CSP announced a sabbatical program for established clinician–scientists to train at one of the CSPCCs or ERICs for up to one year. The purpose of the program is to support clinician–investigators who wish to secure training time to learn about the conduct of cooperative studies and epidemiologic research.

Concluding Remarks

Partnering with Outside Organizations

In summary, we believe that there are considerable strengths to conducting multicenter clinical trials in

The VACSP has partnered with NIH and industry for many years in conducting multicenter clinical trials. In recent years, a special emphasis has been placed on partnering with outside agencies to enhance the effect of the limited VA research funding, and these efforts have been fruitful. Recent examples of this partnering include: the Digitalis in Heart Failure Trial, sponsored by the VA, NHLBI, and Burroughs–Wellcome Company and conducted in 302 VA and non-VA sites in the US and Canada; a series of trials sponsored by the VA and the National Institute of Drug Abuse (NIDA) to evaluate new treatments for drug abuse; the Prostate Cancer Intervention Versus Observation Trial (PIVOT), sponsored by the VA, Agency for Healthcare Quality and Research (AHQR) and NCI; the Beta-Blocker Evaluation of Survival Trial (BEST), funded by the VA, NHLBI, and industry; the Clinical Outcomes Utilizing Revascularization and Aggressive Drug Evaluation (COURAGE) trial, supported by the VA and 10 pharmaceutical companies; the VA/National Institute of Deafness and Other Communication Disorders (NIDCD) Hearing Aid Clinical Trial; and the Shingles Prevention Study sponsored by the VA, NIH, and a pharmaceutical company.

The VACSP has been working with the American College of Surgeons to promote clinical trials evaluating new surgical operations and technologies. This collaboration has resulted in a VA trial comparing the outcomes of laparoscopic vs. open tensionfree inguinal hernia repair, a trial comparing open tension-free hernia repair vs. watchful waiting funded by AHQR, and a trial comparing pallidotomy vs. deep brain stimulation in Parkinson’s Disease. The VACSP has also issued a program announcement for the development of multinational clinical trials between the VA and the Medical Research Councils of Canada and the UK. As the field of clinical trials matures, it is likely that achievable treatment effect sizes will decrease, necessitating larger and larger trials, or “mega” trials. These types of collaborations will be important in the future, as the larger trials will exceed the capacity of any single clinical trials program.

Table 7

Strengths of the VACSP

Related to VA health care system • Large veteran population willing to participate in research, well-represented by minority groups • Largest integrated healthcare system in US with 172 medical centers under single administrative system • High-quality physician–investigators • National administrative databases that allow for tracking of patients • Supportive management in VA Headquarters • System of local research offices and IRBs at participating sites that facilitate multicenter research Related to the CSP • Quality and experience of the coordinating centers • Well-established mechanisms for conducting multi-site trials • Planning process usually produces tightly focused, cost-effective protocols • Rigorous review process by HRC and CSEC • Guidelines and SOPs for conducting trials • Multiple levels of protection of research subjects • Ability to conduct trials with high power and generalizability, so the impact on changing health care practices is maximized compared with other research programs

14

Cooperative Studies Program, US Department of Veterans Affairs

Table 8

Limitations of the VACSP

[3]

Related to VA health care system • • • •

Primarily male population, limiting generalizability of results Large studies in female and childhood diseases are not possible Changes in the health care system, including aging and declining of veteran population, decentralization and consolidation of facilities Reduction in dedicated research time for physician–investigators

Related to the CSP • • •

Long duration from submission of planning request to publication of main results raises the risk of study becoming outdated Limitation of funding Limited capacity to conduct mega trials within VA system

[4]

[5]

[6]

[7]

[8]

the VA health care system, as enumerated in Table 7. There are also some acknowledged limitations of the Program, some of which can be addressed in the future (Table 8). This article has described the history, organization and productivity of a clinical trials program designed as an integral part of a large health care system. The biostatistical and pharmacy positions in the Program are ideal from the standpoint that these people are integrally involved in the research from beginning to end and play a major role in the conduct of the trials. The Program is an example of how clinician–investigators and methodologists can work together successfully to design and conduct largescale clinical research.

[9]

[10]

[11]

Acknowledgments We are extremely indebted to the foresight and support of the US Congress, Executive Branch, and VA management, and to the dedication of the health care providers and veteran patients in the VA system to enable us to carry out this important research.

[12]

[13]

References [1]

[2]

Anonymous (1997). Cooperative Studies Program Guidelines for the Planning and Conduct of Cooperative Studies. Office of Research and Development, Department of Veterans Affairs, Washington, DC. Ball, J., Klett, C.J. & Gresock, C.J. (1959). The Veterans Administration study of prefrontal lobotomy, Journal of Clinical and Experimental Psychopathology 20, 205–217.

[14]

[15]

Barnwell, J.B., Bunn, P.A. & Walker, A.M. (1947). The effect of streptomycin upon pulmonary tuberculosis, American Review of Tuberculosis 56, 485–507. Boden, W.E., O’Rourke, R.A., Crawford, M.H., et al. (1998). Outcomes in patients with acute non-Q-wave myocardial infarction randomly assigned to an invasive as compared with a conservative management strategy, New England Journal of Medicine 338, 1785–1792. Boyko, E.J., Koepsell, T.D., Gaziano, J.M., et al. (2000). U.S. Department of Veterans Affairs medical system as a resource to epidemiologists, American Journal of Epidemiology 151, 307–314. Caffey, Jr, E.M., Diamond, L.S., Frank, T.V., et al. (1964). Discontinuation or reduction of chemotherapy in chronic schizophrenics, Journal of Chronic Diseases 17, 347–358. Caffey, Jr, E.M., Galbrecht, C.R. & Klett, C.J. (1971). Brief hospitalization and aftercare in the treatment of schizophrenia, Archives of General Psychiatry 24, 81–85. Casey, J.F., Bennett, I.F., Lindley, C.J., et al. (1961). Drug therapy in schizophrenia: a controlled study of the relative effectiveness of chlorpromazine, promazine, phenobarbital, and placebo, Archives of General Psychiatry 4, 381–389. Casey, J.F., Hollister, L.E., Klett, C.J., et al. (1961). Combined drug therapy of chronic schizophrenics: a controlled evaluation of placebo, dextroamphetamine, imipramine, isocarboxazid, and trifluoperazine added to maintenance doses of chlorpromazine, American Journal of Psychiatry 117, 997–1003. Casey, J.F., Lasky, J.J., Klett, C.J. & Hollister, L.E. (1960). Treatment of schizophrenic reactions with phenothiazine derivatives: A comparative study of chlorpromazine, triflupromazine, mepazine, prochlorperazine, perphenazine and phenobarbital, American Journal of Psychiatry 117, 97–105. Cohen, N.L., Waltzman, S.B., Fisher, S.G., et al. (1993). A prospective, randomized study of cochlear implants, New England Journal of Medicine 328, 233–237. Cohn, J.N., Archibald, D.G., et al. (1986). Effect of vasodilator therapy on mortality in chronic congestive heart failure. Results of a Veterans Administration Cooperative Study, New England Journal of Medicine 314, 1547–1552. Cohn, J.N., Johnson, G., Zeische, S., et al. (1991). A comparison of enalapril with hydralazine isorbide dinitrate in the treatment of chronic congestive heart failure, New England Journal of Medicine 325, 303–310. Department of Veterans Affairs Laryngeal Cancer Study Group (1991). Induction chemotherapy plus radiation compared with surgery plus radiation in patients with advanced laryngeal cancer. New England Journal of Medicine 324, 1685–1690. Ezekowitz, M.D., Bridges, S.L., James, K.E., et al. (1992). Warfarin in the prevention of stroke associated

Cooperative Studies Program, US Department of Veterans Affairs

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

[27]

[28]

with nonrheumatic atrial fibrillation, New England Journal of Medicine 327, 1406–1412. Fudala, P.J., Vocci, F., Montgomery, A. & Trachlenberg, A.I. (1997). Levomethyl acetate (LAAM) for the treatment of opioid dependence: a multisite, open-label study of LAAM safety and an evaluation of the product labeling and treatment regulations, Journal of Maintenance in the Addictions 1, 9–39. Goldman, S., Copeland, J., Moritz, T., et al. (1989). Saphenous vein graft patency 1 year after coronary artery bypass surgery and effects of antiplatelet therapy. Results of a Veterans Administration Cooperative Study, Circulation 80, 1190–1197. Goldman, S., Copeland, J., Moritz, T., et al. (1991). Starting aspirin therapy after operation. Effects on early graft patency. Circulation 84, 520–526. Goldstein, G., Materson, B.J., Cushman, W.C., et al. (1990). Treatment of hypertension in the elderly: II. Cognitive and behavioral function. Results of a Department of Veterans Affairs Cooperative Study, Hypertension 15, 361–369. Gorham, D.R., Pokorny, A.D. & Moseley, E.C. (1964). Effects of a phenothiazine and/or group psychotherapy with schizophrenics, Diseases of the Nervous System 25, 77–86. Hamilton, J.D., Hartigan, P.M., Simberkoff, M.S., et al. (1992). Early versus later zidovudine therapy of patients with symptomatic human immunodeficiency virus infection: results of a randomized, double-blind VA Cooperative Study, New England Journal of Medicine 326, 437–443. Hammermeister, K., Sethi, G.K., Henderson, W.G., et al. (1999). Outcomes 15 years after valve replacement with a mechanical versus a bioprosthetic valve: final report of the VA randomized trial, Journal of the American College of Cardiology 36, 1152–1158. Henderson, W.G. (2000). Is it becoming more difficult to attain target sample sizes in clinical trials? Presentation at the 21st Annual Meeting of the Society for Clinical Trials. Toronto, Canada, April 16–19. Henderson, W.G. (1980). Some operational aspects of the Veterans Administration Cooperative Studies Program from 1972–1979, Controlled Clinical Trials 1, 209–226. Henderson, W.G., Demakis, J., Fihn, S.D., et al. (1998). Cooperative studies in health services research in the Department of Veterans Affairs, Controlled Clinical Trials 19, 134–148. Hobson, R.W., Weiss, D.G., Fields, W.S., et al. (1993). Efficacy of carotid endarterectomy for asymptomatic carotid stenosis, New England Journal of Medicine 328, 221–227. James, K.E. (1980). A model for the development, conduct, and monitoring of multicenter clinical trials in the Veterans Administration, Controlled Clinical Trials 1, 193–207. Kathe, B.A., Chan, Y.-K., Buehler, D.A., et al. (1981). Protection of patient rights and welfare in the VA

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39]

[40]

[41]

15

Cooperative Studies Program, Controlled Clinical Trials 2, 267–274. Kaufman, J.S., Reda, D.J., Fye, C.L., et al. (1998). Subcutaneous compared with intravenous epoetin in patients receiving hemodialysis, New England Journal of Medicine 339, 578–583. Klett, C.J. & Caffey, Jr, E.M. (1972). Evaluating the long-term need for antiparkinson drugs by chronic schizophrenics, Archives of General Psychiatry 26, 374–379. Lasky, J.J., Klett, C.J., Caffey, Jr, E.M., et al. (1962). Drug treatment of schizophrenic patients: a comparative evaluation of chlorpromazine, chlorprothixene, fluphenazine, reserpine, thioridazine and triflupromazine, Diseases of the Nervous System 23, 698–706. Lavori, P.W., Sugarman, J., Hays, M.T. & Feussner, J.R. (1999). Improving informed consent in clinical trials: a duty to experiment, Controlled Clinical Trials 20, 187–193. Lepor, H., Williford, W.O., Barry, M.J., et al. (1996). The efficacy of terazosin, finasteride, or both in benign prostatic hyperplasia, New England Journal of Medicine 335, 533–539. Lewis, H.D., Davis, J.W., Archibald, D.G., et al. (1983). Protective effects of aspirin against acute myocardial infarction and death in men with unstable angina: results of a Veterans Administration Cooperative Study, New England Journal of Medicine 309, 396–403. Light, R.W., O’Hara, V.S., Moritz, T.E., et al. (1990). Intrapleural tetracycline for the prevention of recurrent spontaneous pneumothorax. Results of a Department of Veterans Affairs Cooperative Study, Journal of the American Medical Association 264, 2224–2230. Ling, W., Charuvastra, C.V., Kaim, S.C. & Klett, C.J. (1976). Methyl acetate and methadone as maintenance treatment for heroin addicts: a Veterans Administration cooperative study, Archives of General Psychiatry 33, 709–720. Materson, B.J., Cushman, W.C., Goldstein, G., et al. (1990). Treatment of hypertension in the elderly: I. Blood pressure and clinical changes. Results of a Department of Veterans Affairs Cooperative Study, Hypertension 15, 348–360. Materson, B.J., Reda, D.J., Cushman, W.C., et al. (1993). Single-drug therapy for hypertension in men. A comparison of six antihypertensive agents with placebo, New England Journal of Medicine, 328, 914–921. Mayberg, M.R., Wilson, S.E., Yatsu, F., et al. (1991). Carotid endarterectomy and prevention of cerebral ischemia in symptomatic carotid stenosis, Journal of the American Medical Association 266, 3289–3294. Niewoehner, D., Erbland, M.L., Deupree, R.H., et al. (1993). Effect of systemic glucocorticoids on exacerbations of chronic obstructive pulmonary disease, New England Journal of Medicine 340, 1941–1947. Peduzzi, P., Guarino, P., Donta, S., et al. (2000). Design of an informed consent study to evaluate the utility of a

16

[42]

[43]

[44]

[45]

[46]

[47]

[48]

[49]

Cooperative Studies Program, US Department of Veterans Affairs focus group consent document in the VA cooperative study. A randomized multicenter controlled trial of multi-modal therapy in veterans with Gulf War illness (CSP 470). Presentation at the 21st Annual Meeting of the Society for Clinical Trials. Toronto, Canada. April 16–19. Peduzzi, P., Kamina, A. & Detre, K. (1998). Twenty-two year follow-up in the VA cooperative study of coronary artery bypass surgery for stable angina, American Journal of Cardiology 81, 1393–1399. Prien, R.F., Gillis, R.D. & Caffey, Jr, E.M. (1973). Intermittent pharmacotherapy in chronic schizophrenia, Hospital and Community Psychiatry 24, 317–322. Rosenheck, R., Cramer, J., Xu, W., et al. (1997). A comparison of clozapine and haloperidol in hospitalized patients with refractory schizophrenia, New England Journal of Medicine 337, 809–815. Rubins, H.B., Robins, S.J., Collins, D., et al. (1999). Gemfibrozil for the secondary prevention of coronary heart disease in men with low levels of highdensity lipoprotein cholesterol, New England Journal of Medicine 341, 410–418. Saudek, C.D., Duckworth, W.C., Giobbie-Hurder, A., et al. (1996). Implantable insulin pump vs. multipledose insulin for non-insulin-dependent diabetes mellitus. A randomized clinical trial, Journal of the American Medical Association 276, 1322–1327. Spechler, S.J. & the Department of Veterans Affairs Gastroesophageal Reflux Disease Study Group (1992). Comparison of medical and surgical therapy for complicated gastroesophageal reflux disease in veterans, New England Journal of Medicine 326, 786–792. Streptomycin Committee (1947). The effects of streptomycin on tuberculosis in man, Journal of the American Medical Association 135, 634–641. Takaro, T., Hultgren, H.N., Lipton, M.J., et al. (1976). The VA cooperative randomized study of surgery for coronary arterial occlusive disease. II. Subgroup with significant left main lesions, Circulation 54, (Suppl. III), 107–117.

[50]

[51]

[52]

[53]

[54]

[55]

[56]

[57]

The Digitalis Investigation Group (1997). The effect of digoxin on mortality and morbidity in patients with heart failure, New England Journal of Medicine 336, 525–533. The VA Cooperative Variceal Sclerotherapy Group (1991). Prophylactic sclerotherapy for esophageal varices in male alcoholics with cirrhosis: a randomized single blind multi-center clinical trial, New England Journal of Medicine, 324, 1779–1784. The Veterans Administration Systemic Sepsis Cooperative Study Group (1987). Effect of high-dose glucocorticoid therapy on mortality in patients with clinical signs of systemic sepsis, New England Journal of Medicine 317, 659–665. The Veterans Affairs Total Parenteral Nutrition Cooperative Study (1991). Perioperative total parenteral nutrition in surgical patients, New England Journal of Medicine 325, 525–532. VA Cooperative Study Group on Antihypertensive Agents (1967). Effects of treatment on morbidity in hypertension. Results in patients with diastolic blood pressures averaging 115 through 129 mm Hg, Journal of the American Medical Association 202, 1023–1034. VA Cooperative Study Group on Antihypertensive Agents (1970). Effects of treatment on morbidity in hypertension. II. Results in patients with diastolic blood pressure averaging 90 through 114 mm Hg, Journal of the American Medical Association 213, 1143–1152. Wasson, J.H., Reda, D.J., Bruskewitz, R.C., et al. (1995). A comparison of transurethral surgery with watchful waiting for moderate symptoms of benign prostatic hyperplasia, New England Journal of Medicine 332, 75–79. Weinberger, M., Oddone, E.Z., Henderson, W.G., et al. (1996). Does increased access to primary care reduce hospital readmissions?, New England Journal of Medicine 334, 1441–1447.

WILLIAM G. HENDERSON, PHILIP W. LAVORI, PETER PEDUZZI, JOSEPH F. COLLINS, MIKE R. SATHER & JOHN R. FEUSSNER

COORDINATING COMMITTEE A Coordinating Committee is a committee that a sponsor may organize to coordinate the conduct of a multicenter trial. If a coordinating committee and/or coordinating investigator(s) are used in multicenter trials, their organization and/or selection is the responsibility of the sponsor.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

COORDINATING INVESTIGATOR The Coordinating Investigator is an investigator who is assigned the responsibility for the coordination of the investigators at different centers that participate in a multicenter trial. If a coordinating committee and/or coordinating investigator(s) are used in multicenter trials, their organization and/or selection is the responsibility of the sponsor.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

CORONARY DRUG PROJECT

2.2 Outcome Variables The primary outcome variable was all-cause mortality for the entire follow-up period, with secondary outcomes of CHD and CV death, recurrent definite nonfatal MI, the combination of CHD death or definite nonfatal MI, cerebral stroke, and others (2).

PAUL L. CANNER Maryland Medical Research Institute Baltimore, Maryland

By 1960, evidence had accrued that linked elevated blood lipid levels with increased incidence of coronary heart disease (CHD). At the same time, the pharmaceutical industry was developing drugs that were effective in reducing blood cholesterol in persons with hyperlipidemia. The time had come to assess whether reduction of lipid levels would be effective in the treatment and possible prevention of CHD. In November 1960, proceedings were started that culminated in a line item in the Federal budget for funding in 1966 of the Coronary Drug Project (CDP) (1). 1

2.3 Sample Size and Duration of Follow-up From March 1966 to October 1969, 53 Clinical Centers (in the United States and Puerto Rico) recruited a total of 8341 patients— about 1100 in each of the five drug groups and 2789 in the placebo group (2). The 2.5 to 1 ratio of patients in the placebo group relative to each drug group was designed to minimize the total sample size while achieving a specified power relative to each of the five drug–placebo comparisons (3,4). Patients were followed until July 1974 with clinic visits and examinations every 4 months for a minimum of 5 years, a maximum of 8.5 years, and a mean of 6.2 years per patient on their assigned treatment regimen (5).

OBJECTIVES

The CDP was a randomized, double-blind, placebo-controlled clinical trial of the efficacy and safety of five lipid-modifying agents in men with previous myocardial infarction (MI) (2). The patients were followed for a minimum of 5 years to determine whether pharmaceutical modification of blood lipids would lead to improved survival and a reduction in cardiovascular (CV) mortality and morbidity. A secondary objective was to identify baseline prognostic factors for CV mortality and morbidity in the placebo group of patients.

2.4 Eligibility Criteria A prospective participant in the CDP had to be a male aged 30 to 64 years with electrocardiogram-documented evidence of an MI that occurred not less than 3 months previously. Insulin-dependent diabetics and persons already on lipid-modifying medications at time of entry were excluded (2). 2.5 The CDP Aspirin Study

2 STUDY DESIGN AND METHODS 2.1 Treatments

Three CDP treatment regimens were discontinued early because of adverse effects. The patients in these three groups who were eligible and willing (1529 patients altogether) were randomized into a short (10 to 28 months, mean 22 months) double-blind trial of 972 mg/day aspirin and placebo. As with the CDP, the primary outcome variable for the CDP Aspirin Study was all-cause mortality (6).

CDP patients were randomly allocated to six treatment arms: mixed conjugated equine estrogens at two dosage levels (2.5 and 5.0 mg/day), clofibrate (1.8 g/day), dextrothyroxine (6.0 mg/day), nicotinic acid (or niacin, 3.0 g/day), and a lactose placebo (3.8 g/day) (2). These treatments were dispensed in identical-appearing capsules (9 per day at full dosage). Both the patients and the clinical staff were blinded as to each patient’s treatment group except that side effects of the estrogen and niacin treatments tended to unblind these treatment groups.

2.6 Administrative Structure In addition to 53 clinical centers, the CDP had a Coordinating Center (at the present usually

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CORONARY DRUG PROJECT

called Data Coordinating Center; University of Maryland, Baltimore), a Central Laboratory (Centers for Disease Control, Atlanta), an ECG Reading Center (University of Minnesota, Minneapolis), and a Drug Procurement and Distribution Center (Public Health Service Supply Service Center, Perry Point, Maryland). Linkage of these centers with the National Heart, Lung, and Blood Institute (NHLBI) was provided through a Medical Liaison Officer, members of the Biometrics Research Branch, and a Health Scientist Administrator for medical, biostatistical, and budgetary matters, respectively. A Steering Committee composed of representatives of the Clinical Centers, Coordinating Center, Central Laboratory, ECG Reading Center, and NHLBI provided scientific direction for the study at the operational level (7). 2.7 Data and Safety Monitoring A Data and Safety Monitoring Committee (DSMC) composed of representatives from the Coordinating Center, Central Laboratory, ECG Reading Center, and NHLBI, the Chairman and Vice-Chairman of the Steering Committee, one outside statistician, and three clinicians who were not participants in the Clinical Centers met at least twice a year to review the treatment group data and make recommendations with regard to continuation or discontinuation of the study treatment groups. A Policy Board, which was composed of a clinical pharmacologist, a biostatistician, and three clinicians/cardiologists who had no direct involvement in the CDP, acted in a senior advisory capacity on policy matters throughout the duration of the study. The Policy Board reviewed and either ratified or disapproved recommendations from the DSMC (7,8). This model of both a DSMC and an independent Policy Board has evolved into the present-day Data and Safety Monitoring Board (DSMB) made up of scientists who are independent of the study. The members of the CDP DSMC and Policy Board wished not to be blinded as to identification of the study treatments in the data monitoring reports. They recognized that assimilating hundreds of pages of tables and graphs on a great variety of safety and efficacy outcomes in a short period was difficult

enough without their being blinded to treatment group identification as well. With treatment group blinding, significant patterns in the data with respect to treatment efficacy or safety might be missed easily. Furthermore, they realized that decisions concerning treatment efficacy are not symmetrical with those concerning treatment safety, with more evidence required for early stopping for efficacy than for safety. 3 RESULTS Three CDP treatment groups were terminated early, that is, before the scheduled end of the study; these groups were both estrogen groups and the dextrothyroxine group (9–11). The two remaining drug groups—clofibrate and niacin—and the placebo group were followed until the scheduled conclusion of the trial (5). 3.1 High Dose Estrogen In May 1970, a decision was reached to discontinue the 5.0 mg/day estrogen (ESG2) group because of an increased incidence of cardiovascular events. One major consideration in the deliberations over the ESG2 findings had to do with possible bias in diagnosing definite nonfatal MI and nonfatal thromboembolic events because of the treatment being unblinded in a large percentage of the patients because of feminizing side effects of the medication. Several special analyses were carried out to assess the possibility and extent of such bias. These analyses pertained to (1) incidence and duration of hospitalization for cardiac problems; (2) incidence of subsequent cardiovascular death in patients with definite nonfatal MI since entry; (3) incidence of several nonfatal cardiovascular events ranked in order of severity, counting only the single most serious event for a given patient; and (4) comparison of the centrally read electrocardiographic findings taken in connection with new MI events for the ESG2 and placebo groups. These analyses did not yield any evidence in support of the hypothesis of overdiagnosis of nonfatal MI in the ESG2 group (9,12). At its meeting of May 13, 1970, the DSMC reviewed the subgroup analyses shown in

CORONARY DRUG PROJECT

3

Table 1. Mortality and Morbidity in the High Dose Estrogen (ESG2) and Placebo Groups, Coronary Drug Project

Event

ESG2

Placebo

Risk groupa

n

%

n

%

z-value

All 1 2 All 1 2

1118 738 380 1022 684 338

8.1 5.1 13.9 6.2 6.7 5.0

2789 1831 958 2581 1689 892

6.9 6.1 8.5 3.2 2.9 3.7

1.33 −0.95 3.02 4.11 4.30 1.05

Total mortality Definite nonfatal MI

a Risk 1 = men with one MI without complications prior to entry into the study; Risk 2 = men with more than one previous MI or one MI with complications prior to entry. Source: Coronary Drug Project Research Group (12); reprinted from Controlled Clinical Trials © 1981, with permission from Elsevier.

Table 1. For this analysis, Risk 1 comprised men with one previous MI without complications prior to entry into the study and Risk 2 included men with either more than one previous MI or one MI with complications prior to entry. For the total group the z-value for the ESG2-placebo difference in total mortality was 1.33 and the corresponding z-value for definite nonfatal MI was 4.11. [A z-value is defined here as a drug-placebo difference in proportions of a given event, divided by the standard error of the difference; z-values of ±1.96 correspond to a conventional P-value of 0.05. However, given the multiple treatment groups, multiple endpoints (here, total mortality and definite nonfatal MI), and multiple reviews of the data during the course of the study, it was judged necessary to require much larger z-values than these to establish statistical significance (13,14)]. The DSMC agreed to discontinue the ESG2 treatment in Risk 2 patients only, but the Policy Board voted to discontinue the entire ESG2 treatment group (9,12). 3.2 Dextrothyroxine In October 1971, a decision was reached to discontinue the dextrothyroxine (DT4) treatment group in the CDP, based primarily on a higher mortality in the DT4 group compared with placebo. Although the z-value (1.88) for the difference did not achieve the conventional P < 0.05 level of statistical significance, and even less so when taking into account the five treatment versus placebo group comparisons, the DSMC members did not think that

conventional significance levels necessarily applied to adverse treatment effects. The deliberations that led to the decision for discontinuation focused largely on the question of whether the excess mortality was present consistently throughout the total group of DT4-treated patients or whether it was concentrated in certain subgroups (10). Table 2 gives the observed DT4 and placebo group findings for total mortality in subgroups defined by baseline risk categorization, history of angina pectoris, and ECG heart rate. Within each higher risk subgroup, there was a substantially higher mortality in the DT4 group than in the placebo group. Conversely, DT4 showed a somewhat lower mortality than placebo in the three lower risk subgroups. These subgroups that showed adverse effects of DT4 were identified following a survey of 48 baseline variables. Because no a priori hypotheses concerning DT4 effects in defined subgroups had been specified, the observed effects were treated as a posteriori findings in the evaluation of their statistical significance. The statistical analysis of these subgroup findings lay primarily in two directions (10). First, because the observed subgroup findings emerged from an analysis that involved 48 different baseline variables, it was desirable to determine whether the observed differences were any greater than might be expected by chance alone from evaluation of 48 variables. Computer simulations showed that the observed interactions with treatment for baseline heart rate and for history of

4

CORONARY DRUG PROJECT

Table 2. Percentage of Deaths in Selected Subgroups, Dextrothyroxine (DT4) and Placebo Groups, Coronary Drug Project Baseline Characteristic Risk Group,a 8/1/71 Risk 1 Risk 2 History of angina pectoris, 8/1/71 Negative Suspect/definite ECG heart rate, 8/1/71 <70/min ≥70/min Combination,b 8/1/71 Subgroup A Subgroup B Combination, 10/1/70 Subgroup A Subgroup B Combination in interval, 10/1/70–8/1/71 Subgroup A Subgroup B

DT4

Placebo

n

%

n

%

z-value

719 364

10.8 22.5

1790 925

11.0 15.4

−0.11 3.06

440 643

7.7 19.6

1142 1573

9.9 14.4

−1.33 3.06

576 494

9.5 21.3

1482 1194

10.7 14.7

−0.74 3.32

460 623

6.5 20.9

1210 1505

9.9 14.6

−2.17 3.58

460 623

4.1 16.4

1210 1505

7.7 11.2

−2.60 3.29

441 521

2.5 5.4

1117 1337

2.4 3.8

0.09 1.50

a Risk 1 = men with one MI without complications prior to entry into the study; Risk 2 = men with more than one previous MI or one MI with complications prior to entry. b Subgroup A = men with baseline ECG heart rate <70/min. and either Risk 1 or with no history of angina pectoris prior to entry; Subgroup B = men with either baseline ECG heart rate ≥70/min or Risk 2 plus history of suspect or definite angina pectoris prior to entry. Source: Coronary Drug Project Research Group (10); reprinted from Journal of the American Medical Association © 1972, American Medical Association; all rights reserved.

angina pectoris both fell in the 5% tail of the simulated distribution of maximum interaction effects. Thus, the observed difference in the effects of DT4 on mortality between the two subgroups defined by these two variables likely was real. The second approach to the statistical evaluation of subgroup findings focused on two rather complicated subgroups based on entry heart rate, history of angina pectoris, and risk categorization (Table 2). In the data report prepared for a previous DSMC meeting, these two subgroups were defined as a result of a trial and error process of maximizing the treatment–subgroup interaction with respect to total mortality. For Subgroup A the z-value for the DT4-placebo difference was −2.60 and for Subgroup B it was +3.29 (Table 2). The z-value for interaction was 3.96 for these subgroups. Because this subgroup was constructed solely because of analyzing the data many different ways, there was inadequate statistical

evidence that this would indeed be the best subgroup to stop. Hence, the patients were followed for another few months to observe whether DT4 continued to do poorly in Subgroup B and well in Subgroup A. During this additional follow-up period, in Subgroup B, DT4 continued to show a greater than 40% higher mortality than the placebo group (5.4% vs. 3.8%), thus justifying the choice of this subgroup as one in which DT4 treatment should be discontinued. However, in Subgroup A, there was no longer a mortality benefit with DT4 (2.5% vs. 2.4%; Table 2) (10). As a result of these and other data analyses, the DSMC, at its meeting on October 21, 1971, approved motions to discontinue DT4 medication in both Subgroup A and Subgroup B. These decisions were ratified by the CDP Policy Board (12). 3.3 Low Dose Estrogen In March 1973, a decision was reached to discontinue the 2.5 mg/day estrogen (ESG1)

CORONARY DRUG PROJECT

5

Table 3. Projection of Future Mortality Experience, Low Dose Estrogen (ESG1) and Placebo Groups, Coronary Drug Project

A. Current % deaths B. Current survivors C. % deaths at end of study, 1.96 SE difference D. Future % deaths given 1.96 SE difference at end of study

ESG1

Placebo

19.9 (219/1101) 882 21.1 (232/1101) 1.5 (13/882)

18.8 (525/2789) 2,264 24.0 (670/2789) 6.4 (145/2264)

Source: Coronary Drug Project Research Group (12); Controlled Clinical Trials © 1981, with permission from Elsevier.

group (11). This decision was based on an excess incidence of venous thromboembolism, an excess mortality (not statistically significant) from all cancers, and a small, statistically insignificant excess of total mortality in the ESG1 group compared to the placebo group. As of February 1, 1973, 19.9% of the patients in the ESG1 group had died, compared with 18.8% in the placebo group (Table 3) (11). This difference was not statistically significant, and there was no clear evidence that the drug was doing definite harm. The DSMC posed the question: What is the possibility that this trend could reverse itself with ESG1 showing a statistically significant beneficial effect with respect to all-cause mortality at the end of the study in summer 1974? For this to happen, future mortality in the two groups would have to be 1.5% for ESG1 compared with 6.4% for placebo (Table 3, line D). Because this outcome was considered to be extremely unlikely given the experience to date, this analysis plus the other considerations noted earlier led to the early discontinuation of the ESG1 group in 1973 (12). Years later, a striking parallel was noted in the mortality findings between the ESG1 versus placebo comparison in the CDP in men and the estrogen/progestin versus placebo comparison in the Heart and Estrogen/ progestin Replacement Study in women. In both studies, excess mortality was observed in the treated group in the first year or two followed by a slight mortality benefit in subsequent years (15).

3.4 Clofibrate At the conclusion of the CDP, the mortality of the clofibrate group was almost identical to that of the placebo group (25.5% vs. 25.4%) (5). However, it was not always this way. The DSMC reviewed interim reports of treatment versus placebo differences in all-cause mortality at 2-month intervals throughout the course of the study. For the clofibrate-placebo comparison, on three occasions during the first 30 months of the study the z-value exceeded the −1.96 boundary (signifying a conventional P-value less than 0.05) (12). If the DSMC had decided to stop the study and declare clofibrate therapeutically efficacious on the basis of these early ‘‘statistically significant’’ results, then it is evident that in retrospect it would very likely have been a wrong decision. However, the DSMC realized that the chances of finding significant differences in the absence of true differences became much higher than 5%—perhaps as high as 30 or 35%—when the data were examined repeatedly over time for such differences (16) and when five different drug groups were being compared with placebo at each of these time points. Several statistical methods were used in the CDP to take account of the repeated analysis of treatment effects in the decisionmaking process (13,14,17–20). The use of each of these methods required a much more extreme z-value than −1.96 early in the study to conclude that a statistically significant difference had been found. 3.5 Niacin The results in the niacin group at the end of follow-up on study medication in 1974

6

CORONARY DRUG PROJECT

were only modestly encouraging. Patients taking niacin experienced about 10% and 26% decreases in serum cholesterol and triglycerides, respectively, over 5 years of follow-up. Also, substantial reductions were observed in the niacin group in definite nonfatal MI (z = −3.09) and the combination of CHD death or definite nonfatal MI (z = −2.77) at the end of the trial. However, for the primary outcome—all-cause mortality—the findings for the niacin and placebo groups were almost identical. Thus, the following conclusion was reported by the CDP Research Group: ‘‘The Coronary Drug Project data yield no evidence that niacin influences mortality of survivors of myocardial infarction; this medication may be slightly beneficial in protecting persons to some degree against recurrent nonfatal myocardial infarction. However, because of the excess incidence of arrhythmias, gastrointestinal problems, and abnormal chemistry findings in the niacin group, great care and caution must be exercised if this drug is to be used for treatment of persons with coronary heart disease’’ (5). Nine years later, the NHLBI provided funding to the CDP Coordinating Center to conduct a mortality follow-up of CDP patients. The main interest was to ascertain whether an excess of cancer deaths had occurred in the two estrogen groups. The answer to that question turned out to be ‘‘no.’’ However, most surprisingly, a highly significant reduction in all-cause mortality was observed in the niacin group compared with placebo (52.0% vs. 58.2%; z = −3.52) as well as to all other CDP treatment groups (21). The survival curves for the niacin and placebo groups were nearly identical for the first 6 years of follow-up before starting to diverge (21). No clear explanation can be provided for the very long lag period in mortality benefit. Possibly, counterbalancing adverse effects of niacin had occurred—such as a significantly higher incidence of atrial fibrillation and other cardiac arrhythmias in the niacin group (5)—while patients were taking the drug. The CDP data leave unsettled the question of whether benefit or the suppression of benefit might derive from taking niacin for more than 6 years.

Patients who took niacin experienced an elevation of plasma fasting and 1-hour glucose levels over time—an apparent adverse effect of niacin. For this reason, the use of niacin in diabetic patients has been restrained, despite the encouraging findings with respect to 15-year mortality. However, recent analyses of the CDP data indicate no reduced mortality benefit in patients with higher plasma glucose levels at baseline or with increased glucose levels at 1 year (22). A similar outcome has been observed with respect to the metabolic syndrome (23). 3.6 The CDP Aspirin Study With a mean follow-up of 22 months, the allcause mortality in the aspirin and placebo groups was 5.8% and 8.3%, respectively (z = −1.90) (6). 3.7 Mortality by Level of Adherence Although clofibrate ultimately showed no benefit overall, study investigators as well as outside observers (24) raised the question as to whether the patients who adhered well to the clofibrate treatment regimen showed any benefit with respect to mortality and cardiovascular morbidity. Clofibrate and placebo group patients were classified according to their cumulative adherence—the estimated number of capsules actually taken as a percentage of the number that should have been taken according to protocol during the first 5 years of follow-up or until death, if earlier. For those with ≥80% adherence, 5-year mortality was nearly identical in the clofibrate and placebo groups (15.0% vs. 15.1%; Table 4). For those with <80% adherence, 5year mortality was somewhat lower for clofibrate (24.6%) than for placebo (28.2%). However, the most astounding finding came from focusing on the two figures for the placebo group: 15.1% mortality for good adherers versus 28.2% for poor adherers. The z-value for this difference was −8.12 (P = 4.7 × 10−16 ). Adjustment for 40 baseline characteristics reduced the mortality difference between good and poor adherers only to 16.4% versus 25.8% (z = −5.78). In conclusion, it is doubtful that any valid conclusions can be drawn from analyses like these because there is no way of ascertaining precisely how or

CORONARY DRUG PROJECT

7

Table 4. Five-Year Mortality in the Clofibrate and Placebo Groups, According to Cumulative Adherence to Protocol Prescription, Coronary Drug Project Clofibrate Adherencea <80% ≥80% Total study group

Placebo

n

% mortalityb

n

% mortalityb

357 708 1065

24.6 (22.5) 15.0 (15.7) 18.2 (18.0)

882 1813 2695

28.2 (25.8) 15.1 (16.4) 19.4 (19.5)

aA

patient’s cumulative adherence was computed as the estimated number of capsules actually taken as a percentage of the number that should have been taken according to the protocol during the first 5 years of follow-up or until death (if death occurred during the first 5 years). b The figures in parentheses are adjusted for 40 baseline characteristics. Source: Coronary Drug Project Research Group (25); reprinted from New England Journal of Medicine © 1980 Massachusetts Medical Society; all rights reserved.

why the patients in the treated and control groups have selected themselves or have become selected into the subgroups of good and poor adherers (25). Analyses of drugplacebo differences in mortality by change in serum cholesterol over time were similarly shown to give anomalous and uninterpretable results (25).

3.8 Studies of the Natural History of CHD As noted earlier, the CDP placebo group was sized 2 1/2 times larger—a total of 2789 patients—than any of the five drug groups so as to optimize the sample size for five drugplacebo comparisons. Another benefit of such a large placebo group was the ability to carry out several data analytical studies of the natural history of CHD in the placebo group, particularly, the evaluation of the relationship of 5-year mortality and cardiovascular morbidity to various risk cardiovascular risk factors measured or observed at baseline, either singly or in combinations (26–35).

3.9 Evaluation of Data from External Quality Control Programs Split samples of blood specimens were sent to the Central Laboratory in a blinded fashion to assess the technical error (measurement error at the laboratory plus any specimen deterioration that may have taken place during storage and shipment) of laboratory

determinations. Similarly, ECGs were resubmitted in a blinded fashion to the ECG Reading Center to assess the ECG reading variability. After attempting to select the most meaningful measure of technical error or reading variability among different possibilities, the CDP statisticians developed a new measure that related the measurement error to the total variability among patients of a particular laboratory determination or ECG reading. In addition, for ECG characteristics, such as Q/QS patterns, ST-depression, and T-wave patterns, for which significant worsening of ECG findings was compared between each treatment group and placebo, the duplicate readings of the same ECGs were used to determine ‘‘significant worsening’’ from the first to the second reading. In this way the amount of ‘‘significant worsening’’ from the baseline to a follow-up ECG attributable to reading variability could be determined (36). 4 CONCLUSIONS AND LESSONS LEARNED 4.1 Making Decisions with Respect to Treatment Efficacy and Safety Arriving at a decision for early termination of a treatment group or of an entire clinical trial, because of either beneficial or adverse results, is a complex process. It may involve, among other things, the need to (1) determine whether the observed treatment differences are likely to represent real effects and are not caused by chance; (2) weigh the importance of different response variables, some possibly trending in favor of the treatment and some

8

CORONARY DRUG PROJECT

against it; (3) adjust for differences in distributions of baseline characteristics among the treatment groups; (4) discern possible biases (because the study was not double blind) in the medical management of patients or in the diagnosis of events; and (5) evaluate treatment effects in subgroups of the study participants (12). 4.2 Assessment of Treatment Efficacy by Follow-up Responses to Treatment Analyses of drug–placebo differences in mortality by adherence to CDP treatment regimen or by change in serum cholesterol over time were shown to give anomalous and uninterpretable results. The reason is that there is no way of ascertaining precisely how or why the patients in the treated and control groups have selected themselves or have become selected into the subgroups of good and poor adherers or of cholesterol responders or nonresponders (25). 4.3 Assessment of Technical Error of Laboratory and ECG Determinations New methods for evaluating the technical error of Central Laboratory determinations and ECG reading variability of the ECG Reading Center were developed, relating these sources of error to the total variability among study patients in these laboratory determinations or ECG characteristics (36). 4.4 Impact of Treatment Group Findings on the Practice of Medicine Of the five original treatment regimens evaluated in the CDP, only niacin remains as a viable lipid-modifying regimen. Because of the long-term mortality benefit of immediaterelease niacin shown in the CDP, there is renewed interest among cardiologists in prescribing this medication for modification of serum lipid levels in persons at risk for CHD. Also, more recently, more clinical trials of immediate-, sustained-, and extendedrelease niacin, in combination with other medications that impact blood lipids, have been conducted (37–39). The results of the short-duration study of aspirin and placebo in patients whose CDP treatment regimens had been discontinued have led to several additional studies

of aspirin in patients with CHD (40,41) and a resultant promotion of this popular medication to the forefront of prevention and treatment of MI. REFERENCES 1. W. J. Zukel, Evolution and funding of the Coronary Drug Project. Control. Clin. Trials 1983; 4: 281–312. 2. Coronary Drug Project Research Group, The Coronary Drug Project. Design, methods, and baseline results. Circulation 1973; 47: I1–I79. 3. P. L. Canner and C. R. Klimt, Experimental design features of the Coronary Drug Project. Control. Clin. Trials 1983; 4: 313–332. 4. C. W. Dunnett, Multiple comparison procedures for comparing several treatments with a control. J. Am. Statist. Assoc. 1955; 50: 1096–1121. 5. Coronary Drug Project Research Group, Clofibrate and niacin in coronary heart disease. JAMA 1975; 231: 360–381. 6. Coronary Drug Project Research Group, Aspirin in coronary heart disease. J. Chron. Dis. 1976; 29: 625–642. 7. P. L. Canner and J. Stamler, Organizational structure of the Coronary Drug Project. Control. Clin. Trials 1983; 4: 333–343. 8. P. L. Canner, Monitoring of the data for evidence of adverse or beneficial treatment effects in the Coronary Drug Project. Control. Clin. Trials 1983; 4: 467–483. 9. Coronary Drug Project Research Group, The Coronary Drug Project. Initial findings leading to modifications of its research protocol. JAMA 1970; 214: 1303–1313. 10. Coronary Drug Project Research Group, The Coronary Drug Project. Findings leading to further modifications of its protocol with respect to dextrothyroxine. JAMA 1972; 220: 996–1008. 11. Coronary Drug Project Research Group, The Coronary Drug Project. Findings leading to discontinuation of the 2.5 mg/day estrogen group. JAMA 1973; 226: 652–657. 12. Coronary Drug Project Research Group, Practical aspects of decision making in clinical trials: The Coronary Drug Project as a case study. Control. Clin. Trials 1981; 1: 363– 376. 13. P. Armitage, C. K. McPherson, and B.C. Rowe, Repeated significance tests on accumulating data. J. Royal Statist. Soc. A 1969; 132: 235–244.

CORONARY DRUG PROJECT 14. P. L. Canner, Monitoring treatment differences in long-term clinical trials. Biometrics 1977; 33: 603–615. 15. N. Wenger, G. L. Knatterud, and P. L. Canner, Early risks of hormone therapy in patients with coronary heart disease. JAMA 2000; 284: 41–43. 16. P. Armitage, Sequential Medical Trials, 2nd ed. New York: Wiley, 1975. 17. J. Cornfield, Sequential trials, sequential analysis and the likelihood principle. Am. Statist. 1966; 20: 18–23. 18. J. Cornfield, A Bayesian test of some classical hypotheses—with applications to sequential clinical trials. J. Am. Statist. Assoc. 1966; 61: 577–594. 19. J. Cornfield, The Bayesian outlook and its application. Biometrics 1969; 25: 617–657. 20. M. Halperin and J. Ware, Early decision in a censored Wilcoxon two-sample test for accumulating survival data. J. Am. Statist. Assoc. 1974; 69: 414–422. 21. P. L. Canner, K. G. Berge, N. K. Wenger, J. Stamler, L. Friedman, R. J. Prineas, and W. Friedewald, for the Coronary Drug Project Research Group, Fifteen year mortality in Coronary Drug Project patients: Long-term benefit with niacin. J. Am. Coll. Cardiol. 1986; 8: 1245–1255. 22. P. L. Canner, C. D. Furberg, M. L. Terrin, and M. E. McGovern, Benefits of niacin by glycemic status in patients with healed myocardial infarction (from the Coronary Drug Project). Am. J. Cardiol. 2005; 95: 254–257. 23. P. L. Canner, C. D. Furberg, and M. E. McGovern, Benefits of niacin in patients with versus without the metabolic syndrome and healed myocardial infarction (from the Coronary Drug Project). Am. J. Cardiol. 2006; 97: 477–479. 24. A. R. Feinstein, Clinical biostatistics. XLI. Hard science, soft data, and the challenges of choosing clinical variables in research. Clin. Pharmacol. Ther. 1977; 22: 485–498. 25. Coronary Drug Project Research Group, Influence of adherence to treatment and response of cholesterol on mortality in the Coronary Drug Project. N. Engl. J. Med. 1980; 303: 1038–1041. 26. Coronary Drug Project Research Group, The prognostic importance of the electrocardiogram after myocardial infarction: Experience in the Coronary Drug Project. Ann. Intern. Med. 1972; 77: 677–689.

9

27. Coronary Drug Project Research Group, Prognostic importance of premature beats following myocardial infarction: Experience in the Coronary Drug Project. JAMA 1973; 223: 1116–1124. 28. Coronary Drug Project Research Group, Left ventricular hypertrophy patterns and prognosis. Experience postinfarction in the Coronary Drug Project. Circulation 1974; 49: 862–869. 29. Coronary Drug Project Research Group, Factors influencing long-term prognosis after recovery from myocardial infarction—threeyear findings of the Coronary Drug Project. J. Chron. Dis. 1974; 27: 267–285. 30. Coronary Drug Project Research Group, Serum uric acid: Its association with other risk factors and with mortality in coronary heart disease. J. Chron. Dis. 1976; 29: 557–569. 31. Coronary Drug Project Research Group, The prognostic importance of plasma glucose levels and of the use of oral hypoglycemic drugs after myocardial infarction in men. Diabetes 1977; 26: 453–465. 32. Coronary Drug Project Research Group, The natural history of myocardial infarction in the Coronary Drug Project: Long-term prognostic importance of serum lipid levels. Am. J. Cardiol. 1978; 42: 489–498. 33. Coronary Drug Project Research Group, Cigarette smoking as a risk factor in men with a prior history of myocardial infarction. J. Chron. Dis. 1979; 32: 415–425. 34. Coronary Drug Project Research Group, The natural history of coronary heart disease: Prognostic factors after recovery from myocardial infarction in 2789 men. The 5-year findings of the Coronary Drug Project. Circulation 1982; 66: 401–414. 35. Coronary Drug Project Research Group, Highdensity lipoprotein cholesterol and prognosis after myocardial infarction. Circulation 1982; 66: 1176–1178. 36. P. L. Canner, W. F. Krol, and S. A. Forman, External quality control programs in the Coronary Drug Project. Control. Clin. Trials 1983; 4: 441–466. 37. G. Brown, J. J. Albers, L. D. Fisher, S. M. Schaefer, J.T. Lin, C. Kaplan, X. Q. Zhao, B. D. Bisson, V. F. Fitzpatrick, and H. T. Dodge, Regression of coronary artery disease as a result of intensive lipid-lowering therapy in men with high levels of apolipoprotein B. N. Engl. J. Med. 1990; 323: 1289–1298. 38. B.G. Brown, X. Q. Zhao, A. Chait, L. D. Fisher, M. C. Cheung, J. S. Morse, A. H. Dowdy, E. K. Mariino, E. L. Bolson, P. Alaupovic, et

10

CORONARY DRUG PROJECT

al., Simvastatin and niacin, antioxidant vitamins, or the combination for the prevention of coronary disease. N. Engl. J. Med. 2001; 345: 1583–1592. 39. A. J. Taylor, L. E. Sullenberger, H. J. Lee, J. K. Lee, K. A. Grace, Arterial Biology for the Investigation of the Treatment Effects of Reducing Cholesterol (ARBITER) 2. A doubleblind, placebo-controlled study of extended release niacin on atherosclerosis progression in secondary prevention patients treated with statins. Circulation 2004; 110: 3512–3517. 40. Aspirin Myocardial Infarction Study Research Group, A randomized controlled trial of aspirin in persons recovered from myocardial infarction. JAMA 1980; 243: 661–669. 41. P. L. Canner, An overview of six clinical trials of aspirin in coronary heart disease. Stat. Med. 1987; 6: 255–263.

FURTHER READING Coronary Drug Project Research Group, The Coronary Drug Project: Methods and lessons of a multicenter clinical trial. Control. Clin. Trials 4: 273–536. This monograph includes the following chapters: 1. Brief description of the Coronary Drug Project and other studies; 2. Evolution and funding of the Coronary Drug Project; 3. Experimental design features; 4. Organizational structure of the study; 5. Role of the National Institutes of Health; 6. Role and Methods of the Coordinating Center; 7. Role and Methods of the Central Laboratory; 8. Role and Methods of the ECG Reading Center; 9. Role and Methods of the Drug Procurement and Distribution Center; 10. Role and Methods of the Clinical Centers; 11. Design of data forms; 12. External quality control programs; 13. Monitoring of the data for evidence of adverse or beneficial treatment effects; 14. Further aspects of data analysis; 15. Closing down the study; 16. Impact of the Coronary Drug Project on clinical practice.

CROSS REFERENCES Adherence Data Monitoring Committee Early Termination - Study Stopping Boundaries Subgroup Analysis

CORRELATION

association to a straight line. If the variables X and Y are random variables with a joint probability distribution, then the correlation coefficient is defined as

PETER ARMITAGE University of Oxford Oxford, UK

ρ=

In a rather loose sense, two characteristics or variables are said to be correlated if changes in one variable tend to be accompanied by changes in the other, in either the same or the opposite direction. Thus, the incidence of ischemic heart disease is positively correlated with the softness of drinking water, since many epidemiologic studies have shown that the incidence tends to be higher in areas with softer water; and, conversely, the incidence is negatively correlated with water hardness. In view of the more specific definitions to be discussed below, it would perhaps be preferable to use the term association for this informal usage. Correlation implies a linear relationship with superimposed random variation; association often loosely implies a monotone relationship, but the term may also be applied to nominal data where rank order is undefined. For an account of the early history of the term correlation, see (10), especially pp. 297–299. The term was current during the middle of the nineteenth century, but its statistical usage is rightly attributed to Francis Galton, who initially used the spelling ‘‘co-relation’’. Galton was concerned with the correlation between characteristics of related individuals; for example, between an individual’s height and the mean height of the two parents. The correlation coefficient emerged from Galton’s work, after further elucidation by F.Y. Edgeworth and Karl Pearson, to become a central tool in the study of relationships between variables, especially (in Pearson’s work) between physiological and behavioral measurements on human beings.

σXY , σX σY

(1)

where σ XY , σ X , and σ Y are, respectively, the covariance of X and Y and the standard deviations of X and Y. The value of ρ is bounded between −1 and +1, taking these extreme values only when there is a linear functional relation between X and Y. Thus, if Y = α + βX exactly (as, for instance, with temperatures recorded in Fahrenheit for Y and centigrade for X), then ρ = 1 if β > 0 (as in this example), and ρ = −1 if β < 0. Biologic variables are not normally connected by linear functional relations, and the correlation coefficient usually lies between the two extremes. There is a close connection between the concept of correlation and that of linear regression. Let β Y.X be the slope of the linear regression of Y on X, and β X.Y that of the regression of X on Y. Then βY.X =

σXY , σX2

βX.Y =

σXY σY2

and, from (1), β Y.X β X.Y = ρ 2 . Since ρ 2 ≤ 1, |β Y.X |≤|1/β X.Y | (the latter expression being the slope of the regression of X on Y in a diagram with Y as the ordinate), equality being achieved only for perfect correlation. Thus, the two regression lines are in general inclined at an angle. When the correlation coefficient ρ = 0, both β Y.X and β X.Y are zero, and the two regression lines are at right angles. For a particular value of X, define Y 0 to be the value predicted by the linear regression of Y on X. Then the variance of the residuals about regression, E[(Y − Y 0 )2 ] is equal to σY2 (1 − ρ 2 ). One interpretation of the correlation coefficient is, therefore, the fact that its square is the proportion of the variance of one variable that is ‘‘explained’’ by linear regression on the other. The relationship is symmetric, being equally true for the other regression.

1 THE PRODUCT– MOMENT CORRELATION COEFFICIENT The product–moment correlation coefficient (normally abbreviated to correlation coefficient) is a measure of the closeness of the

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

2

CORRELATION

THE SAMPLE CORRELATION

The correlation coefficient may also be defined, in similar manner, for a finite set of n paired quantitative observations (x1 , y1 ), . . . , (xn , yn ). The correlation coefficient, denoted now by r, may be calculated as
(xi − x)(yi − y) r = 
1/2 .
(xi − x)2 (yi − y)2

(2)

Here, x and y are the mean values of the xi and yi , respectively, and the summations run from 1 to n. The basic properties of the sample correlation coefficient are essentially those outlined above for random variables. The relationship with regression now applies to the slopes of the least squares regression lines, by.x and bx.y . The squared correlation coefficient, r2 , is the proportion of the total sum of squares (yi − y)2 ‘‘explained’’ by the regression of y on x. Some scatter diagrams representing simple situations are shown schematically in Figure 1. In Figures 1(a) and (e) the points lie on a straight line, and r = +1 and −1, respectively. In Figure 1(c) the variation in one variable is approximately independent of the value of the other variable, and r = 0. In Figures 1(b) and (d) there is an intermediate degree of correlation, the variance of one variable being reduced when the value of the other variable is fixed, so 0 < r < 1 for (b) and −1 < r < 0 for (d). The sample correlation coefficient given in (2) is sometimes referred to as the Pearson product–moment correlation coefficient, the reference here being to Karl Pearson (7). It provides the method of calculation for any finite set of paired values, and invites consideration of its sampling error. 3

SAMPLING ERROR

Suppose that the n pairs of observations are drawn at random from a bivariate distribution of random variables X and Y. The sampling distribution of r will depend on the characteristics of the parent distribution, in particular on the population correlation

Figure 1. A schematic representation of scatter diagrams with regression lines, illustrating different values of the correlation coefficient. Reproduced from 1 by permission of Blackwell Science, Oxford

coefficient ρ. In general, r is a consistent but biased estimator of ρ, but the bias is of order 1/n and likely to be small except for very small values of n. More specific results are available if stronger assumptions are made about the nature of the parent distribution, and the traditional assumption is that of a bivariate normal distribution. This model was widely used in the early work of Galton and Karl Pearson; appropriately enough, since it provides a reasonable description of many of the biometric variables studied by them. Under the bivariate normal assumption, the distribution of r depends only on ρ and n. The density was first derived in 1915 by Fisher (4) and subsequently tabulated by David (3). If ρ = 0, then the statistic (n − 2)1/2 r (1 − r2 )1/2

(3)

CORRELATION

follows a Student’s t distribution with n − 2 degrees of freedom, and can be used to test the null hypothesis that ρ = 0. In fact, this test is valid more generally, for the standard model for linear regression, in which the values x of X are chosen arbitrarily but Y is distributed normally with constant variance around a linear function of x. Returning to the bivariate normal model, Fisher derived a variance-stabilizing transformation of r,   1+r 1 , z = tanh−1 r = log 2 1−r the distribution of which approaches normality more rapidly than that of r, as the sample size, n, increases. Asymptotically, E(z) = tanh−1 ρ, and, approximately, var(z) = 1/(n − 3). An alternative transformation (9) provides a generalization of (3): for ρ = 0, the statistic (n − 2)1/2 (r − ρ) [(1 − r2 )(1 − ρ 2 )]1/2 follows approximately a t distribution with n − 2 degrees of freedom. For further details about the distribution of r, see [6, Chapter 10]. 4

INTRACLASS CORRELATION

Suppose that observations on a single variable y are arranged in n groups, each containing m observations, and that there is reason to expect possible differences in the mean level of y between groups. If such differences exist, observations in the same group will tend to be positively correlated. This phenomenon is called intraclass correlation. Fisher (5) illustrates this for the case m = 2 by referring to measurements on pairs of brothers. He distinguishes between situations in which the brothers fall into labeled categories such as ‘‘elder’’ and ‘‘younger’’, and those in which no such categorization is required. In the first case, there are two variables—measurements for elder and younger brothers—and the standard product–moment correlation coefficient may be calculated. This is called the interclass correlation. In the second case, each pair would enter into the calculation twice, since they

3

are not naturally ordered, and the denominator of the correlation coefficient may be based on a single sum of squares about the mean for all the 2n observations. This is the intraclass correlation. It is interesting to note that Fisher used the term ‘‘class’’ to denote the possible labeling categories (‘‘elder’’ and ‘‘younger’’ here), whereas many modern writers use it to denote the groups into which the observations are clustered (‘‘sibships’’ here). The intraclass correlation coefficient, rI , may be calculated as a modified variant of (2) for all the nm(m − 1) pairs of observations in the same group, each pair being counted twice. In the cross product in the numerator in (2), all deviations are taken from the mean, y, of all mn observations. Since each observation appears m − 1 times in the cross product, the denominator of (2) is m − 1 times the sum of squares of deviations of all mn observations about y. An equivalent formula for rI clarifies the relation between intraclass correlation and variation between the group means. Denote by yi the mean for the ith group, and by v the variance of the mn observations with divisor mn rather than mn − 1. Then

rI =

m


(yi − y)2 − nv (m − 1)nv

,

the summation running from 1 to n. It follows that 1 ≤ rI ≤ 1, − (m − 1) the lower limit of −1/(m − 1) being achieved when all the yi , are equal, and the upper limit of 1 when there is no variation within the groups so that (yi − y)2 = nv. Data of the type considered here would normally be analyzed by a one-way analysis of variance, and, as Fisher (5) showed, there is a close connection between the two approaches. If, in the analysis of variance, the mean squares between and within groups are denoted by s2b and s2w , respectively, then

rI =

s2b −



n (n−1)

s2b + (m − 1)





s2w 

n (n−1)

, s2w

4

CORRELATION

so, for large n, rI ∼

s2b − s2w s2b + (m − 1)s2w

.

Equivalently, rI ∼ (F − 1)/(F + m − 1), where F is the usual variance ratio statistic, s2b /s2w . Two approaches may be followed in discussion of the sampling error of the intraclass correlation coefficient, using either finite population theory as is usual for, or the random effects model more usual in biologic applications. The first approach assumes that the n groups are randomly selected from a larger set of N, and that each set of m observations within a group is randomly selected from M. If the intraclass correlation coefficient for the finite population of MN observations is denoted by ρ I , the sample value rI may be regarded as an estimator of ρ I . The random effects model effectively assumes infinite values for M and N. The group means are assumed to be distributed with a variance component σ 2b , and the withingroup deviations with a component σ 2w . Then ρI =

σ 2b σ 2b

+ σ 2w

.

Inferences about ρ I may be made using standard results for the F distributions in the one-way analysis of variance. Some examples of the use of intraclass correlation as a descriptive tool are as follows: 1. In sample survey theory, to indicate the correlation between observations in the same cluster due to systematic between-cluster variation. 2. In statistical genetics, to indicate the correlations in genetic traits between members of the same family: see the articles on familial correlations (which deals in detail with the situation in which the family groups vary in size) and genetic correlations and covariances. The numerical values of intraclass correlation coefficients are more meaningful in genetics than in most other applications, because of the predictions of Mendelian theory,

although the predicted values for familial correlations, for example between siblings, may be distorted by the additional effects of environmental correlation. 3. In studies of the reliability of repeated measurements, or the agreement between observers in measuring characteristics of the same subject. Note that when the same observers are used for each subject, there may be systematic differences in the level of recording for different observers, and the oneway analysis of variance analogue is no longer valid (2).

5 SOME GENERALIZATIONS The concepts underlying the product– moment correlation coefficient may be generalized or modified in various ways. The (x, y) pairs may not be closely linearly related, but may nevertheless be perfectly associated through a nonlinear relation. If this relation is monotone, the observations will be ranked in the same order by both x and y. Two commonly used coefficients of rank correlation are Spearman’s ρ (essentially the product–moment correlation of the ranks), and Kendall’s τ (based on the number of discrepancies in the ranking of paired observations by the two variables). Both coefficients are bounded by the values −1 and +1 for perfect negative and positive agreement between the rankings. When there are more than two quantitative variables, the correlations between pairs play an important part in various methods of multivariate analysis. In multiple linear regression, two generalizations of r are commonly used. The multiple correlation coefficient, R, generalizes a property of r, in that the proportion of the variance of the dependent variable y that is ‘‘explained’’ by the multiple regression on x1 , . . . , xk is R2 . (Since the squared form carries the essential information, R is never given a negative sign.) The partial correlation coefficient measures the product–moment correlation between y and one of the predictor variables, xi say, when all the other predictors are kept constant.

CORRELATION

It is therefore useful in assessing the separate effects of different predictors, especially if they are themselves closely correlated. In the early work of the Galton–Pearson school, much effort was put into the estimation of correlation coefficients when the observed values were nominal—perhaps even binary—but were supposed to represent divisions of some underlying, unobserved, continuous variables. The correlation between the presumed continuous variables had to be estimated from the discrete observations. It was usually assumed that the underlying distribution was bivariate normal. For two binary classifications, the measure is called tetrachoric correlation. When one variable is binary and the other is quantitative, the measure is called biserial correlation. These methods are less frequently used now, as alternate models for categorical data seem more appropriate. 6

INTERPRETATION

The high profile assumed by the concept of correlation during the early part of the twentieth century has now largely vanished. This is partly due to the emergence of more penetrating methods of statistical analysis. In particular, the emphasis has gradually moved away from an index measuring a degree of association, to an attempt to describe more explicitly the nature of that association. In a word, the emphasis has moved from correlation toward regression. Apart from this general shift in viewpoint, there are some specific problems in the interpretation of correlation coefficients, which need to be taken into account in any data analysis in which they are used: 1. Correlation does not imply causation. Two variables may be highly correlated because they are both causally related to a third variable or a group of such variables, and yet have no causative relation to each other. Relations of this type are often called nonsense or spurious correlations. Often, the intervening variable is time. That is, two variables x and y may both be steadily increasing with time, over a certain time period,

5

or one may be increasing while the other decreases. The two variables are then likely to be highly correlated. For instance, during the first two-thirds of the twentieth century, imports of tobacco into the UK increased steadily, as did the number of divorces granted. The two variables, measured in successive decades, are highly correlated. It would certainly not be correct to assume that either variable caused the other. Nonsense correlations are among the most prevalent causes of injudicious inferences from statistical data by the general public and the media. 2. Correlation measures closeness to a linear relationship. Two variables may be very closely associated by a nonlinear relation, and yet have a low correlation coefficient. As an extreme example, in Figure 2 (reproduced from (8)) is shown a scatter diagram between randomly drawn standard normal deviates and their squares. The population correlation coefficient is exactly zero, but it would have been entirely wrong to assume a lack of dependence. Independent random variables have zero correlation; but zero correlation does not imply independence. 3. The correlation between two biologic variables may be affected by selection of particular values of one variable. For instance, if X and Y have a distribution approximating to a bivariate normal distribution, with correlation coefficient ρ, restriction of the range of X by removal of extreme values in both directions will tend to decrease the correlation coefficient below the original value ρ. Thus, the correlation between height and age of children is higher for the age range 5–12 years than for the range 7–8 years. Conversely, omission of central values of X with retention of extreme values will tend to increase the correlation coefficient. This phenomenon may make it difficult to compare correlation coefficients in different populations differing in the degree and type of selection.

6

CORRELATION

REFERENCES

Figure 2. A scatter plot of Y = X 2 for 100 standard normally distributed random numbers and their squares. Reproduced from 8 by permission of Wiley, New York

4. The effect of sampling variation is often underestimated, so that undue importance is given to moderately high correlations based on few observations. The upper 5 percentile of the distribution of |r| when the population value ρ = 0, from (3), is 0.878 for n = 5, and 0.632 for n = 10. In this sense, moderately large correlations from small numbers of observations are inherently unreliable.

1. Armitage, P., Berry, G. & Matthews, J. N. S. (2002). Statistical Methods in Medical Research, 4th Ed. Blackwell Science, Oxford. 2. Bartko, J. J. (1966). The intraclass correlation coefficient as a measure of reliability, Psychological Reports 19, 3–11. 3. David, F. N. (1938). Tables of the Correlation Coefficient. Cambridge University Press, Cambridge. 4. Fisher, R. A. (1915). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population, Biometrika 10, 507–521. 5. Fisher, R. A. (1925). Statistical Methods for Research Workers. Oliver & Boyd, Edinburgh. 6. Patel, J. K. & Read, C. B. (1982). Handbook of the Normal Distribution. Marcel Dekker, New York. 7. Pearson, K. (1896). Mathematical contributions to the theory of evolution, III: regression, heredity and panmixia, Philosophical Transactions of the Royal Society of London, Series A 187, 253–318. 8. Rodriguez, R. N. (1982). Correlation, in Encyclopedia of Statistical Sciences, Vol. 2, S. Kotz & N. L. Johnson, eds. Wiley, New York, pp. 193–204. 9. Samiuddin, M. (1970). On a test for an assigned value of correlation in a bivariate normal distribution, Biometrika 57, 461–464. 10. Stigler, S. M. (1986). The History of Statistics. Belknap Press, Cambridge, Mass.

COST-EFFECTIVENESS ANALYSIS

short review of definitions and design issues will be given.

MORVEN LEESE Institute of Psychiatry—Health Service and Population Research Department, London, UK

1

DEFINITIONS AND DESIGN ISSUES

In health economics, evidence from both clinical and economic perspectives is considered together. The result might be to prioritize interventions, set research priorities, or devise a definitive recommendation about whether a specific intervention should be adopted as being both clinically effective and good value for money. The first step in a health economics study would be to gather and present costs and clinical consequences side by side, without them necessarily being entered into a formal costeffectiveness analysis (a process referred to as a ‘‘cost-consequence’’ analysis). Costeffectiveness analysis (CEA) in a clinical trial, as discussed in this article, is a more specific analysis that takes this process a step further. It typically aims to provide information about the value for money of a small number of competing treatments, often just two—the new intervention and its comparator. This comparison is usually expressed through the incremental cost effectiveness ratio (ICER), which is the cost per unit of benefit obtained from the intervention compared with the comparator. A related measure is the incremental net benefit (INB), which is the overall net cost for a given willingness-topay per unit of benefit. The INB has recently become much more widely used for inferential purposes than the ICER, because of analytical problems that develop from the ICER’s ratio form. However, the comparison made directly within the clinical trial framework may provide only partial information on cost-effectiveness (1). Consequently, modeling is often used to generalize the results of a CEA to time frames or patient groups other than those used in the trial. The various approaches to making and generalizing cost-effectiveness comparisons in clinical trials are described later in this article. First, a

Cost-effectiveness analysis is a method for comparing the costs and clinical outcomes of competing courses of action, which allows health providers and policy makers to choose between them within budgetary constraints. The procedures to be compared may be drug treatments, psychological interventions, or health-management systems. It is usually regarded as an essential component of a clinical trial and regulatory bodies may require the submission of economic information as part of the process of reimbursement (2). Cost-effectiveness analysis allows different procedures to be compared in terms of cost per unit of outcome. Cost utility analysis (CUA) is similar to cost effectiveness analysis (and is sometimes considered to be a branch of it), except that outcomes are measured in generalized units of utility, rather than in disease-specific units such as symptom reduction scores. CUA enables mortality and morbidity to be combined into a single measure, for example as quality-adjusted life years (QALYs), and allows the comparison of treatments for different diseases to be placed on a common basis. Cost minimization analysis (CMA) is also used in economic evaluation. It is only relevant where competing procedures can be regarded as having equal benefit, in which case it makes sense to compare their costs directly. CMA is less commonly used in clinical trials than CEA or CUA, because equivalence of treatments is difficult to establish (3). Both the terms ‘‘cost effectiveness analysis’’ and ‘‘cost benefit analysis’’ (CBA) are sometimes used to refer to any kind of economic evaluation, but they should be distinguished. CEA differs from CBA in that the latter measures both costs and consequences in monetary terms, whereas in CEA the consequences are measured in terms of a common health outcome. CBA, which is

1.1 The Various Types of Analysis used in Economic Evaluation

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

COST-EFFECTIVENESS ANALYSIS

described in another article, tends to be used in a general context where very different health technologies, often more than two, are to be compared with each other and/or with other modes of expenditure within an overall budget (such as the national economy). 1.2 The Economic Perspective Culyer (4) defines the economic perspective as ‘‘the viewpoint adopted for the purposes of an economic appraisal (cost effectiveness, costutility studies and so on) which defines the scope and character of the costs and benefits to be examined, as well as other critical features, which may be social value-judgemental in nature, such as the rate of discount.’’ The perspective of the patient would be concerned with, for example, symptom reduction or loss of income because of illness. The perspective of the health provider would include the costs of drugs or hospital beds, and the perspective of society might include loss of taxes because of illness-related unemployment. If appropriate, the analysis can be presented from more than one perspective. It should be said, however, that in many clinical trials a limited perspective is forced onto the investigator, especially if the economic component of a clinical trial has been ‘‘piggybacked’’ on to an existing trial design. Funding bodies may insist on a particular perspective for submissions. For example, the National Institute for Health and Clinical Excellence in England and Wales specifies this to be all health outcomes on individuals, and costs to the National Health Service and Personal Social Services (5). The U.S. Public Health Service panel requires a societal perspective (6). 1.3 Choice of Comparator From the health economist’s point of view, the choice of comparator would ideally be current best practice (if it can be adequately defined), for example, one that has been shown to be most effective or cost effective in the past. However, the comparator might simply be the currently most widely used treatment. The choice would depend on the scenario in which the economic decision is to be made. Placebo controls would rarely be appropriate; exceptions occur in which the new drug is to

be used as an adjunctive therapy to supplement rather than to substitute an existing intervention (7), or if it is necessary to obtain regulatory approval.

1.4 Setting and Timescale The economic perspective will influence the setting and length of time to which the economic analysis applies (the analytic horizon). The ideal time horizon from an economic point of view would be that over which the intervention being evaluated might have an effect on costs and benefits. In practice, however, shorter time horizons—such as the duration of the effectiveness trial—are chosen for practical and cost reasons. When the relevant data cannot be measured directly during the clinical trial, it is common practice to extrapolate the results by modeling. This process may entail additional data gathering outside the trial, either from surveys or published literature. If the results are to be generalized to different patient groups (for example from an urban setting to a rural setting), then information on costs and relevant patient characteristics may need to be gathered from the different settings so that modeling can be undertaken. A consideration for both costs and benefits is discounting to value future costs to the present day (8). A typical rate would be 3.5% per annum for costs. Discounting health outcomes can also be applied and a similar rate to that used for costs would usually be appropriate for consistency. (Note that inflation adjustment for costs is a different procedure that may also be applied, in addition to discounting.) The net-present value of the streams of costs and benefits, and hence the cost-effectiveness ratio, may be affected by discounting even when the same timescales are involved. However, it becomes a particular issue when benefits operate over different timescales to costs, for example where a current treatment is compared with a preventive measure. In practice, discounting is not considered in many clinical trials, on the basis that the rate would apply equally in both arms.

COST-EFFECTIVENESS ANALYSIS

1.5 Sample Size As a rule, sample sizes are larger for an economic evaluation than for the corresponding clinical evaluation. This larger sample size occurs because many possible sources of variation exist for usage indicators and costs, so that standard deviations are generally very high. It is worth noting that in cluster-randomized designs, sample sizes might be even larger. Not only must the number of individuals be increased according to the design effect, but also the minimum number of clusters should be increased because of the special features of health economics data (9). Simulation or bootstrapping approaches for sample sizes based on the ICER have been developed (10). A Bayesian approach has also been suggested (11), although it has not been very widely adopted possibly because of its relative complexity. A straightforward classic approach based on the INB is described later. For an extensive discussion of approaches to sample size calculation, see Willan and Briggs (12). 1.6 Economic Analysis Plans Glick et al. (13) give guidelines on the content of an economic analysis plan. The plan can either be drawn up in parallel with the main analysis plan or incorporated within it. In any case, the two plans should be as consistent as possible, for example, in the treatment of missing values and choice of primary clinical outcome and whether to use an intention to treat approach. However, some differences are likely to occur. Additional disaggregated outcomes may be specified, as would technical economic issues such the economic perspective and sources of cost data. A description of any intended adjustment of trial-based estimates through modeling would also be included in the economic analysis plan. 2 COST AND EFFECTIVENESS DATA 2.1 The Measurement of Costs The measurement of costs involves determining the resources used and placing monetary values on them via unit costs. Resource use is multiplied by these unit costs to obtain total cost per patient. If costs are common between two alternatives, then they may be

3

omitted from the analysis, bearing in mind that it would restrict more general comparisons in the future. Resources can be direct (e.g., drugs, hospitalization, travel to hospital) or indirect (e.g., lifetime medical costs, sickness absence). Data collection methods may be informal if the costs are simple (e.g., pharmacy notes on hospital case reports) or via especially designed questionnaires, if complex. Examples of the latter are the CSSRI-EU (14), used for psychiatric studies, in which aggregated resource use over a sixmonth period is noted from case records or patient interviews, and the cost diary (15), in which the patient records all use of resources over a period of time such as a week or month. Different methods can result in different estimates (16), and validity and accuracy checks are advisable. Unit costs can be found from several sources, which include the health economics literature and charges made by health providers. See Glick et al. (17) for an extended discussion of unit cost estimation. 2.2 The Measurement of Effectiveness Although mortality and morbidity, for example symptom or quality of life scores, may be used as clinical effectiveness measures, generic health utility-based outcome measures such as QALYs (18) tend to be preferred for economic evaluations. This preference occurs because they take into account both quantity and health-related quality of life, they allow for patient or societal preferences about outcomes (utilities), and they can be used to compare treatments in different contexts. (A cost-effectiveness analysis using such measures would then take the form of a cost-utility analysis.) Utility-based outcome measures are typically derived as follows. Patients describe their state of health, using categorical responses to several items using a standard scale, via a self- or intervieweradministered quality-of-life questionnaire. These responses are mapped on to a single score, which usually ranges from 1 for perfect health to 0 for dead (although it is feasible for some states to be regarded as worse than death and have negative values). The mapping process uses previously obtained usage (or preference) weights that

4

COST-EFFECTIVENESS ANALYSIS

reflect public or patient preferences as to different health states. These methods are typically derived from large samples, either of the general population or patients with the disease of interest. A survey would normally provide weights to compute QALYs in many individual studies and would only be repeated when a questionnaire was to be used in a new patient group. Ryan et al. (18,19) discuss methods for deriving preference weights, which include the following widely used techniques: time trade-off, standard gamble, and Visual Analog Score methods. The QALY is computed as the product of the usage score and length of time over which QALYs are to be accumulated. This value might be the length of the trial, survival within the trial or a normal lifespan projected into the future. One QALY is equivalent to a year of perfect health or two years in a health state valued at 0.5. Because QALYs can be derived from a variety of scales and/or sets of preference weights, it is advisable to specify the particular ‘‘flavor’’ of QALY when reporting results. 2.3 Quality-of-Life Scales A criterion for the choice of a quality-of-life scale might be its ability to produce usage scores and hence QALYs. Two examples are as follows: The EQ5D (20) (previously known as the EuroQol) is a public-domain self-administered generic measure of healthrelated quality of life that can produce QALYs. It uses a five-domain classification system: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. Each domain is scored as follows: no health problems = 1, moderate health problems = 2, and extreme health problem = 3. Each of the 243 unique health states thus has an associated descriptor that ranges from 11111 for perfect health to 33333 for the worst possible state. The HUI (21) is self- or intervieweradministered. The HUI2 consists of seven attributes, each with three to five levels that range from highly impaired to normal: sensation (vision, hearing, and speech), mobility, emotion, cognition, self-care, pain, and fertility. The HUI3 has eight attributes (vision, hearing, speech, ambulation, dexterity, emotion, cognition, and pain) each with five to six levels.

3 THE ANALYSIS OF COSTS AND OUTCOMES 3.1 The Comparison of Costs Whereas the balance between cost and effectiveness is generally the key issue in any evaluation, an analysis of the relative costs of competing interventions may be of interest in itself, and it would be the primary analysis in a cost-minimization study. Most cost data is skewed and, in the past, nonparametric methods or transformation have been suggested to deal with this situation. It is now generally recognized that the arithmetic mean cost is of prime interest (because it can be used to produce total costs for populations), and it has led to the suggestion of bootstrapping of the mean costs and group differences (22) or the use of generalized linear models. Nixon and Thompson (23) have compared a number of parametric models (such as the Normal, gamma, log-normal, and log-logistic distributions) from the point of view of estimating the mean. They conclude that large samples are required to distinguish between these models, because the fit in the tails is important, and advise conducting sensitivity analyses with different model choices. 3.2 The Incremental Cost Effectiveness Ratio Balancing cost and effectiveness can be achieved through the Incremental Cost Effectiveness Ratio (ICER), which is defined as ICER =
C/
E

(1)

where
C is the difference in cost between the two interventions to be compared, and
E is the difference in effectiveness. The
E,
C combinations can be displayed on the cost effectiveness plane, in which the ICER corresponding to any combination is the slope of the line joining the point to the origin, as shown in Fig. 1. This figure also shows 1000 bootstrapped resamples from the distribution of
E and
C. The four quadrants in the plane have different interpretations. Quadrant SE shows ‘‘dominance’’ of the treatment (where it is more effective and less costly and therefore the obvious choice); quadrant NW shows dominance of the comparator. In the other

COST-EFFECTIVENESS ANALYSIS

60,000

NW quadrant

NE quadrant

l = £40,000

40,000 Difference in costs (£)

5

l = £30,000 20,000

0

−20,000 SW quadrant −0.5

SE quadrant 0.0 0.5 Difference in effects (QALYs)

1.0

Figure 1. A cost Effectiveness Plane, showing 1000 bootstrapped resamples from a distribution of cost and effect differences (means £10,000 for costs and 0.25 for QALYs). The point estimate of the ICER (£40,000) is indicated as a bold dot. Lines corresponding to maximum willingness to pay values of £30,000 and £40,00 are shown.

two quadrants, the choice between the alternatives depends on the maximum willingness to pay, λ. This value is indicated by a line whose slope is λ, so that in quadrant NE any combination that lies below the line would be regarded as cost effective at that value of λ. Points in quadrant SW, where
E is negative, would often be regarded as unacceptable because a new treatment which was less effective would often be rejected for ethical reasons even if less costly. Several problems occur in analyzing ICERs. One problem is that a single value may be consistent with different combinations of
E and
C lying in different quadrants and therefore having different interpretations. A negative ratio is consistent with a more effective and less costly treatment and with a less effective and more costly treatment. Furthermore, this ratio is undefined at the origin, and it has a skewed distribution. These features mean that analytical solutions for estimating confidence intervals may be problematic even when the point estimate is unambiguous, because they may span different quadrants. Confidence limits for the

ICER have been the subject of much investigation (24). One solution to confidence interval estimation, which assumes that
C and
E have a joint normal distribution, is based on Fieller’s method (25,26), the interval being given by: 2 ] [
C ·
E − z2α/2 σEC
 2 2 {[
E,
C − z2α/2 σEC ] − [
E2 − z2α/2 σE2 ] ± ·[
C2 − z2α/2 σC2 ]}


E2 − z2α/2 σE2 (2) where z2α/2 is the critical point from the standard normal distribution (α = 0.05 for 95% limits), σ E is the standard deviation of the effect differences, σ C is the standard deviation of the cost differences, and σ 2 EC is the covariance between them. Confidence interval estimation can also be performed by ordering the slopes associated with bootstrapped estimates of
E and
C, which determine the bottom and top percentiles for the ICER. However, when the limits span the axes, the problems with ICERs mentioned above may still occur. Bootstrapped

6

COST-EFFECTIVENESS ANALYSIS

estimates on the cost-effectiveness plane (as shown in Fig. 1) are nevertheless informative in illustrating the range of values consistent with the data. 3.3 The Incremental Net Benefit The Incremental Net Benefit (INB) approach (27) can avoid the problems associated with the ICER by a simple rearrangement of the ICER as follows: INB =
C − λ
E

(3)

where
C the difference in costs is,
E is the difference in effects and λ is the maximum willingness to pay. The variance of the INB, σB2 , is given by 2 σB2 = σC2 + σE2 /λ2 − σEC /λ

(4)

This formulation is expressed in units of cost; it can of course also be expressed in terms of units of effectiveness. The advantage of the INB is that it avoids the use of ratios. In most cases, it can be treated as approximately normally distributed so that standard statistical methods can be used to analyze it, including sample size estimation and OLS regression. The disadvantage is that a choice of λ needs to be made, which is not always easy. Sample size calculations based on INB are straightforward, although they may need to be performed for a range of figures for λ. The sample size per arm for a single-sided test at critical level α and power β is given by 2 2 + σBT )/δ 2 N = (z1−α + z1−β )2 (σBS

(5)

where z1−α and z1−β are the critical values from the standard normal distribution, δ is the minimum clinically relevant value for the INB, and σ bS and σ bT are the standard deviations for the INBs in the standard and treatment group, respectively. Estimates of standard deviations for the costs and effects can be obtained from pilot studies or the literature. The correlation between cost and effectiveness is reported in the literature rarely; if such data are unavailable, then it is advisable to use a range of plausible correlations. As expected, sample size for estimating cost effectiveness approaches that for estimating the effectiveness alone as λ increases,

and also as the correlation between cost and effectiveness increases. 3.4 The Cost Effectiveness Acceptability Curve The Cost Effectiveness Acceptability Curve (CEAC) plots the probability of a treatment being cost effective in relation to a comparator, against a range of plausible values for λ. For example, in Fig. 1 the proportion of bootstrapped estimates of
E and
C lying below the line with slope λ = £30,000 is about 40%. By varying λ, a nonparametric CEAC can be built up as in Fig. 2. It has been proposed that the proportion is interpreted as the probability of the new intervention being cost effective at willingness-to-pay λ. However, controversy surrounds the use of the term ‘‘probability.’’ Briggs (28), among others, argues that it only makes sense in Bayesian terms. Nevertheless, this concept has provided a widespread and useful method of summarizing the evidence regarding cost effectiveness. Note that the proportion where the solid line cuts the y axis is the single-sided P value for the cost difference (because the value of λ is 0, and the INB is determined solely by the cost). Similarly, the probability to which the solid line tends is 1 minus the P value for the difference in effectiveness. Indeed, rather than bootstrapping estimates, such curves can be produced via the P values from analytical tests of INB = 0 versus INB > 0 for different values of λ. Parametric CEACs can also be computed by integrating an assumed joint distribution for
E and
C or by parametric bootstrapping (26,29). In Fig. 2, all combinations of
E and
C that lie under the λ -line are included in the calculation of the proportion. However, one might wish to exclude negative
Es in the SW quadrant on ethical grounds. Severens et al. (30) point out that the CEAC might need to be adapted to deal with points in the SW quadrant if the willingness to accept a health loss differed from the willingness to pay for health gain (the lines would have different slopes in the two quadrants). O’Brien and Briggs (26) illustrate CEACs along with a related curve in which the INB is plotted against λ. In such a plot, the points where

COST-EFFECTIVENESS ANALYSIS

7

Percentage bootstrapped samples cost effective

100%

80%

60%

40%

20%

0% 0

20000 40000 60000 80000100000 120000140000160000180000 200000 220000

Maximum willingness to pay in £ (λ) Figure 2. A Cost Effectiveness Acceptability Curve based on the bootstrapped resamples shown in Figure 1. The solid line indicates cost effectiveness of the intervention; the dotted line denotes cost effectiveness of the comparator. The vertical line indicates the point estimate of the ICER (corresponding to a probability of cost effectiveness of 50%).

the 95% CIs for the INB cross the λ axis are identical to the confidence limits for the ICER. Fenwick et al. (29) give an extended discussion of the use and interpretation of CEACs.

required. These analytical considerations are now discussed, and two published examples are summarized that illustrate the range of techniques available. 4.1 Missing Data

4 ROBUSTNESS AND GENERALIZABILITY IN COST-EFFECTIVENESS ANALYSIS Ensuring the robustness and generalizability of economic conclusions as far as possible is a key issue in cost-effectiveness evaluations. The special characteristics of economic data may need to be addressed. Classic sensitivity analysis would be considered as essential; this method seeks to ensure that the conclusions of an economic evaluation are robust and do not depend heavily on the values of particular parameters or model assumptions. It may also be necessary to account for imbalance in the arms of the trial, to reduce bias in an overall estimate, or to make recommendations for a particular subgroup (typically using regression techniques as in the analysis of clinical data). Finally, extrapolation into the future through modeling may be

Economic evaluations frequently entail missing values (because of dropouts or failure to follow up participants), censored values, and treatment switches. In a discussion of possible solutions to the problem, Briggs et al. (31) point out that (1) complete case analysis is inefficient and possibly biased, (2) available case analysis lead to difficulties in using standard statistical methods and can preclude the estimation of total cost per patient, and (3) na¨ıve imputation methods fail to reflect variability in the data. They illustrate more sophisticated methods in which the appropriate level of variability can be incorporated into summary statistics, based on the variation between several complete imputed data sets (multiple imputations). These authors also give several references to methods appropriate for attrition that is not at random and a list of software for missing value treatment.

8

COST-EFFECTIVENESS ANALYSIS

4.2 Censored Data Censored data may occur when accumulated cost or survival is measured over a fixed period. Whereas survival itself can be estimated using usual life table techniques, the same is not true of quality adjusted survival (QALYs) or cost. Standard survival techniques need to be modified for such data because the costs or QALYs accumulated by the censoring point will be correlated with those accumulated at death: Those who are censored are likely to have smaller values, and censoring is thus informative. Two methods that have been proposed to deal with such data are as follows. The direct (Lin) method (32) is based on patients’ cost histories over subintervals of the follow-up, with costs or QALYs in each interval weighted by the probability of surviving to the start of the interval. Inverse probability weighting (33) also involves weighting costs in subintervals, but weights are based on the probability of surviving to the actual censoring point rather than the start of the interval. It is more flexible than the direct method for that reason and also because adjustment for covariates is possible. 4.3 Treatment Switches In addition to missing or censored data, another threat to the robustness of conclusions is ‘‘off protocol’’ treatment switching. Torrance et al. (34) discuss the implications of switches different scenarios (e.g., treatment to comparator and vice versa). The traditional analytical approach is intention to treat, in which patients are analyzed (so long as data are available) as if they were in the arm to which they were randomized. This approach compares the strategies of offering the treatments under consideration. In economic evaluations that are aimed at ‘‘reallife’’ scenarios, such switches might not be discouraged, and the interest might be in the true treatment effect rather than the effect of an offer of treatment. In that case, specialized analytical approaches are available (35). 4.4 Multicenter Trials and Pooling Multicenter trials may be undertaken to maximize the number of patients available for a

clinical study and to improve generalizability of the clinical findings. If the centers are in different countries, then purchasing power parities can be used to equalize the purchasing power of different currencies. Drummond and McGuire (7), who give a full discussion of issues in costing in this situation, advise presenting all costs in their raw state, as well as any final adjusted values. A problem for economic evaluations in multicenter trials is that whereas clinical effects might reasonably be expected to transfer from one center to another, it is less likely to be true for costs because both resource use patterns and unit costs may differ by center (36). This analysis applies particularly in multinational studies. Thompson et al. (37) discuss the use of multilevel gamma models for modeling costs taking account of both patient and center heterogeneity. Drummond and Pang (38) discuss the issues involved in pooling cost-effectiveness estimates and the stage at which this should be performed (e.g., pooling all data before calculating ICERs or pooling the ICERs themselves). They point out that homogeneous effects for the clinical outcome as measured on an odds ratio scale (e.g., when the outcome is mortality) may be heterogeneous when measured on the additive scale necessary for cost-effectiveness estimates, especially if the baseline rates differ by center. Furthermore, treatment-by-center interactions may have limited power to detect such heterogeneity in economic data. Decisions about pooling are often difficult to make, and it is advisable to present detailed center-specific summary statistics as well as any pooled estimates of the costs, the ICERs, or the INBs. 4.5 Classic Sensitivity Analysis Examples of assumptions that would be entered into a sensitivity analysis would be the choice of distributional model used to compare mean costs or the use of common or center-specific unit costs in multicenter trials. Parameters are varied and the base case analysis is re-run so that the impact of uncertainty in these parameters or assumptions can be assessed. Typically, extreme values for parameters (rather than the best estimates as used for the base-case) are substituted

COST-EFFECTIVENESS ANALYSIS

and the analysis is rerun. Favorable or unfavorable scenarios may be devised in this way, and the substitution may be performed either one parameter or several at a time. Walker and Fox-Rushby (39) give guidance on how to plan standard sensitivity analyses. Probabilistic sensitivity analysis entails assigning a distribution to parameters and simulating data sets consistent with these distributions. Briggs (40) discusses some issues in choosing appropriate distributions for these parameters. 4.6 Regression Models Regression-based modeling can also be used to investigate costs or cost effectiveness in relation to patient characteristics. Motivation includes adjustment for baseline imbalance in randomized groups and exploratory analysis of factors associated with varying levels of costs or cost effectiveness. In the former case, the adjusted estimate of cost effectiveness is expected to be less biased because of imbalance than estimates from traditional methods. In the latter case, separate summary statistics and CEACs can be presented for different subgroups as predicted by the model. In the case of modeling costs alone, generalized linear models with gamma errors might be appropriate to deal with skewness in the data. Poisson or zero-inflated Poisson models can be used for count-based resource use data (41). For modeling cost effectiveness, the INB is much easier to deal with than the ICER because ordinary least-squares (OLS) regression can be used, although such models would need to be estimated for a range of willingness to pay values. Willan et al. (42) show how seemingly unrelated regression can be used; this method, which recognizes that different sets of covariates may be to be included for cost and for effectiveness, consists of two separate regression models linked through the individual patient error terms. 4.7 Markov Models to Extrapolate Over Time The Markov model can be used to extrapolate results from clinical trials over much longer periods, even whole lifetimes. For example, it can be used to link an intermediate outcome (such as blood pressure) with a longer term

9

outcome (such as mortality). Markov models are derived from a mutually exclusive set of health states (e.g., alive or dead) with transitions among states that occur at regular intervals; each type of transition has an associated transition probability. These probabilities may be estimated from published data, for example, mortality rates. The minimum time that a patient can spend in any state determines the cycles through which the model passes, and costs and utilities of passing one cycle in each state are assigned. Expected values for costs and utilities are calculated by multiplying the proportion of patients in each state by the corresponding cost and utility values and summing across both states and cycles. Monte Carlo simulation can be employed to assign distributions to the parameters of the model, so that uncertainty in the final expected values can be assessed. Lang and Hill (43) provide an overview of modeling techniques such as these. 4.8 Examples of Modeling and Sensitivity Analysis Oostenbrink et al. (44) combine a Markov model with a probabilistic sensitivity analysis in a study of the cost effectiveness of bronchodilators. The model was structured around transitions in disease severity and the number and type of exacerbations, with cycles (after the first) being one month over the course of a year. The data for this came from trials in six centers. Dirichlet, beta, and gamma distributions were used to represent uncertainty in transitions between disease states, exacerbations, and resource use, respectively. Monte Carlo simulations of costs and effects were shown on the cost-effectiveness plane. Sensitivity analyses included changing transitions and exacerbation probabilities, as well as utilities assigned to disease states. Hoch et al. (45) show how regression can be used in conjunction with the INB to identify the marginal benefits for various subgroups of patients with particular characteristics and to adjust for baseline imbalance in a trial of a program in assertive treatment for people with severe mental illness compared with usual community services. The INB

10

COST-EFFECTIVENESS ANALYSIS

was fitted first as a regression on arm and including the covariates of age, a functioning score, and ethnic group (the latter differing between treatment arms at baseline). This formula produced an overall CEAC adjusted for these variables. Finally, an interaction between treatment group and the covariates was included, which allowed CEACs to be presented for the black and white groups separately. REFERENCES 1. M. J. Sculpher, K. Claxton, M. Drummond, and C. McCabe, Whither trial-based economic evaluation for health care decision making? Health Econ. 2006; 15: 677–687. 2. R. S. Taylor, M. F. Drummond, G. Salkeld, and S. D. Sullivan, Inclusion of cost effectiveness in licensing requirements of new drugs: the fourth hurdle. BMJ 2004; 329: 972–975. 3. A. H. Briggs and B. J. O’Brien, The death of cost minimisation analysis? Health Econ. 2004; 10: 179–184. 4. A. Culyer, The Dictionary of Health Economics. Cheltenham, UK: Edward Elgar, 2005. 5. NICE. Guide to the Methods of Technology Appraisal (reference N0515). 2004. National Institute for Health and Clinical Excellence, UK. 6. M. R. Gold, J. E. Siegel, L. B. Russell, and M. C. Weinstein, Cost-effectiveness in health and medicine. New York: Oxford University Press, 1996. 7. M. E. Drummond and A. McGuire, Economic Evaluation in Health Care. Merging Theory with Practice. Oxford, UK: Oxford University Press, 2001, p. 236. 8. D. H. Smith and H. Gravelle, The practice of discounting in economic evaluation of health care interventions. Int. J. Technol Assess. Health Care. 2001; 17: 236–243. 9. T. N. Flynn and T. J. Peters, Conceptual issues in the analysis of economic data from cluster randomised trials. J. Health Services Res. Policy 2005; 10: 97–102. 10. M. J. Al, B. A. Van Hout, B. C. Michel, and F. F. H. Rutten, Sample size calculations in economic evaluation. Health Econ. 1998; 7: 327–335. 11. A. O’Hagan and J. W. Stevens, Bayesian methods for design and analysis of costeffectiveness trials in the evaluation of health

care technologies. Stats. Methods Med. Res. 2001; 11: 469–490. 12. A. R. Willan and A. H. Briggs, Statistical Analysis of Cost-Effectiveness Data. Chichester, UK: Wiley, 2006. 13. H. Glick, D. Polsky, and K. Schulman, Trialbased economic evaluations: an overview of design. In: M. Drummond and A. McGuire (eds.), Economic Evaluation in Health Care: Merging Theory with Practice. Oxford, UK: Oxford University Press, 2001, pp. 113–140. 14. D. Chisholm, M. R. Knapp, H. C. Knudsen, F. Amaddeo, L. Gaite, B. van Wijngaarden, Client socio-demographic and service receipt inventory--European version: development of an instrument for international research. EPSILON Study 5. Br. J. Psychiatry. 2000;supplementum(39):s28–33. 15. M. E. J. B. Goossens, M. P. M. H. Ruttenvan M¨olken, J. W. S. Vlaeyen, and S. M. J. P. van der Linden, The cost diary: a method to measure direct and indirect costs in costeffectiveness research. J. Clin. Epidemiol. 2000; 53: 688–695. 16. S. Byford, M. Leese, M. Knapp, H. Seivewright, S. Cameron, V. Jones, K. Davidson, and P. Tyrer, Comparison of alternative methods of collection of service use data for the economic evaluation of health care interventions. Health Econ. 2006. 17. H. A. Glick, S. M. Orzol, J. F. Tooley, D. Polsky, and J. O. Mauskopf, Design and analysis of unit cost estimation studies: How many hospital diagnoses: how many countries? Health Econ. 2003; 12: 517–527. 18. G. W. Torrance, Measurement of health state utilities for economic appraisals: a review. J. Health Econ. 1986; 5: 1–30. 19. M. Ryan, D. A. Scott, C. Reeves, A. Bate, E. R. van Teijlingen, E. M. Russell, M. Napper, and C. M. Robb, Eliciting public preferences for healthcare: a systematic review of techniques. Health Technol. Assess. 2001; 5: 1–186. 20. EuroQol Group, EuroQol - a new facility for the measurement of health-related quality of life. Health Policy. 1990; 16: 199–208. 21. J. Horsman, W. Furlong, D. Feeny, and G. Torrance, The Health Utilities Index (HUI(r)): concepts, measurement properties and applications. Health Quality Life Outcomes. 2003;(1). Available at: http://www.hqlo.com/content/1/1/54. 22. J. A. Barber and S. G. Thompson, Analysis of cost data in randomised controlled trials: an application of the non-parametric bootstrap. Stat. Med. 2000; 19: 3219–3236.

COST-EFFECTIVENESS ANALYSIS

11

23. R. M. Nixon and S. G. Thompson, Parametric modelling of cost data in medical studies. Stat. Med. 2004; 23: 1311–1331.

Schulman, Estimating country-specific costeffectiveness from multinational clinical trials. Health Econ. 1998; 7: 481–493

24. D. Polsky, H. A. Glick, R. Willke, and K. Schulman, Confidence intervals for costeffectiveness ratios: a comparison of four methods. Health Econ. 1997; 6: 243–252.

37. S. G. Thompson, R. M. Nixon and R. Grieve, Addressing the issues that arise in analysing multicentre cost data, with application to a multinational study. J. Health Econ. 2006; 25: 1015–1028.

25. A. R. Willan and B. J. O’Brien Confidence intervals for cost-effectiveness ratios: an application of Fieller’s theorem. Health Econ. 1996; 5: 297–230. 26. B. J. O’Brien and A. H. Briggs, Analysis of uncertainty in health care cost-effectiveness studies: an introduction to statistical issues and methods. Stat. Methods Med. Res. 2002; 11: 455–468. 27. A. A. Stinnett and J. Mallahy, Net health benefits: a new framework for the analysis of uncertainty in cost-effectiveness analysis. Medical Decision Making. 1998; 18(suppl):S68–S80. 28. A. H. Briggs, A Bayesian approach to stochastic cost-effectiveness analysis. Health Econ. 1999; 8: 257–261. 29. E. Fenwick, B. J. O’Brien, and A. Briggs, Costeffectiveness acceptability curves: facts, fallacies and frequently asked questions. Health Econ. 2004; 13: 405–415. 30. J. L. Severens, D. E. M. Brunenberg, E. A. L. Fenwick, B. O’Brien, and M. Joore, Costeffectiveness acceptability curves and a reluctance to lose. Pharmacoeconomics 2005; 23: 1207–1214. 31. A. Briggs, T. Clark, J. Wolstenholme, and P. Clarke, Missing . . . presumed at random: cost-analysis of incomplete data. Health Econ. 2003; 12: 377–392. 32. D. Y. Lin, E. J. Feuer, R. Etzioni, and Y. Wax, Estimating medical costs from incomplete follow-up data. Biometrics 1997; 53: 419–434. 33. A. R. Willan, D. Y. Lin, R. J. Cook, and E. B. Chen, Using inverse-weighting in costeffectiveness analysis with censored data. Stat. Methods Med. Resl. 2002. 34. G. W. Torrance, M. F. Drummond, and V. Walker, Switching therapy in health economics trials: confronting the confusion. Med. Decision Making 2003; 335–339 35. G. Dunn, M. Maracy, and B. Tomenson, Accounting for non-compliance in clinical trials. Stat. Methods Med. Res. 2005; 14: 369–395. 36. R. J. Willke, H. A. Glick, D. Polsky, and K.

38. M. Drummond and F. Pang, Transferability of economic evaluation results. In: M Drummond and A. McGuire (eds.), Economic Evaluation in Health Care: Merging Theory with Practice. Oxford, UK: Oxford University Press, 2001. 39. D. Walker and J. Fox-Rushby, How to do (or not to do) . . . Allowing for uncertainty in economic evaluations: sensitivity analysis. Health Policy Planning. 2001; 16: 435–443. 40. A. Briggs, Probabilistic analysis of costeffectiveness models: statistical representation of parameter uncertainty. Value in Health. 2005; 8: 1–2. 41. A. M. Jones, Applied Econometrics for Health Economists – A Practical Guide. London: Office of Health Economics, 2001. 42. A. R. Willan, A. H. Briggs, and J. S. Hoch, Regression methods for covariate adjustment and subgroup analysis for non-censored costeffectiveness data. Health Econ. 2004; 13: 461–475. 43. D. Lang and S. R. Hill, Use of pharmaceconomics in prescribing research. Part 5 modelling – beyond clinical trials. J. Clin. Pharm. Therapeut. 2003; 28: 433–439. 44. J. B. Oostenbrink, M. P. M. H. Ruttenvan M¨olken, B. U. Monz, J. M. FitzGerald, Probabilistic Markov model to assess the cost-effectiveness of bronchodilator therapy in COPD patients in different countries. Value in Health. 2005; 8: 32–46. 45. J. S. Hoch, A. H. Briggs, and A. R. Willan, Something old, something new, something borrowed, something blue: a framework for the marriage of health econometrics and costeffectiveness analysis. Health Econ. 2002; 11: 415–430.

FURTHER READING A registry of published cost effectiveness analyses and preference weights for various diseases is available at the Tufts –New England Medical Center Cost Effectiveness Registry. http://www.tufts-nemc.org/CEARegistry/. The web page of the International Society for Pharmacoeconomics and Outcomes Research gives

12

COST-EFFECTIVENESS ANALYSIS

an extremely useful summary of national guidelines for health evaluation, including information on discount rates, sources of cost information, preferred method of analysis, and so on. http://www.ispor.org/PEguidelines/index.asp. Unit costs for the US, the UK (The Department of Health) and internationally (the World Health Organisation- WHO-CHOICE) may be found from the following websites, which also give general guidance and/or information on publications about cost-effectiveness analysis: http://www.cms.hhs.gov http://www.pssru.ac/uk http://www.who.int/choice Purchasing Power Parities (PPPs) may be found from the Organisation for Economic Cooperation and Development website. http:// www.oecd.org.

CROSS-REFERENCES cost benefit analysis bootstrapping treatment by center interaction quality of life intention to treat

COVARIATES

the usefulness of covariates for the analysis of clinical data. The overwhelming majority of clinical literature concentrates on investigating the prognostic effect of covariates in observational studies, and for almost every disease a large number of prognostic factors are examined. Statistical methods used for the analysis vary considerably, and the statistical community is far from a consensus as to the most sensible approach. (Some of these issues are discussed in the separate article on prognostic factor studies.) In the context of prognostic factor studies in oncology, several analysis strategies are discussed in Schumacher et al. (1) and McShane & Simon (2). Differences in approach, assumptions, and results are illustrated. The severe weaknesses in reporting methods and results of prognostic factor studies are discussed by Riley et al. (3), and several resulting difficulties of a systematic summary of the effect of a prognostic factor are discussed by Altman (4). When analyzing a randomized trial, the discussion centers around the advantages and disadvantages of adjusting the treatment effect for covariates and also the proper selection of covariates for this purpose. Some consensus has been reached on these issues, and the evolving recommendations from various groups are available (5–8). Aside from the overall results of a RCT, clinicians are interested in whether the treatment effect is similar in all patients or whether heterogeneity has been reached in the effect in specific subgroups of patients. Finding subgroups for which a treatment is more effective could be an important step toward the aim of individualizing treatments, a topic of growing interest. Specifically, as our ability to measure and to understand the biological mechanisms of gene expression data becomes more advanced, the definition of prognostic subgroups has become one of the most promising areas of research. However, a review in four major medical journals in 1997 demonstrated severe problems in subgroup analyses (9). Most trials presented baseline comparability in unduly large tables, and about half the trials inappropriately used significance tests for baseline comparison.

WILLI SAUERBREI Institute of Medical Biometry and Medical Informatics University Medical Center Freiburg Freiburg, Germany

1 UNIVERSAL CHARACTER OF COVARIATES In clinical trials, covariates are used for many purposes. They are often called explanatory variables, predictor variables, independent variables, prognostic variables, or prognostic factors. Covariates can be used in all phases of clinical studies, including defining the patient population, calculating the required sample size, allocating treatment in randomized and nonrandomized trials, adjusting the estimate of a treatment effect, defining subgroups considered for investigations of interactions with treatment, helping in the biological understanding of a disease and its potential treatment, and developing prediction rules. Covariates can be demographic factors such as country or race, or biological and genetic factors such as sex or blood group, or factors that may change over time such as blood pressure or tumor size, or characteristics of patients that are relevant for the course of the disease and the effect of a treatment. Covariates are measured on different scales, such as binary (sex), categorical (histologic tumor type), ordered categorical (tumor grade), or continuous (age, height). Because of their universality and general applicability, covariates play a central role in clinical research, in randomized controlled trials (RCT), and in nonrandomized trials (NRT). Beyond this, covariates are used in medical decision making and health policy decisions (see 2.3). The measurement of covariates can be simple, cheap, and yield high data quality (e.g., sex), but it can also be very complicated, expensive, and imperfect. The scale of the measurement and the quality of the data can have a serious influence on the use and

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

COVARIATES

Two-thirds of the reports presented subgroup findings, but mostly without appropriate statistical tests for interaction. Many reports put too much emphasis on subgroup analyses that commonly lacked statistical power. These studies were conducted before gene expression data became available in trials; although the enormous amount of new covariate type data offers new possibilities and hope for improving prediction and treatment of patients, it also poses new challenges to the design and analysis of clinical trials. The broad range of possible uses and different types of issues, such as the controversies surrounding gene expression data, are beyond the scope of this discussion; instead, we focus on some of the most important aspects and relevant applications. To this end, we will implicitly assume the situation of a phase III trial or a large observational study. We consider different uses of covariates in controlled clinical trials where they play a central role in all parts of the trial: for example, definition of the patient population, RCT treatment allocation, adjustment of the estimate of the treatment effect, and definition of subgroups for the investigation of interaction. Continuous covariates also need specific attention; in application, they are often changed to a categorical form. Simplicity is an important reason for categorization, but categorization introduces several critical issues. Finally, we will examine some issues of reporting and summary assessment of prognostic markers. It will become obvious that severe improvements are required for a better use of marker data in clinical trials. 2

USE OF COVARIATES IN CLINICAL TRIALS

2.1 Patient Population To select an adequate population in the planning phase of a new clinical trial, knowledge about the effect of covariates is required. Prospective randomized trials are usually designed for risk-adapted patient populations where covariates define the main inclusion criteria. Exclusion criteria are applied to exclude patients with extreme values or rare conditions. Age is a specific covariate that is

often used to define a population by excluding young (e.g., below 18 years) or old (e.g., above 70 years) patients. In recent years, several trials have been designed specifically to investigate treatments in older patients. With more specific knowledge about treatment responsiveness, covariates are used to restrict the trial population to patients who are more likely profit from the therapy. In most recent trials investigating hormonal therapies in breast cancer, for example, patients must have a positive estrogen receptor. In the design of a new trial, investigators must find a compromise between internal and external validity. Internal validity aims to eliminate the possibility of biases in the design and conduct of the trial, whereas external validity considers the extent to which the results of a trial provide a correct basis for generalizations to other circumstances. For eligibility, consideration should be given both to patients who are likely to benefit from treatment and to the desired generalizability of the results. If the eligibility criteria are very narrow, the generalizability of the study is compromised; if they are too broad, the effectiveness of treatment may be masked by the inclusion of patients who have little chance of responding (10). In this respect, covariates can also play a central role in deciding whether a trial is useful and whether it is ethical. The targeted population may be too small for a comparative trial; for example, a study of early stage disease would require a very large sample size because the expected event rates in a reasonable time period are low. Another problem derives when the knowledge about risk factors is insufficient to identify which patients have a severely increased risk; only those patients would be suitable candidates for inclusion in the population for a prevention trial (11). 2.2 Treatment Allocation in a RCT Covariates play an important role in allocating treatment to patients in a RCT. Usually, allocation is implemented in one of two ways. Most often used is preallocation: a randomization list is prepared in advance, giving a new patient the next treatment on the list.

COVARIATES

To use covariates for a stratified randomization, different lists are required for each strata. Also common is a form of postallocation: the patient is recruited and entered into the trial, then—possibly after covariate information has been taken into account and perhaps using some automated telephone or Web-based system—the investigator is told the treatment or (in a double-blind trial) the number of the pack to be given to the patient (12). To achieve balance with respect to the covariates suspected to have an influence on the outcome, it is common practice to use them as stratification factors in the randomization or to use dynamic techniques such as minimization (13) or biased coin designs (14). Often, the study center may be the first stratification factor used. In trials with a large number of participating centers, however, practicality may argue against using further stratification criteria because exact balance becomes impossible if several centers enter only a small number (say less than five ) patients. This is also the case when several covariates are used as stratification factor, resulting in a large number of strata (15). Too many strata also incur an organizational cost; they may complicate the process of treatment allocation when it becomes necessary to have the information on covariates available at the time of randomization (7). Randomized blocks is the most commonly used technique. Using either preallocation or postallocation, this approach ensures that the treatments are allocated in the chosen proportions for each block. The block size should be unknown to the investigators. Employing a separate list for each stratum, it can only balance for covariates by stratification. In general, it is advisable to

3

incorporate the stratification criteria in the analysis, although exceptions may be made if the strata become very small and strata are not themselves predictive of the outcome (7). Another popular technique is covariateadaptive randomization (16), of which minimization (13) seems to be used most often. However, Senn (12) argues as to whether all such schemes really involve randomization and refers to this approach by the more general term ‘‘allocation based on covariates’’ (ABC). The patient must be entered first into the trial and values need to be reported first, before the central trial office can then announce the allocation, so the method of postallocation alone is suitable using either telephone or (nowadays) Webbased allocation. A common form of ABC is ‘‘minimization,’’ an attempt to balance trials as best possible by using prognostic information. Senn (12) discusses the advantages and disadvantages of ABC schemes and points out that their advantages have been overstressed in the literature. He argues that the decision to adjust for prognostic factors has a far more dramatic effect on the precision of the inferences than the choice of an ABC or randomization approach. 2.3 Adjustment of the Estimate of a Treatment Effect in a RCT To demonstrate the importance of stratification or adjustment for covariates of the estimate of a treatment effect in a randomized trial, we want to use a well-known and extreme theoretical example for the analysis of survival time data from Peto et al. (17). Data are given in Table 1. The univariate analysis gives for the effect of treatment a P-value of 0.252 when using the logrank test. The P-value for the effect of the prognostic factor is <0.0001. Stratifying

Table 1. Hypothetical Data of Patients Treated With Either A or B and One Prognostic Factor Treatment

Prognostic factor

Survival times

A

normal elevated normal elevated

8, 220, 365 + , 852 + , 1296 + , 1328 + , 1460 + , 1976 + 8, 52, 63, 63 70, 76, 180, 195, 210, 632, 700, 1296, 1990 + , 2240 + 13, 18, 23

B

Note: + : censored observation. Source: Peto et al. Br J Cancer. 1977; 35: 1–39.

4

COVARIATES

for the prognostic factor the logrank test for treatment indicates an effect with a P-value of 0.016. Assuming proportional hazards, the estimated hazard ratio for treatment is 1.77 (95% confidence interval, 0.65–4.81) in a univariate Cox model, but it increases to 3.47 (1.07–11.22) when adjusting for the prognostic factor and even to 4.32 (1.19–15.75) in a stratified Cox model. For a more realistic, but still hypothetical, example demonstrating the effect of imbalance of a prognostic variable on the estimated treatment effect, see Altman (18). Randomized controlled trials are the gold standard for the comparison of two or more treatments because they promote comparability among the study groups. In an observational study, such comparability can only be attempted by adjusting for matching on known covariates, with no guarantee or assurance, even asymptotically, of control for other baseline patient characteristics. Even for unknown important covariates, randomization ensures comparability with a high probability, and it provides a probabilistic basis for an inference from the observed results when considered in reference to all possible results (16). However, it is obvious that some imbalance between baseline covariates will often remain after randomization. To describe the patient population and to demonstrate baseline comparability between treatment groups, most trials present covariate information in a table. These tables are often unduly large, and Assmann et al. (9) found that about half the trials inappropriately used significance tests for baseline comparison. In randomized trials, treatment groups are indeed random samples, and any differences observed between them are necessarily due to chance, so the use of hypothesis tests is highly questionable. The only use of significance testing for baseline differences is to check the randomization process; however, there is only minimal power for it (12, 19). As demonstrated with the introductory example, stratification or adjustment of the treatment estimate for covariates is an important issue involving several elements. If a covariate has a strong prognostic effect,

adjustment can be quite important even if the imbalance between groups is only small. Such imbalance can work in either direction, masking or overstating the true treatment difference (19). Reducing the variance of the estimated treatment effect by adjusting for covariates correlated with outcome is a further important statistical issue of the adjustment for baseline covariates. Usually, most of the arguments are discussed in the context of a linear model and later transferred to generalized linear models or models for survival data. For the latter models, some issues differ; however, most practical consequences are similar (20–23). 2.3.1 General Recommendations. Concerning issues of which covariates to select and when to select them, some consensus was reached by the Committee for Proprietary Medicinal Products (6). Their ‘‘Points to Consider’’ comprised the following recommendations (here, numbered Points 1 to 13). Not all statisticians may agree to all of the following points, so acceptance by the statistical community is still under debate. However, these points were meant to clarify and harmonize issues on the use of baseline covariates in clinical trials: P1. Stratification may be used to ensure balance of treatments across covariates; it may also be used for administrative reasons. The factors that are the basis of stratification should normally be included as covariates in the primary model. P2. Variables known a priori to be strongly, or at least moderately, associated with the primary outcome and/or variables for which there is a strong clinical rationale for such an association should also be considered as covariates in the primary analysis. The variables selected on this basis should be prespecified in the protocol or the statistical analysis plan. P3. Baseline imbalance observed post hoc should not be considered an appropriate reason for including a variable as covariate in the primary analysis.

COVARIATES

P4. Variables measured after randomization and so potentially affected by the treatment should not normally be included in the primary analysis. P5. If a baseline value of a continuous outcome measure is available, then this should usually be included as a covariate. P6. Only a few covariates should be included in a primary analysis. P7. In the absence of prior knowledge, a simple functional form (usually either linearity or dichotomizing a continuous scale) should be assumed for the relationship between a continuous covariate and the outcome variable. P8. The validity of model assumptions must be checked when assessing the results. P9. Whenever adjusted analyses are presented, results of the treatment effect in subgroups formed by the covariates (appropriately categorized, if relevant) should be presented to enable an assessment of the validity of the model assumptions. P10. Sensitivity analyses should be preplanned and presented to investigate the robustness of the primary results. Discrepancies should be discussed and explained. P11. The primary model should not include treatment by covariate interactions. If substantial interactions are expected a priori, the trial should be designed to allow separate estimates of the treatment effects in specific subgroups. P12. Exploratory analyses may be carried out to improve the understanding of covariates not included in the primary analysis. P13. A primary analysis, unambiguously prespecified in the protocol or statistical analysis plan, correctly carried out and interpreted, should support the conclusions that are drawn from the trial. Since there may be a number of alternative valid analyses, results based on prespecified analyses will carry most credibility.

5

2.3.2 Selection of Covariates for Adjustment. In many diseases, a long list of potentially important covariates are considered when a clinical trial being designed and analyzed. There is general agreement that only a few covariates should be included in a primary analysis (P6). However, it is less obvious how to select these covariates. For linear models with continuous outcomes, some methodological arguments are presented to guide selection of specific covariates (7). Covariates for which adjustment is to be carried out should be identified at the design stage and specified in the study protocol (P2). During the analysis phase, variable selection procedures should not be used to select covariates. Analysis strategies based on preliminary tests of covariate imbalance reduce the power of the analysis to detect a treatment difference where such a difference exists. Post hoc tests of covariates associated with outcome have a different danger in that they may bias the estimates of the treatment effect. If, by chance, a prognostic factor is severely imbalanced, problems concerning the best way to analyze and to interpret the results may appear. Adjusting or not for this factor should hardly influence the estimate of the treatment effect if the prognostic value is only weak. Fortunately, the critical case is rare in which a strong prognostic factor is a priori not considered for the final analysis and, at the same time, is severely imbalanced between treatment groups. Presenting sensitivity analyses and interpreting the results carefully are the only general recommendations for these cases. Covariates specified a priori for adjustment are often based on clinical knowledge and the expected distribution in the trial. For the linear regression model, a more formal approach for the choice of covariates to include in the analysis plan for a clinical trial is presented by Raab et al. (7). From the analysis of previous trials, they need to obtain the multiple correlation between the outcome and each potential set of covariates considered for inclusion and the number of patients planned to include in the current trial as well. Where practical, the design should aim to balance these factors across treatments.

6

COVARIATES

2.3.3 Binary or Survival Outcomes. Unfor tunately, some aspects are known to be different for randomized studies with binary and survival outcomes, usually analyzed in the framework of a logistic regression model or a Cox regression model, respectively (7). In particular, in the Cox model, the omission of a balanced covariate leads to a mixture model that is no longer a proportional hazards model (20) and results in a downward bias of the treatment effect estimate (21, 22). The same is true in the situation when the covariate is not completely omitted from the model but rather is included in a categorized form (23). Furthermore, the variance of the treatment effect estimate in an analysis omitting a balanced covariate is smaller than in the adjusted analysis. Despite the gain in precision due to covariate omission, there is a loss of power due to an underestimation of the treatment effect, which is much more important. Therefore, it is recommended to adjust for important prognostic factors even in randomized treatment comparisons. However, in the case of the Cox model, this recommendation is based on totally different arguments than in the case of the classic linear model. The practical consequences of adjusting for a prognostic covariate may be similar to the linear case (23). Robinson and Jewell (24) showed that the asymptotic power of the test for the treatment effect in logistic regression is always increased by adjusting for a prognostic covariate. This is due to an increase of the size of the adjusted treatment effect rather than to a reduction of its standard error. 2.3.4 Examples Demonstrating Influence of Adjustment. There are several published RCTs showing that adjustment or stratification for covariates can have a severe influence on the result. In a randomized trial, the Canadian Clinical Trials Group compared two adjuvant chemotherapy regimens in 716 postmenopausal women with early breast cancer (25). They used a stratified block randomization procedure with three stratification factors. Treatments were well balanced in these three factors. However, slight imbalances were observed in two other important prognostic factors. Comparing the overall survival for the two treatments based

on the logrank test gave a statistically nonsignificant P-value of 0.11, ignoring the cova riates. A stratified logrank test considering the three factors used for randomization gave a P-value of 0.034. The P-value was additionally reduced to 0.028 by adjusting the stratified analysis for the two further factors observed to be slightly imbalanced (8, 26). The analysis using the three prespecified randomization criteria is certainly sensible; however, the value of the analysis including the two additional factors is questionable because these factors were determined in a data-dependent way (P3). Christensen et al. (27) present another example showing the influence of adjustment in a randomized trial. The unadjusted analysis comparing azathioprine versus placebo in patients with primary biliary cirrhosis gave P = 0.2. There was some imbalance in serum bilirubin, which is a very strong prognostic variable in such patients. The azathioprine group had higher levels on average and hence a worse prognosis. Using a stratified logrank analysis, the P-values for therapy varied with the number of strata but were always smaller compared with the P-value from the unadjusted analysis. With one, four, and eight strata, the P-values were 0.1, 0.07, and 0.05, respectively. This illustrates the increased power of a stratified analysis, but it also shows that for a highly prognostic covariate it might not be feasible to have enough strata to remove the effect of imbalance. Adjusting the treatment effect by use of regression modeling is then preferable (18). 2.4 Define Subgroup and Investigation of Interactions Covariates are often used to define subgroups and to investigate whether treatment effects are heterogeneous in subgroups of the patient population. The statistical term for heterogeneity of this type is interaction. In a series of short statistical notes, Matthews and Altman (28–30) illustrated that the approach of analyzing data separately in each subgroup is incorrect. They show in two simple examples that it is important to compare effect sizes and not P-values between subgroups. In an assessment of 50 reports from randomized clinical trials, Assmann et al. (9)

COVARIATES

found that two thirds of the reports presented subgroup findings without appropriate statistical tests for interaction. Many reports put too much emphasis on subgroup analyses, which commonly lack statistical power. Many trials confined subgroup attention to just one baseline factor, but five trials examined more than six factors; often, subgroup differences were studied for more than one endpoint. Less than half of subgroup-analysis reports used statistical tests of interactions, which directly examine whether the treatment difference in an outcome depends on the patient’s subgroup. Most other reports relied instead on P-values for treatment difference in each separate subgroup. However, because trials are usually powered to detect an overall effect, subgroup analysis must suffer from a lack of power. Assman et al. (9) stated that, of all various multiplicity problems in clinical trials, subgroup analysis remains the most overused and overinterpreted. Even though these problems have been elucidated (31, 32), trial reports continue to employ many subgroup analyses. This reflects an intellectually important issue: real treatment differences may depend on certain baseline characteristics. Thus, such data exploration is not bad on its own, provided that investigators (and readers) consider it as hypothesis generation only. Because of low power, researchers are inevitably left in doubt as to whether a suggestive result from a subgroup analysis (e.g., with a P-value of the test of interaction between P = 0.05 and P = 0.15) is simply due to chance or merits further investigation. Even after reporting an interaction that is not statistically significant, investigators may still overinterpret subgroup findings. As stated in P11 of the ‘‘Points to Consider,’’ treatment by covariate interactions should not be included in a primary model, but possible expected interactions should be specified a priori. Obviously, this affects interpretation of a trial result. An example with a strong interaction in a prespecified subgroup analysis was reported in Jonat et al. (33). In a large randomized trial for adjuvant treatment of premenopausal breast cancer patients, chemotherapy was

7

compared with hormone therapy. The primary analysis of disease-free survival for all 1614 patients showed that chemotherapy was superior to hormone therapy (hazard ratio 1.18, P = 0.029). However, in a prespecified analysis, a highly statistically significant interaction between treatment and estrogen-receptor status was observed (P = 0.0016). Therefore, the planned estrogenreceptor subgroups were considered separately. There was a large advantage for the chemotherapy arm in the small (304 patients) estrogen-receptor–negative group (hazard ratio 1.76, P = 0.0006), but equivalence for the two treatments (hazard ratio 1.01; 95% confidence internal, 0.84–1.20) in the larger estrogen-receptor–positive subgroup. Investigations of a treatment interaction with a continuous covariate add another difficulty. In general, the covariate is categorized, which raises issues as to the number of categories (most often two are used) and the selection of a cutpoint. Some issues discussed below regarding categorization or determination of the functional form for a continuous covariate transfer to the investigation of interactions. To avoid a substantial loss of information by categorizing a continuous variable in two or three groups, several approaches have been proposed recently (34, 35). For specific examples, they have demonstrated their usefulness, but they all need further investigation of their properties. 3 CONTINUOUS COVARIATES: CATEGORIZATION OR FUNCTIONAL FORM? 3.1 Categorization Often variables are measured in a continuous form, but changing them into a categorical form is common practice in many analyses. An important reason is that categorization makes it easier for clinicians to use covariate information in making treatment decisions (36). Another reason is that the functional form of the influence of a covariate is often unknown, and an analysis based on categorization is easier. Further, categorization allows graphical presentation (e.g., in survival analysis, Kaplan-Meier curves can be presented).

8

COVARIATES

In particular, values of a variable are frequently divided into two groups. Categorization enables researchers to avoid strong assumptions about the relation between the marker and risk. However, this comes at the expense of throwing away information. The information loss is greatest with only two groups, although this approach is very common. Often splitting at the sample median is used (37). It is well known that the results of analyses can vary if different cutpoints are used for splitting. In recent years, there has been increasing interest in investigating various cutpoints and choosing the one that corresponds to the most statistically significant correlation with outcome. In other words, the cutpoint defining ‘‘low’’ and ‘‘high’’ risk is chosen such that it minimizes the P-value relating the prognostic factor to outcome. The cutpoint so chosen is often termed ‘‘optimal,’’ but this description is inadvisable because of the well-known problem of multiple testing. When a series of statistical tests, each with a prespecified nominal type 1 error—for example, 5%—is performed on the same data, this procedure leads to a global error rate for the whole procedure that might be much higher than 5%. Theoretical arguments (38, 39) and results from simulation studies (37, 40) demonstrate that the false-positive rate can be inflated to values exceeding 40% when a nominal level of 5% is used. Altman et al. (37) preferred to call this procedure the ‘‘minimum P-value approach.’’ They provide a correction, valid for large sample sizes, of the minimal P-value to allow for the multiple testing. For practical applications, a simple approximation for the corrected P-value is given by
 Pcor ≈ −1.63Pmin 1 + 2.35 loge Pmin for ε = 10%
 Pcor ≈ −3.13Pmin 1 + 1.65 loge Pmin for ε = 5% where Pmin denotes the minimum P-value of the logrank statistic, taken over the selection interval characterized by the proportion ε of the smallest and largest values of the prognostic factor that are not considered as potential cutpoints. This approximation works well

for small minimum P-values (0.0001 < pmin < 0.1). For example, to reach a value Pcor = 0.05 requires Pmin = 0.002 when ε = 10% and even Pmin = 0.001 when ε = 5%. Using the correction formula above, the corrected Pvalue for the optimal cutpoint was 0.403 in an example investigating the prognostic value of S-phase fraction (SPF) in breast cancer patients, in contrast to the minimum P-value of 0.037 which would indicate a statistically significant influence of SPF (37). The ‘‘optimal cutpoint approach’’ leads to other serious problems. Most likely different cutpoints will be termed ‘‘optimal’’ in different studies or subpopulations. Analyzing the data in multivariate analysis may further change the optimal cutpoint. This was illustrated by Altman et al. (37). Results of an analysis based on the optimal cutpoint may show impressive prognostic effects, but it must be kept in mind that the effect can be heavily overestimated. For a discussion of this issue and possible ways to correct for ¨ it, see Schumacher et al. (41) and Hollander et al. (42). For a more detailed discussion on categorizing a continuous covariate, see Mazumdar and Glassman (36) and Mazumdar et al. (43). Categorizing a continuous covariate into a small number of groups or having an ordered categorical variable with at least three groups raises an issue how to code the variable for the analysis. The importance of this coding is often underrated in analyses. Using a test for trend, the specific choice of scores (e.g., 1, 2, 3 or 1, 2, 4, or scores that are the median of each category) used for the coding has an influence on the P-value. Based on the coding, a covariate may be statistically significant or not. Using dummy coding for an ordered variable with several groups, the coding may reflect the ordered nature of the variable, or it may only reflect a categorized variable ignoring the ordering. Using variable selection strategies to derive a final multivariable model coding can also have a substantial influence. A model that does not reflect the true influence of a variable may be selected if coding does not reflect the scale of the covariate.

COVARIATES

It has to be stressed that categorizing continuous covariates has often serious disadvantages and should be avoided. For the development of a prognostic model, we propose a better approach to work with continuous covariates. 3.2 Determination of the Functional Relationship Traditionally, for continuous covariates, either a linear functional relationship or a step function after grouping is assumed. Problems of the latter approach are issues about defining cutpoints, overparameterization, and loss of efficiency (some of which we have just discussed). In any case, a cutpoint model is often an inadequate way to describe a smooth relationship. The assumption of linearity may be incorrect, leading to a misspecified final model in which a relevant variable may not be included (e.g., because the true relationship with the outcome is non-monotonic) or in which the assumed functional form differs substantially from the unknown true form. An alternative approach is to keep the variable continuous and to allow some form of nonlinearity. Instead of using quadratic or cubic polynomials, a more general family of parametric models have been proposed by Royston and Altman (44) that is based on so-called fractional polynomial (FP) functions. Here, usually one or two terms of the form Xp are fitted, the exponents p being chosen from a small preselected set S of integer and noninteger values. Although only a small number of functions is considered (besides no transformation (P = 1), the set S includes seven transformations for FPs of degree 1 (FP1) and 36 for FPs of degree 2 (FP2); FP functions provide a rich class of possible functional forms leading to a reasonable fit to the data in many situations. Royston and Altman (44) dealt mainly with the case of a single predictor, but they also suggested and illustrated an algorithm for fitting FPs in multivariable models. By combining backward elimination with the search for the most suitable FP transformation for continuous predictors, Sauerbrei and Royston (45) propose modifications to this

9

multivariable FP (MFP) procedure. A further extension of the MFP procedure aims to reflect basic knowledge of the types of relationships to be expected between certain predictors and the outcome. In their study on prognostic factors for patients with breast cancer, Sauerbrei et al. (46) illustrated this approach in a simple way by comparing results with multivariable model building by either assuming linearity for continuous factors or by working with categorized data. Stability investigations with bootstrap resampling have shown that MFP can find stable models, despite the considerable flexibility of the family of FPs and the consequent risk of overfitting when several variables are considered (47). 4 REPORTING AND SUMMARY ASSESSMENT OF PROGNOSTIC MARKERS For many diseases, a large number of prognostic markers can be considered. Systematic reviews and meta-analytical approaches to identifying the most valuable prognostic markers are needed because conflicting evidence relating to markers is usually published across a number of studies. Obviously, this requires a number of important steps, starting with systematic reports of all studies. For randomized trials, the CONSORT (Consolidated Standards of Reporting Trials) statement (48) was developed and has improved reporting. Weaknesses of the original version were addressed in a revised version (49, 50). Covariates play only a minor role in this document, which requests that baseline demographic and clinical characteristics of each treatment group should be reported. Furthermore, details should be given for subgroup analyses and adjusted analyses. Covariates are not mentioned in the revised template of the CONSORT diagram. Reporting of prognostic factor studies is hardly addressed in the literature and a meta-analytic approach to assess the prognostic value of a marker is still an exception. To investigate the practicality of such an approach, an empirical investigation of a systematic review of tumor markers for neuroblastoma was performed by Riley et al. (3). They identified 260 studies of prognostic markers, which considered 130 different

10

COVARIATES

markers. They found that the reporting of these studies was often inadequate, in terms of both statistical analysis and presentation. There was also considerable heterogeneity for many important clinical/statistical factors. These problems restricted both the extraction of data and the meta-analysis of results from the primary studies, limiting feasibility of the evidence-based approach. Recently, reporting recommendations for tumor marker prognostic studies (REMARK) were published by a NCI-EORTC Working Group on Cancer Diagnostics (51). The investigators suggested guidelines to provide relevant information about the study design, preplanned hypotheses, patient and specimen characteristics, assay methods, and statistical analysis methods. In addition, the guidelines suggested helpful presentations of data and important elements to include in reports. The goal of these guidelines is to encourage transparent and complete reporting so that the relevant information will be available to others to help them evaluate the usefulness of the data and understand the context in which the conclusions apply (51). It is hoped that the quality of primary studies and their reporting of results improve with clear conclusions and policy recommendations formed about prognostic markers. Altman and Lyman (52) have proposed guidelines for both conducting and evaluating prognostic marker studies, including the need for prospective registration of studies. Alongside these, Riley et al. (3) have developed simple guidelines on how to report results to facilitate both interpretation of individual studies and the undertaking of systematic reviews, meta-analysis, and, ultimately, evidence-based practice. They recommend that research groups collaborate and identify the most commonly used prognostic tools in current practice so that adjusted results of new prognostic marker studies can use consistent sets of adjustment factors. The availability of full individual patient data, including all markers considered, is the most viable way to produce valid and clinically useful evidence-based reviews and metaanalyses. Individual patient data would limit the large problems of poor and heterogeneous reporting that have been observed, and also reduce the potential impact of reporting bias.

Prospective registration of studies alongside the availability of individual patients data would also help restrict the potential for publication bias. These guidelines all point to researchers working together toward planned pooled analyses, currently a particularly important concept for epidemiological research (53). However, summary assessments of prognostic factor studies based on individual patient data are still an exception. Concerning the current situation, Altman (4) lists several problems with systematic reviews of prognostic studies from publications: Studies are difficult to identify. Negative (nonsignificant) results may not be reported (publication bias). Reporting of methods is inadequate. Study designs vary. Most studies are retrospective. Inclusion criteria vary. Quality assessments lack recognized criteria. Studies employ different assays/measurement techniques. Methods of analysis vary. Studies use differing methods of handling of continuous variables (some data dependent). Studies use different statistical methods of adjustment. Studies adjust for different sets of variables. Quantitative information on outcome is inadequately reported. Presentation of results varies (e.g., survival at different time points). This list clearly underlines the necessity of individual patient data for a sensible summary of prognostic factor studies. REFERENCES ¨ 1. M. Schumacher, N. Hollander, G. Schwarzer, and W. Sauerbrei, Prognostic factor studies. In: J. Crowley and D. P. Ankerst (eds.), Handbook of Statistics in Clinical Oncology, 2nd ed. Boca Raton, FL: Chapman & Hall/CRC Press, 2006, pp. 289–333.

COVARIATES 2. L. McShane and R. Simon, Statistical methods for the analysis of prognostic factor studies. In: M. K. Gospodarowicz, D. E. Henson, R. V. P. Hutter, B. O’Sullivan, L. H. Sobin, and C. Wittekind (eds.), Prognostic Factors in Cancer, 2nd ed. New York: Wiley-Liss, 2001, pp. 37–48. 3. R. D. Riley, K. R. Abrams, A. J. Sutton, P. C. Lambert, D. R. Jones, et al., Reporting of prognostic markers: current problems and development of guidelines for evidence based practice in the future. Br J Cancer. 2003; 88: 1191–1198. 4. D. G. Altman, Systematic reviews of studies of prognostic variables. In: M. Egger, S. Davey, and D. Altman (eds.), Systematic Reviews in Health Care: Meta-Analysis in Context. London: BMJ Publishing, 2001, pp. 228–247. 5. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E9 Statistical Principles for Clinical Trials. Current Step 4 version, 5 February 1998. Available at: http://www.ich.org/LOB/media/ MEDIA485.pdf 6. Committee for Proprietary Medicinal Products (CPMP), Points to consider on adjustment for baseline covariates. Stat Med. 2004; 23: 701–709. 7. G. Raab, S. Day, and J. Sales, How to select covariates to include in the analysis of a clinical trial. Control Clin Trials 2000; 21: 330–342. 8. D. Tu, K. Shalay, and J. Pater, Adjustment of treatment effect for covariates in clinical trials: statistical and regulatory issues. Drug Inf J. 2000; 34: 511–523. 9. S. Assmann, S. Pocock, L. Enos, and L. Kasten, Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet. 2000; 255: 1064–1069. 10. S. Green, J. Benedetti, and J. Crowley, Clinical Trials in Oncology 2nd ed. Boca Raton, FL: Chapman & Hall/CRC, 2003. 11. M. Baum, Shooting sacred cows. Cancer Futures 2003; 2: 273–278. 12. S. Senn, Added values—controversies concerning randomization and additivity in clinical trials. Stat Med. 2004; 23: 3729–3753. 13. S. J. Pocock, and R. Simon, Sequential treatment assignment with balancing of prognostic factors Biometrics. 1975; 31: 103–115. 14. B. Efron, Forcing a sequential experiment to be balanced. Biometrika 1971; 58: 403–417.

11

15. T. M. Therneau, How many stratification factors are too many to use in a stratification plan? Control Clin Trials. 1993; 4: 98–108. 16. W. Rosenberger and J. Lachin, Randomization in Clinical Trials. Theory and Practice. New York: Wiley, 2002. 17. R. Peto, M. C. Pike, P. Armitage, Peto R, Pike MC, Armitage P, N. E. Breslow NE, D. R. Cox DR, et al., Design and analysis of randomized clinical trials requiring prolonged observations of each patient: II. Analysis and examples. Br. J. Cancer. 1977; 35: 1–39. 18. D. G. Altman, Comparability of randomised groups. Statistician. 1985; 34: 125–136. 19. D. G. Altman, Covariate imbalance, adjustment for. In: Redmond, C. and Colton, T., (eds.), Biostatistics in Clinical Trials. New York: Wiley, 2001, pp. 122–127. 20. M. Schumacher, M. Olschewski, and C. Schmoor, The impact of heterogeneity on the comparison of survival times Stat Med. 1987; 6: 773–784. 21. M. H. Gail, S. Wieand, and S. Piantadosi, Biased estimates of treatment effect in randomized experiments with nonlinear regressions and omitted covariates. Biometrika. 1984; 71: 431–444. 22. I. Ford, J. Norrie, and S. Ahmadi, Model inconsistency, illustrated by the Cox proportional hazards model. Stat Med. 1995; 14: 735–746. 23. C. Schmoor and M. Schumacher, Effects of covariate omission and categorization when analysing randomized trials with the Cox model. Stat Med. 1997; 16: 225–237. 24. L. D. Robinson and N. P. Jewell, Some surprising results about covariate adjustment in logistic regression models. Int Stat Rev. 1991; 58: 227–240. 25. M. N. Levine, V. H. Bromwell, K. I. Pritchard, B. D. Norris, L. E. Shepherd, et al., Randomized trial of intensive cyclophosphamide, epirubicin, and fluorouracil in premenopausal women with node-negative breast cancer. J Clin Oncol. 1998; 16: 2651–2658. 26. C. D. Atkins, Adjuvant chemotherapy with CEF versus CMF for node-positive breast cancer. J Clin Oncol. 1998; 16: 3916–3917. 27. E. Christensen, J. Neuberger, and J. Crowe, Beneficial effect of azathioprine and prediction of prognosis in primary biliary cirrhosis. Gastroenterology. 1985; 89: 1084–1091. 28. D. G. Altman and J. N. S. Matthews, Interaction 1: heterogeneity of effects. BMJ 1996; 313: 486.

12

COVARIATES

29. J. N. S. Matthews and D. G. Altman, Interaction 2: compare effect sizes not P values. BMJ. 1996; 313: 808. 30. J. N. S. Matthews and D. G. Altman, Interaction 3: How to examine heterogeneity. BMJ. 1996; 313: 862. 31. S. Yusuf, J. Wittes, J. Probstfield, and H. Tyroler, Analysis and interpretation of treatment effects in subgroups of patients in randimised clinical trials. JAMA. 1991; 266: 93–98. 32. S. J. Pocock, M. D. Hughes, and R. J. Lee, Statistical problems in the reporting of clinical trials: a survey of three medical journals. N Engl J Med. 1987; 317: 426–432. 33. W. Jonat, M. Kaufman, W. Sauerbrei, R. Blamey, J. Cuzick, et al., for Zoladex Early Breast Cancer Research Association Study. Goserelin versus cyclophosphamide, methotrexate, and fluorouacil as adjuvant therapy in premenopausal patients with nodepositive breast cancer: The Zoladex Early Breast Cancer Research Association Study. J Clin Oncol. 2002; 20: 4628–4635. 34. M. Bonetti and R. Gelber, A graphical method to assess treatment-covariate interactions using the Cox model on subsets of the data. Stat Med. 2000; 19: 2595–2609. 35. P. Royston, and W. Sauerbrei, A new approach to modelling interactions between treatment and continuous covariates in clinical trials by using fractional polynomials. Stat Med. 2004; 23: 2509–2505. 36. M. Mazumdar and J. R. Glassman, Categorizing a prognostic variable: review of methods. Code for easy implementation and applications to decision-making about cancer treatments. Stat Med. 2000; 19: 113–132. 37. D. G. Altman, B. Lausen, W. Sauerbrei, and M. Schumacher, Dangers of using ’’optimal’’ cutpoints in the evaluation of prognostic factors. J Natl Cancer Inst. 1994; 86: 829–835. 38. R. Miller, D. Siegmund, Maximally selected chi square statistics. Biometrics. 1982; 38: 1011–1016.

¨ 42. N. Hollander, W. Sauerbrei, and M. Schumacher, Confidence intervals for the effect of a prognostic factor after selection of an ‘‘optimal’’ cutpoint. Stat Med. 2004; 23: 1701–1713. 43. M. Mazumdar, A. Smith, and J. Bacik, Methods for categorizing a prognostic variable in a multivariable setting. Stat Med. 2003; 22: 559–571. 44. P. Royston and D. G. Altman, Regression using fractional polynomials of continuous covariates: parsimonious parametric modelling (with discussion). Appl Stat. 1994; 43: 429–467. 45. W. Sauerbrei and P. Royston, Building multivariable prognostic and diagnostic models: Transformation of the predictors by using fractional polynomials [corrections published in J R Stat Soc Ser A Stat Soc. 2002; 165: 399–400], J R Stat Soc Ser A Stat Soc. 1999; 162: 71–94. 46. W. Sauerbrei, P. Royston, H. Bojar, C. Schmoor, M. Schumacher, and the German Breast Cancer Study Group (GBSG), Modelling the effects of standard prognostic factors in node positive breast cancer. Br J Cancer 1999; 79: 1752–1760. 47. P. Royston and W. Sauerbrei, Stability of multivariable fractional polynomial models with selection of variables and transformations: a bootstrap investigation. Stat Med. 2003; 22: 639–659. 48. C. Begg, M. Cho, S. Eastwood, R. Horton, D. Moher, et al., Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996; 276: 637–639. 49. D. Moher, K. Schulz, and D. Altman, The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann Intern Med. 2001; 134: 657–662. 50. D. Altman, K. Shulz, D. Moher, M. Egger, F. Davidoff, et al., The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med. 2001; 134: 663–664.

40. S. G. Hilsenbeck and G. M. Clark, Practical P-value adjustment for optimally selected cutpoints. Stat Med. 1996; 15: 103–112.

51. L. M. McShane, D. G. Altman, W. Sauerbrei, S. E. Taube, M. Gion, G. M. Clark, for the Statistics Subcommittee of the NCI-EORTC Working on Cancer Diagnostics, REporting recommendations for tumor MARKer prognostic studies (REMARK). J Natl Cancer Inst. 2005; 97: 1180–1184.

¨ 41. M. Schumacher, N. Hollander, and W. Sauerbrei, Resampling and cross-validation techniques: a tool to reduce bias caused by model building? Stat Med. 1997; 16: 2813–2827.

52. D. G. Altman and G. H. Lyman, Methodological challenges in the evaluation of prognostic factors in breast cancer Breast Cancer Res Treat. 1998; 52: 289–303.

39. B. Lausen and M. Schumacher, Maximally selected rank statistics. Biometrics. 1992; 48: 73–85.

COVARIATES 53. M. Blettner, W. Sauerbrei, B. Schlehofer, T. Scheuchenpflug, C. Friedenreich, Traditional reviews, meta-analyses and pooled analyses in epidemiology. Int J Epidemiol 1999; 28: 1–9.

13

COX PROPORTIONAL HAZARDS MODEL

survival functions are compared across multiple groups or subpopulations, a commonly used model is to assume that hazard ratios are proportional across groups or subpopulations over time. This model is called the Cox proportional hazards model, which we will discuss in detail.

JIANWEN CAI and DONGLIN ZENG University of North Carolina at Chapel Hill, North Carolina

The Cox proportional hazards model, which is also referred to as Cox regression, proportional hazards regression, or relative-risk regression, is a regression procedure in the study of failure times that may be censored. It was originally proposed by Cox (1). It has become a standard method for dealing with censored failure time data and has been widely used in clinical trials and other biomedical research. Much methodological work has been motivated by this model. The aim of this article is to provide an overview on the Cox proportional hazards model. We will first present the model in its original form and discuss the interpretation and estimation of the regression parameters. We will also present the estimation of the cumulative hazard function and the survival function after a Cox model fit. Extension to the stratified model, approximations for handling tied failure-time data, incorporation of time-dependant covariates, as well as ways to check proportional hazards assumption, will be discussed.

1.2 Model and Parameter Interpretation The Cox proportional hazards model specifies the hazard rate λ (t; Z) for an individual with covariate vector Z as λ(t|Z) = λ0 (t) exp{β  Z}, t ≥ 0

(1)

where β is a p-vector of unknown regression coefficients, and λ0 (t) is an unknown and unspecified non-negative function. The function λ0 (t) is referred to as the baseline hazard function, and it is the hazard rate for an individual with a covariate vector Z = 0. Model (1) is a semiparametric model, in that the effect of the covariates on the hazard is explicitly specified, whereas the form of the baseline hazard function is unspecified. The regression coefficient represents the log-hazard ratio for one unit increase in the corresponding covariate given that the other covariates in the model are held at the same value. For example, consider a clinical trial that compares treatment A with treatment B. Let Z be an indicator for the treatment group A and Age represent the age of the subject. The proportional hazards model for studying the effect of treatment after adjusting for age is λ(t|Z, Age) = λ0 (t) exp (β trt Z + β age Age). The coefficient β trt is the log-hazard ratio for comparing treatment A to treatment B for subjects of the same age, whereas β age is the log-hazard ratio for 1 year increase in age for patients within the same treatment group.

1 MODEL AND ESTIMATION 1.1 Hazard Rate Function Let T denote a random variable that represents failure time, and let S(t) = P(T ≥ t) be the survival function. The hazard rate function for T is then defined as λ(t) = lim h−1 P h↓0
 t ≤ T < t + h |T ≥ t . That is, λ(t) denotes the instantaneous rate of failure at time t. Mathematically, the hazard rate function uniquely determines the distribution of T. t Specifically, S(t) = exp{− 0 λ(s) ds}. The hazard rate function is used commonly in analyzing censored survival data. For example, when comparing the survival distributions of two groups, one can use the ratio of the hazard functions from two groups. The latter is called a hazard ratio. When

1.3 Partial Likelihood Typical failure time data are subject to right censoring, in which some study subjects are only observed to survive beyond some time points. The censoring mechanism is usually assumed to be independent censoring; that is, the probability of failing at [t, t + dt) given all failure and censoring information as well

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

COX PROPORTIONAL HAZARDS MODEL

as all information on the covariates up to time t is the same as the probability of failing at [t, t + dt) given the subject did not fail up to time t and all information on the covariates up to time t. Let vj (j = 1, . . . , n) denote the observed failure and censoring times for a sample of n subjects and t1 ≤ · · · ≤ tk denote the observed distinct failure times. Under independent censoring and assuming no tied-failure times exist, the estimation of the regression parameter in Model (1) can be carried out by applying standard asymptotic likelihood procedure to the ‘‘partial’’ likelihood function [Cox (2)] L(β) =

k 







eβ Zi 

l∈Ri eβ Zl

i=1

,

(2)

where Zi is the covariate vector for the subject failing at ti ; Zl is the corresponding covariate vector for the lth member of the risk set Ri where Ri = {j:vj ≥ ti , j = 1, . . . n}. In fact, the ith factor in Model (2) is precisely the probability that the subject with the covariate vector Zi fails at ti given the failure, censoring, and covariate information prior to ti on all subjects in the sample. The ˆ is obtained estimator for β, denoted by β, by maximizing the partial likelihood, which is equivalent to solving the score equation U(β) = 0, provided finite maximum exists, where the score is: U(β) = ∇β log L(β) =

k  

Zi − Ei (ti ; β)



(3)

i=1

with



Ei (ti ; β) =

l∈Ri Zl eβ Zl 

l∈Ri eβ Zl

The partial likelihood given in Model (2), which differs from the marginal or conditional likelihood, was first introduced by Cox (2). It was later justified [c.f., Johansen (5)] that Model (2) is also the profile likelihood function for β. Hence, standard theory for parametric likelihood function is applicable under suitable regularity conditions. For 2 ˆ is a conlogL(β) example, the inverse of −∇ββ sistent estimator of the asymptotic variance ˆ The Wald’s test, score test, or likeof β. lihood ratio test for comparing two nested models can also be carried out using the partial likelihood. The formal justification for the asymptotic properties for βˆ was given in Andersen and Gill (6) using the martingale theory. 1.4 Cumulative Hazard Function Estimation The cumulative hazard function 0 (t) = t 0 λ0 (s) ds can be estimated by ˆ 0 (t) = 



1

ti ≤t

l∈Ri eβˆ Zl



(4)

after the Cox model fit [Oakes (7); Breslow (8)]. In the special case of one homogeneous population (i.e., no covariates exist), The estimator in (4) reduces to the Nelson-Aalen estimator for cumulative hazard function. As a result from S0 (t) = e−0 (t) , the baseline survival function S0 (t) can be estimated ˆ by Sˆ 0 (t) = e−0 (t) . To estimate the cumulative hazard function and the survival function ˆ 0 (t) or for any given values of Z, using  ˆ we have (t|Z) ˆ ˆ 0 (t)eβˆ  Z and Sˆ 0 (t) and β, = βˆ  z ˆ S(t|Z) = [Sˆ 0 (t)]e .

.

Note that U(β) is the sum of random variables that have a conditional, and hence an unconditional, mean of zero so that U(β) = 0 provides an unbiased estimating equation for β. It was pointed out that in the special case of the two sample problems, U(0) reduces to the log-rank test [Mantel (3); Peto and Peto (4)] when there are no tied observed failure times. Iterative procedures, such as the Newton-Raphson method, are used commonly to solve the score equation.

2 PRACTICAL ISSUES IN USING THE COX MODEL 2.1 Tied Data When failure time is continuous, although it is impossible to observe tied times in theory, tied data can develop in practice because of imprecise measure of times. The true time ordering of the ties is not observed. The partial likelihood function in Model (2) needs to be modified to reflect this ordering. Particularly, the exact partial likelihood function should be given to account for all possible

COX PROPORTIONAL HAZARDS MODEL

orderings at the tied failure times. However, such calculation can be cumbersome. For example, suppose three patients with subscripts 1, 2, 3, from a total of n patients, died on the same day after randomization. Because 3! = 6 is the possible ordering of these 3 patients, we denote each of the ordering sequences as

3



eβ Z1  βZ   e 1 + eβ Z2 + eβ Z3 + · · · + eβ  Zn 

eβ Z2  ×  βZ   2 e 1 + eβ Z2 + eβ Z3 /3 + · · · + eβ  Zn 

× 





eβ Z1 + eβ Z2

eβ Z3  .   + eβ Z3 /3 + · · · + eβ Zn

A1 = {1, 2, 3}, A2 = {1, 3, 2}, A3 = {2, 1, 3},

Empirically, Breslow approximation A4 = {2, 3, 1}, A5 = {3, 1, 2}, and A6 = {3, 2, 1}. works well when the number of ties is relatively small; Efron’s approximation yields Then the term contributing to the partial results closer to the exact results than Breslikelihood function from that day, that is, low’s approximation with only trivial increase the chance of observing these three patients in computer time. Most software uses Bres6 dying on the same day, is equal to  P(Ai ). low approximation as default. i=1

Particularly, 

P(A1 ) =





eβ Z1 + eβ Z2

eβ Z1  + eβ Z3 + · · · + eβ  Zn 

×



eβ Z2 Z β + e 3 + · · · + eβ  Zn



eβ Z3 , Z β + e 4 + · · · + eβ  Zn

eβ Z2



×

eβ Z3



P(A2 ) =

 eβ Z1

+

 eβ Z2

eβ Z1   + eβ Z3 + · · · + eβ Zn 

×



eβ Z2 +

eβ Z3  + · · · + eβ Zn

 eβ Z3



×

eβ Z2 , Z  β + e 4 + · · · + eβ Zn



eβ Z2

and so on. Clearly, the number of these terms increases dramatically with the number of ties in factorial fashion. For instance, with 5 ties, 5! = 120 possibilities can be calculated. In practice, to avoid the cumbersome calculation for the exact partial likelihood, approximations have been proposed. Breslow (9) suggests treating the sequentially occurring ties as if they were distinct. Therefore, in the previous example, the corresponding term in the partial likelihood function is 





eβ Z1 +β Z2 +β Z3  βZ 3 .   e 1 + eβ Z2 + eβ Z3 + · · · + eβ  Zn Another approximation is suggested by Efron (10), in which the corresponding term in the partial likelihood function is

2.2 Time-Dependent Covariates Time-dependent covariates refer to covariates which may vary over time. They are usually classified into external and internal [Kalbfleisch and Prentice ((11), section 6.3)]. Particularly, external time dependent covariates are those which are not dependent on a subject’s survival for their value at any time. They can be obtained even if the subject has died. Some examples for external timedependent variables are ‘‘fixed’’ variable (i.e., non–time-dependent), amount of pollution or radiation in an area, and number of years since birth. Comparatively, internal timedependent covariates are those variables that can only be measured or only have meaning when subject is alive. Examples include white blood count at time t, systolic blood pressure at time t, and spread of cancer at time t. The Cox model can handle both external and internal covariates. Particularly, for time-dependent covariates Z(t), the Cox model assumes 

λ( t| Z(t)) = λ0 (t)eβ Z(t) . Under this model, the hazard ratio for comparing subjects with Z(t) versus subjects  ˜ ˜ with Z(t) is equal to eβ (Z(t)−Z(t)) , which is no longer independent of t. Estimation of β can be obtained by maximizing the partial likelihood function    k  eβ Zi (ti ) L(β) =  l∈Ri eβ Zl (ti ) i=1

4

COX PROPORTIONAL HAZARDS MODEL

where Zi (t) is the covariate vector for the subject failing at ti ; Zl is the corresponding covariate vector for the lth member of the risk set Ri where Ri = {j: vj ≥ ti , j = 1, . . . n}. In practice, time-dependent covariates can be introduced to model hazard ratios associated with fixed covariates that change over time. For example, in the Atomic Bomb Survivors Study, the event of interest is leukemia incidence. The follow-up time is the number of years since 1945. Let XE be the indicator for being exposed and we fit the following model: λ(t) = λ0 (t)eβXE +γ XE ·t . Based on this model, the hazard ratio between exposed subjects and nonexposed subjects is λ(t|X E = 1)/λ(t|X E = 0) = eβ+γ t . Thus, this model allows the log hazard to change linearly with time. Additionally, we can test H0 : hazard ratio constant versus H1 : log-hazard ratio changes linearly with time by testing H0 : γ = 0 versus H 1 : γ = 0 in the above model. The time-dependent covariate can also be included to study the effect of a covariate that changes over time. For example, we are interested in the association of smoking and lung cancer incidence. Time is measured by age. Let XE (t) denote the cumulative number of packs smoked by age t. The following model can be fitted: λ(t) = λ0 (t)eβXE (t) . As a result, the hazard ratio that compared a smoker who has smoked 350 packs/year since age 20 with a person of the same age who has never smoked is eβ·350(t−20) if t ≥ 20 and 1 if t < 20. In the situation when β > 0, the hazard ratio increases with time as long as smoker continues to smoke. Although the Cox model with timedependent covariates is flexible in modeling time-varying hazard ratio, internal timedependent covariates are particularly susceptible to being inappropriately controlled, because they often lie in the causal pathway about which one wants to make inferences. For example, in a clinical trial for the effect of immunotherapy in the treatment of metastatic colorectal carcinoma, adjusting for the most recent depressed white blood counts (WBC) would facilitate treatment comparisons among subjects with similar prognoses at any specified time. However, immunotherapy might improve the prognosis by improving WBC over time; adjustment

for WBC over time might remove the apparent effect of treatment, because treated and control subjects with the same WBC might have similar prognoses. In general, when the time-dependent covariate lies in the causal pathway of the treatment on the event process, such time-dependent covariate should not be adjusted in the model if the treatment effect is the focus of interest. Caution needs to be exercised when applying the Cox model with internal time-dependent covariates. 2.3 Stratified Population In many applications, the study population may consist of more than one distinct stratum because of some confounding variable C. Estimation of β without adjusting for confounding variable can lead to misleading results. In the Cox proportional hazards model, there are two ways that one can adjust for the effect of C. To illustrate, we suppose a confounder C has three levels, and let ZE denote the indicator of exposure status in which we are interested. In the first approach, one is to create indicator variables for indicating different levels of C and include the indicator variables as covariates in the model. Specifically, let Zj = 1 if C = j, and Zj = 0, otherwise for j = 2 and 3. To adjust for the effect of C, we fit the following model λ(t) = λ0 (t)eβ2 Z2 +β3 Z3 +βE ZE . Based on the above model, when C = 1, the hazard is λ(t) = λ0 (t)eβE for the exposed group and λ(t) = λ0 (t) for the unexposed group. The hazard ratio that compares the exposed with the unexposed when C = 1 is then HR = eβE . Similarly, within other levels of C, the hazard ratio is also eβE . Note also that the hazard ratio that compared the unexposed group with C = j to the unexposed group with C = 1 is eβj for j = 2 and 3. Because β 2 and β 3 are regression coefficients and are not restricted, the stratum-to-stratum differences in log-hazard ratios for the unexposed groups have different magnitudes, but they are parallel over time. This parallelism is the key assumption for such indicator variable adjustment approach. A second method of stratification is to use a stratified Cox model. Specifically, λ(t) = λ0j (t)eβE XE where j = 1, 2, and 3 correspond to the three different levels of C. Based on

COX PROPORTIONAL HAZARDS MODEL

this stratified model, when C = 1, the hazard is λ(t) = λ01 (t)eβE for the exposed group and λ(t) = λ01 (t) for the unexposed group. The hazard ratio that compares the exposed with the unexposed when C = 1 is then HR = eβE . Similarly, within the other levels of C, the hazard ratio is also eβE . Note that the hazard ratio that compared the unexposed group with C = j to the unexposed group with C = 1 is λ0j (t)/λ01 (t) for j = 2 and 3. Because λ0j (t) (j = 1, 2, 3) are arbitrary functions of time, the stratum-to-stratum differences in log-hazard ratios for the unexposed groups across different level of C are completely arbitrary. Comparing the above two approaches, the first one that uses the indicator variable stratification model is restrictive because it requires the parallel log hazards across strata in the unexposed group; instead, the second approach that uses stratum-specific baseline hazard functions is more robust. However, with increasing number of parameters in the second approach, any test for the null exposure effect is less powerful as compared with the first one if the parallel assumption for the log hazard is satisfied. Additionally, if the confounder C is measured continuously and the strata were formed by grouping values of it, better control for C might be achieved with continuous covariate adjustment as done in the first approach. 2.4 Censoring Mechanism The standard inference for the Cox proportional hazards model is valid only when rightcensoring time satisfies a certain censoring mechanism. In practice, such a censoring mechanism often occurs in the following two kinds of scenarios: 1. Random censorship. The censoring times of the individuals are stochastically independent of each other and of the failure times. Random censorship includes the special case of Type I censoring, in which the censoring time of each individual is fixed in advance, as well as the case in which subjects enter the study at random over time and the analysis is carried out at some prespecified time.

5

2. Independent censorship. At time t and given all information on the covariates up to time t, individuals cannot be censored because they seem to be at unusually high (or low) risk of failure. In terms of formula, it is saying that:  P failure in [t, t + dt) H(t), Z(t)  = P failure in [t, t + dt) survival to t, Z(t) , where H(t) contains the failure and censoring information up to time t. Many usual censoring schemes belong to this class, which includes random censorship. Type II censoring is one special case of independent censoring, in which the study may continue until the dth smallest failure time occurs, at which time all surviving items are censored. Another special case of independent censoring is progressive type II censoring, in which a specific fraction of individuals at risk may be censored at each of several ordered failure times.

2.5 Assessing the Proportional Hazards Assumption One crucial assumption for the proportional hazards model is that the hazard ratio for different values of a time-independent covariate is constant over time. This assumption needs to be checked. Four practical approaches can be implemented to assess the proportional hazards assumption. 2.5.1 Method 1. This approach introduces an interaction term between the covariate being assessed and a specified function of time. Specifically, if we are interested in assessing the proportional hazards assumpof Z, tion for Zj , which is the jth element consider fitting the model λ(t Z) = λ0 (t)  eβ Z+γ Zj Q(t) where Q(t) is a specified function of t [often Q(t) = t]. Thus, to check the proportional hazards assumption, we can test the null hypothesis of γ = 0. This test is sensitive to the type of departure specified by the function Q(t).

6

COX PROPORTIONAL HAZARDS MODEL

2.5.2 Method 2. The second approach is to divide the time axis into k intervals, which are in turn denoted as [0,t1 ),[t1 , t2 ), · · · ,[tK−1 , tK ) with t0 = 0 and tK = ∞. Replace β  Z by K

βk ZI{tk−1 ≤t
model. The heterogeneity of (βˆ1 , βˆ2 , . . . , βˆK ) is an evidence of nonproportionality. 2.5.3 Method 3. The third approach is based on the fact that under the proportional hazards assumption, log(− log S(t Z )) = −β  Z + log(− log S0 (t)), where S(t|Z) is the conditional survival function given Z and S0 (t) = exp {−0 (t)}. Therefore, the survival functions for all levels of Z should be parallel. We can then examine ˆ Z )) for assessing the the plot of log(− log S(t proportional hazards assumption for a single ˆ Z ) is estidiscrete variable Z, where S(t mated nonparametrically or using stratified Cox regression when other covariates need to be controlled for. When Z is continuous, we may stratify Z into Q levels and after estimation within each level, plot log(−logSˆ q (t)) versus t for q = 1, · · · , Q to examine whether the curves are parallel. 2.5.4 Method 4. As model checking for linear regression models is based on residuals, this approach also uses the so-called Schoenfeld residuals. Particularly, for subject i and the jth covariate, Schoenfeld residuals are defined as Lij = Zij (ti ) − Zj (ti ), where Zj (t) =

n βˆ  Zl t)  I(tl ≥t)Zlj (t)e

l=1

n βˆ  Zl (t)  I(tl ≥t)e

. If the proportional

l=1

hazards assumption holds, Lij should be a random walk. For assessing general departure from proportional hazards, Schoenfeld residuals can be plotted. The residuals should be randomly scattered about zero if the proportional hazards assumption holds. The trend in the Schoenfeld residuals reflects how the effect of the covariate is varying over time. 2.6 Software The Cox proportional hazards model can be fit in several software programs including R-package (http://www.r-project.org/), Splus

(http://www.insightful.com/), and SAS (http:// www.sas.com/). Especially, both R and Splus use function called ‘‘coxphm,’’ whereas SAS uses procedure ‘‘PHREG.’’ Options in these packages can allow one to choose different ways of handling ties, fit stratified Cox models, and perform tests to assess proportional hazards assumptions. All packages can handle time-dependent covariates as well. 3 EXAMPLE Data are taken from a randomized trial in which men with advanced inoperable lung cancer were randomized to either standard treatment or one test chemotherapy treatment [Kalbfleisch and Prentice (11)]. The failure time of interest was time from randomization to death, and one important question was to examine the effect of comparing two treatments in terms of the hazard rate for death. The data contain 137 patients; 69 patients were randomized to the standard treatment and the remaining to the test chemotherapy treatment. The median survival time is 62 days, and 9 patients were right censored. The Cox proportional hazards model can be used to analyze the data. The covariates used in the regression model include treatment indicator, age at randomization, indicator variables for four histologic type of tumors (squamous, small, adeno, and large), and patient performance status using the ‘‘Karnofsky rating.’’ Efron’s method is used to handle ties. The estimates for the regression coefficient, which maximizes the partial likelihood function, are given in Table 1. It shows that the two treatments have no statistically significant difference on the survival; patients with small or adeno tumors had higher risk than patients with squamous tumor. Patients with a lower Karnofsky rating tended to have higher risk. Particularly, for patients within the same treatment group and with the same Karnofsky rating and age, those with a small tumor had 135.5% higher risk than those with a squamous tumor. Those with an adeno tumor had 225% higher risk than those with a squamous tumor. The baseline cumulative hazard function can be estimated using

COX PROPORTIONAL HAZARDS MODEL

7

Table 1. Analysis of lung cancer data using Cox regression ˆ

βˆ

Covariate Chemotherapy versus standard Small tumor versus squamous tumor Adeno tumor versus squamous tumor Large tumor versus squamous tumor Karnowsky rating Age

0.303 0.856 1.179 0.402 −0.034 −0.009

the expression in Equation (4), and Fig. 1 displays the estimated baseline cumulative hazard function versus time. To assess the proportional hazards assumption, we calculate the Schoenfeld residuals and plot in Fig. 2. Each subplot in the figure plots the Schoenfeld residuals associated with each covariate versus time. The figure indicates that no obvious departure from the proportional hazards assumption is observed.

4 EXTENSION OF THE COX PROPORTIONAL HAZARDS MODEL

ˆ se(β)

1.354 2.355 3.250 1.495 0.968 0.991

0.206 0.271 0.296 0.283 0.005 0.009

P-value 0.14 0.00 0.00 0.15 0.00 0.33

problems. We list some important extensions below. 1. Extension to the counting process. The counting process is defined as a stochastic process of time t and it counts number of failures by time t. Thus, singlefailure time data are analyzed by a special counting process with only one event occurring. However, the counting process is more general, and its definition includes recurrent failures. If we denote this process as N(t), similar to hazard function, then we can define socalled intensity function given covari
−1 ate Z as λ(t|z)  = limh↓0 h E N(t + h)− N(t) H(t), Z where H(t) is the history process up to time t. With covariates Z, one natural extension of the Cox

0

2

Baseline cumulative hazards 4 6 8 10

12

Since its introduction, the Cox proportional hazards model has been extended from analyzing univariate failure time data to several

eβ

0

200

400 600 Time in days

800

1000

Figure 1. Estimated baseline cumulative hazard function over time.

1000

0

200

400

600

800

1000

400 600 Time in days

800

1000

0

200

400 600 Time in days

800

1000

Residue (age) −0.3 −0.1 0.1 0.3

200

Residue (large vs squamous) −6 −2 2 6

Time in days

0

Residual (kps)

Residue (small vs squamous) −8 −4 0 4

800

−0.20 −0.05 0.10

400 600 Time in days

Residual (adeno vs squamous) −5 0 5

200

0

200

400 600 Time in days

800

1000

0

200

400 600 Time in days

800

1000

Figure 2. Schoenfeld residuals for assessing proportional hazards assumption.

COX PROPORTIONAL HAZARDS MODEL

Residual (treatment) −4 0 2 4

8

0

COX PROPORTIONAL HAZARDS MODEL

proportional hazards model is the proportional intensity model which assumes λ(t Z ) = λ0 (t) exp{β  Z}. The partial likelihood function, which is the same as before, can be constructed to estimate and make inference on β. Martingale theory based on counting process were developed [c.f., Andersen and Gill (6); Fleming and Harrington (12)] and have been fruitful in obtaining large sample results for event time models. 2. Extension to multivariate failures. Another extension of the Cox proportional hazards model is to model multivariate failure times [Wei et al (13); Hougaard(14)]. In this case, multivariate events, such as death, time to relapse, and so on, are of interest. An event-specific Cox regression model can be used to model the hazard function of each event, each with a different baseline hazard function and covariate effect. A pseudo-partial likelihood function, which is the simple product of each partial likelihood from each event, can be maximized to estimate covariate effect, whereas the variance needs to be estimated using sandwiched form. 3. Extension to clustered failures. In clustered failure time data, subjects may or may not experience the same type of event but they may be correlated because subjects are from the same cluster. The Cox proportional hazards model can be used to model each individual’s hazard function; however, an unobserved cluster-specific frailty is introduced into each model to account for within-cluster correlation. Such a model is called frailty model for clustered failures [Clayton and Cuzick (15); Hougaard (14)]; but it is essentially an extension of the Cox proportional hazards model. 4. Extension to different data structures. The Cox proportional hazards model has also been applied to analyze different data structures. Examples

9

include left-truncation data [Wang (16)]; interval-censored data [Huang (17), Sun (18)], in which failure time is not observed directly but is observed within some intervals; and case-cohort and case-control design [Breslow and Day (19)], in which subjects are sampled based on failure outcomes.

REFERENCES 1. D. R. Cox, Regression models and life-tables. J. Royal Stat. Soc. B 1972; 34: 187–202. 2. D. R. Cox, Partial likelihood. Biometrika 1975; 62: 269–276. 3. N. Mantel, Evaluation of survival data and two new rank order statistics arising in its consideration. Cancer Chemother. 1966; 50: 163–170. 4. R. Peto and J. Peto, Asymptotically efficient rank invariant test procedures (with discussion). J. Royal Stat. Soc. A, 1972; 135: 185–206. 5. S. Johansen, An extension of Cox’s regression model. Internat. Stat. Rev. 1983; 51: 258–262. 6. P. K. Andersen and R. D. Gill, Cox’s Regression model for counting processes: a large sample study. Ann. Stat. 1982; 10: 1100–1120. 7. D. Oaks, Contribution to discussion of paper by D. R. Cox. J. Royal Stat. Soc. B 1972; 34: 208. 8. N. E. Breslow, Contribution to discussion of paper by D. R. Cox. J. Royal Stat. Soc. B 1972; 34: 216–217. 9. N. E. Breslow, Covariance analysis of censored survival data. Biometrics 1974; 30: 89–99. 10. B. Efron, The efficiency of Cox’s likelihood function for censored data. J. Am. Stat. Assoc. 1977; 72: 557–565. 11. J. D. Kalbfleisch and R. L. Prentice, The Statistical Analysis of Failure Time Data. New York: Wiley, 2002. 12. R. R. Fleming and D. P. Harrington, Counting Processes and Survival Analysis. New York: Wiley, 1991. 13. L. J. Wei, D. Y. Lin, and L. Weissfeld, (1989), Regression analysis of multivariate incomplete failure time data by modeling marginal distributions. J. Am. Stat. Assoc. 1989; 84: 1065–1073. 14. P. Hougaard, Analysis of Multivariate Survival Data. New York: Springer, 2000.

10

COX PROPORTIONAL HAZARDS MODEL

15. D. Clayton and J. Cuzick, Multivariate generalizations of the proportional hazards model (with discussion). J. Royal Stat. Soc. A, 1985; 148: 82–117. 16. M. C. Wang, Hazards regression analysis for length-biased data. Biometrika 1996; 83: 343–354. 17. J. Huang, Efficient estimation for the proportional hazard model with interval censoring. Ann. Stat. 1996; 24: 540–568. 18. J. Sun, The Statistical Analysis of Intervalcensored Failure Time Data. New York: Springer, 2006. 19. N. E. Breslow and N. E. Day, Statistical Methods in Cancer Research: The Design and Analysis of Cohort Studies. Lyon, France: International Agency for Research on Cancer, 1987.

FURTHER READING P. K. Andersen, Borgan, R. D. Gill, and K. Keiding, Statistical Models Based on Counting Processes. New York: Springer-Verlag, 1993. R. R. Fleming, and D. P. Harrington, Counting Processes and Survival Analysis. New York: Wiley, 1991. P. Hougaard, Analysis of Multivariate Survival Data., New York: Springer, 2000. J. D. Kalbfleisch, and R. L. Prentice, The Statistical Analysis of Failure Time Data. New York: Wiley, 2002. T. M. Therneau and P. M. Grambsch, Modeling Survival Data: Extending the Cox Model. New York: Springer, 2000. J. Sun, The Statistical Analysis of Intervalcensored Failure Time Data. New York: Springer, 2006.

CROSS-REFERENCES Censoring Hazard rate Hazard ratio Logrank Test Survival Analysis

CRONBACH’S ALPHA

alpha, in general, is not a measure of reliability, although this is commonly reported as such in the psychology research literature, for the assumptions underlying that claim are seldom satisfied. Consider the following two examples: Example 1. Suppose the m items each relate to the different systems that might be primarily affected by syphilis (the last one ‘‘Other’’), with each item to be answered 0 (No) or 1 (Yes). Then for those without syphilis, the answer is No to all items, and for those with syphilis, the answer is Yes to one and only one item. The total score then is an indicator of the presence of syphilis in the individuals in the population and may have nearly perfect inter-rater or test–retest reliability. However, Cronbach’s alpha will be near zero, because answering Yes to one item precludes answering Yes to any other. In this case, the m items do not all measure exactly the same characteristic and the errors are not independent or identically distributed. Thus, Cronbach’s alpha near zero in this kind of a situation is not necessarily an indication that the total score is poor. Here Cronbach’s alpha near zero is an indication that the items are not addressing the same characteristic in the same way; i.e., inconsistency exists among the items. In many cases, the types of items used in measure development for randomized clinical trials deliberately attempt to tap into different, relatively independent facets of the construct the measure seeks, which is a situation that might well lead to Cronbach’s alpha near zero, but also to a reliable and valid measure. Example 2. Suppose that for each administration of the test, the subject is assigned a random number, and then asked m times: What number do you have? Then, if subjects comply, Cronbach’s alpha should be 1, and yet the average of the m responses is a random number with zero reliability over separate administrations of the test. A Cronbach’s alpha that is too close to 1 may indicate redundancy among the items or correlated errors, and it is not necessarily an indication that the score reliably measures any subject characteristic. Such a situation presents a difficulty in documenting a measure for use

HELENA CHMURA KRAEMER Stanford University, California

The underlying model for Cronbach’s alpha is that the response on the jth item for subject i is modeled by Xij = µ + ξi + ηj + εij where ξ i is the characteristic of interest for subject i, ηj is the fixed mean for item j, and εij j = 1,2, . . . ,m are errors terms that are independent of each other and of ξ i . Then if the score for subject i is the sum or average of the m items, under this model, the interrater reliability (q.v.) of that score is given by Cronbach’s alpha, α, where α=

mρ (m − 1)ρ + 1

where ρ is Variance (ξ i )/Variance (X ij ) (in this model, it is assumed to be the same for each item). 1

ESTIMATION OF ALPHA

To estimate α, one draws a sample of N subjects from the population of interest and administers the m items to each subject. Then, using a two-way analysis of variance (ANOVA; N subjects by m items), one can estimate α with Fs − 1 Fs where FS is the F-statistic to test the subject effect in the ANOVA with (N − 1) and (m − 1)(N − 1) degrees of freedom. Although the motivation for Cronbach’s alpha is based on considerations of the reliability of the score based on the sum or average of the m items, in practice, it is usually appropriate only to cite Cronbach’s alpha as a measure of internal consistency among the items used to compute that score. Cronbach’s

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

CRONBACH’S ALPHA

in a randomized clinical trial, for presenting redundant items merely places greater measurement burden on the patients in the trial with no increase in the quality or quantity of information available to be used to answer the research questions of the trial. When m items are presented to a subject at one time, as they usually are, it is difficult to ensure that the errors are uncorrelated and such correlated errors tend to inflate Cronbach’s alpha. Consequently, Cronbach’s alpha indicates how well the multiple items tap into a single construct (including correlated errors), always a measure of interitem consistency, but not necessarily a measure of reliability of the score obtained by summing those items. 2

ITEM-TEST CORRELATIONS

Generally, statistical computer programs that estimate Cronbach’s alpha also examine the item–test correlations in two ways, which is a useful addendum to Cronbach’s alpha. First, the correlation between each of the m items and the total score is computed. Since the item appears in both variables correlated in this way, that correlation coefficient might be inflated, particularly if the number of items is relatively small. Alternatively, the correlation between each of the m items and the score based on the remaining m – 1 items is computed. Ideally, if all the items measure the same construct and contribute about equal information, as hypothesized in the underlying model for alpha, each item–test correlation will be reasonably and about equally as strong. If an item seems to have substantially smaller correlation than others, that suggests an inconsistency with those other items. If, in fact, the remaining items are valid measures of the construct of interest, the impact of removing that item would be to improve the quality of the score as a measure of that construct. However, it is possible that the one or two items that seem to be inconsistent with the others are the most reliable and valid measures of the construct of interest, whereas the others measure some other construct. Then removing the inconsistent few items may actually detract from the quality of the score

as a measure of the construct of interest. For this reason, before seeking to estimate how internally consistent a set of items is, there should be some empirical support for the reliability and validity of each of the items for the construct of interest. 3 CONSIDERATIONS IN THE USE OF ALPHA Computing and reporting Cronbach’s alpha is only one of many ways available to document the quality of a score based on summing or averaging a set of m items. For example, psychometricians often perform factor analysis on the set of items. If such a factor analysis indicated that one major factor underlied the set of items, and a factor structure that placed approximately equal weight on each item for that factor, this would automatically indicate that Cronbach’s alpha should be quite high. On the other hand, should factor analysis indicate the presence of multiple independent factors, or one factor with very disparate weights on the m items, it is likely that Cronbach’s alpha would be low to moderate. Another interesting alternative is item response theory (2). In this situation, it is assumed that there is one underlying latent (unmeasurable) construct underlying the response to each item. The responses to each item (or some transformation of the response) are assumed to have a distribution determined by a linear function of that underlying construct. However, the slope and intercept of the linear model relating a response to each item may be item-specific. The intercept is an indication of the ‘‘difficulty’’ of the item and of the slope of the ‘‘discrimination’’ of the item. If an item has zero discrimination, then responses on that item are of no value in estimating the latent construct. Such an item is best removed. The stronger the discrimination, the more informative is the response on that item to the level of the latent construct. At the same time, on two items with the same discrimination, the same answer may indicate higher levels of the underlying construct than the other, and this is reflected in the difficulty of the item. The ideal set of items to measure that latent construct would include items at varying difficulty to ensure

CRONBACH’S ALPHA

sensitivity to variations at all levels of the latent construct. The goal would be to estimate the value of the latent construct for each subject from the responses on the m items. Cronbach’s alpha and item response theory both assume a single underlying latent construct. However, item response theory attempts to estimate that latent construct, whereas users of Cronbach’s alpha simply sum or average the items, putting equal weight on items that are more or less difficult, and have better or poorer discrimination. Coefficient alpha clearly plays an important role in measure development, a process that logically predates use of such measures in randomized clinical trials. REFERENCES 1. L. J. Cronbach, Coefficient alpha and the internal structure of tests. Psychometrika 1951; 16:297–334. 2. S. E. Embretson and S. Reise, Item Response Theory for Psychologists. Mahwah, N.J.: Erlbaum, 2000.

CROSS-REFERENCES Interrater Reliabilitiy Reliability Analysis

3

CROSSOVER DESIGN

crossover trial. Therefore, the main reason for considering the use of a crossover design is the ability to produce sufficiently high precision with comparatively small numbers of subjects. Indeed, examples can be suggested in which the precision from a crossover trial is orders of magnitude greater than a parallel group study of equivalent size. Crossover trials have also proved to be important in dose-ascending studies of pharmacokinetics of new drug substances in healthy volunteers, for which the requirements above are broadly satisfied. This potential gain in precision comes with a price, however. Obviously such a design cannot be used with treatments that irreversibly change the condition of the subject, such as treatments that are curative. Once treatment has ceased, the subject must return to the original condition, at least approximately. For the same reason, such designs are only appropriate for conditions that are reasonably stable over time. For example, crossover designs have often been used in the past for chronic conditions like angina and asthma. When using such a design, the possibility always remains that some consequence of earlier treatment may still be influential later in the trial. In the crossover context, this is called a carryover effect. In principle, such effects may literally be caused by a physical carryover of a drug, or to a less-direct cause such as a psychological effect of prior treatment. The key point is that subjects in the second and later periods of a crossover trial have systematic differences in their past experience in the trial. This potential source of bias is akin to confounding in an epidemiological study and implies that to some extent the analysis of data from a crossover trial will inevitably rely more on assumptions and modeling, and less directly on the randomization, than a conventional parallel-group study. This issue is particularly apparent in the so-called twoperiod, two-treatment or 2 × 2 design. This design is the simplest and arguably, the most commonly used design in a clinical setting. In this design, each subject receives two different treatments that we conventionally label as A and B. Half the subjects are randomized

MICHAEL G. KENWARD London School of Hygiene and Tropical Medicine London, United Kingdom

1

INTRODUCTION

In the familiar parallel-group design, each subject is randomly allocated to one of the experimental interventions or treatments. In contrast, in a crossover design, each subject is randomized to a sequence of treatments, which is a special case of a repeated measures design. Despite this difference, the goal of a crossover study, in nearly all settings remains the same as that of a parallel-group study: to compare the effects of the single treatments not the effects of the sequences to which the subjects are randomized. Thus, in a crossover trial, subjects are not randomized to the interventions under comparison, and it has important implications for the design, analysis, and interpretation. The main advantage of a crossover design over the parallel group is the opportunity it provides to compare the effects of treatments within subjects. That is, any component of an individual’s response that is consistent across time is removed from the treatment comparison. The degree of such consistency is measured by the within-subject (or intraclass) correlation. As a consequence, the precision of the treatment effect from a crossover study will be larger than that from a parallel group study with the same numbers of subjects, and it will have the greatest advantage when the within-subject correlation is high. In this respect, the crossover study has much in common with a blocked or matched study, in which the subject is the blocking/matching variable. A second advantage in terms of precision per subject that is less often remarked in this context is that each subject contributes observations under more than one treatment. So, although a simple baseline measurement can be used to remove between-subject variability in a parallel group trial, the precision will still be substantially less than the corresponding

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CROSSOVER DESIGN

to receive A first and then, after a suitably chosen period of time, cross over to B. The remaining subjects receive B first and then cross over to A. Because this particular design is used so commonly and because it raises very special issues in its own right, the next section is devoted specifically to it. Many other so-called higher order designs exist with more than two periods, and/or treatments, and/or sequences. The choice of such designs and associated analyses are considered in the remaining sections. Much of what follows is elaborated in two standard texts on crossover trials: Senn (1) and Jones and Kenward (2), and we refer the reader to these for more details.

out a priori an overall change in response between the two periods: a so-called period effect. Hence, on design considerations alone we must adjust for a period effect. It is the requirement for this adjustment than means that, in general, any crossover design must have at least as many sequences as treatments. Fortunately the adjustment is very simple to do in practice, particularly so in the 2 × 2 design. Denote the averages from the four sequence-by-period means by Y ij . The B-A mean treatment difference adjusted for period effect (τ say) is then estimated as

2 THE TWO-PERIOD, TWO-TREATMENT DESIGN

To make inferences about τ , we note that if we define the within-subject differences Dik = Yi2k − Yi1k , then τˆ is simply the mean difference of these between the two sequence groups: 1 τˆ = (D2. − D1. ). 2

In the two-period, two-treatment, or 2 × 2, design, each subject is randomized to one of two treatment sequences: A then B, or B then A. We use the term period for the time interval in which the subject receives a particular treatment, hence it is a two-period design. Often, an additional period occurs between the two treatment periods in which subjects do not take either treatment. This period is called the washout period, and its purpose is largely to lessen the likelihood of bias caused by carryover. We return below to such concerns, as well as to the incorporation into the analysis of measurements made in the washout period. To begin, we assume that no carryover effect is present and no treatmentby-period interaction has occurred; we focus on the use of the measurements made only in the treatment periods. To represent these measurements, we introduce some notation. Let Yijk denote the outcome from subject k; (k = 1, . . . , ni ) on sequence i; (i = 1, 2) in period j; (j = 1, 2). It is assumed that ni subjects are randomized to the ith sequence. This outcome might, for example, be a summary measure from all or part of the relevant treatment period or a measure from a particular time, often the end, of the period. When the treatment effect is estimated from these measurements, it is necessary to acknowledge the repeated measures structure of the design, which implies that we cannot rule

τˆ =

1 (Y 12. + Y 21. − Y 11. − Y 22. ). 2

(1)

For inferences we can therefore use standard two sample procedures on the differences Dik . Most commonly, the standard two-sample t-test framework is used to provide test statistics, confidence intervals, and power calculations. For the latter, it must be remembered that the relevant variance is that of the within-subject differences Dik , not the raw observations Yijk . Alternatively, nonparametric procedures can be applied to the same differences. We therefore observe that with complete data, under the assumption of no carryover or treatment-by-period interaction, the analysis for the 2 × 2 design is a very straightforward application of a two-sample procedure applied to the within-subject differences. However, concern has existed in this setting about the carryover assumption. If a carryover effect exists, then the simple estimate in Equation (1) will be biased for τ . An early and influential paper by Grizzle (3) made the suggestion that the analysis should therefore begin with a test for the presence of such a carryover and, only if insignificant, should the estimator in Equation (1) be used. If it is found significant, then the treatment effect should be estimated from the first period data only (i.e., Y 21. − Y 11. ).

CROSSOVER DESIGN

This two-stage procedure was widely adopted. However, it suffers from two very serious problems that mean, in practice, it should be avoided. The first problem lies with the power of the initial test for carryover that, in the 2 × 2 trial, turns out to be equivalent to a test for treatment-by-period interaction. This test parallels that for the direct treatment effect above, but it is calculated using the subject sums Sik = Yi1k + Yi2k in place of the differences Dik , and there is no factor of 1/2. Suppose that an observation Yijk has variance σ 2 and the within-subject correlation is equal to ρ. Then V(Dik ) = (1 − ρ)σ 2 and V(Sik ) = (1 + ρ)σ 2 . It follows that the ratio of the variances of the carryover estimator and the the direct treatment estimator is R=2+

ρ 1−ρ

As the within-subject correlation increases, this ratio grows without limit. In a typical crossover setting the within-subject correlation is large, and the trial is powered for the direct treatment effect, which leads to comparatively small numbers of subjects. Consequently, the power for the carryover effect is often very low indeed. As an example, consider the 2 × 2 crossover trial on treatment for asthma described in Reference 2, section 2.10. This study had 17 subjects; the outcome was a lung function measure: forced expired volume in one second. In this trial, ρ was estimated as 0.96, with corresponding R equal to 28.6. As a consequence, 28.6 times as many subjects (476) would have been required to estimate the carryover effect with the same precision as the direct-treatment effect and, in terms of power in the actual sample, the test for carryover is effectively useless. Because of the typical low power of the test for carryover, it was suggested in the original two-stage proposal (3) that it might be made at a higher nominal size, such as 10%. However, any such modification is insufficient in most cases to rescue this test, and negative results from such low-powered tests convey no useful information. The second major problem with the twostage procedure was pointed out by Freeman

3

(4). Because of the statistical dependence between the preliminary test for carryover and the first-period treatment comparison, the two-stage procedure leads to bias in the resultant treatment estimator and increases the probability of making a Type I error. In other words, the actual significance level of the direct treatment test is higher than the nominal one chosen for the test. The firstperiod comparison would only be unbiased were it always used or chosen for a reason independent of it. Unfortunately, this is not the case in the two-stage procedure, in which it is only used after a significant (hence large) carryover. Although attempts have been made to circumvent this problem, all solutions involve introducing information about the carryover effect that is not contained in the data. Given that a crossover trial will be powered for within-subject estimation of the treatment effect and will not have the sensitivity to provide useful information on the carryover, it has been argued that the best approach to this problem is to take all practical steps to avoid possible carryover. Examples of these steps include using washout periods of adequate length between the treatment periods and then, assuming that this effect is negligible, using the simple estimator in Equation (1) and the associated analysis. In turn, this method requires a good working knowledge of the treatment effects, which will most likely be based on prior knowledge of, and/or the mechanisms behind, these examples. Many 2 × 2 trials have additional measurements taken before the first treatment period: at the end of the so-called run-in period and at the end of the washout period. These measurements are called baseline measurements, noting that in contrast to a parallel group study such a baseline measurement (the ones from the washout period) can be collected after randomization. Although it is tempting to introduce these baselines into the analysis in a fashion analogous to a parallel-group design, for example using change from baseline as response and/or with baselines as covariates, some care is needed in their introduction. Because of the within-subject nature of the primary analysis, these baselines do not have the usual role familiar from parallelgroup studies. Between-subject variability as

4

CROSSOVER DESIGN

existing in the treatment periods is already eliminated from the treatment comparison; for the baselines to make a contribution over and above this they must explain additional variability. It will happen only when the correlation between an outcome Yijk and its associated baseline is considerably greater than between the two outcomes Yi1k and Yi2k . This outcome might happen for example if the time interval between the treatment measurements is much greater than between baseline and associated treatment measurement. If the correlations among the four measurements from a subject are all similar, then not only will no improvement in precision be obtained from using baselines in the analysis, but also their introduction is likely to actually decrease precision. For example, if all within-subject correlations are exactly equal, then the variance of the treatment estimator obtained using the change from baseline as a response is exactly double that of the estimator (1) that uses only the treatment outcomes. The baselines have much greater potential value when estimating the carryover effect because they can be used to eliminate between-subject variability from this method. In some settings, this method can lead to a considerable increase in precision. However, the difficulty associated with the two-stage procedure is not eliminated. Now, two potential types of carryover exist: the original one associated with the second treatment period, and an additional one associated with the end of the washout period. Before one uses the baselines to examine the former, the status of the latter needs to be established, which leads to an even more complicated sequential procedure with additional potential for bias (5). In addition, the gain in precision afforded by inclusion of baseline information will not bring it to the same level as that of the direct treatment comparison. Again the advice is not to attempt exploration of a possible carryover effect as a part of the primary treatment comparison. Related issues apply to the use of a genuine baseline covariate, that is, one that is measured before randomization only. Details on the incorporation of such covariates into the analysis are given in Reference 2, Section 2.11. The very simple analyses so far described rest on the assumption that

the data are continuous. The analog of the two-sample t test of the differences Dik for binary data is given by the Mainland-Gart test, which is a simple extension of McNemar’s test that incorporates adjustment for a possible period effect. The Mainland-Gart test is just that for association in a 2 × 2 contingency table that is constructed as follows. When the outcome is binary, each subject presents one of four possible responses for the two periods (Yi1k , Yi2k ): (0,0), (0,1), (1,0), and (1,1). The so-called nonpreference responses (0,0) and (1,1) are discarded, and the columns of the table are correspond to the preference responses: (0,1) and (1,0). The rows are then given by the two sequence groups:

Sequence AB BA

Outcome (0,1) (1,0) n11 n12 n21 n22

In this table, n11 is, for example, the number of subjects who responded with a 0 in period 1 and a 1 in period 2 in sequence group AB. The test for treatment effect, ignoring carryover, is then the test for association in table, for example, using Pearson’s chisquared statistic. If numbers in this table are small, as they often are in a crossover trial especially after discarding the (0,0) and (1,1) responses, then a small sample exact procedure such as the Fisher exact test can be used. Other outcomes, such as ordinal and count, are best handled using appropriate model-based methods, to which we return briefly below. Sample size calculations in such categorical settings are less straightforward however than for continuous outcomes. Choices must be made about the scale on which the treatment effect is to be measured, and an appropriate measure of within-subject dependence must be introduced. The actual calculations will be based on analytical approximations or simulation methods. Finally, we note that the simplicity of the analyses met so far rests on the completeness of the data. If subjects drop out of the trial, then some singleton measurements will be left. We may choose to leave these measurements out of the analysis and use the same simple procedures with

CROSSOVER DESIGN

the completers. In terms of precision, if the within-subject correlation is high, then these singleton measurements contribute very little to the direct treatment effect, and little precision is lost through their omission. However, if we do wish to incorporate them in the analysis, then methods are needed that can combine within- and between- subject information, and suitable tools for this are met below when general models for higher order designs are discussed. 3

HIGHER ORDER DESIGNS

It has been recognized that it is best to assume that any carryover effect is negligible when using the 2 × 2 crossover design. In most settings, the design provides insufficient information on such a carryover to allow either adjustment for it or elimination of it. Several higher order designs for two treatments have been proposed that circumvent this problem but only at the expense of strong assumptions that concern the nature of the carryover effect. These designs have extra sequences and/or extra periods. For example, extending the design to three periods means that the following three alternative designs with two sequences may be used: ABB/BAA, ABA/BAB, or AAB/BAA. In the absence of carryover, these designs are effectively equivalent for estimating the direct treatment effect. However, more importantly, they also allow the treatment effect to be estimated using within-subject information after adjustment for a particular form of simple carryover effect, which is something that is impossible with the 2 × 2 design. In these circumstances, the three designs behave rather differently in terms of precision. It turns out that the first design, ABB/BAA, is most efficient among the three, and it provides an estimate of the adjusted treatment effect of useful precision. At first sight, this design seems to provide a solution to the main problem associated with the 2 × 2 design. However, this advantage depends on strong and potentially unrealistic assumptions about the form of the carryover, namely 1) that the carryover effect is the same whether carryover is occurring into A or into B and 2) it affects the immediate following period

5

only. In general, we call carryover effects that satisfy these constraints, ‘‘simple firstorder.’’ This example demonstrates a general problem with the design of higher order crossover design: Many arrangements are possible, and some can be shown to have particularly good mathematical properties under specific modeling assumptions (especially concerning carryover). However, these assumptions are themselves usually not open to sufficiently precise assessment from the data generated by the design, so the same main issue surrounding the 2 × 2 design persists, albeit in a more-complicated and lessaccessible form. For this reason, great care is needed in interpreting the term ‘‘optimal design’’ in this setting. The higher order twotreatment designs can be extended in several ways with more sequences and periods, which allows additional effects to be estimated, such as treatment-by-period interaction, secondorder carryover, and carryover-by-treatment interaction. They are not widely used in practice however, and the main role of higher order two treatment designs is in the form of three-period designs in bioequivalence trials (Reference 2, Chapter 7). For a design with more than 2 treatments (t say) it is usual, where practical, for each subject to receive all treatments, which implies that the number of periods (p say) is equal to t if each subject receives each treatment once. In addition, because adjustment for period effects is the norm, it is important for reasons of efficiency to balance treatments within periods. A design in which this balance is exact is called a Latin square design. In such a design, every subject receives each treatment, and each treatment occurs equally often in each period. This example demonstrates four sequences, four treatments (A, B, C, and, D), and four periods:

Sequence 1 2 3 4

1 A D C B

Period 2 3 B C A B D A C D

4 D C B A

A design based on this scheme would have a multiple of four subjects in total with one

6

CROSSOVER DESIGN

quarter randomly assigned to each sequence. Provided carryover is not a concern, any such square would be equally efficient for estimation of the direct treatment effects, and similar arrangements can be constructed with any number of treatments. If adjustment is to be made for carryover, then this design is very poor. It can be observed that each treatment is preceded by the same treatment in the three sequences in which it does not occur first, which means that treatment and carryover effects are highly nonorthogonal, and adjustment causes a sever loss in precision. This issue can be largely resolved by choosing a special Latin square arrangement in which every treatment is preceded equally often by each other treatment. For example, in the fou-treatment case we might consider the following design:

Sequence 1 2 3 4

1 A B C D

Period 2 3 B C D A A D C B

4 D C B A

Latin square designs with such balance are called Williams Square designs (6). In fact, they are only square when t is even; for t odd, a Williams square design has 2t sequences. Examples of Williams squares designs for a range of values of t are given in Reference 2, Chapter 4. Even if adjustment for carryover is not to be used in the analysis, these designs have the advantage of having a degree of robustness to carryover: An unadjusted analysis will produce less biased estimates of the direct-treatment effects even when carryover is present in the data compared with the earlier Latin square design. However, as above, these remarks refer strictly to a specific form of carryover: simple first order. Many alternative forms are conceivable, and a design that is chosen with one type in mind may be poor for another. Not all higher order crossover designs are appropriate for undifferentiated sets of treatments, as are the Williams squares. Particular designs are to be used for comparisons with a control and for factorial designs. A special type is used when the treatments are

increasing doses of a drug, and for safety reasons a higher dose cannot be experienced before a lower one. Details of these and other special designs can be found in Reference 2, Chapter 4. It is a good choice where possible to have each subject experience each treatment in a crossover trial, as in a Williams square. Such designs are sometimes called complete by analogy with a complete-block design. However, it is sometimes not possible, often because of practical constraints over the length of the trial, and then so-called incomplete designs are required for which p < t. Each subject receives only a subset of the treatments. An example with t = 3 and p = 2 is as follows:

Sequence 1 2 3 4 5 6

Period 1 2 A B B A A C C A B C C B

To balance out the allocation of treatments, it is usual for such designs to have more sequences than treatments. These arrangements induce considerable nonorthogonality between subjects and treatments, and consequently, adjustment for subject effects has a detrimental effect on precision. Such designs are inefficient, which implies, among other things, that potentially important information on the treatment effects are contained in the subject sums, which is not the case with complete designs. To recover such information special methods of analysis are needed, which brings us to consideration of general analysis tools for crossover designs. 4 MODEL-BASED ANALYSES The simple two-sample analyses that were used above with complete data from the 2 × 2 design need generalization for other designs and for incomplete data. Because the data from crossover trials are examples of repeated-measures data, we can draw on the

CROSSOVER DESIGN

body of analysis tools available for this setting (7,8). Accordingly, with continuous data, linear regression models for dependent data can be used. For discrete data, the appropriate analogs of these are used. In many settings with continuous data, we can in fact use very simple, ordinary, regression models by analogy with randomized-block analyses. With crossover trials, one normally expects fairly stable responses over time, and therefore it is often sufficient to capture the withinsubject dependence through the introduction into a linear model of a simple subject effect. This effect can be treated as fixed or random. To understand the implications of choosing one or other of these, we need to consider the source of the information on treatment effects in the given crossover design. It is natural and convenient to divide this information into two independent components: the between- and within-subject information. The between-subject information is that provided by the sums (or means) of all the observations from each subject. By contrast, the withinsubject information is that contained among all differences of observations from each subject. The information available from these two sources depends on both the design and the model fitted. It is also affected by the size of the within-subject correlation: as the size increases, the variability of the betweensubject sums grows. In terms of models, in many settings, a very simple, or minimal, form will suffice, which will include, in addition to the subject effects, categorical effects for period and treatment. On occasion, it will be augmented by carryover effects and, more rarely, other terms. In a fully balanced Latin square design, with the minimal model, all information on treatments is contained withinsubjects. When some form of imbalance is introduced (for example, because of dropout, the introduction of additional model terms, or the use of an incomplete design), then some information on treatment effects will move to the between-subjects stratum. We are now in a position to consider the choice of fixed- or random-subject effects. The use of fixed-subject effects, that is, treating the subject effects as any other linear model parameter, implies that all betweensubject information is discarded. However,

7

the resulting model is an example of a standard linear regression model, and all the familiar regression procedures are available, including exact small sample F- and t-based inferences. Moreover, in such analyses, no assumption needs to be made about the distribution of the subject effects among subjects implying an additional degree of robustness in these analyses. They can be observed as crossover versions of conventional randomized block analyses in which subjects are blocks. Many advantages exist to keeping the crossover analysis within this framework, not least the simplicity. We should regard this as the approach of choice provided it is appropriate, that is when there is negligible between-subject information. In contrast, the use of random-subject effects allows the recovery of the between-subject information, but it moves the analysis framework to that of the linear mixed model. This model is less robust that the fixed-effects model because of the need to assume some distribution for the subject effects. In addition, small-sample inferences must rely on approximate methods (9), whose properties become less reliable in very small samples. Hence, this approach should be used only when strictly necessary, and nonnegligible between-subject information is available for recovery, with sufficiently large sample sizes. Considerable nonorthogonality is needed to make this worthwhile; examples of such settings are inefficient designs (e.g., with p < t) and trials with a high proportion of withdrawals. For the reasons given above, this will not be the case in the typical efficient, complete, crossover design with high withinsubject correlation. Hence, careful consideration needs to be given to the data, design, and other components of the analysis model before choosing the random-subject effects approach. In certain settings, this approach is appropriate, but these settings tend in practice to be the minority. Another point is that if baseline or covariates that are measured at the beginning of each treatment period are introduced into the analysis (within-subject covariates), then the use of random, as opposed to fixed, subject effects will lead to bias in the estimated treatment comparisons.

8

CROSSOVER DESIGN

Similar issues apply when considering analyses for discrete data, with some important distinctions. First, asymptotic arguments become the norm with the discrete data analogs of these analyses. This argument implies, among other things, that the fixed-subject model cannot be used in the same way as above, by simply estimating an effect for each subject. Instead, so-called conditional likelihood analyses are the proper analogs of these and are available only for certain models: namely logistic regression for a binary outcome (Reference 2, Section 6.2), multivariate logistic regression for a nominal outcome (Reference 2, Section 6.4), and log-linear regression for a Poisson (count) outcome (Reference 2, Section 6.5.1). No conditional likelihood analogy exists, for example, for a proportional odds model for an ordinal outcome. Random-effects models can be used however with all such classes. A second important distinction concerns the variety of classes of model available for non-normal outcomes, in contrast to the almost automatic use of the linear regression basis of models for continuous data. This distinction is largely a consequence of the nonlinearity in the link between the model effects, such as treatment and period, and the mean outcome that is necessary in such non-normal settings. The distinction between two important classes of models, the so-called marginal and subject-specific, is especially relevant in the repeated measures setting and needs to be fully appreciated when analyzing discrete crossover data. These classes are discussed in the crossover setting in Reference 2, Chapter 6 and in a more general repeated measures setting in Reference 9. A brief consideration other types of outcome, such as event time, is given in Reference 2. Section 6.5. REFERENCES 1. S. J. Senn, Cross-over Trials in Clinical Research, 2nd ed. New York: Wiley, 2002. 2. B. Jones and M. G. Kenward, Design and Analysis of Cross-over Trials, 2nd ed. London: Chapman & Hall/CRC, 2003. 3. J. E. Grizzle, The two-period change-over design and its use in clinical trials. Biometrics 1965; 21: 467–480.

4. P. Freeman, (1989) The performance of the twostage analysis of two treatment, two period crossover trials. Stat. Med. 1989; 8: 421–432. 5. M. G. Kenward and B. Jones, The analysis of data from 2x2 cross-over trials with baseline measurements. Stat. Med. 1987; 6: 911–926. 6. E. J. Williams, Experimental designs balanced for the estimation of residual effects of treatments. Aust. J. Scientif. Res. 1949; 2: 149–168. 7. G. Verbeke and G. Molenberghs, Linear Mixed Models for Longitudinal Data. New York: Springer, 2000. 8. G. Molenberghs and G. Verbeke, Models for Discrete Longitudinal Data. New York: Springer, 2005. 9. M. G. Kenward and J. H. Roger, Small sample inference for fixed effects estimators from restricted maximum likelihood. Biometrics 1997; 53: 983–997.

Table 1. Plan of 2 × 2 Trial

CROSSOVER TRIALS

The aim of medical research is to develop improved treatments or cures for diseases and medical ailments. Part of that research involves comparing the effects of alternative treatments with a view to recommending those that should be used in practice. The treatments are compared using properly controlled randomized clinical trials∗ . In such trials the treatments are given either to healthy volunteers (in the early phase of development) or to patients (in the later phases of development). We will refer to patients, volunteers, or whoever is being compared in the trial as the subjects. Two types of design are used in these trials: the parallel-group design and the crossover design. In order to explain these we will consider trials for comparing two treatments A and B. The latter might be different ingredients in an inhaler used to treat asthma attacks or two drugs used to relieve the pain of arthritis, for example. In a parallel-group trial the subjects are randomly divided into two groups of equal size. Everyone in the first group gets A, and everyone in the second group gets B. The difference between the treatments is usually estimated by the difference between the group means. In a crossover trial the subjects are also randomly divided into two groups of equal size. (In agriculture and dairy science, crossover trials often are referred to as changeover trials.) Now, however, each subject gets both treatments for an equal period of time. In the first group the subjects begin by getting A for the first period and then cross over to B for the second period. Each subject in the second group begins with B in the first period and then crosses over to A for the second period. The basic plan of this design is given in Table 1. This type of crossover trial uses two treatment sequences AB and BA and is usually referred to as the 2 × 2 trial. The main advantage the crossover trial has over the parallel-group trial is that the two treatments are compared within subjects as opposed to between subjects. That is, the

Group

Period 1

Period 2

1 2

A B

B A

2 × 2 trial provides two repeated measurements on each subject, and the difference between these is used to estimate the difference between A and B. In this way each subject ‘‘acts as his or her own control’’ and any variability between subjects is eliminated. As the variability within subjects is usually much smaller than that between subjects, a relatively precise estimate of the treatment difference is obtained. In contrast, the treatment difference in the parallel-group trial is estimated by taking differences of measurements taken on different subjects, and so is based on between-subject variability. As a consequence the crossover trial requires far fewer subjects than a parallel-group trial to achieve equivalent power to detect a particular size of treatment difference. A detailed comparison of the two types of design is given by Grieve [18]. If yij is the mean for period j in group i, then for the 2 × 2 trial the within-subjects estimator of the A − B treatment difference is D = 12 [(y11 − y12 ) − (y21 − y22 )]. Obviously, crossover trials are not suitable for treatments that effect a cure. A basic assumption is that subjects will be in the same state at the start of the second period as they were in at the start of the first period. Therefore, it is essential to ensure that the effect of the first treatment is not present at the start of the second period. One way of achieving this is to separate the two active periods by a washout period of sufficient length to ensure that the effects of the first treatment have disappeared by the start of the second period. Any effect of previous treatment allocation that affects the second period is a carryover effect. If τA and τB denote the effects of treatments A and B, respectively, and λB and λB denote the carryover effects of treatments A and B, then in the presence of unequal carryover effects the expected value of D, the within-subjects estimator of the treatment

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

CROSSOVER TRIALS

difference, is (τA − τB ) − 12 (λA − λB ), i.e., D is biased. If the carryover difference is of the same sign as the treatment difference, then D underestimates the true treatment difference. Therefore, if a significant treatment difference is detected, it is still appropriate to conclude that the treatments are different, because the trial has detected an even smaller difference between the treatments than anticipated at the planning stage. In the basic 2 × 2 trial it is not possible to estimate the difference between the carryover effects using within-subject information. This is because the estimate is based on differences of subject totals. However, for designs with more than two periods or treatment sequences it is possible to estimate the carryover difference using within-subject information. Also, in the basic 2 × 2 trial the carryover difference is completely confounded with the group difference and the treatmentby-period interaction. Both the parallel-group trial and the crossover trial are usually preceded by a runin period, when subjects are acclimatized and, perhaps, are monitored for eligibility. Response measurements are usually taken during the run-in period of the parallel-group trial and during the run-in and washout periods of the 2 × 2 trial. The run-in and washout measurements can be used to estimate the carryover difference in the 2 × 2 trial, and the run-in measurements can be used to improve the precision of the parallel-group trial. However, even when run-in measurements are used, the parallel-group trial still falls short of the crossover trial as far as precision of estimation is concerned and so needs more subjects than the crossover trial to achieve comparable power. In the general case of t treatments, a crossover trial consists of randomly dividing the subjects into s groups and assigning a different sequence of treatments to each group. The sequences are of length p, corresponding to the p periods of the trial, and some of the sequences must include at least one change of treatment. The choice of sequences depends on the number of treatments and periods and on the purposes of the trial. Examples of designs for t = 3 and t = 4 treatments are given later.

2 × 2 CROSSOVER TRIAL

In the absence of run-in and washout measurements, a standard analysis for this design follows the two-stage approach of Grizzle [21] and Hills and Armitage [24]. In the first stage, a test, based on the subject totals, is done to determine if the carryover effects are equal. If they are not significantly different (usually at the 10% level), then in the second stage the within-subjects test for a treatment difference, based on D, is done (usually at the 5% level). If the first-stage test is significant, then the test for a treatment difference uses only the data collected in the first period of the trial, i.e., is based on y11 − y21 . This analysis has been criticized, particularly because the actual significance level can be much higher than the nominal level of 5% [14,46]. However, some improvement in the performance of this two-stage procedure is possible if measurements are available from the runin and washout periods [25,28]. The best advice is to base the analysis on the assumption that carryover effects are either absent or equal for both treatments and to proceed directly to the test for a treatment difference that uses within-subject comparisons. This assumption would need to rely on prior knowledge or washout periods of adequate length. As noted above, the within-subjects estimate of the treatment difference is biased downwards if carryover effects are different, but the extent of the bias will not be great unless the carryover difference is large. The detailed views of a working group of the Biopharmaceutical Section of the American Statistical Association∗ are given in Peace [42, Chap. 3]. Their view is that the 2 × 2 design is not the design of choice if carryover effects exist. A Bayesian approach to the analysis is described in Grieve [16,19,20], and further discussion on analysis can be found in Jones and Kenward [26, §2.13] and Senn [46, Chap. 3]. Overall, if significant carryover effects are likely to occur, then the 2 × 2 design is best avoided if possible. A number of better designs for two treatments are mentioned below.

CROSSOVER TRIALS

HIGHER-ORDER DESIGNS FOR TWO TREATMENTS The disadvantages of the 2 × 2 design can be overcome if more periods or sequences are used, given certain assumptions about the behavior of the carryover effects. The twoperiod design with four sequences AA, BB, AB, and BA enables the treatment difference to be estimated within subjects even if carryover effects are present and different. However, in common with all two-period designs in general, the estimate of the treatment difference is inefficient, and it is better to use at least three periods. For three periods the recommended design has two sequences ABB and BAA, and for four periods it has four sequences AABB, BBAA, ABBA, and BAAB.

3

Table 2. Balanced Design for t = 3, p = 3 Sequence No.

Treatment Sequence

1 2 3 4 5 6

ABC ACB BAC BCA CAB CBA

Table 3. Balanced Design for t = 4, p = 4 Sequence No.

Treatment Sequence

1 2 3 4

ABCD BDAC CADB DCBA

DESIGNS FOR THREE OR MORE TREATMENTS There is a great variety of designs for three or more treatments. The choice of design will be determined by the purpose of the trial, the number of permissible periods or sequences, and other practical constraints. An important feature of these designs is that carryover differences can be estimated using withinsubject information. In a variance-balanced design the variance of the estimated difference between any two treatments, allowing for subjects, periods, and any carryover effects, is the same whichever pair of treatments is considered. Plans of such designs for t = 3 and t = 4 and t periods are given below. For properties of balanced and nearly balanced designs see CHANGEOVER DESIGNS [41]. A general introduction including a review, tables of designs, discussion of optimality, and choice of design are given in Jones and Kenward [26, Chap. 5]. More recent reviews are given in Afsarinejad [1] and Matthews [37]. Study of the optimality of crossover designs has generally concentrated on a model that has fixed effects for the subject, period, treatment, and carryover effects and independent within-subject errors that have constant variance. It is mostly assumed that if carryover effects can occur they are only of first order, i.e., last for only one period. Most results in the literature refer to universal optimality [27]; a useful review is given in Matthews [37].

Closely linked to optimality are the concepts of uniformity and balance. A design is uniform if each treatment occurs equally often in each period and each treatment is allocated equally often to each subject. A design is (combinatorially) balanced if every treatment follows every other treatment equally often. Balanced uniform designs are universally optimal for the estimation of treatment and carryover effects [22,23,6]. When p = t, balanced uniform designs are the Williams designs (Williams [50]) and can be constructed easily using the algorithm given by Sheehe and Bross [48]. Examples of these designs are given in Tables 2 and 3 for t = 3 and t = 4, respectively. If every treatment follows every other treatment, including itself, equally often, the design is strongly balanced, and strongly balanced uniform designs are universally optimal [6]. A simple way of generating a strongly balanced design is to repeat the last period of a Williams design to give a design with p = t + 1. These designs are variance-balanced and have the additional property that the treatment and carryover effects are orthogonal. For those combinations of t, s, and p for which a variance-balanced design does not exist, it may be possible to construct a partially balanced design∗ . In such a design the variances of the estimated treatment differences are not all the same but do not vary

4

CROSSOVER TRIALS

much. This makes them attractive in practice, as they usually require fewer periods or sequences than a fully balanced design. Such designs are tabulated in Jones and Kenward [26] and in Ratkowsky et al. [45]. Another potentially important group of designs is where the treatments are made up of factorial combinations of two or more ingredients. For example, the four treatments A, B, C, and D might correspond to all possible combinations of two ingredients X and Y, where each ingredient can occur either at a high or a low level. Here designs that are efficient for estimating the main effects∗ or the interaction∗ of X and Y can be constructed [13].

ANALYSIS OF CONTINUOUS DATA Crossover data are examples of repeated measurements∗ ; that is, they consist of a set of short sequences of measurements. The observations from one subject will typically be correlated, which needs to be accommodated in the analysis. Continuous crossover data are most commonly analyzed using a conventional factorial linear model and analysis of variance∗ . The model will almost invariably include terms for period and treatment effects. Other terms may be included as required, such as first- and higher-order carryover effects, treatment-by-period interaction, and treatment-by-carryover interactions, although for some designs there may be aliasing among these, and the inclusion of more than a small number of such terms can seriously reduce the efficiency of the analysis. For all terms except carryover, the definition of appropriate factor levels is straightforward. Construction of factors for the latter is not obvious, because there are observations for which these effects cannot occur, for example those in the first period. One simple solution for this is to deliberately alias part of the carryover effect with period effects. For example, for a first-order carryover factor, levels follow treatment allocation in the preceding period, except for observations in the first period, when any factor level can be used provided it is the same in all sequences. After adjustment for the period term, this factor

gives the correct sums of squares and degrees of freedom for the first-order carryover. Within-subject dependence is normally allowed for by the inclusion of fixed subject effects in the linear model. Essentially, a randomized block analysis is used with subjects as blocks. In the case of the 2 × 2 trial, this analysis reduces to a pair of t-tests, each comparing the two sequence groups (Hills and Armitage [24]). For the treatment effect, the comparison is of the within-subject differences, and for the carryover–treatment-byperiod interaction, it is of the subject totals. Baseline measurements contribute little to the efficiency of the direct treatment comparison, but may substantially increase that of the carryover. Analyses for higher-order two-treatment two-sequence designs can be expressed in terms of t-tests in a similar way. For designs in which treatment effects are not orthogonal to subjects (for example, when t > p, or generally when a carryover term is included), there exists some treatment information in the between-subject stratum; this is lost when fixed subject effects are used. It has been suggested that this between-subject (interblock) information should be recovered through the use of random subject effects. Restricted maximum likelihood∗ (REML) is an appropriate tool for this. However, small, well-designed crossover trials are not ideal for the recovery of interblock information∗ : most of the treatment information lies in the within-subject stratum, between-subject variability will typically be high, and the reliance on asymptotic estimates of precision means that standard errors of effects can be seriously underestimated. The use of random subject effects implies a simple uniform covariance structure for the sequence of observations from a subject. A more general structure can be used, for example, to allow for a serial correlation pattern, but such analyses are not widely used, and there is little evidence as yet to suggest that these models are needed for routine crossover data. Such modeling may be important, however, when there are repeated measurements within treatment periods, and the multivariate linear model provides an appropriate framework for this setting.

CROSSOVER TRIALS

Nonparametric methods of analysis for crossover data are not well developed, apart from the two-treatment two-sequence designs in which the t-tests can be replaced by their nonparametric analogues. In other designs the occurrence of different treatment sequences precludes a straightforward application of an orthodox multivariate rank test, while the use of the rank transformation followed by a conventional analysis of variance is best avoided [8]. Senn [46] develops an ad hoc test based on a combination of twosequence tests, and a general review is given in Tudor and Koch [49]. ANALYSIS OF DISCRETE DATA We consider a binary response first, coded 0/1. As with continuous data, the 2 × 2 trial forms a simple special case. Each subject then provides one of four categories of joint response: (0,0), (0,1), (1,0), and (1,1). Given ni subjects on sequence i, the data from such a trial can be summarized in a 2 × 4 contingency table∗ as in Table 4. Two tests for treatment effect (assuming no carryover effect) are based on the

Table 4. Binary Data from a 2 × 2 Trial. Joint Response Sequence

(0,0)

(0,1)

(1,0)

(1,1)

AB BA

n11 n21

n12 n22

n13 n23

n14 n24

Table 5. Contingency Table for Mainland–Gart Test Sequence

(0,1)

(1,0)

AB BA

n12 n22

n13 n23

Table 6. Contingency Table for Prescott’s Test Sequence

(0,1)

(0,0) or (1,1)

(1,0)

AB BA

n12 n22

n11 + n14 n21 + n24

n13 n23

5

entries in this table. The Mainland–Gart is the test for association in the 2 × 2 contingency table [35,15] as given in Table 5. This involves the data from only those subjects who make a preference. Prescott’s test [43] introduces the pooled nonpreference data, and it is the test for linear trend in the 2 × 3 table given as Table 6. The test for carryover–treatment×period interaction is the test for association in the table involving only the nonpreference outcomes [2] as given in Table 7. Conventional chi-square, likelihoodratio, or conditional exact tests can be used with these tables. These tests are quick and simple to use. Unfortunately, they do not generalize satisfactorily for higher-order designs and ordinal responses, and they are awkward to use when there are missing data. Recent developments in modeling dependent discrete data have made available a number of more flexible model-based approaches that are applicable to crossover data. Generalized estimating equation (GEE) methods can be used to fit marginal models to binary data from any crossover design, whether data are complete or not [52,51, 30,31] and have been extended for use with ordinal data [32,31]. A marginal, or population-averaged, model defines the outcome probabilities for any notional individual in the population under consideration for the given covariate values (treatment, period, and so on). It is marginal with respect to the other periods and, provided a crossover trial is used to draw conclusions about constant as opposed to changing treatment conditions, can be regarded as the appropriate model from which to express conclusions of most direct clinical relevance. The simpler forms of GEE (GEE1) are comparatively straightforward to use, but may provide poor estimates of precision in small trials. Extended GEE methods (GEE2) are more complicated, but give better estimates of precision. The full likelihood for a marginal model cannot be expressed in closed form for p > 2, so likelihood-based analyses require considerably more elaborate computation (e.g., [32,3]). In contrast to marginal models, subjectspecific models include subject effect(s) that

6

CROSSOVER TRIALS

determine an individual’s underlying outcome probabilities. Other effects, such as period and direct treatment, modify these subject-specific probabilities, and generally these effects will not have the same interpretation as their marginal-model analogues. Marginal representations of probabilities can be obtained from subject-specific models by taking expectations over the distribution of the subject effects, but only in special cases will the treatment–covariate structure of the subject-specific model be preserved. In analyses using subject-specific models, subject effects cannot be treated as ordinary parameters as with continuous data, because the number of these effects increases at the same rate as the number of subjects. This implies that estimates of other effects will be inconsistent, a generalization of the well-known result for matched case-control studies. Two alternative approaches can be used: conditional likelihood and random subject effects. If, for binary data∗ , a logistic regression∗ model is used, or, for categorical data∗ , a model based on generalized (or adjacentcategory) logits, then a conditional likelihood analysis can be used in which the subject effects are removed through conditioning on appropriate sufficient statistics [29]. In the binary case these statistics are the subject totals; in the categorical case, the subject joint outcomes ignoring the order. The application of this approach to binary data from the two-period two-treatment design produces the Mainland–Gart test. The conditional likelihood can be calculated directly, or the whole analysis can be formulated as a log-linear analysis for a contingency table∗ of the form of the 2 × 4 table above, with appropriate extension for other designs and for categorical outcomes. One advantage of this approach is the availability of conditional exact tests when sample sizes are very small.

Table 7. Contingency Table for Hills and Armitage’s Test Sequence

(0,0)

(1,1)

AB BA

n11 n21

n14 n24

The two main disadvantages are (1) the discarding of between-subject information in the process of conditioning, which precludes a population-averaged interpretation of the results, and (2) the use of generalized logits, which are not ideal for ordinal categorical outcomes. If the subject effects are assumed to follow some distribution, typically the normal, then the likelihood for the model can be obtained through numerical integration∗ [11,12]. In general such analyses are computationally intensive, but are usually manageable for crossover trials, for which sample sizes are commonly small. The inferences from such models are subject-specific, but populationaveraged summary statistics, for example marginal probabilities, can be produced using integration. Numerical integration can be avoided through the use of an approximate or hierarchical likelihood in place of the full marginal likelihood [4,34]. However, the consistency of such procedures is not guaranteed for all sample configurations, and the smallsample properties of the resulting analyses for crossover data have not yet been explored.

CONCLUDING REMARKS There is a large and diverse literature on the statistical aspects of crossover trials, which reflects their extensive use in medical research. There are at present three books on the subject (Jones and Kenward [26]; Senn [46]; Ratkowsky et al. [45]) and several reviews. The literature is scattered over numerous journals and conference proceedings, e.g., [5,7,10,33,36]. A particularly useful review is given in Statist. Methods Med. Res., 3, No. 4 (1994). In addition to medicine, crossover trials are used in areas such as psychology [39], agriculture [40], and dairy science. An industrial example is given by Raghavarao [44].

CROSSOVER TRIALS

REFERENCES 1. Afsarinejad, K. (1990). Repeated measurements designs—a review. Commun. Statist. Theory and Methods, 19, 3985–4028. 2. Armitage, P. and Hills, M. (1982). The twoperiod cross-over trial. Statistician, 31, 119–131. 3. Balagtas, C. C., Becker, M. P., and Lang, J. B. (1995). Marginal modelling of categorical data from crossover experiments. Appl. Statist., 44, 63–77. 4. Breslow, N. E. and Clayton, D. G. (1993). Approximate inference in generalized linear models. J. Amer. Statist. Ass., 88, 9–24. 5. Carriere, K. C. and Reinsel, G. C. (1992). Investigation of dual-balanced crossover designs for two treatments. Biometrics, 48, 1157–1164. 6. Cheng, C. -S. and Wu, C-F. (1980). Balanced repeated measurements designs. Ann. Statist., 6, 1272–1283. Correction (1983), 11, 349. 7. Chi, E. M. (1992). Analysis of cross-over trials when within-subject errors follow an AR(1) process. Biometrical J., 34, 359–365. 8. Clayton, D. and Hills, M. (1987). A two-period cross-over trial. In The Statistical Consultant in Action, D. J. Hand and B. S. Everitt, eds. Cambridge University Press. 9. Cochran, W. G., Autrey, K. M., and Cannon, C. Y. (1941). A double change-over design for dairy cattle feeding experiments. J. Dairy Sci., 24, 937–951. 10. Cornell, R. G. (1991). Non-parametric tests of dispersion for the two-period crossover design. Commun. Statist. Theory Methods, 20, 1099–1106. 11. Anon. (1985–1990). EGRET: Epidemiological, Graphics, Estimation and Testing Package. Statistics and Epidemiology Research Corp., Seattle. 12. Ezzet, F. and Whitehead, J. (1991). A random effects model for ordinal responses from a cross-over trial. Statist. Med., 10, 901–907. 13. Fletcher, D. J., Lewis, S. M., and Matthews, J. N. S. (1990). Factorial designs for crossover clinical trials. Statist. Med., 9, 1121–1129. 14. Freeman, P. R. (1989). The performance of the two-stage analysis of two-treatment, twoperiod cross-over trials. Statist. Med., 8, 1421–1432. 15. Gart, J. J. (1969). An exact test for comparing matched proportions in crossover designs. Biometrika, 56, 75–80.

7

16. Grieve, A. P. (1985). A Bayesian analysis of the two-period cross-over trial. Biometrics, 41, 979–990. Correction (1986), 42, 456. 17. Grieve, A. P. (1987). A note on the analysis of the two-period crossover design when period–treatment interaction is significant. Biometric J., 29, 771–775. 18. Grieve, A. P. (1990). Crossover vs parallel designs. In Statistics in Pharmaceutical Research, D. A. Berry, ed. Marcel Dekker, New York. 19. Grieve, A. P. (1994). Extending a Bayesian analysis of the two-period crossover to allow for baseline measurements. Statist. Med., 13, 905–929. 20. Grieve, A. P. (1994). Bayesian analyses of twotreatment crossover studies. Statist. Methods Med. Res., 4, 407–429. 21. Grizzle, J. E. (1965). The two-period changeover design and its use in clinical trials. Biometrics, 21, 467–480. 22. Hedayat, A. and Afsarinejad, K. (1978). Repeated measurements designs, I. In A Survey of Statistical Design and Linear Models, J. N. Srivastava, ed. North-Holland, Amsterdam, pp. 229–242. 23. Hedayat, A. and Afsarinejad, K. (1978). Repeated measurements designs, II. Ann. Statist., 6, 619–628. 24. Hills, M. and Armitage, P. (1979). The twoperiod cross-over clinical trial. Brit. J. Clin. Pharm., 8, 7–20. 25. Jones, B. and Lewis, J. A. (1995). The case for cross-over trials in phase III. Statist. Med., 14, 1025–1038. 26. Jones, B. and Kenward, M. G. (1989). Design and Analysis of Crossover Trials. Chapman & Hall, London. (This text takes a broad view with emphasis on crossover trials used in medical research. Both theory and practice are covered in some detail. The analysis of repeated measurements both between and within periods is considered. Methods for analyzing binary and categorical data are described as well as methods for continuous data.) 27. Kiefer, J. (1975). Construction and optimality of generalized Youden designs. In A Survey of Statistical Design and Linear Models, J. N. Srivastava, ed. North-Holland, Amsterdam, pp. 333–341. 28. Kenward, M. G. and Jones, B. (1987). The analysis of data from 2 × 2 cross-over trials with baseline measurements. Statist. Med., 6, 911–926.

8

CROSSOVER TRIALS

29. Kenward, M. G. and Jones, B. (1991). The analysis of categorical data from cross-over trials using a latent variable model. Statist. Med., 10, 1607–1619. 30. Kenward, M. G. and Jones, B. (1992). Alternative approaches to the analysis of binary and categorical repeated measurements. J. Biopharm. Statist., 2, 137–170.

45.

31. Kenward, M. G. and Jones, B. (1994). The analysis of binary and categorical data from crossover trials. Statist. Methods Med. Res. 3, 325–344. 32. Kenward, M. G., Lesaffre, E., and Molenberghs, G. (1994). An application of maximum likelihood and generalized estimating equations to the analysis of ordinal data from a longitudinal study with cases missing at random. Biometrics, 50, 945–953.

46.

33. Laserre, V. (1991). Determination of optimal designs using linear models in crossover trials. Statist. Med., 10, 909–924. 34. Lee, Y. and Nelder, J. A. (1996). Hierarchical generalized linear models. J. R. Statist. Soc. B, 58, 619–678. 35. Mainland, D. (1963). Elementary Medical Statistics, 2nd ed. Saunders, Philadelphia. 36. Matthews, J. N. S. (1990). The analysis of data from crossover designs: the efficiency of ordinary least squares. Biometrics, 46, 689–696. 37. Matthews, J. N. S. (1994). Multi-period crossover designs. Statist. Methods Med. Res., 4, 383–405. 38. Molenberghs, G. and Lesaffre, E. (1994). Marginal modelling of ordinal data using a multivariate Plackett distribution. J. Amer. Statist. Ass., 89, 633–644. 39. Namboodiri, K. N. (1972). Experimental design in which each subject is used repeatedly. Psychol. Bull., 77, 54–64. 40. Patterson, H. D. (1950). The analysis of change-over trials. J. R. Statist. Soc. B, 13, 256–271. 41. Patterson, H. D. (1982). Change over designs. In Encyclopedia of Statistical Sciences, S. Kotz, N. L. Johnson, and C. B. Read, eds. Wiley, New York, pp. 411–415. 42. Peace, K. E. (1990). Statistical Issues in Drug Research and Development. Marcel Dekker, New York. 43. Prescott, R. J. (1979). The comparison of success rates in cross-over trials in the presence of an order effect. Appl. Statist., 30, 9–15. 44. Raghavarao, D. (1989). Crossover designs in

47.

48.

49.

50.

51.

industry. In Design and Analysis of Experiments, with Applications to Engineering and Physical Sciences, S. Gosh, ed. Marcel Dekker, New York. Ratkowsky, D. A., Evans, M. A., and Alldredge, J. R. (1993). Crossover Experiments. Marcel Dekker, New York. (The contents of this text are presented from the viewpoint of someone who wishes to use the SAS statistical analysis system to analyze crossover data. Two nonstandard features are the way designs are compared and the approach suggested for the analysis of categorical data. Extensive tables of designs, which would otherwise be scattered over the literature, are included.) Senn, S. (1993). Cross-over Trials in Clinical Research. Wiley, Chichester. (This text is mainly written for biologists and physicians who want to analyze their own data. The approach is nontechnical, and explanations are given via worked examples that are medical in nature. A critical view is expressed on mathematical approaches to modeling carryover effects.) Senn, S. (1994). The AB/AB crossover: past, present and future? Statist. Methods Med. Res., 4, 303–324. Scheehe, P. R. and Bross, I. D. J. (1961). Latin squares to balance for residual and other effects. Biometrics, 17, 405–414. Tudor, G. and Koch, G. G. (1994). Review of nonparametric methods for the analysis of crossover studies. Statist. Methods Med. Res., 4, 345–381. Williams, E. J. (1949). Experimental designs balanced for the estimation of residual effects of treatments. Austral. J. Sci. Res., 2, 149–168. Zeger, S. L. and Liang, K. -Y. (1992). An overview of models for the analysis of longitudinal data. Statist. Med., 11, 1825–1839.

52. Zhao, L. P. and Prentice, R. L. (1990). Correlated binary regression using a quadratic exponential model. Biometrika, 77, 642–648. See also CHANGEOVER DESIGNS; CLINICAL TRIALS —II; and REPEATED MEASUREMENTS.

M. G. KENWARD B. JONES

DATA MINING

records will have been collected for monitoring and treating patients, but can subsequently be analyzed en masse in the search for previously unsuspected relationships and causes of disease). Of course, there is no reason—apart from the expense of collecting large data sets—why data should not be collected specifically to answer a particular question, but then the analysis is a more standard statistical one. The excitement of data mining is also partly a consequence of this secondary nature: it suggests that there is valuable information concealed within the data one already has, simply waiting for someone to tease it out. Unfortunately, the ‘‘simply’’ part of this exercise is rather misleading. Indeed, if it was simple, it would doubtless already have been done. One of the problems is that large data sets necessarily have a great deal of structure in them, but this structure has three major sources in addition to the target one of ‘‘important, real, undiscovered structure’’. These three sources are data contamination, chance occurrences of data, and structure which is already known to the database owner (or, if not explicitly articulated as known, sufficiently obvious once it has been pointed out to be of no genuine interest or value—such as the fact that married people come in pairs). The first and second of these are sufficiently important to warrant some discussion. It is probably not too much of an exaggeration to say that all data sets are contaminated or distorted in some way, though with small data sets this may be difficult to detect. With large data sets, it means that the data miner may triumphantly return an unusual pattern which is simply an artifact of data collection, recording, or other inadequacies. Brunskill (1) describes errors that occurred in the coding of birth weights: 14 oz recorded as 14 lb, birth weights of one pound (1 lb) being read as 11 lb, and misplaced decimal points for example, 510 gms recorded as 5100 gms. Note that all of these errors yield overreporting of birth weights. A data mining exercise might therefore report an unusual excess of

DAVID J. HAND Imperial College London, London, UK

Data mining is a new discipline, which has sprung up at the confluence of several other disciplines, stimulated chiefly by the growth of large databases. The basic motivating stimulus behind data mining is that these large databases contain information which is of value to the database owners, but this information is concealed within the mass of uninteresting data and has to be discovered. That is, one is seeking surprising, novel, or unexpected information and the aim is to extract this information. This means that the subject is closely allied to exploratory data analysis. However, issues arising from the sizes of the databases, as well as ideas and tools imported from other areas, mean that there is more to data mining than merely exploratory data analysis. Perhaps, the main economic stimulus to the development of data mining tools and techniques has come from the commercial world: the promise of money to be made from data processing innovations is a familiar one, and huge commercial databases are now rapidly growing in size, as well as in number. However, there is also substantial scientific and medical interest: philosophers of science have remarked that advances and innovation often occur when a mismatch between the data and the predictions of a theory occurs, and nowadays to detect such mismatches often requires extensive analysis of large data sets. Examples of areas of scientific applications of data mining include astronomy (3) and molecular biology (8). Genomics, proteomics, microarray data analysis, and bioinformatics, in general, are areas that are making extensive use of data mining tools. Apart from the sizes of the data sets, one of the distinguishing features of data mining is that the data to which it is applied are often secondary. That is, the data will typically have been collected to answer some other question, or perhaps secondarily in the course of pursuing some other issue (medical

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

DATA MINING

high birth weights (indeed, we might hope that a successful analysis would report this), but an excess that was of no real interest. Indeed, it seems that this may have happened (2,7). Digit preference is another cause of such curiosities, and is only detectable in large data sets. Wright and Bray (9) describe this occurring in measurements of nuchal translucency thickness, and Hand et al. (5) describe it in blood pressure measurements. In statistics, one is often able to cope with data inadequacies by extending the model to cope with them. Thus, for example, distorted sampling may be allowed for by including a model for the case selection process, and incomplete vectors of measurements may be handled via the EM algorithm. However, such strategies can only be adopted if one has some awareness (and understanding) of the data contamination mechanism. In data mining, with secondary data, this is often not the case. Often, such problems are ignored—with obvious potential for misleading conclusions. Statistics tackles the possibility of spurious (chance) patterns arising in the data using tools which estimate the probability of such structures arising by chance, merely as a consequence of random variation; that is, with hypothesis and significance tests. Unfortunately, with large data sets, and large sets of possible patterns being sought, the opportunity for the discovery of apparent structures is clearly great. This means that the statistical approach cannot be readily applied. Instead, data miners simply define score functions (for the ‘‘interestingness’’ or ‘‘unusualness’’ of a pattern), without any probability interpretation, and pass those patterns that show the largest such scores over to an expert for evaluation. This description reveals the process nature of data mining. Data mining is not a ‘‘one-off’’ exercise, to be done and finished with. Rather, it is an ongoing process: one examines a data set, identifies features of possible interest, discusses them with an expert, goes back to the data in the light of these discussions, and so on. The score functions may be the same as the criteria used in statistical model fitting, without the probabilistic interpretation, or there may be other criteria. An illustration of the

differences in perspective is given by regression analysis. A statistician may find the maximum likelihood estimates of the parameters, assuming a normal error distribution. In contrast, a data miner may adopt the sum of squared residuals as a score function to use in choosing the parameters. Since maximizing the likelihood based on a normal error distribution leads to the sum of squares criterion, these two approaches yield the same result—but they start from different positions. The statistician has a formal model in mind, while the data miner is simply aiming to find a good description of the data. Of course, the distinction is not a rigid one—there is overlap between the two perspectives. This example does show how central the concept of modeling is to the statistician. In contrast, data miners tend to place much more emphasis on algorithms. Given the essential role of computers in data mining, this algorithmic emphasis is perhaps not surprising. Moreover, when data sets are very large, the popular statistical algorithms may become impracticable (tools that make repeated passes through the data, for example, may be out of the question with a billion data points). Computers are, of course, also important for statistics, but many statistical techniques can be applied on small data sets without computers—many were originally developed that way. One consequence of the stress on algorithms is that it may be difficult to describe exactly what model is being fit to the data. This can have adverse consequences. For example, cluster analysis is widely used in data mining, but without careful thought about the nature of the procedure, it can be difficult to be clear about what sort of ‘‘clusters’’ are being found. Thus, compact structures may be appropriate in some situations (e.g. to produce compact summarizing descriptions, with the clusters being represented by ‘‘central’’ points), while in others, elongated shapes may be desirable, in which neighboring points in the same cluster are similar but distant ones are not. Without an awareness of the type of structure that the method reveals, inappropriate conclusions could be drawn: a species could be incorrectly partitioned on a dimension in which it has substantial variability.

DATA MINING

In the above, we have used phrases such as ‘‘model’’, ‘‘structure’’, and ‘‘pattern’’ without defining them. Hand et al. (4) define a model as a large scale summary of a set of data (i.e. as the standard statistical notion of a model), and a pattern as a small scale, local structure. A Box–Jenkins decomposition of a time series (see ARMA and ARIMA Models) is a model, whereas a conjunction of values, which occasionally repeats itself (e.g. a petit mal seizure in an EEG trace), is a pattern. Models are the staples of statistics, but patterns are something with which it has generally not been concerned (there seem to be three main exceptions: the study of scan statistics, of spatial disease clusters, and of outliers, all concerned with local anomalies). An examination of the data mining literature shows that both models and patterns are important, but narrow views of data mining sometimes fail to recognize the diversity of the tools used. Thus, for example, it is sometimes claimed that data mining is merely the application of recursive partitioning methods (e.g. tree classifiers), but this is a parody of the breadth of the field. Likewise, the viewpoint sometimes proposed in the econometric literature, that data mining is merely an elaborate and extensive form of model search, fails to recognize the various other kinds of data mining activities that go on. A large number of different kinds of tools are used in data mining— reflecting the eclecticism of its origins. Some recent ones in pattern detection, culled from the data mining literature with no particular objective other than to indicate the diversity of different kinds of methods are: tools for characterizing, identifying, and locating patterns in multivariate response data; tools for detecting and identifying patterns in two dimensional displays (such as fingerprints); identifying sudden changes over time (as in patient monitoring); and identifying logical combinations of values that differ between groups. Some examples of important tools in model building in data mining (again chosen with no particular aim other than to illustrate the range of such methods) include: recursive partitioning, cluster analysis, regression modeling, segmentation of time series into a small number of segment types, techniques for condensing huge (tens of billions of data points)

3

data sets into manageable summaries, and collaborative filtering, in which transactions are processed as they arrive, so that future transactions may be treated in a more appropriate manner. Much statistical theory is aimed at producing valid inferences from a sample to some population (real or notional) from which the sample has been drawn. This might be so that one can make comparative statements about the populations, or for forecasting, or for other reasons. These methods are also appropriate in data mining, provided one has a sample and that it has been drawn in a probabilistic way (so that one knows the probability of each object appearing in the sample). Going further than this, in many data mining applications one has available data on the entire population (for example, all chemical molecules in a particular class) and then, in model building data mining applications, analyzing a sample from the data set may be a sensible way to proceed. In contrast, however, in a pattern detection exercise, it will typically be necessary to analyze the entire data set: if one is seeking those data points that are anomalous, there is no alternative to examining every data point. It is clear that data mining will be of increasing importance as time progresses. However, the importance should not conceal the difficulties. Finding unsuspected structures in large data sets and identifying those that are due to phenomena of genuine interest and not merely arising from data contamination or due to chance is by no means a trivial exercise. Issues of theory, of data management, and of practice all arise. General descriptions of data mining include those of Fayyad et al. (4) and Hand, Mannila, and Smyth (6). REFERENCES 1. Brunskill, A. J. (1990). Some sources of error in the coding of birth weight, American Journal of Public Health 80, 72–73. 2. David, R. J. (1980). The quality and completeness of birthweight and gestational age data in computerized birth files, American Journal of Public Health 70, 964–973. 3. Fayyad, U. M., Djorgovski, S. G. & Weir, N. (1996). Automating the analysis and cataloging

4

DATA MINING

of sky surveys, in Advances in Knowledge Discovery and Data Mining, U. M. Fayyad, G. Piatetsky-Shapiro, P. Smyth & R. Uthurusamy, eds. AAAI Press, Menlo Park, pp. 471–493. 4. Fayyad, U. M., Piatetsky-Shapiro, G., Smyth, P. & Uthurusamy, R., eds. (1996). Advances in Knowledge Discovery and Data Mining. AAAI Press, Menlo Park. 5. Hand, D. J., Blunt, G., Kelly, M. G. & Adams, N. M. (2000). Data mining for fun and profit, Statistical Science 15, 111–131.

6. Hand, D. J., Mannila, H. & Smyth, P. (2000). Principles of Data Mining. MIT Press. 7. Neligan, G. (1965). A community study of the relationship between birth weight and gestational age, in Gestational Age, Size and Maturity, Vol. 19. Clinics in Developmental Medicine Spastics Society Medical Education Unit, pp. 28–32. 8. Su, S., Cook, D. J. & Holder, L. B. (1999). Knowledge discovery in molecular biology: identifying structural regularities in proteins, Intelligent Data Analysis 6, 413–436. 9. Wright, D. E. & Bray, I. (2003). A mixture model for rounded data, The Statistician 52, 3–13.

DATA MONITORING COMMITTEE

following the revelation of horrific experiments performed on inmates of the Nazi concentration camps during World War II. Additional motivation for effort in this area came from the work of Henry Beecher, whose 1966 publication on ethical lapses in clinical research received wide attention (4). This document was followed by the Declaration of Helsinki (5), first issued in 1964 with numerous subsequent revisions, and, in the United States, the report of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research (the ‘‘Belmont Report’’), issued in 1979 (6). The modern era of attention to protection of human subjects, beginning in the mid-twentieth century, coincided with the emergence of the randomized clinical trial as the preferred method of evaluating effects of medical treatments. The concept of selecting treatments for individuals essentially by the flip of a coin prompted its own ethical concerns, particularly in areas of serious and difficult-to-treat diseases such as cancer, but many issues developed in regard to the proper conduct of trials. One key issue, the approaches to which are still evolving, is the need to monitor the emerging results of a clinical trial to ensure that participation remains safe and appropriate. In the earliest days of the randomized clinical trial, the monitoring of emerging data was performed routinely by the study sponsors (pharmaceutical companies or government funding agencies). The investigators themselves, or at least the investigators who were part of the study leadership, were also often involved, especially for studies funded by the National Institutes of Health (NIH). But an awareness was developing that when difficult judgments had to be made about emerging results, involvement in the study could threaten the objectivity of these judgments. In 1967, a major NIH-commissioned assessment of then-current clinical trials methodology was reported by a committee headed by Dr. Bernard Greenberg of the University of North Carolina (7). One item recommended in this report was the establishment of a policy advisory board that would include individuals with relevant expertise

SUSAN ELLENBERG University of Pennsylvania Philadelphia, Pennsylvania

Data Monitoring Committees (DMCs) are oversight groups for clinical trials. All investigations of medical interventions require ongoing monitoring. Monitoring is required first and foremost to ensure that any emerging safety issues are identified and dealt with as rapidly as possible. Monitoring is also important to ensure that the study is being implemented appropriately, that patients are being entered as anticipated, treatment is being administered according to the study protocol, the dropout and ineligibility rates are not excessive, and so forth. In most trials, monitoring functions are performed by the sponsors of the study together with the individuals who are carrying out the study. In some trials, however, particularly those trials in which treatment strategies are compared to determine impact on survival and other serious outcomes, DMCs [also often called data and safety monitoring boards (DSMBs)] traditionally have been established to review the accumulating data on a regular basis and make recommendations to the study sponsor regarding continuation, termination, or modification. Over the last decade, the use of DMCs has increased, both in governmentsponsored and industry-sponsored trials. Two complementary books are available that provide information on DMC operations; one focuses on principles and practices (1), and one is a series of case studies that describe how DMCs operated and the issues they faced in specific trials (2). 1 EVOLUTION OF DMCS AS A COMPONENT OF CLINICAL TRIALS Research with human subjects has a very long history, but the widespread consensus on the need for mechanisms to protect research subjects is relatively recent. The Nuremberg Code (3), issued in 1949, was a simple statement of principles that emerged

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

DATA MONITORING COMMITTEE

not otherwise involved in the study to provide advice about important decisions affecting the conduct of the study. The entity now known as a data monitoring committee, as a data and safety monitoring board, or as any of several other similar appellations, evolved from this recommendation. In the remainder of this article, I will use the term ‘‘data monitoring committee (DMC),’’ as an umbrella term that includes all variants. By the late 1970s, the use of DMCs was fairly standard in major randomized trials sponsored by many institutes of the NIH as well as the Veterans Administration and other federal research agencies. Around this time, statistical methods were developed to accommodate the need to conduct regular interim analyses throughout the duration of a study without inflating the Type 1 error rate by providing multiple opportunities to declare a positive result 8–11. These approaches rapidly gained widespread use in the large multicenter trials sponsored by NIH. The tensions and urgencies surrounding the early clinical trials of treatments for HIV/AIDS, beginning in the late 1980s, led to increased attention to operational aspects of DMCs (12–14). By that time, emerging recognition of problems in cancer trials likely attributable to the widely accessible interim results (15) led to the establishment of DMCs for the NCI-funded cancer cooperative groups. By the mid-1990s, use of DMCs for randomized studies funded by federal agencies had become standard, and in 1998 the NIH issued its first formal policy on the use of such committees in its extramural research programs (16). Trials sponsored by industry have also increasingly engaged such committees to oversee many trials, particularly Phase 3 trials intended to serve as the primary basis for marketing approval of drugs to treat serious illness. DMCs are not required by the Food and Drug Administration (FDA) except for the very limited set of trials done in emergency research settings for which exception from the informed consent requirement has been approved (17). Nevertheless, the value of simultaneously ensuring optimal oversight of patient safety (by having outside experts review the interim data) and ensuring and

maintaining trial integrity (by keeping individuals with a financial interest in the trial outcome blinded to the interim data) was recognized by both FDA and industry scientists, and the use of DMCs for industry trials has increased substantially since the early 1990s. In 2006, FDA issued a detailed guidance document on the use of DMCs in trials subject to FDA regulation (18). Today, DMCs are widely used in clinical trials sponsored both by industry and by government agencies and are often used even in early phase, uncontrolled studies when the treatment is novel and risks may be unusually high. 2 ROLES AND RESPONSIBILITIES OF DMCS The primary responsibility of a DMC is to protect study subjects from risks that exceed the level anticipated when the study was initiated and described to subjects during the informed consent process. To fulfill this responsibility, a DMC will receive reports from the study statistician at regular intervals, will convene to discuss the interim data, and will make recommendations to the study sponsor regarding any actions that may be needed. Such recommendations might range from informing subjects about a new potential side effect that has been observed to changing dose or schedule or to terminating the study early. A recommendation for early termination could result from a variety of patterns of emerging data, including a level of benefit that is already definitive, an observed magnitude of risk that is unlikely to be outweighed by benefits, or a consistent similarity in event rates such that demonstrating an advantage of one or the other treatment is very unlikely even if the trial were continued to its planned completion target (futility). Of course, a DMC often notes that no changes are warranted and recommends that the trial continue as designed. Another critical responsibility of a DMC is protecting the integrity of the study. A DMC that is the only entity with access to interim data provides assurance that the study design and/or conduct will not change as a result of emerging results in a way that would bias the results or cause the study

DATA MONITORING COMMITTEE

to fail to provide useful information. For example, if investigators are aware of the interim data and these data begin to suggest an advantage for one treatment, then they may decide, long before the data are definitive, to stop entering subjects into the study and thereby prevent the study from producing conclusive results. Similarly, a sponsor aware of interim data might suggest a change in the primary endpoint midway through the study if the data on that endpoint are more suggestive of a benefit than the data on the originally specified primary endpoint. If neither sponsors nor investigators have access to interim data, changes in design and conduct that are intended to promote a more efficient and/or informative study can be initiated without concern that these changes were motivated by interim results and therefore could cause problems in interpreting the final statistical analyses. A DMC will also generally monitor the quality of the study as evidenced by dropout rate, timeliness of information, ineligibility rate, adherence to the protocol, and so forth, and make recommendations regarding issues that could threaten the study’s informativeness. Although quality issues should be monitored closely by the sponsor and lead investigator(s), it is often helpful to have an outside perspective on these matters that can give the trial leadership an extra push in regard to lagging recruitment, delayed data submissions, and other operational issues. 3

DMC MEMBERSHIP

DMC members must have the same type and magnitude of expertise as the trial leadership so the sponsor and investigators are comfortable that the DMC can fulfill its mission of protecting study subjects and make good judgments about stopping, continuing, or modifying the trial. In most cases, this involves clinical expertise in the area(s) relevant to the trial and statistical expertise in the methods of clinical trial design, analysis, and interim monitoring. For example, a study of a new drug to treat rheumatoid arthritis would include rheumatologists and at least one statistician, all of whom would be expected to be familiar with clinical trials methodology. In studies with multiple

3

components or with complex treatment regimens, multiple areas of clinical expertise may be required. For example, the DMC for the Women’s Health Initiative required clinicians with expertise in women’s health, including specialists in cardiology, cancer, gynecology, and bone metabolism (19). For trials with complex designs, it is often advisable to include more than a single statistician, as multiple perspectives may be found on statistical methods and interpretation of data, just as multiple viewpoints may develop on optimal treatment in a specific situation. In addition, nonphysician scientists, such as immunologists, pharmacologists, engineers (for medical device trials), or epidemiologists may be important DMC members for some trials. Trials that raise particularly difficult ethical issues and trials that have the potential to have a major impact on public health often include a bioethicist on their DMCs. The role of a bioethicist is more than that of a subject advocate; all DMC members generally consider themselves to be serving that role. The bioethicist can be especially helpful in framing the difficult decisions a DMC may have to make and may facilitate the development of a consensus view. Trials in some areas routinely include a community representative on a DMC for trials that affect a particular community. For example, clinical trials conducted by the NCIfunded cooperative oncology groups must include a lay representative on their DMCs (20). Such individuals may contribute an extremely valuable perspective, particularly when difficult decisions must be made. It is important to be certain, however, that maintaining confidentiality of interim data will not be a problem for such a member, who may have many connections with individuals whose choice of treatment may be influenced by the results of the trial. Other issues besides type of expertise should be considered in constituting a DMC. Evaluation of potential conflicts of interest is extremely important. Some conflicts are obvious, such as being in a position to benefit financially from a particular trial outcome, either through place of employment (e.g., the sponsoring company) or investment holdings.

4

DATA MONITORING COMMITTEE

Other conflicts, especially nonfinancial conflicts, may be trickier. Should someone serve on a DMC if her spouse is an investigator in the trial or works for the sponsor? What about her brother or a close friend? Should someone on a speaker’s bureau for the pharmaceutical sponsor be on the DMC? What about someone who was the lead investigator on a Phase 1 study that had promising results and led to more investigations? (If the product ultimately is deemed successful, then that Phase 1 study will be referenced far more frequently than if the product ultimately is found ineffective or harmful.) Other considerations for DMC member selection include representation of participating countries or regions in international trials, representation of population subsets who may be the focus of the trial (it would seem strange, for example, to have an allfemale DMC overseeing a trial that evaluates a treatment for testicular cancer or to have an all-white DMC overseeing a trial in the area of sickle-cell anemia), and prior DMC experience. Although it is neither feasible nor sensible to require that all DMC members have such experience, it is important that at least some members do. Finally, DMC members should be prepared to give DMC meetings their highest priority. DMCs cannot work effectively if its members are casual about their attendance and participation. DMCs range in size. A general principle is that the DMC should be as small as possible while including all the required areas of expertise. The primary advantage of a small committee is the ease of arranging meetings; this advantage can be particularly important when an unscheduled emergency meeting is necessary. (Of course, from a sponsor’s perspective, the costs are lower for a small committee!) Many DMCs, even for major trials, include only three members, typically two clinicians and one statistician. Trials that require members with a variety of types of expertise may require more members. Also, DMCs that monitor multiple trials, such as DMCs for cooperative group or network studies, typically require more members because multiple studies increase the range of expertise required. The choice of DMC chair is a critical one. A DMC chair definitely should have had

prior DMC experience. Successful chairs can have any of the types of scientific expertise required for the study the DMC is monitoring; the key characteristic required is the ability to manage discussion and to keep it constructive and on a path toward consensus. A DMC chair is often selected because of his/her eminence in the professional community, but such individuals are often extremely busy, with many commitments. It is essential for the trial sponsor to have an understanding with the DMC chair that DMC meetings must be given the highest priority. 4 OPERATIONAL ISSUES 4.1 Charter The operational principles for a DMC should be laid out in a charter that is provided and agreed to by all DMC members before the initiation of review. The charter will address issues such as expected meeting schedule, meeting format, statistical approach to interim monitoring, determination of conflicts of interest, handling of meeting minutes, types and format of reports to the committee, and other aspects of DMC processes. Sample charters that can serve as templates for most DMCs have been published (1,21). 4.2 Meeting Format DMC meetings are often arranged in segments. One segment, usually called the ‘‘open session,’’ permits the study sponsor and/or lead investigator(s) to discuss the status of the study with the DMC and bring to the committee’s attention any issues that may be relevant to its deliberations. For example, a sponsor may be conducting multiple studies of a product in a variety of settings and may wish to inform the DMC of findings in these other studies. Another segment, a ‘‘closed session,’’ is typically limited to the DMC and the statistician who has prepared the interim analyses. In the closed session, the interim results are presented to the DMC, with opportunity for discussion. The DMC may also request an ‘‘executive session’’ in which only DMC members would participate. In many studies sponsored by federal agencies, it is the practice for representatives of the sponsoring agency to attend closed

DATA MONITORING COMMITTEE

sessions; in some agencies, executive sessions in which such representatives would be excluded are not permitted. In trials sponsored by industry, it is rare for sponsors to attend closed sessions or have access to the interim data. 4.3 Meeting Schedule It is optimal for DMCs to have their initial meeting before the initiation of the study. It is important for all DMC members to have a thorough understanding of the study design and goals and of the plan for monitoring the interim data. In some cases, DMCs may have useful suggestions for modifications, particularly with respect to the monitoring plan, that the sponsor may wish to consider. In other cases (which should occur only very rarely), a DMC member may realize that he/she is not comfortable with the study as designed; in such circumstances, if the design cannot be changed, this member may resign from the committee and be replaced before any review of interim data. Meeting schedule/frequency once the study is underway depends on the rapidity with which information becomes available. Interim analyses may be scheduled based on chronologic time (e.g., every year for the duration of the study) or, more typically, on the proportion of information on the primary outcome that has been achieved. The study may plan for only a few interim analyses with the potential of early termination for higherthan-expected benefit or clear futility, but it may be necessary for the DMC to meet at times other than when these formal interim analyses are scheduled to allow for regular review of safety data and of study operational issues. Certainly, any trial that develops particular safety concerns would likely arrange for frequent DMC review of the accumulating safety data over and above the scheduled times for formal interim analysis. Most DMCs meet at least on a semi-annual basis to make sure that the safety oversight and review of study quality can occur frequently enough so that problems can be identified and rectified quickly. It may happen on occasion that the data reviewed at a regularly scheduled interim analysis are on the margin of decision point.

5

That is, data may suggest that termination or modification may be appropriate, but the DMC may not yet be certain that such a recommendation is warranted. In such situations, a DMC may be reluctant to wait until the next regularly scheduled analysis time for another opportunity to recommend changes in the study, and an extra meeting therefore may be scheduled. Statistical monitoring approaches have been developed that account for such ‘‘extra looks’’ at the accumulating data while still controlling Type 1 error (11,22). 4.4 Other Operational Issues The process for coming to a decision is optimally one of developing consensus; every attempt should be made to come to a mutually satisfactory agreement on recommendations without resorting to formal votes. Many DMC meetings are convened by teleconference. This method may be adequate for routine meetings to review safety data but without formal interim analyses presented; for meetings at which decisions may be made about continuing or terminating a study on the basis of a scheduled interim analysis, or when concern about emerging safety issues is to be discussed, it is preferable to meet in person. The initial meeting of a DMC should be held in person if at all possible so that board members have the opportunity to get to know each other to facilitate interaction at later meetings that may need to be held by teleconference. Unscheduled meetings of the DMC, particularly ‘‘emergency’’ meetings held to discuss new reports of serious and unexpected safety issues, may have to be held by teleconference because of the practical limitations in traveling committee members to a central site on short notice. Some debate exists over the way the interim comparative data should be presented to a DMC. Some individuals prefer to keep the data coded so that DMC members see the results by arm but do not know which arm is which. This practice, however, would require separate coding of safety and efficacy data in any trial in which one treatment has a known excess of a certain side effect; if multiple side effects are more common on one arm than another, it might even be necessary to

6

DATA MONITORING COMMITTEE

use different codes for each adverse event reported. Any attempt to maintain blinding of DMC members, whether one or multiple codes are employed, will present major obstacles to a DMC trying to weigh risks and benefits effectively. Therefore, many experienced DMC members have advocated that DMCs have access to unblinded data (23,24). Motivated by the increased use of DMCs and the recognition that much variability has developed in how they operate, the Health Technology Assessment Programme of the United Kingdom’s National Health Service commissioned a detailed study of the ways in which accumulating data are monitored and decisions are made about possible changes in ongoing trials. The report of this project, entitled DAMOCLES (Data Monitoring Committees: Lessons, Ethics, Statistics) was issued in 2005 (26); several papers reporting specific aspects of the project have been published (27–30). 5 INDEPENDENCE OF THE MONITORING PROCESS Although much consensus has developed regarding practices of clinical trial DMCs, differing perspectives are found on a variety of DMC operational aspects. One major area of debate relates to issues of independence, not just of DMC members but also of the entire process. As noted earlier, the theme of independence is an important one in DMC establishment and operation. The individuals who review the interim data and make judgments about whether the trial should continue as designed, whether changes should be made, or whether the trial should terminate early, should not have any major financial or other type of stake in the trial outcome, as this could render any recommendations suspect. Additionally, individuals with a financial or other major stake in the outcome (primarily, the commercial sponsor) are generally thought to best avoid having any access to the interim data, as such access will render suspect any change in design or analytical plan the sponsor might wish to make for the most valid reasons. So, where are the controversies?

5.1 Government versus Commercial Sponsors Representatives of federally sponsored research programs often play what are essentially ex officio roles on DMCs of trials sponsored by their agencies. Such individuals do not have any financial stake in the outcome; also, they have a responsibility to ensure the protection of study subjects and the appropriate expenditures of federal dollars. For these reasons, it is often argued that representatives of government agencies do have a role to play in evaluating the interim data from ongoing trials funded by their agencies. Another perspective is that, although such individuals have no financial stake in the outcome, they may have a major professional stake in the outcome if they proposed the study and/or were responsible for preliminary data that suggested that the study be undertaken. In addition, and just as for commercial sponsors, if new information becomes available from external sources that suggests that the study design should be altered in some way during its course, the sponsor representative may know the impact of such a change on the ultimate study conclusions and therefore no longer can consider the argument for making the change in a completely objective manner. 5.2 Statisticians Performing the Interim Analyses Other than DMC members, clearly one other person must be aware of the interim comparisons, and that is the statistician (or statistical group) who performs the interim analyses, develops the report, and presents the data to the DMC. What are the independence issues for the reporting statistician? If the statistician works for the commercial sponsor of the study, then one might worry about overt or subtle pressure on the statistician during the course of the study to indirectly reveal data trends to others in the company. Another concern is that if the statistician doing the interim analysis is the same statistician who works with the trial leadership and sponsor on design and conduct of the trial, that statistician may be put in a difficult position if a change in design partway through the trial is proposed by the sponsor and/or study leadership and the statistician is aware of whether

DATA MONITORING COMMITTEE

such a change will have an impact on the ultimate trial conclusions. In this situation, the statistician would not be able provide objective advice, uninfluenced by the interim data, as it is impossible to be uninfluenced by information of which one is aware. For this reason, it has been recommended that the statistician who performs the interim analyses and reports to the DMC be different from the trial’s primary statistician, who was involved in the design, will advise on conduct issues, and will be the primary statistical author on the trial report (31–34). A strong counterargument has been made, to the effect that such an arrangement could lead to a suboptimal monitoring process if the statistician doing the interim analysis is less knowledgeable about the study and therefore less able to engage fully with the DMC when concerns develop (35,36). REFERENCES 1. S. Ellenberg, T. R. Fleming, and D. L. DeMets, Data Monitoring Committees in Clinical Trials: A Practical Perspective. Chichester, U. K.: John Wiley & Sons, 2002. 2. D. L. DeMets, C. D. Furberg, and L. M. Friedman, Data Monitoring in Clinical Trials: A Case Studies Approach. New York: Springer, 2006. 3. The Nuremberg Code. In: R. J. Levine (ed.), Ethics and Regulation of Clinical Research, 2nd ed. Baltimore, MD: Urban and Schwarzenberg, 1986, pp. 425–426. 4. H. K. Beecher, Ethics and clinical research. N. Engl. J. Med. 1966;274:1354–1360. 5. World Medical Association, World Medical Association Declaration of Helsinki. Ethical principles for medical research involving human subjects. September 10, 2004. Available: http://www.wma.net/e/policy/b3.htm. Accessed November 29, 2007. 6. The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, The Belmont Report. Ethical principles and guidelines for the protection of human subjects of research. April 18, 1979. National Institutes of Health Office of Human Subjects Research. Available: http: //www.nihtraining.com/ohsrsite/guidelines/ belmont.html. Accessed November 29, 2007. 7. Organization, review, and administration of cooperative studies (Greenberg Report): A

7

report from the Heart Special Project Committee to the National Advisory Heart Council. May 1967. Controlled Clinical Trials 1988;9:137–148. 8. J. L. Haybittle, Repeated assessment of results in clinical trials of cancer treatments. British J. Radiology. 1971;44:793–793. 9. S. J. Pocock, Group sequential methods in the design and analysis of clinical trials. Biometrika. 1977;64:191–199. 10. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics. 1979;35:549–556. 11. K. K. G. Lan and D. L. DeMets, Discrete sequential boundaries for clinical trials. Biometrika. 1983;70:659–663. 12. S. S. Ellenberg, M. W. Myers, W. C. Blackwelder, and D. F. Hoth. The use of external monitoring committees in clinical trials of the National Institute of Allergy and Infectious Diseases. Stat. Med. 1993;12:461–467. 13. S. S. Ellenberg, N. Geller, R. Simon, and S. Yusuf, (eds.), Practical issues in data monitoring of randomized clinical trials (workshop proceedings). Stat. Med. 1993;12:415–616. 14. D. L. DeMets, T. R. Fleming, R. J. Whitley, J. F. Childress, S. S. Ellenberg, M. Foulkes, K. H. Mayer, J. R. O’Fallon, R. B. Pollard, J. Rahal, M. Sande, S. Straus, L. Walters, and P. Whitley-Williams, The data monitoring board and acquired immune deficiency syndrome (AIDS) clinical trials. Controlled Clinical Trials. 1995;16:408–421. 15. S. J. Green, T. R. Fleming, and J. R. O’Fallon, Policies for study monitoring and interim reporting of results. J. Clinical Oncology. 1987;5:1477–1484. 16. National Institutes of Health, NIH policy for data and safety monitoring. June 10, 1998. Available: grants.nih.gov/grants/guide/noticefiles/not98-084.html.. Accessed November 29, 2007. 17. Code of Federal Regulations, Title 21, Part 50.24. 18. U.S. Food and Drug Administration, Guidance for clinical trial sponsors, Establishment and operation of clinical trial data monitoring committees. March 27, 2006.. Available: http://www.fda.gov/cber/gdlns/clindatmon .pdf. Accessed November 29, 2007. 19. The Women’s Health Initiative Study Group, Design of the Women’s Health Initiative clinical trial and observational study. Controlled Clinical Trials. 1998;19:61–109.

8

DATA MONITORING COMMITTEE

20. Policy of the National Cancer Institute for data and safety monitoring of clinical trials. Available: http://deainfo.nci.nih.gov/ grantspolicies/datasafety.htm. 21. DAMOCLES Study Group, NHS Health Technology Assessment Programme. A proposed charter for clinical trial data monitoring committees: helping them to do their job well. Lancet. 2005;365:711–722. 22. K. K. G. Lan and D. L. DeMets, Changing frequency of interim analyses in sequential monitoring. Biometrics. 1989;45:1017– 1020. 23. C. L. Meinert, Masked monitoring in clinical trials–blind stupidity? N. Engl. J. Med. 1998;338:1381–1382. 24. T. R. Fleming, S. Ellenberg, and D. L. DeMets, Monitoring clinical trials: issues and controversies regarding confidentiality. Stat. Med. 2002;21:2843–2851. 25. A. M. Grant, D. G. Altman, A. B. Babiker, M. K. Campbell, F. J. Clemens, J. H. Darbyshire, D. R. Elbourne, S. K. McLeer, M. K. Parmar, S. J. Pocock, D. J. Spiegelhalter, M. R. Sydes, A. E. Walker, and S. A. Wallace, DAMOCLES study group, Issues in data monitoring and interim analysis of trials. Health Technol. Assess. 2005;9:1–238, iii–iv. 26. M. R. Sydes, D. J. Spiegelhalter, D. G. Altman, A. B. Babiker, and M. K. B. Parmar, DAMOCLES Study Group, Systematic qualitative review of the literature on data monitoring committees for randomized controlled trials. Clinical Trials. 2004;1:60–79. 27. M. R. Sydes, D. G. Altman, A. B. Babiker, M. K. B. Parmar, and D. Spielgelhalter, DAMOCLES Group, Reported use of data monitoring committees in the main published reports of randomized controlled trials: a cross-sectional study. Clinical Trials. 2004;1:48–59. 28. F. Clemens, D. Elbourne, J. Darbyshire, and S. Pocock, DAMOCLES Group, Data monitoring in randomized controlled trials: surveys of recent practice and policies. Clinical Trials. 2005;2:22–33. 29. A. E. Walker and S. K. McLeer, DAMOCLES Group, Small group processes relevant to data monitoring committees in controlled clinical trials: an overview of reviews. Clinical Trials. 2004;1:282–296. 30. S. S. Ellenberg, and S. L. George, Should statisticians reporting to data monitoring committees be independent of the trial sponsor and leadership? Stat. Med. 2004;23:1503–1505.

31. J. P. Siegel, R. T. O’Neill, R. Temple, G. Campbell, and M. A. Foulkes, Independence of the statistician who analyses unblinded data. Stat. Med. 2004;23:1527–1529. 32. M. Packer, J. Wittes, and D. Stump, Terms of reference for data and safety monitoring committees. Am. Heart J. 2001;141:542–547. 33. D. L. DeMets and T. R. Fleming, The independent statistician for data monitoring committees. Stat. Med. 2004;23:1513–1517. 34. S. J. Pocock, A major trial needs three statisticians: why, how and who? Stat. Med. 2004;23:1535–1539. 35. S. Snapinn, T. Cook, D. Shapiro, and D. Snavely, The role of the unblinded sponsor statistician. Stat. Med. 2004;23:1531–1533. 36. J. Bryant, What is the appropriate role of the trial statistician in preparing and presenting interim findings to an independent Data Monitoring Committee in the U.S. Cancer Cooperative Group setting? Stat. Med. 2004;23:1507–1511.

FURTHER READING S. Ellenberg, T. R. Fleming, and D. L. DeMets, Data Monitoring Committees in Clinical Trials: A Practical Perspective. Chichester, U. K.: John Wiley & Sons, 2002. D. L. DeMets, C. D. Furberg, and L. M. Friedman, Data Monitoring in Clinical Trials: A Case Studies Approach. New York: Springer, 2006. A. M. Grant, D. G. Altman, A. B. Babiker, M. K. Campbell, F. J. Clemens, J. H. Darbyshire, D. R. Elbourne, S. K. McLeer, M. K. B. Parmar, S. J. Pocock, D. J. Spiegelhalter, M. R. Sydes, A. E. Walker, and S. A. Wallace, the DAMOCLES study group, Issues in data monitoring and interim analysis of trials. Health Technol. Assess. 2005;9(7): 1–238.

CROSS-REFERENCES Alpha-spending function Benefit-Risk Assessment Futility Analysis Group Sequential Designs Interim Analysis Stopping Boundaries Trial Monitoring Lan-DeMets Alpha-Spending Function Early Termination–Study

DATA SAFETY AND MONITORING BOARD (DSMB) A Data Safety and Monitoring Board (DSMB) is an independent committee composed of community representatives and clinical research experts that reviews data while a clinical trial is in progress to ensure that participants are not exposed to undue risk. A DSMB may recommend that a trial be stopped if there are safety concerns or if the trial’s objectives have been achieved.

This article was modified from the website of the National Institutes of Health (http://clinicaltrials. gov/ct/info/glossary) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

DATA STANDARDS

1

THE STANDARDS ENVIRONMENT

CDISC is now a nonprofit standards development organization that has been working collaboratively with numerous groups for the past decade to fulfill its initial mission to establish global standards for the acquisition, exchange, submission, and archiving of clinical research data and metadata. Its current mission is to develop and support global, platform-independent data standards that enable information system interoperability to improve medical research and related areas of healthcare. In 2001, CDISC established a charter agreement with Health Level Seven (HL7), which is a healthcare standards development organization (2); a commitment was made between the two organizations to harmonize their standards to facilitate the flow of information between healthcare (electronic health records) and clinical research. This harmonization is being done through the development of a Biomedical Research Integrated Domain Group (BRIDG) Model, which represents the clinical research domain in the context of the HL7 Reference Information Model (RIM). In addition, CDISC was recently approved as a Liaison A organization to the International Standards Organization (ISO) Technical Committee 215 for healthcare standards. Unfortunately, the standards environment is far more complex than three organizations, because many additional standards organizations are developing standards for various purposes from transferring digital images (Dicom) to purchase order information for consumers (X12). Other types of standards are not for data interchange but for such purposes as the quality of care or the content of clinical reports (ICH). Fortunately, CDISC has a ‘‘niche’’ in the clinical research arena and has made a concerted effort not to duplicate efforts of others, rather to ensure productive collaboration.

EDWARD HELTON, SAS REBELLA KUSH, CDISC

In 1997, the use of technology for clinical research outside of the statistical programming area was really in its infancy. Yet, a handful of individuals were well ahead of the times. One individual who came into clinical research from a major chemical company had worked with all sites in a large site network to develop standard electronic source documents and was seeking a means to transfer the source data electronically into electronic case report forms and then to sponsors in a standard format. Another person who came from the defense industry had developed a site-management system that had been rolled out to hundreds of sites such that all of these sites were collecting important clinical research information in a standard format, particularly for subject recruitment and management of studies. A third individual was among the first to implement (not pilot) electronic data capture within the clinical research industry. These individuals recognized a true value for standards to facilitate the exchange of information among research stakeholders. Unfortunately, they could not find sponsors ready to accept electronic data in a standard manner, and no industry interchange standards were in use. Hence, a challenge was put forth in July 1997 to the industry to help develop such standards. Despite the response that biopharmaceutical companies were struggling to set standards within their own companies, much less industry standards, there was sufficient interest to form a volunteer group to initiate the development of industry-wide data interchange standards. The Clinical Data Interchange Standards Consortium (CDISC) was initiated at an inaugural meeting in September of that year (1). At that point, CDISC entered the realm of Standards Development Organizations in a complex global standards environment.

1.1 CDISC Collaborations and Standards Development Although CDISC has been working closely with HL7 since 2001 as a partnering standards development organization, additional

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

DATA STANDARDS

alliances and collaborative working relationships have contributed to the progress of CDISC over the past decade of its existence. It is impossible to list all the organizations with which CDISC has worked; however, the regulatory authorities in the U.S. [Food and Drug Administration (FDA)] and Europe [European Medicines Agency (EMEA)] and Ministry of Health Labor and Welfare have been very meaningful and important relationships for CDISC. In addition, CDISC works with groups such as the pharmaceutical manufacturers associations in the United States, Europe, and Japan (PhRMA, EFPIA, and JPMA); national health institutions and academia; other nonprofit organizations such as the Critical Path Institute and the Pharmaceutical Safety Institute; and other organizations in the healthcare and research arenas. The CDISC teams are multidisciplinary and the members are volunteers from many different companies, which include biopharmaceutical organizations, device companies, academia, technology, and service providers. The volunteers and alliance partners are extremely important to developing standards through an open, consensus-based process (3). 1.2 Benefits of the CDISC Standards A robust business case study by Gartner (4), with support from PhRMA and CDISC, found that the CDISC standards, when implemented from the beginning of a clinical study, can significantly improve processes, thus saving time and cost (∼60% of the nonsubject participation time, with a 70–90% savings in the start-up stage). Additional benefits beyond cost and time savings included increased data quality; improved ability to integrate data, which facilitates data reuseability in knowledge warehouses to improve science, marketing, and safety surveillance; streamlining data interchange among partners and communication among project team members; facilitating regulatory submissions; and enabling transfer and integration of data from disparate tools/technologies. The implementation of data standards seems to be a key strategic initiative of several global companies in the biopharmaceutical industry because it will position them better to use

electronic health records for clinical research in the future. 1.3 CDISC Standards and Healthcare The CDISC has a Healthcare Link initiative that is an integral part of the CDISC strategy. Through this initiative, CDISC has performed several projects with inherent goals to (1) streamline the ability of investigative site personnel to do research within the workflow of clinical care and (2) to enter data electronically only once for multiple downstream purposes, which include safety reporting, clinical trial registration, and results reporting. The CDISC Healthcare Link projects include eSource Data interchange; ‘‘Single Source,’’ which has now been extended to an integration profile; and the BRIDG Model. The eSource Data Interchange (eSDI) project (5) resulted in a document that explains the value and viable means of collecting data only once, electronically, for use in research and clinical care. Specifically, recommendations for eSource systems are used to conduct clinical research and as well as examples of various scenarios that can accomplish eSDI in the context of the existing guidelines and regulations for regulated research. ‘‘Single Source’’ is the basis for one eSDI scenario. This proof-of-concept, conducted at Duke University Medical Center and Duke Clinical Research Institute, demonstrated a viable standards-based opportunity for data entered once to be used for patient care and clinical research. It laid the foundation for the development of an integration profile, working with Integrating the Healthcare Enterprise. The profile, called Retrieve Form for Data Capture (6), was demonstrated in five use cases in January 2007, which include clinical trials, trial registry, biosurveillance, safety reporting, imaging, and lab data (7). It is now being applied in actual implementations in hospital settings, with use cases to support data collection within an electronic health record system for downstream safety reporting to regulatory authorities, academic institutions, NIH and institutional review boards, and for use in protocol-based regulated clinical research.

DATA STANDARDS

The BRIDG Model mentioned previously is the means to link healthcare and protocoldriven clinical research semantically. It was initiated by CDISC primarily to ensure harmonization of its own standards in addition to linking these with the healthcare standards of HL7. It is a domain analysis model that represents clinical research in the context of the HL7 Reference Information model. This open model is now a collaborative initiative, currently with the FDA, the National Cancer Institute, CDISC, and HL7 as the primary stakeholders. This model has been adopted by HL7 Regulated Clinical Research Information Management (RCRIM) Technical Committee (8) as its domain analysis model and the basis for all of the HL7 RCRIM standards. 1.4 CDISC Standards—Basic Types and Value CDISC has two basic types of standards today—content standards that define the information, metadata, and terminology and a transport standard that can carry the standard content, which is the CDISC XMLbased Operational Data Model (ODM) (9). The content standards are harmonized in the BRIDG model (or are in the process of being harmonized). In certain cases, additional transport standards can be used to carry CDISC standard content, such as XMLbased HL7 messages or even ASCII or SAS. The critical aspect is that the content is semantically consistent and that it is harmonized and represented in an overarching model—the BRIDG model in this case. Semantic interoperability allows for the use of different transport mechanisms and various technologies and tools to exchange information in a meaningful and efficient manner. This quality is the value of standards— to support the seamless exchange of information among systems in research and healthcare. 2 AN OVERVIEW OF KEY CLINICAL RESEARCH STANDARDS The CDISC data standards were developed ‘‘starting with the end in mind.’’ It was important to ensure that the CDISC standards

3

would readily support the end goal of a warehouse and/or an integrated report, particularly a regulatory submission. That being said, the business case emphasized the benefit of collecting the data in the same standard format such that the information would readily flow from end to end. To be efficient and make use of the varied expertise of the CDISC volunteers, the CDISC standards were developed in parallel. As noted in the prior section, the BRIDG model has served to ensure that the CDISC standards are a harmonized and compatible set, in addition to serving as the link with healthcare standards. Figure 1 depicts the key CDISC standards and how they support clinical research data flow from end to end, with the content standards being ‘‘carried’’ by the transport standard, ODM. It also indicates the linkage with electronic health records. Table 1 briefly describes the key standards relevant to clinical research. 3 ADOPTION OF CLINICAL RESEARCH STANDARDS AND BEST PRACTICES 3.1 Adoption of CDISC Standards The latest survey on adoption of CDISC standards, which was conducted by Tufts Center for Drug Development in 2007, indicates that adoption has continued to climb since the standards were first made available (10). The adoption logically follows in the order they were released with the Study Data Tabulation Model (SDTM) being used by 69% of those global companies surveyed; SDTM is being piloted by 22%, and 8% plan to implement in the future, which leaves only 1% of those surveyed without implementation in progress or anticipated. The Operational Data Model (ODM) is the next most widely adopted CDISC standard, which is used by 32% of those surveyed; it is being piloted by 32% and 34% are planning to implement the ODM. Downloads of the CDISC SDTM have numbered over 11,000 in approximately 40 different countries. The global reach of CDISC also extends to events in India, China, Australia, along with well-established CDISC Coordinating Committees in Europe and Japan. The CDISC standards and/or initiative outcomes have been included in

4

DATA STANDARDS

Table 1. Clinical Research Standards: This Table Lists Key Standards in the Clinical Research Arena, Including those from Healthcare that Relate to Clinical Research. Standard

Type

Reference Information Model (RIM)

Object Model

Description

An object model created by HL7 as a pictorial representation of the clinical data (domains); the RIM identifies the life cycle of events that an HL7 message (transport standard) will carry, explicitly representing the connections between information carried in the fields of HL7 messages. Biomedical Research Domain Open, collaborative model initiated by CDISC that will Integrated Domain Group Analysis bridge the gap between biomedical research and Model (BRIDG) Model healthcare through the harmonization of standards for research with the relevant healthcare standards of HL7 through the RIM; currently governed by the CDISC, HL7, NCI, and FDA. SDTM Content The CDISC standard for regulatory submission of Standard case-report form data tabulations from clinical research studies; referenced as a specification in Final FDA Guidance for eSubmissions. Operational Data Model Transport The XML-based CDISC standard for the acquisition, (ODM) Standard exchange, reporting or submission, and archival of CRF-based clinical research data. Analysis Data Model (ADaM) Content The CDISC content standard for regulatory submission of Standard analysis datasets and associated files. Laboratory Data Model (LAB) Content The CDISC standard for data transfers of laboratory Standard information between clinical laboratories and study sponsors/CROs; this content standard has four different potential implementations (transport methods): HL7 message, ODM XML, SAS XPT, ASCII. Case Report Tabulation Data Content and The XML-based CDISC standard referenced by the FDA Definition Specification Transport as the specification for the data definitions for CDISC (CRTDDS) – define.xml Standard SDTM datasets. This standard is an extension of the ODM. Standard for the Exchange of Content An extension of the CDISC SDTM standard for Nonclinical Data (SEND) Standard submission of data from preclinical (nonclinical) or animal toxicology studies. Protocol Representation Content The standard supporting the interchange of clinical trial Standard protocol information; a collaborative effort with Health Level Seven (HL7) Trial Design Model (TDM) Content The CDISC standard that defines the structure for Standard representing the planned sequence of events and the treatment plan of a trial; a subset of SDTM and Protocol Representation. Clinical Data Acquisition Content A CDISC-led collaborative initiative to develop the Standards Harmonization Standard content standard for basic data collection fields in case (CDASH) report forms; CDASH is the collection standard based on SDTM as the reporting standard. Terminology Controlled The controlled standard vocabulary and code sets for all Vocabulary CDISC standards (Content Standard) Glossary Dictionary of The CDISC dictionary of terms and their definitions Term related to the electronic acquisition, exchange, and Definitions reporting of clinical research information; a list of acronyms and abbreviations is also available. The standard name, type, and a brief description are included. More information is available at: www.cdisc.org; www.bridgmodel.org, www.hl7.org.

DATA STANDARDS

ments, streamlining the clinical trial process) #34–53

strategic planning documents in Japan and in documents for EMEA inspectors in Europe (11).

• Harnessing Bioinformatics # 46–53 • Moving Manufacturing into the 21st

3.2 Critical Path Initiative

Century (#54–66) • Developing Products to Address Urgent

In March 2004, through the leadership of Dr. Janet Woodcock, the FDA published a report entitled ‘‘Innovation/Stagnation: Challenge and Opportunity on the Critical Path to New Medical Products’’ (12). This report was a unique effort on the part of the FDA to bring attention to the need to modernize the current medical product evaluation and development methodologies to reap public health benefits. The resulting Opportunities List (13) outlines 76 opportunities for public and private partnerships that could potentially help bring the fast-paced biomedical discovery realm closer to therapeutic development, which has a much slower pace. The opportunities fall into the following categories:

Public Health Needs (#67–71) • Specific At-risk Populations – Pediatrics

(#72–76) The principal activities of CDISC and HL7 fall into the second category of streamlining clinical trials with Opportunities #44 and #45. In addition, the CDISC has an FDA pilot in progress that is addressing the At-risk Population of Pediatrics through Opportunity #72. Opportunity #45 is Development of Data Standards. It states ‘‘Lack of standardization is not only inefficient; it multiplies the potential for error.’’ It also refers to CDISC as paving the way with its SDTM and that ‘‘HL7 and CDISC are working to create standards that can be used for the exchange, management, and integration of electronic healthcare information to increase the effectiveness and efficiency of healthcare delivery.’’

• Better Evaluation Tools (e.g., biomark-

ers) - #1–33 • Streamlining Clinical Trials (e.g., trial

designs, patient response measure-

Data Flow Using CDISC Standards and Linking Clinical Research and Healthcare

ODM (transport)

Electronic Health Record

SDTM and Analysis Data (content) Protocol Information (content)

Patient Info HL7 and/or ODM XML

Clinical Trial Data

Protocol Representation Trial Design (SDTM) Analysis Plan

Clinical Trial Protocol

HL7 and/or ODM XML

Source Data (other than SDTM/CRF data)

Administrative, Tracking, Lab Acquisition Info

Patient Info

ODM XML

5

Clinical (CRF or eCRF) Trial Data (CDASH as defined by STDM)

(e)Source Document

ODM XML

CRF, Analysis Data

Operational & Analysis Databases

Integrated Reports ODM XML Define .xml

STDM Data, Analysis Data, Metadata

Warehouses, Reporting, Regulatory Submissions

Figure 1. Data Flow Using CDISC Standards; Clinical Research Link to Healthcare. This figure shows the different CDISC standards and how they work together to support the flow of clinical research from protocol through reporting for data warehouses and electronic regulatory submissions. It also shows the linkage point between clinical research and healthcare (electronic medical records).

6

DATA STANDARDS

Specifically, #45 calls for Consensus on Standards for Case Report forms. These standards for data collection will facilitate the way that investigators and clinical study personnel and data managers complete and manage the data entered on paper or electronic case report forms. ‘‘Differences in case report forms across sponsors and trials creates opportunities for confusion and error.’’ To address this opportunity, CDISC, with the encouragement of the FDA and the Association of Clinical Research Organizations, formed a Collaborative Group to provide strategic direction and help fund this initiative. Since then, several hundred volunteers have been working to establish the Clinical Data Acquisition Standards Harmonization (CDASH) standards for data collection (14). These standards are based on the SDTM. Opportunity #72 is Better Extrapolation Methods and Best Practices in Pediatrics. It explicitly calls for the creation of an integrated set of pediatric data for quantitative analysis to exploit past experiences based on adults. ‘‘Analysis of such a database could reveal best practices for other aspects of pediatric trial designs as well, enabling sponsors to avoid repeating less useful or inefficient trial designs. As a result, fewer children would be exposed to unnecessary or suboptimal clinical studies.’’ CDISC is working with the FDA on a pilot to use the CDISC standards for integration of data from numerous pediatric studies to evaluate the value of using standards in this context. 3.3 CDISC Pilots with FDA Currently, CDISC is conducting or has conducted four pilots with the FDA to implement the CDISC standards in a controlled and educational manner. One pilot initiated in 2007 involves the Standard for the Exchange of Nonclinical Data (SEND) and standards for pharmacogenomic data developed through the CDISC LAB team and HL7. ‘‘The National Center for Toxicological Research (NCTR) in collaboration with CDER staff is supporting the SEND pilot by hosting SEND data and making them available to CDER reviewers. The goal of NCTR’s participation is to identify key bottlenecks and limitations of current eSubmission

strategies and identify best practices to facilitate data flow at each stage of the data submission pipeline. This activity includes evaluating the Janus data model for SEND data’’ (15). Another FDA pilot that uses CDISC standards is with the Operational Data Model (ODM). This pilot will go through 2008 with six sponsors who have volunteered to submit ODM XML files instead of paper/PDF case report forms. The ODM offers a transport standard for providing CRF data in XML format, along with a standard audit trail that adheres to the 21CFR11 regulations (16). The SDTM/ADaM pilot was the first CDISC-FDA pilot, and it focused generally on eSubmissions. This pilot was initiated in 2006 and was designed to test the effectiveness of data submitted to the FDA using CDISC standards in meeting the needs and the expectations of both medical and statistical FDA reviewers. In doing so, the project also assessed the data structure/architecture, resources, and processes needed to transform data from legacy datasets into the SDTM and ADaM formats and to create the associated metadata and CRTODS. The report from this pilot is now posted openly on the CDISC website so that others can learn from the experiences of the pilot team. The follow-on pilot from this first eSubmission pilot specifically addresses the integration of safety data from multiple pediatric studies (as noted for Opportunity #72 in the Critical Path Opportunity List). This pilot, which is called the Integrated Safety Data Pilot, has a major objective to explore and evaluate best practices for the processes associated with the use of standards for facilitating regulatory eSubmissions and safety assessments. The pilot will evaluate data integration, workflow and processes, and semantic interoperability, as well as standard analysis and reporting. These two eSubmission pilots are addressed in more depth in the next two sections of this article.

DATA STANDARDS

4 DATA STANDARDS FOR A FINAL STUDY REPORT (FDA: THE FORMAT AND CONTENT OF THE FULL INTEGRATED CLINICAL AND STATISTICAL REPORT OF A CONTROLLED CLINICAL STUDY AND ICH/FDA E3: STRUCTURE AND CONTENT OF CLINICAL STUDY REPORTS) An earlier guideline from the FDA was provided in 1988 for the Format and Content of the Clinical and Statistical Sections of New Drug Applications, in which a strong contextual description of an integrated clinical and statistical report was described. This contextual format was used as a basis for the next generation of an integrated clinical and statistical report developed by the International Conference on Harmonization (ICH). Through a rigorous review and approval process, global adoption of the ICH E3 Guideline: Structure and Content of Clinical Study Reports was adopted by the United States, Japan, and Europe in 1996 (17). This guideline would presently be viewed as a standard metadata or context for a final study report (FSR). If one looks at the Table of Contents of the E3, there is congruency with the standard data domains in SDTM such as demography, vital signs, medical history, exposure, labs, concomitant therapy/medications, adverse events, inclusion/exclusion criteria, and so on. It is important to note that the E3 does not provide extensive standard granularity or data content (e.g., required standard variables, values, units, data definitions, terminology, etc.). A clarification of the requirements of a standard FSR was provided by the ICH eCTD, which was adopted by the FDA April 2006 as Providing Regulatory Submissions in Electronic Format—Human Pharmaceutical Product Applications and Related Submissions Using the eCTD Specifications (18). The FDA removed all previous guidances for the New Drug Application effective 1 January 2008. The true significance of the FDA eCTD Guidance is the strong references to the CDISC data content models or standards and the required file structure. The electronic Common Technical Document contains five modules, and the Module 5 contextual structure provides the Clinical Study Reports Folder requirements. Very briefly

7

presented here are initial comments regarding the requirements for a FSR (note the reference to the E3 metadata structure). 4.1 Study Reports Typically, clinical study reports are provided as more than one document based on the ICH E3 guidance document when providing a study. The individual documents that should be included in a study report are listed below: • Synopsis (E3 2) • Study report (E3 1, 3–15) • Patients excluded from the efficacy stud-

ies (E3 16.2.3) • Demographic data (E3 16.2.4) • Compliance and/or drug concentration

data (E3 16.2.5) • Individual efficacy response data (E3

16.2.6) • Adverse event listings (E3 16.2.7) • Listing of individual laboratory mea-

surements by patient (E3 16.2.8) • Case report forms (E3 16.3) • Individual patient data listings (CRTs)

(E3 16.4) The e-CTD Guidance provides some of the very best data content requirements and is presented in the associated document on the FDA website called the Study Data Specifications. For example, the specifications for organizing study datasets and their associated files in folders found in the Study Data Specification document are summarized in Figure 2. In parallel with the dataset file structure the requirements for the use of CDISC data content models such as SDTM (or SEND), CRTDDS (define.xml), or using the HL7 normative standard for creating the annotated ECG waveform data files are also provided in the eCTD Datasets Study Data Specification document. Most importantly, we have entered a new era for the use and application of standard data content that will greatly enhance a more useful application of the previously available standard FSR metadata or context. Also, to support standard data collection, the e-CTD Guidance provides a contextual requirement

8

DATA STANDARDS

for case report forms that are enhanced by the previously described CDISC CDASH model. Again, we have not previously had standardized case report forms in the industry; having standard CRFs based on SDTM content will improve the workflow and process for the reporting of both safety and efficacy data. To demonstrate that all these contextual and content standards would actually work (using the eCTD file structure, the context, or metadata of the E3 and the related requirements for CDISC standards in the Study Data Specification), CDISC and the FDA performed a pilot for the generation, submission, and review of a mock FSR by the FDA. The pilot submission package included one abbreviated study report that documented the pilot project team’s analyses and reporting of the legacy clinical trial data. The purpose of providing a study report was to test the summarizing of results and the linking to the metadata, as well as providing results or findings for the regulatory review team to review and/or reproduce. Accompanying the study report were the tabulation datasets, analysis datasets, Define.xml files that contain all associated metadata, an annotated case report form (aCRF), and a reviewer’s guide. The report used data from a Phase II clinical trial provided by Eli Lilly and Company. The trial data was mapped to SDTM as the simple analytic view of the raw or observed data to begin the process described above for generating a standard report. The report and results of the SDTM/ADaM CDISC/FDA Pilot are posted on the CDISC website for use by the industry.

5 STANDARD DATA FOR REGULATED AND REQUIRED WRITTEN ADVERSE EVENT SAFETY REPORTS [INDIVIDUAL CASE SAFETY REPORT (ICSR) AND PERIODIC SAFETY UPDATE REPORT (PSUR)] Reporting drug safety events and data to regulatory authorities generally is classified into two categories that we normally describe as preapproval (under an investigational drug application) and postmarket approval reports (under a marketing approval license). An expedited safety report for either preapproval or postapproval reporting is described as a transactional process. Meaning from the time of the adverse event to the submission to a regulatory agency considerable data is gathered, processed, evaluated, and reported as required by the appropriate laws or regulations. The preapproval reports have the benefit of, more often than not, having parallel control data. As we know, the postapproval reports frequently do not have supportive control data but do generally have a much larger database. The postapproval adverse event analysis expands the risk management aspect of reporting, and it becomes more strategic after approval because of a lack of control data, the presence of frequent uncontrolled concomitant medications (polytherapy, polypharmacy), and unpredicted dosing and off-label use. Continual assessment in relationship to pharmacological class and comparator medications becomes essential. Standard data and collection greatly enhance this process and will build the relationship to translational medicine and clinical pharmacology. They will also support the need to correlate phenotype to biomarkers expressed by the genome as signals of

Figure 2. Specifications for organizing study datasets and their associated files in folders.

DATA STANDARDS

severe or unexpected untoward events. Historically, efforts were made to standardize safety data reporting by generating the ICH Medical Dictionary for Regulatory Activities (MedDRA) and the ICH E2B(R3)/E2BM2 (Data Element for Transmission of Individual Case Safety Reports). The MedDRA provides a standard international terminology for describing an adverse event with a hierarchal and standardized structure of terminology from system organ classification to preferred, verbatim, and lowest level term. The standard element of transmission of an expedited safety report (IND Initial Written Report) on adverse and unexpected events in the preapproval arena uses the MedWatch 3500A form. This 3500A form and standard elements are used in the FDA/ICH supported ICSR (Individual Case Safety Report) for postmarketing expedited reports as well. These expedited safety reports using the MedDRA and E2BM2 standard data elements are summarized in the PSUR (Periodic Safety Report) or in an IND Annual Safety Report or any time a Standard Summary Safety report is required by the FDA. An HL7 message is being tested to carry the ICSR content. The requirement for standardized safety reporting is growing exponentially to enable integration of large datasets from multiple sources, and it is driven by increasing standardization by the global regulatory authorities. For example, the FDA’s Sentinel Network for a federated network of both private and public safety databases will best be served by standard data and standard data collection. The FDA has taken a strong position that the content standard will be CDISC data models. Similarly, collaborative efforts are led by the NIH to standardize safety data reporting and monitoring across federal agencies using the BAER (Basal Adverse Event Report). These NIH, NCI, FDA, CDISC, and HL7 have worked together to harmonize their various AE reporting standards within the BRIDG model to support interoperability and to facilitate information exchange from preapproval through postmarketing for a therapy. CDISC has led these harmonization activities. The NIH reporting using BAER will be in parallel with the new FDA initiatives for MedWatch Plus and

9

the new FDA Adverse Event Reporting System. Also, the FDA’s four recent Guidances for Pre-Approval Safety, Risk Minimization Action Plans, Pharmacovigilance, and Pharmacoepidemology and the Good Review Practice, Conducting a Clinical Safety Review of a New Product Application and Preparing a Report on the Review, are all soon to embrace electronic-data and SDTM (the simple analytic view of the observed data). A new pilot between the FDA and CDISC regarding the application of CDISC data standards to support integrated safety data analysis was initiated in 2007 (19). The overall mission of the second iteration of the CDISC/FDA Pilot (the Integrated Safety Data Submission pilot) project team is to demonstrate that a safety data submission (including safety analysis plan and key safety statistical analyses) created using the CDISC standards will meet the needs and expectations of FDA reviewers in conducting an integrated and appropriately directed safety review of data from multiple studies, compounds, and sponsors. During the process of preparing, aggregating, and analyzing the data, the project team will assess the applicability of the CDISC standards to integrated data, identifying any issues/questions to be addressed by the CDISC standards development teams. A major objective of this pilot is to explore and evaluate best practices for the processes associated with the use of standards for facilitating regulatory eSubmissions and safety assessments. The pilot will evaluate data integration, workflow and process, semantic interoperability, and standard analysis and reporting. Practice datasets from eight clinical studies from multiple submissions that span several products have been deidentified and altered for confidentiality and to include intentional signals for analysis by the pilot team. The planned deliverable is submission of an integrated data summary that would support a safety review and analysis. Last, it is in the scope of the pilot team’s mission to serve the FDA Critical Path Opportunities Number 44—Propose Regulations to Require Electronic Submission of Study Data, Number 45—Streamlining Clinical Trial Data Collection and Number 72—Develop Pediatric Trials Database.

10

DATA STANDARDS

The CDISC standards are now available in production versions to support the transport, management and analysis, submission, and archive of electronic clinical research data. Standards to support data collection and protocol representation will be in production versions in 2008. CDISC will continue to support the FDA pilots and industry implementations of these standards to augment the best practices and case study results that have already been compiled. Other areas, including terminology, will also continue to be enhanced and harmonized into the BRIDG model for the open support of systems that can become increasingly interoperable within clinical research and with healthcare. Continued contributions from the industry will only serve to make these standards more valuable in streamlining processes, which improves data quality and ultimately enhances patient safety. REFERENCES 1. Clinical Data Interchange Standards Consortium. Available: http://www.cdisc.org/. 2. Health Level Seven. Available: http://www. hl7.org/. 3. http://www.cdisc.org/about/bylaws pdfs/ CDISC-COP-001-StandardsDevelopmentFeb2006.pdf. 4. C. Rozwell, R. Kush, and E. Helton, Save time and money. Appl. Clin. Trials 2007; 16(6). 70 5. eSource Data Interchange Group, Leveraging the CDISC Standards to Facilitate the use of Electronic Source Data within Clinical Trials, V 1.0, 20 2006. Available: http://www.cdisc.org/eSDI/eSDI.pdf. 6. Retrieve Form for Data Capture (RFD) Integration Profile. Available: http://wiki.ihe.net/ index.php?title=Retrieve Form for Data Capture. 7. Interoperability Showcase Press Release. Available: http://www.cdisc.org/news/10-042007 PR29 HIMSS Interoperability Showcase.pdf. 8. Regulated Clinical Research Information Management (RCRIM) Technical Committee. Available: www.hl7.org/Special/committees/rcrim/index.cfm. 9. Operational Data Model. Available: http:// www.cdisc.org/models/odm/v1.3/index.html.

10. D. Borfitz, Survey: New Mindset about eClinical Technologies, Standards. eCliniqua. 2007. 11. The Draft Reflection Paper on Expectations for Electronic Source Documents used in Clinical Trials, European Medicines Agency. Available: http://www.emea.europa.eu/Inspections/ docs/50562007en.pdf. 12. Food and Drug Administration, Innovation/ Stagnation: Challenge and Opportunity on the Critical Path to New Medical Products. Available: http://www.fda.gov/oc/initiatives/ criticalpath/whitepaper.pdf. 13. Food and Drug Administration, Critical Path Opportunities List. Available: http://www.fda. gov/oc/initiatives/criticalpath/reports/opp list.pdf. 14. Clinical Data Acquisition Standards Harmonization. Available: http://www.cdisc.org/ standards/cdash/index.html. 15. SEND Pilot. Available: http://www.fda.gov/oc/ datacouncil/send.html. 16. Title 21 Food and Drugs Chapter; Food and Drug Administration, Department of Health and Human Services, Part 11 Electronic Records; Electronic Signatures. Available: http:// www.accessdata.fda.gov/scripts/cdrh/cfdocs/ cfCFR/CFRSearch.cfm?CFRPart=11. 17. International Conference on Harmonisation, ICH Harmonised Tripartite Guideline: Structure and Content of Clinical Study Reports. Available: http://www.ich.org/LOB/media/ MEDIA479.pdf. 18. Guidance for Industry: Providing Regulatory Submissions in Electronic Format—Human Pharmaceutical Product Applications and Related Submissions Using the eCTD Specifications. Available: http://www.fda.gov/ cder/guidance/7087rev.pdf. 19. CDISC SDTM/ADaM Pilot Team, CDISC SDTM/ADaM Pilot Project-Project Report, January 2008. Available: http://www.cdisc. org/publications/SDTMADaMPilotProjectReport.pdfhttp://www.cdisc.org/publications/ SDTMADaMPilotProjectReport.pdf.

CROSS-REFERENCES Metadata

DERMATOLOGY TRIALS

study was performed to support registration of the drug.

STEVEN R. FELDMAN ADELE R. CLARK

2

Wake Forest University School of Medicine Winston-Salem, North Carolina

The study was a 4-week, multicenter, randomized, double-blinded, vehicle-controlled, parallel-group comparative study of clobetasol propionate 0.05% spray in subjects with moderate to severe psoriasis (5).

Skin diseases are among the most common human afflictions. These conditions have as much impact on patients’ quality of life as other major medical conditions (1). As the skin forms the interface with the environment, it has complex and important immune functional roles. Perhaps as a result, numerous different inflammatory skin diseases exist. One of the most common is psoriasis. Many new psoriasis treatments are being developed based on advancements in our understanding of the immune system (2). Topical corticosteroids continue to be used for psoriasis. Although topical treatment offers the opportunity to use potent medications with little risk of systemic side effects, topical treatments are inconvenient (3,4). New delivery systems are being developed. A trial of a topical clobetasol spray formulation presents a good model for dermatology trials (5). 1

STUDY DESIGN

2.1 Study Drug Clobetasol propionate 0.05% spray (Clobex spray; Galderma Laboratories, L.P., Fort Worth, TX) and the spray vehicle were compared. A key issue in topical treatment studies is that the vehicle may have important effects of its own, which includes direct effects on the skin and indirect effects on the delivery of the active drug through the skin. ‘‘Inactive’’ components of the vehicle can be irritating and can cause allergic reactions. Moreover, patients’ adherence to treatment will be affected by the vehicle (4,6). In this study, the vehicle was composed of alcohol, isopropyl myristate, sodium lauryl sulfate, and undecylenic acid (7).

OBJECTIVES 2.2 Study Population

The objective of this study was to assess the efficacy and safety of clobetasol propionate spray 0.05% in the treatment of moderate to severe plaque-type psoriasis (5). Clobetasol is a corticosteroid with very potent anti-inflammatory properties. Clobetasol is already available in ointment, cream, gel, and other formulations. Dermatologic dogma suggests that moisturizing vehicles are best suited for psoriasis treatment (6). Yet patients may perceive such vehicles as messy and inconvenient (6). An easy to use, nonmessy clobetasol formulation was developed. The primary hypothesis of this study was that more subjects treated with clobetasol propionate spray would achieve success than would subjects treated with placebo spray (5). The study would provide an assessment of the efficacy of the spray that could be compared with results in the literature for traditional clobetasol ointment products. This Phase III

In all, 120 subjects with moderate to severe psoriasis were recruited. Subjects could be of either sex, had to be at least 18 years of age, and had to present with an area of plaque psoriasis that covered at least 2% of the body surface area. The face, scalp, groin, axillae, and other intertriginous areas were excluded because clobetasol is too potent to be used on these areas. Subjects were required to have a target lesion with a severity (accounting for scaling, erythema, and plaque elevation) of at least 3 (moderate) on a scale ranging from 0 (none) to 4 (severe/very severe). Including plaques of this degree of severity is important to help define the psoriasis as moderate to severe. It also gives room to define improvement to clear (0) or almost clear (1) as success. Women of childbearing potential had to have a negative urine pregnancy test and had to agree to use an effective nonprohibited

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

DERMATOLOGY TRIALS

form of birth control for the duration of the study. Prior to entering the study, subjects had to respect treatment-specific wash-out periods as defined by the protocol regarding any topical and systemic treatments known to affect psoriasis, which includes immunomodulatory therapies and exposure to natural or artificial UV radiation. Normally, this duration is about 2 weeks for topical treatments, 1 month for phototherapy or oral systemic medications, and 3 months for long acting agents, which include some biologic treatments. As patients wash out from topical medication, some flaring of the disease can occur at study entry; nearly all trials want to evaluate subjects with stable disease. Fortunately, patients tend to be noncompliant and are already largely off the medications they should be using. In the case of psoriasis, seasonal variations in disease can occur. Thus, it tends to be easier to recruit patients in the late fall and winter when the disease tends to be worse. If the treatment lasts 6 months, however, patients in placebo groups tend to improve as the weather warms and they begin to have ultraviolet light exposure. 2.3 Treatment Half the subjects were randomized to twicedaily use of clobetasol propionate spray 0.05% for 4 weeks and the other half to vehicle spray. Topical clobetasol is highly effective, so a 2- to 4-week study interval is often sufficient to observe efficacy. The study period may be included in the FDA-approved drug label. A short period can be used in marketing to promote the high efficacy of a product. A longer duration can be useful in marketing the safety of the product. In short-term studies such as this one, a vehicle control group may be entirely reasonable. In longer term studies (3 months or more), patients in the vehicle or placebo control arm may switch over to active treatment after a predetermined period. This arrangement helps promote recruitment to the study as well as retention in the placebo arm. The crossover is particularly important for studies of patients with more severe disease, as they might not be willing to risk entering a

study and being on placebo, nor may they be willing to wait several months on placebo if no ‘‘carrot’’ of active drug is offered at the end of the trial. Subjects were also instructed to separate applications by at least 8 hours, not to wash the treated area for at least 4 hours after treatment, and not to apply the study medication within 4 hours prior to study visits. One should recognize that patients are often noncompliant to topical treatments, even in a clinical trial (8,9). Without objective confirmation, researchers have little confidence that the recommendations were followed. The treatment phase was followed by a 4-week treatment-free follow-up period. This period is required to assess how persistent the effect of the product is and whether the condition has any tendency to rebound when the product is discontinued (10). 2.4 Efficacy Assessments The study was evaluated at baseline and weeks 1, 2, 4, and 8 (follow-up 4 weeks after the end of treatment). The frequent follow-up in the clinical trial promotes better adherence than is observed in clinical practice (11). Thus, many medications seem to work better in the clinical trial than in clinical practice. The investigators assessed scaling, erythema, plaque elevation, and overall disease severity using a 5-point scale (0 = clear, 1 = almost clear, 2 = mild, 3 = moderate, 4 = severe/very severe). Although investigator assessments of severity tend to be more objective than subjects’ self-assessments, considerable subjectivity may occur in investigators’ evaluation of these parameters. This subjectivity may be a cause of apparent placebo responses in dermatologic trials (12). In some studies, training sessions are held at investigator meetings to reduce variation between investigators’ efficacy assessments, yet even then a wide range of responses may occur. Using the same evaluator may be more critical in dermatology trials than in studies that use more objective tests as outcomes. Subjects’ assessments of pruritus were also assessed. Dermatology trials often include a measure of subjects’ quality of life; the quality of life measure confirms that objective changes in skin lesions actually result in improvement in the patient’s life (13).

DERMATOLOGY TRIALS

Use of success rates as a primary response measure facilitates intent to treat analyses and shows regulators that the drug makes a meaningful improvement for subjects. In this study, success was defined as a grade of 2 (mild) or less on the 5-point scale at week 2 and defined as a grade of 1 (almost clear) or less on the same scale at the end of treatment at week 4. Many trials of treatments for moderate to severe psoriasis employ the Psoriasis Area and Severity Index (PASI). Although the PASI provides a quantitative assessment of severity, the scores are not easy to interpret clinically, and the measure is not used much in clinical practice. Using ‘‘clear or almost clear’’ as a success measure gives clinicians a parameter that is meaningful in their care of patients. 2.5 Safety Assessments Safety was evaluated by reported adverse events, which include skin atrophy, telangiectasia, burning/stinging, and folliculitis. Clinical signs and symptoms of hypothalamus–pituitary–adrenal (HPA) axis suppression were also assessed. This method is an insensitive way to assess HPA axis suppression. Adrenal stimulation tests are used in other studies to assess more fully the potential for HPA axis suppression (14). 2.6 Statistical Analyses The intent-to-treat population (all subjects enrolled and randomized) was used to assess success rates. Safety evaluations included all subjects who received treatment. The success rate of all parameters was calculated for each time point using a Cochran–Mantel–Haenszel test to confirm the superiority of clobetasol propionate spray 0.05%. Statistical significance of the above efficacy parameters was based on two-tailed tests of the null hypothesis, which resulted in P values of .05 or less. 3 RESULTS 3.1 Efficacy At 2 weeks, 52 subjects (87%) achieved success with clobetasol propionate versus 17 subjects (28%) treated with the spray vehicle (P < 0.001). Despite the more restrictive

3

criteria for success at week 4 than at week 2 (subjects had to be almost or completely clear at week 4, but mild disease or better was considered success at week 2), 47 subjects (78%) achieved success with 4 weeks of treatment with clobetasol propionate compared with 2 subjects (3%) treated with vehicle (P < 0.001). 3.2 Safety Burning/stinging was reported by 14 subjects (23%) within 15 minutes after the first application of clobetasol propionate and by 13 subjects (22%) after application of the vehicle; most reports were mild in severity. No cutaneous signs of skin atrophy, telangiectasia, or folliculitis were detected in either group throughout the duration of the study. No adrenal suppression was identified. Although the subjects tolerated burning and stinging well in the trial, many patients in clinical practice will not be so tolerant of side effects. Side effects seem to be better tolerated in study settings for several reasons. First, research subjects may be highly motivated. Second, frequent follow up visits are required so subjects may be thinking they have to put up with the irritation for a shorter period of time than the typical clinic patients does. Third, at the return visits the study staff may act as cheerleaders to help keep the subject motivated to use the medication. Fourth, continuing to be in the study may mean continuing to collect remuneration. One wonders sometimes whether some subjects stay in the study even when they have stopped using the medication. 4

CONCLUSIONS

The clobetasol propionate 0.05% spray product was highly effective for moderate to severe plaque-type psoriasis and was superior to its vehicle (5). As is typical of psoriasis trials, the study was powered to assess efficacy. Assessing safety is far more difficult. In this group of subjects, the treatment was safe and well tolerated. Superpotent corticosteroids are often labeled for only up to 2 weeks of continuous use. The 4-week treatment period in this study yielded greater clinical benefit with no detectable change in the safety profile. Nevertheless, our certainty in safety is limited by

4

DERMATOLOGY TRIALS

the size of the study, and the study provides no evidence on which to base conclusions about longer term safety profile. Dermatologic dogma suggests that moisturizing, occlusive ointment preparations are most appropriate for treating a dry, scaly condition like psoriasis (15,16). Furthermore, dogma suggests that using keratolytic treatments (such as topical salicylic acid) should be used to reduce scale and enhance penetration of topical psoriasis treatments (15,16). The high efficacy in this study of a nonmoisturizing product used without any keratolytic agent goes contrary to established dogma. The effectiveness of superpotent topical corticosteroids in plaque-type psoriasis probably depends more on whether the patient applies the product than the moisturizing, occlusive, or keratolytic properties of the vehicle. The spray formulation of clobetasol propionate 0.05% is easy to apply and may provide a valuable alternative for specific patient groups.

REFERENCES

6. T. S. Housman, B. G. Mellen, S. R. Rapp, A. B. Fleischer Jr., and S. R. Feldman, Patients with psoriasis prefer solution and foam vehicles: a quantitative assessment of vehicle preference. Cutis 2002; 70: 327–332. 7. Clobex® Spray Prescribing Information. Fort Worth, TX: Galderma Laboratories, L.P., 2005. 8. J. Krejci-Manwaring, M. G. Tusa, C. Carroll, F. Camacho, M. Kaur, D. Carr, A. B. Fleischer Jr., R. Balkrishnan, S. R. Feldman, Stealth monitoring of adherence to topical medication: adherence is very poor in children with atopic dermatitis. J. Am. Acad. Dermatol. 2007; 56: 211–216. 9. C. L. Carroll, S. R. Feldman, F. T. Camacho, J. C. Manuel, and R. Balkrishnan, Adherence to topical therapy decreases during the course of an 8-week psoriasis clinical trial: commonly used methods of measuring adherence to topical therapy overestimate actual use. J. Am. Acad. Dermatol. 2004; 51: 212–216. 10. K. B. Gordon, S. R. Feldman, J. Y. Koo, A. Menter, T. Rolstad, G. Krueger, Definitions of measures of effect duration for psoriasis treatments. Arch. Dermatol. 2005; 141: 82–84.

1. S. R. Rapp, S. R. Feldman, M. L. Exum, A. B. Fleischer Jr, and D. M. Reboussin, Psoriasis causes as much disability as other major medical diseases. J. Am. Acad. Dermatol. 1999; 41: 401–407.

11. S. R. Feldman, F. T. Camacho, J. KrejciManwaring, C. L. Carroll, and R. Balkrishnan, Adherence to topical therapy increases around the time of office visits. J. Am. Acad. Dermatol. 2007; 57: 81–83.

2. E. M. Berger and A. B. Gottlieb, Developments in systemic immunomodulatory therapy for psoriasis. Curr. Opin. Pharmacol. 2007; 7: 434–444.

12. J. Hick and S. R. Feldman, Eligibility creep: a cause for placebo group improvement in controlled trials of psoriasis treatments. J. Am. Acad. Dermatol. 2007; 57: 972–976.

3. K. K. Brown, W. E. Rehmus, and A. B. Kimball, Determining the relative importance of patient motivations for nonadherence to topical corticosteroid therapy in psoriasis. J. Am. Acad. Dermatol. 2006; 55: 607–613.

13. S. R. Feldman and G. G. Krueger Psoriasis assessment tools in clinical trials. Ann. Rheum. Dis. 2005; 64(suppl 2): ii65–68; discussion ii69–73.

4. K. G. Bergstrom, K. Arambula, and A. B. Kimball, Medication formulation affects quality of life: a randomized single-blind study of clobetasol propionate foam 0.05% compared with a combined program of clobetasol cream 0.05% and solution 0.05% for the treatment of psoriasis. Cutis 2003; 72: 407–411. 5. M. T. Jarratt, S. D. Clark, R. C. Savin, L. J. Swinyer, C. F. Safley, R. T. Brodell, and K. Yu, Evaluation of the efficacy and safety of clobetasol propionate spray in the treatment of plaque-type psoriasis. Cutis 2006; 78: 348–354.

14. J. L. Jorizzo, K. Magee, D. M. Stewart, M. G. Lebwohl, R. Rajagopalan, and J. J. Brown, Clobetasol propionate emollient 0.05 percent: hypothalamic-pituitary-adrenal-axis safety and four-week clinical efficacy results in plaque-type psoriasis. Cutis 1997; 60: 55–60. 15. T. P. Habif, Psoriasis and Other Papulosquamous Diseases. Clinical Dermatology: A Color Guide to Diagnosis and Therapy. Philadelphia, PA: Mosby, 2004. 16. A. Rook, T. Burns, S. Breathnach, N. Cox, and C. Griffiths, Rook’s Textbook of Dermatology. 2004.

DERMATOLOGY TRIALS

FURTHER READING J. Garduno, MJ. Bhosle, R. Balkrishnan, and S. R. Feldman, Measures used in specifying psoriasis lesion(s), global disease and quality of life: a systematic review. J. Dermatol. Treat. 2007; 18: 223–242. C. E. Griffiths, C. M. Clark, R. J. Chalmers, A. Li Wan Po, and H. C. Williams, A systematic review of treatments for severe psoriasis. Health Technol. Assess. 2000; 4: 1–125. C. Hoare, A. Li Wan Po, and H. Williams, Systematic review of treatments for atopic eczema. Health Technol. Assess. 2000; 4: 1–191. K. T. Hodari, J. R. Nanton, C. L. Carroll, S. R. Feldman, and R. Balkrishnan, Adherence in dermatology: a review of the last 20 years. J. Dermatol. Treat. 2006; 17: 136–142. I. A. Lee and H. I. Maibach, Pharmionics in dermatology: a review of topical medication adherence. Am. J. Clin. Dermatol. 2006; 7: 231–236. H. P. Lehmann, K. A. Robinson, J. S. Andrews, V. Holloway, and S. N. Goodman, Acne therapy: a methodologic review. J. Am. Acad. Dermatol. 2002; 47: 231–240.

CROSS-REFERENCES Dosage Form Outcomes Center for Drug Evaluation and Research (CDER) Controlled Clinical Trials

5

DESIGNS WITH RANDOMIZATION FOLLOWING INITIAL STUDY TREATMENT

1. In the evaluation of long-term efficacy of a drug, long-term placebo treatment is not always feasible, as e.g. for cancer or psychiatric diseases. If an active control is available, non-inferiority studies might be considered, but these may require large sample sizes, and the determination of a non-inferiority margin can be difficult. 2. For major depression, several stages of treatment can be distinguished: treatment of acute symptoms, and for patients who respond, the continuation phase (to prevent relapse), and the maintenance phase (to prevent recurrence). Parallel-group, placebo-controlled trials aimed at evaluating relapse and/or recurrence include the so-called extension studies. These studies have an initial short-term, parallel–group, placebocontrolled phase. The second stage includes only stage I responders, who continue the treatment with the same medication under double-blind conditions. This design has the ethical advantage that only responders are exposed to study treatment in the second stage, but the possibility of imbalance between treatment groups (i.e., active treatment responders and placebo responders) at the start of the extension stage cannot be excluded, especially following differential dropout in stage I. This was noted by Storosum et al. (3) as a major methodological shortcoming of this design, making interpretation of extension study results difficult. 3. For progressive diseases like Alzheimer’s or Parkinson’s disease, it is important to determine the effect of a treatment on symptoms (short term) and on disease progression (long term). This could be easily addressed in the gold standard design if markers for the progression of the diseases were available, but such markers have not yet been established. Instead, both effects have to be inferred from the patient’s performance status over time. In the

CORNELIA DUNGER-BALDAUF Novartis Pharma AG Basel, Switzerland,

When an innovative treatment is being investigated, reliable evaluation of its benefit for patients is required. The double-blind, randomized, controlled clinical trial with parallel treatment groups has become widely accepted as the gold standard to achieve this goal. In such a trial, patients are initially randomized to a specific treatment or regimen on which they remain throughout the duration of the study. At the end of the treatment period, the benefit to patients is compared between treatment groups. Despite the proven success of the standard design, there remain scientific questions or situations for which it does not offer the most appropriate solution, such as in diseases with serious morbidity where maintaining a full placebo group to the end of a parallel design trial may be considered to be medically and ethically unsatisfactory. In 1975, Amery and Dony (1) proposed a ‘‘clinical trial avoiding undue placebo treatment’’ to evaluate drugs for the treatment of angina pectoris. According to this design, all patients are initially treated with a test drug. After titration to the desired outcome, responders after the first stage are randomized to either continued treatment or placebo. Amery showed that this design and a more conventional design gave similar results (2). Further situations where the standard trial design is difficult to apply are described in this article, followed by a discussion of the designs that address the difficulties.

1 RATIONALE FOR RANDOMIZATION FOLLOWING INITIAL STUDY TREATMENT Difficulties in applying the standard parallelgroup design were noted in the following situations:

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

DESIGNS WITH RANDOMIZATION FOLLOWING INITIAL STUDY TREATMENT

comparison of treatments at the end of a standard trial, the short-term and long-term effects would not be distinguishable. 4. For a disease with fluctuating symptoms, the effect of continuous treatment observed in a standard trial might be diluted, as patients have symptom-free periods where the efficacy of a drug cannot be established. A parallel-group trial with several periods of intermittent treatment (short-course administration for a predetermined time period after symptom recurrence) can be considered. However, due to potentially different dropout patterns, initially comparable treatment groups might no longer be comparable at the start of repeated treatment such that the comparison of repeated treatment in this design becomes questionable. Extensions of the standard design have been proposed to address these questions. Randomized withdrawal designs can be considered for situations 1 and 2, randomized delayed-start designs for situation 3, and retreatment designs for situation 4. Instead of (re)randomizing patients following initial study treatment (at the start of the second stage), a randomization to treatment sequences could be performed upfront before the trial. However, in the presence of dropouts or in case of non-eligibility of some patients after the first stage, randomization of only the patients who enter the second stage is more likely to generate balanced treatment groups. 2

RANDOMIZED WITHDRAWAL TRIALS

In a randomized withdrawal trial (Figure 1), patients receiving a test treatment for a specified time are randomly assigned to continued treatment with the test drug or placebo (i.e., withdrawal of active therapy) (4). Randomization can be performed for all patients who are available to enter the withdrawal stage, or for a selected population such as responders to initial treatment. If possible, the withdrawal stage would be conducted in a double-blind fashion. The trial could start

with a single open-label treatment, but also with several active treatments. The initial treatment stage can be of any length. The withdrawal stage could be designed to have a fixed duration, or to use a time to event approach (e.g. relapse of depression) where the time of placebo exposure for a patient with poor response would be kept to a minimum. Treatment effects are assessed based on the differences between continued active treatment and placebo. 2.1 Application of the Approach There are various situations where the randomized withdrawal design can be useful. It is important to consider which effects are reflected by the treatment difference in the withdrawal stage. A situation that could be interpreted reasonably well would be characterized by a chronic, stable disease where a treatment improves the patient’s health status by alleviating signs or symptoms, but a major change in health status (e.g., curing the disease or considerable deterioration) is not expected. Upon withdrawal of the test drug, the patient’s response is thought to return to the placebo level. Under these conditions, similar results would be observed in a parallel–group. placebo–controlled, longterm efficacy trial and a randomized withdrawal trial. This may be the case for chronic pain or mild to moderate hypertension where a long-term placebo-controlled trial would be difficult to perform. A randomized withdrawal trial is an alternative with considerably reduced placebo exposure. In non-chronic major depression, the design can be used to evaluate the treatment effect of a test drug in the continuation/ maintenance phase. Following treatment of acute symptoms, responders are randomized to continue with test drug treatment or to receive placebo. The reappearance of symptoms (relapse and/or recurrence) is then compared between treatments (3). Randomized withdrawal can also be employed for dose finding. In a dose-response study of a treatment for hypertensive children (5), patients were assigned to a low, middle, and high dose group at the beginning of a 2-week double-blind, randomized, dose-response period. In the high dose group,

DESIGNS WITH RANDOMIZATION FOLLOWING INITIAL STUDY TREATMENT

3

Active

Randomized withdrawal design

Placebo Randomization

Active

Randomized delayedstart design

Active Active

Placebo Randomization

Placebo

Re-Randomization

Active Active

TFI Placebo Re-Randomization

Randomized Re-treatment design

Placebo

TFI

Active

Randomization

Figure 1. Designs for studies with randomization following initial study treatment. TFI: treatment-free interval.

the initial dose was doubled after 3 days unless adverse events or hypotension were observed. On completion of the dose-response stage, patients entered a 2-week double-blind withdrawal stage, randomized to either their last dose or placebo. The study was useful in assessing effectiveness of the dose groups and the dose-response relationship, and the value of up-titration. The randomized withdrawal design is also suitable to assess how long a therapy should be continued (e.g. post-mastectomy adjuvant treatment for women with operable breast cancer (6)). Multiple withdrawal stages can be planned if more than two therapy durations are studied.

2.2 Advantages and Limitations of the Design An advantage of the randomized withdrawal design is that placebo exposure can often be restricted to a minimum (e.g. in the investigation of long-term efficacy). It is also a way to address scientific questions regarding a multistage course of therapy (e.g. in depression), which would be very difficult to achieve in a placebo-controlled, parallel-group trial. It is important to consider phenomena which may interfere with the interpretation of the results of the withdrawal stage. First, carry-over and rebound effects (exacerbation of symptoms beyond pre-treatment levels upon withdrawal of active treatment) may result in erroneous conclusions on the treatment effect. Second, as the patient can receive different treatments during the

4

DESIGNS WITH RANDOMIZATION FOLLOWING INITIAL STUDY TREATMENT

course of the study, his or her expectation and anticipation of the treatment effect might differ from that in a standard trial. This could prompt the patient to report the experience with the drug differently. In addition, partial unblinding could occur as the patient might perceive the absence of specific characteristic test drug effects when switched to placebo. As another issue, it is to be expected that primarily patients with a good prognosis are moving forward into the withdrawal part of the study, and efficacy and safety assessments are performed on a selected population, where safety issues may tend to be under-reported. 2.3 A Modified Version of a Randomized Withdrawal Design A modified version of a randomized withdrawal design was proposed by Leber (7) to evaluate treatments of a progressing disease (e.g., Alzheimer’s and Parkinson’s disease). There has been increasing interest in determining whether these have, in addition to symptomatic (short-term) effects, the potential to slow the progression of the disease (disease-modifying or long-term effects). Patients are initially randomized to receive the test drug or placebo. After a period of treatment designed to be long enough that disease-modifying effects are manifested, patients who received the test drug in the first stage are randomized to either continued test drug or placebo, and patients randomized to placebo in stage I remain on placebo. The second stage extends over a period of time of sufficient length for symptomatic effects to have dissipated. If any treatment effect evident after the initial stage (short-term and longterm effects) persists (the long-term effect), it will be reflected in the treatment difference of the test drug–placebo and placebo–placebo sequence at the end of the withdrawal stage. The symptomatic effect can be obtained by subtracting the disease-modifying effect from the treatment effect at the end of the first stage (comprises both effects), under the assumption that the symptomatic and disease-modifying effect are approximately additive. The test drug–test drug sequence is not used to distinguish between short-term and long-term effect, but it is essential to keep the trial blinded in both stages, and allows

the estimation of the combined short-term and long-term effect of the second stage. 3 RANDOMIZED DELAYED-START TRIALS A randomized withdrawal design suitable to evaluate treatments of progressive diseases such as Parkinson’s or Alzheimer’s disease has been described above. The randomized delayed-start design is another design tailored to distinguish between a drug’s effect in relieving symptoms and on the course of the disease. In the randomized delayed-start design, patients are initially randomized to placebo or test drug. Those who received placebo in the first stage are re-randomized to placebo or the test drug, while the test drug patients remain on test drug (see Figure 1). Patients in the placebo–test drug sequence are expected to improve in the second stage. If the test drug had no disease-modifying effect, then the patients in the test drug–test drug sequence would have no advantage over the patients in the placebo–test drug sequence, and both groups would be expected to reach the same level of response at the end of the second stage. Thus, any difference by which the patients in the placebo–test drug sequence stay below the stage II level of response in the test drug–test drug sequence is considered to reflect the disease-modifying effect. As the treatment effect of the first period comprises both types of effects, the symptomatic effect can be determined as explained above. 3.1 Advantages and Limitations of the Design As an advantage, the design is suitable to distinguish between symptomatic and diseasemodifying treatment effects, maintaining blinding throughout. It is also suited to determine the second stage effect (comprising the disease-modifying and symptomatic effect). The design has an advantage over the randomized withdrawal design with three sequences in that there is a higher chance of receiving the test drug in the second stage, with a potentially favorable impact on patient retention. On the other hand, both designs involve an overall prolonged placebo exposure, which may give rise to ethical issues if the drug is known or expected to modify the

DESIGNS WITH RANDOMIZATION FOLLOWING INITIAL STUDY TREATMENT

course of the disease. Also, disease-modifying effects currently have to be inferred from performance scores, and the involved differences seem to be small due to an imperfect relationship between the underlying pathologic state and performance. This may require large sample sizes (8). 4

RANDOMIZED RE-TREATMENT TRIALS

A randomized re-treatment design can be used to evaluate treatments for chronic diseases with fluctuating symptoms like irritable bowel syndrome (IBS). The majority of IBS patients report episodes of mild to moderate symptoms, which alternate with periods of low symptom intensity (9). Intermittent treatment (short-course administration for a predetermined time period after symptom recurrence) may be suitable for these patients. The design is directed at evaluating the initial and repeated test-drug effect. The trial should start with a treatmentfree baseline period. This serves to record baseline disease activity, to familiarize the patient with data collection methods like paper or electronic diaries, and to ensure that patients start initial treatment with a similar intensity of symptoms. To evaluate first and repeated use of a test drug, patients are randomized to test drug or placebo for a short-term period, followed by a variable period off treatment with rerandomization to placebo or test drug in a second cycle when symptoms warrant restarting medication (10). Restarting treatment only upon symptomatic relapse is an essential condition; patients could otherwise be at different phases of their symptom cycles if the duration of the treatment-free interval was fixed. The initial and repeated test drug effect can be evaluated based on the treatment difference in the initial period and the treatment difference in the repeated treatment period for those patients who received the test drug as the initial treatment. The repeated treatment results of patients initially assigned to placebo are not used to assess the effects of repeated treatment with the test drug; these patients could therefore be switched to test drug in the second cycle in a blinded fashion (see Figure 1) to improve patient retention.

5

A modification of the design has been applied in a study to evaluate treatment efficacy in IBS with constipation (11). 4.1 Advantages and Limitations of the Design An advantage of the randomized re-treatment design lies in the limited exposure to placebo. Further, all patients who are included in the evaluation of repeated treatment effects receive the test drug as initial treatment. For this reason, the treatment groups at the start of second therapy are much more likely to be comparable than in the standard parallelgroup design, where differential dropout patterns could emerge after initial test drug or placebo treatment. As a disadvantage, sample size tends to be large as only patients initially receiving the test drug contribute to the evaluation of the repeated test-drug effect. Unbalanced randomization can relieve this disadvantage. A concern has been raised that unblinding of the first-stage treatments could occur due to different symptom profiles and different times to recurrence after withdrawal of the test drug or placebo (12) Although not used for treatment comparisons, symptoms should therefore also be monitored and recorded during the treatment-free interval to assess this concern. 5

ENRICHMENT

In some randomized withdrawal trials or retreatment trials, the second stage following initial treatment is designed to include an enriched population, e.g. responders to initial treatment. This may be preferred for ethical reasons, or the scientific question may be defined for responders (like relapse and remission in depression). Enrichment could also mean selecting nonresponders to initial treatment if the efficacy of alternative treatments is being evaluated for patients who do not benefit sufficiently from a specific initial treatment. Even in the absence of a planned enrichment, the second stage of a trial will likely be enriched with patients who can tolerate the drug and who experienced the initial treatment as beneficial. It is important to consider the impact of the selection on the observed

6

DESIGNS WITH RANDOMIZATION FOLLOWING INITIAL STUDY TREATMENT

treatment effect. If the outcomes of the initial and second treatment stage are positively correlated, as is often the case for the test drug–test drug sequence, then those with a better initial value (responders) will more likely have a better value in the second stage, resulting in a higher test drug response level. If the correlation was less strong in the test drug–placebo sequence (this is often plausible due to different treatments, but not necessarily always the case), then the treatment effect would be larger than in the unselected population. While this would be an advantage in terms of sample size requirements, it has to be weighed against the smaller number of patients who enter the second stage and are evaluated for the comparison of treatments. A larger sample size may be required especially if the enriched population includes not more than half of the unselected population. 6

CONCLUDING REMARKS

When a trial with randomization following initial study treatment is considered, it is useful to assess how various effects can influence the treatment difference seen in the second stage and how it might differ from what could be observed in a parallel-group trial (if one was feasible). These possible effects include major changes in the patient’s health status due to treatment, changes in the effect of treatment over time, potential unblinding by the patient’s experience with more than one treatment, and rebound effects and carry-over from the previous stage. The second stage should be planned to be long enough to ensure that carry-over does not influence the results. Taking this into account, trials with randomization after initial treatment can be specifically useful in the following situations:

Efficacy of re-treatment: randomized re-treatment design • Progressive diseases Distinguish between disease-modifying and symptomatic effects: randomized withdrawal design (three treatment sequences), randomized delayed-start design (three treatment sequences) Designs with two stages were discussed above. Trials with more than two stages can be designed to evaluate treatments in specific settings and populations. In a trial with three stages and outcome-driven rerandomization, patients whose benefit from a treatment was unsatisfactory could receive subsequent treatments (13). A randomized withdrawal design and a randomized delayed-start design, each with two periods and three treatment sequences, were proposed to distinguish between disease-modifying and symptomatic effects of treatments for progressing diseases. In contrast, the familiar two-treatment, two-period, crossover design is most useful to establish the symptomatic effects of a stable disease. Another difference is that it usually requires a washout period between the two periods and equal period lengths. A general two-period design with the four treatment sequences—test drug–test drug, test drug–placebo, placebo–test drug, placebo–placebo (complete two-period design [14])—includes the other designs and allows a check of the structural assumptions in addition to the determination of both types of effects. Another part of this general design, the treatment sequences test drug–test drug, test drug–placebo, placebo–test drug, can be used in the context of the one study versus two studies paradigm (15).

• Chronic, stable diseases

Long-term efficacy: randomized withdrawal design Dose finding, including titration: randomized withdrawal design • Diseases with a multistage treatment Efficacy in later treatment stages: randomized withdrawal design • Diseases with fluctuating symptoms

REFERENCES 1. W. Amery and J. Dony, A clinical trial design avoiding undue placebo treatment. J Clinical Pharmacol. 1975; 15: 674–679. 2. R. J. Temple, Special study designs: early escape, enrichment, studies in nonresponders. Commun Stat Theory Methods. 1994; 23: 499–531.

DESIGNS WITH RANDOMIZATION FOLLOWING INITIAL STUDY TREATMENT 3. J. G. Storosum, B. J. van Zwieten, H. D. Vermeulen, and T. Wohlfarth, Relapse and recurrence prevention in major depression: a critical review of placebo-controlled efficacy studies with special emphasis on methodological issues. Eur Psychiatry. 2001; 16: 327–335. 4. The European Agency for the Evaluation of Medicinal Products, Human Medicines Evaluation Unit, Committee for Proprietary Medicinal Products (CPMP). Note for Guidance on the Choice of Control Group in Clinical Trials. CPMP/ICH/364/96. January 2001. Available at: http:///www.emea.europa. eu/pdfs/human/ich/036496en.pdf 5. T. Wells, V. Frame, B. Soffer, W. Shaw, Z. Zhang, et al., A double-blind, placebocontrolled, dose-response study of the effectiveness and safety of enalapril for children with hypertension. J Clin Pharmacol. 2002; 42: 870–880. 6. H. J. Stewart, A. P. Forrest, D. Everington, C. C. McDonald, J. A. Dewar, et al., Randomised comparison of 5 years adjuvant tamoxifen with continuous therapy for operable breast cancer. The Scottish Cancer Trials Breast Group. Br J Cancer. 1996; 74: 297–299. 7. P. Leber, Slowing the progression of Alzheimer disease; methodologic issues. Alzheimer Dis Assoc Disord. 1997; 11 (suppl 5): S10–S21. 8. E. Mori, M. Hashimoto, K. R. Krishan, and M. Doraiswami, What constitutes clinical evidence for neuroprotection in Alzheimer disease. Alzheimer Dis Assoc Disord. 2006; 20: S19–S26. 9. B. Hahn, M. Watson, S. Yan, D. Gunput, and J. Heuijerjans, Irritable bowel syndrome symptom patterns. Dig Dis Sci. 1998; 43: 2715–2718. 10. The European Agency for Evaluation of Medicinal Products, Human Medicine Evaluation Unit, Committee for Proprietary Medicinal Products (CPMP). Points to Consider on the Evaluation of Medicinal Products for the Treatment of Irritable Bowel Syndrome. CPMP/EWP/785/97. March 19, 2003. Available at: http:/www.emea.europa. eu/pfs/human/ewp/078597en.pdf ¨ 11. J. Tack, S. Muller-Lissner, P. Bytzer, R. Corinaldesi, L. Chang, et al., A randomised controlled trial assessing the efficacy and safety of repeated tegaserod therapy in women with irritable bowel syndrome with constipation (IBS-C). Gut. 2005; 54: 1707–1730.

7

12. E. Corazziari, P. Bytzer, M. Delvaux, G. Holtmann, J. R. Malagelada, et al., Clinical trial guidelines for pharmacological treatment of irritable bowel syndrome. Aliment Pharmacol Ther. 2003; 18: 569–580. 13. S. M. Davis, G. G. Koch, C. E. Davis, and L. M. LaVange, Statistical approaches to effectiveness measurement and outcome-driven rerandomizations in the Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) studies. Schizophr Bull. 2003; 29: 73–80. 14. L. N. Balaam, A two-period design with t2 experimental units. Biometrics. 1968; 24: 61–73. 15. G. Koch, Commentaries on statistical consideration of the strategy for demonstrating clinical evidence of effectiveness—one larger vs two smaller pivotal studies. Stat Med. 2005; 24: 1639–1646.

FURTHER READING C. Dunger-Baldauf, A. Racine, and G. Koch, Retreatment studies: design and analysis. Drug Inf J. 2006; 40: 209–217. V. E. Honkanen, A. F. Siegel, J. P. Szalai, V. Berger, B. M. Feldman, and J. N. Siegel, A three-stage clinical trial design for rare disorders. Stat Med. 2001; 20: 3009–3021. M. P. McDermott, W. J. Hall, D. Oakes, and S. Eberly, Design and analysis of two-period studies of potentially disease-modifying treatments. Control Clin Trials. 2002; 23: 635–649.

CROSS-REFERENCES Carry-over Placebo-controlled trial Parallel-group design Randomization Unblinding

DIAGNOSTIC STUDIES

1

2

DIAGNOSTIC STUDIES

CARSTEN SCHWENKE

2.1 Guidelines for Diagnostic Studies

Schering AG SBU Diagnostics & Radiopharmaceuticals Berlin, Germany

Diagnostic studies are clinical trials with diagnostic objectives and outcomes that, in general, follow the same scientific and regulatory rules as clinical studies for any other medical product. The design and conduct of the clinical study as well as the statistical methods are regulated by all relevant guidelines like the ICH-E3, ICH-E6, ICH-E9, and ICH-E10. Nevertheless, some specific topics exist with regard to diagnostic studies, which are covered by special guidelines. The Food and Drug Agency (FDA) and the Committee for Proprietary Medicinal Products (CPMP) of the European Agency for the Evaluation of Medicinal Products (EMEA) provide guidance on the evaluation of medical imaging drugs and diagnostic agents. FDA’s draft guideline for industry (2) on ‘‘Developing Medical Imaging Drugs and Biological Products’’ and CPMP’s ‘‘Points to Consider on the Evaluation of Diagnostic Agents’’ (1) provide considerations on the methodology and data presentation along with explanations about how to define the gold standard and clinical endpoints.

INTRODUCTION

Diagnostic studies are performed to investigate the clinical usefulness of a new diagnostic procedure, for example, a new contrast agent for imaging or a new laboratory test to detect antibodies or antigens in blood. Sometimes, these studies are invented to substitute the standard procedure by an improved or less expensive procedure, which is not inferior in terms of efficacy and has some benefit compared with the standard procedure. Before starting a diagnostic study, first the unit for analysis should be defined in the terms ‘‘experimental unit’’ and ‘‘observational unit.’’ All general considerations on clinical studies, as proposed by ICH and other guidelines, apply to diagnostic studies also, but explicit guidelines also exist for diagnostic studies as described in Section 2.1 (1, 2). So-called ‘‘blinded reader’’ studies were introduced by these guidelines as part of diagnostic studies to assess the main efficacy objective in the clinical study. The clinical usefulness of a new diagnostic procedure is evaluated using three components: the reliability, the validity, and the add-on value. Reliability and validity are defined components that investigate, along with parameters for measurement and analysis tools, the performance of a diagnostic procedure. To measure the benefit of a new diagnostic procedure for the patient, the add-on value is often shown by an increased validity compared with a control. Throughout this article, the parameters of interest are described in the text, whereas the formal definitions are displayed separately in a table. The term ‘‘patient’’ is used for a participant of a clinical study regardless of the presence or absence of the disease under investigation.

2.2 The Cascade to Test the Performance of Diagnostic Procedures: Reliability and Validity In diagnostic studies, three parts of efficacy are of interest to claim the clinical usefulness of a new diagnostic procedure. The first part is the reliability (e.g., the rate of agreement of different raters on the same patient). The second part is the validity (i.e., the ability of the diagnostic procedure to measure what it is supposed to measure). As a third part, the procedure has to provide an add-on value as compared with the ability of diagnosing the disease without the new procedure. As an example, the add-on value of a contrast agent for an imaging device may be shown by comparing images with and without the contrast agent. For the validity of a diagnostic procedure, the true disease status has to be known. When no true status is available,

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

DIAGNOSTIC STUDIES

Kappa Parameter (chapt. 3.1)

Intra-class correlation coefficient Lin’s coefficient of concordance

Reliability Analysis (chapt. 3.2)

Bland-Altman plot McNemar’s test

“Truth” unknow

“Truth” known Sensitivity and Specificity Parameter (chapt. 4.1)

Accuracy

Analysis (chapt. 4.2)

Receiver Operator Characteristic (ROC)

Positive and negative predictive value

Validity

Likelihood ratio

Add-on value Figure 1. Cascade to test the performance of a diagnostic procedure

only the reliability in terms of agreement with a control and the add-on value can be used to show clinical usefulness. In Fig. 1, the cascade to conduct the testing of a new diagnostic procedure is graphically presented. The parameters and analysis tools for the evaluation of reliability and validity are described in Sections 3 and 4. For further reading on diagnostic studies, Knottnerus (3) provides more introductory explanations. 2.3 Evaluation of the ‘‘Truth’’ and Choice of Control A critical design issue in diagnostic studies is the determination of the truth provided by the so-called gold standard, standard of truth, or standard of reference (SOR). As nearly no procedure can provide 100% correct diagnoses, the terms ‘‘gold standard’’ and ‘‘standard of truth’’ are not commonly used any more as they imply an absolutely accurate diagnosis. Today, the term ‘‘standard of

reference’’ is preferred to name a method, which provides the so-called true disease status. In general, the SOR should reflect the best available standard procedure but, in any case, has to be independent from the test procedure to avoid any bias in favor or disfavor of the test procedure. When a CT is under investigation, the standard of reference should be a method different from CT like pathology or MRI to obtain independent results. To avoid any bias, in addition, the results of the test procedure should be unknown when the SOR is evaluated, but also the results of the SOR should be unknown when the test procedure is performed. Examples for SORs are CT, MRI, histopathology, biopsies, ultrasound, intraoperative ultrasound, the final diagnosis, or the clinical outcome, obtained from a follow-up examination. Sometimes the best available method to obtain the truth is inappropriate and unethical, for example, performing a biopsy or histopathology on patients with suspected benign tumors and

DIAGNOSTIC STUDIES

suspected absence of any malignancies. In these cases, the SOR might be suboptimal, less valid, or less reliable. The consequence may be incorrect SOR diagnoses, which may lead to biased estimates for the number of true positive (TP) and true negative (TN) test results (for definitions of TP and TN, see section True Positives, True Negatives, False Positives, False Negatives) and so to a biased estimation of the validity of the procedure. This bias could become unavoidable in ethically performed clinical studies. Therefore, the SOR method has to be chosen in a way that it is ethical and appropriate for the patient and provides information closest to the truth. Any inappropriateness and weakness of the SOR has to be taken into account when interpreting study results. When the distribution of patients with weak SOR is similar across treatment groups, the underestimation may be also similar such that the absolute values of sensitivity and specificity show underestimation, but the estimation of the add-on value of the new diagnostic procedure in terms of relative difference between two groups remains appropriate. Therefore, controls are needed in diagnostic studies to evaluate the performance of a diagnostic procedure. In studies on contrast agents, one may use the images without the contrast agent as control to perform an intraindividual comparison. Also, active controls like other contrast agents, other diagnostic procedures (e.g., CT versus MR) or placebo- or vehicle-controls may be used. The type of control depends on the primary objective of the respective study. In general, the control has to be chosen appropriately to enable the assessment of the clinical usefulness of the procedure under investigation. Therefore, the control should always reflect the current medical practice. To maximize the precision of the estimates and to reduce the variability a paired study design is preferable where the test and the control procedures are performed in the same patient. 2.4 Experimental Unit and Observational Unit In general, an experimental unit is the smallest unit to which one treatment is applied (e.g., a patient who receives the contrast

3

agent). The outcome is obtained from the observational unit, which is the smallest unit for which an observation can be made. When a treatment is applied to a patient and a single observation is made for this patient, then the observational unit and the experimental unit are the same. An example is the treatment of headache, where a drug is given to the patient and the pain relief is observed at a fixed point in time. In diagnostic studies, the experimental unit and the observational unit are often not the same. The diagnostic procedure is applied to the patient as experimental unit, but the observational units are several regions of interest within the patient like different organs, different vessels, or different lesions within an organ. An example is the detection of tumors in the liver by an imaging medical device with contrast agent. Here, the patient is the experimental unit as he receives the contrast agent, but the observational units are multiple focal liver lesions. 2.5 Clustered Data in Diagnostic Studies The fundamental unit for statistical analysis of clinical studies is the observational unit rather than the experimental unit. Often, the observational units are highly correlated within the experimental unit. When, for example, several focal liver lesions were found for a patient and one lesion is a metastasis, then the probability may be high that the other lesions are also metastases. In longitudinal studies, where observations are made for the same subject at several points in time, the measurements are correlated, which has to be taken into account for the analysis. To provide a patient-based analysis and to obtain valid estimates for the error variance of the statistical parameters, the experimental unit is preferred as unit for analysis. A solution is to define the patient to be a cluster of multiple correlated observations. Appropriate statistical methods are available to obtain valid estimates for the treatment effect based on clustered data. Examples are the cluster-adjusted McNemar test (4, 5) and the adjusted X2 -test (6). 2.6 Blinded Reader Studies As stated earlier, for diagnostic studies, special guidelines (1, 2) apply. One particularity,

4

DIAGNOSTIC STUDIES

especially in imaging studies, is the evaluation of the efficacy by multiple independent readers who have to perform a blinded assessment. The blinded read is an artificial scenario to assess the performance of a diagnostic procedure and may not reflect the clinical routine. Nevertheless, it forces the readers to evaluate the images under objective conditions and helps to avoid any bias as a result of nonindependence or access to unblinded information like the medical history, age, or sex of the patient. The relevant guidelines (1, 2) state that the readers have to be ‘‘independent’’ (i.e., not yet involved in the development of the diagnostic procedure or affiliated with the sponsor of the trial or the institutions) where the study is conducted. The ‘‘blindness’’ of the readers is the unawareness of most of the information given in the study protocol (i.e., the treatment identity, demographics, the primary objective, clinical information on the patient, and other information that may influence the evaluation of the images). The limitation of providing very limited study and patientrelated information to the readers is introduced to force the reader to focus on the images alone. At least two blinded readers are requested by the FDA and CPMP (1, 2) to evaluate the images to investigate the interreader and intrareader variability and to allow for a wider generalization of the results. A profound selection and training of certified clinical specialists as blinded readers is needed to minimize the interreader variability.

3 RELIABILITY Reliability is introduced to measure the agreement of different methods. In general, a measurement lacks reliability when it is prone to random error. In the following, the term ‘‘method’’ is used as a synonym for the diagnostic procedure from which the observations are obtained and for which the agreement to another method is of interest. Examples for such methods are different diagnostic procedures, multiple readers of images, different laboratory tests, or different raters in psychological studies. Cohen’s Kappa κ, the intraclass correlation coefficient (ICC), Lin’s concordance coefficient rLin , and the Bland-Altman plot can be used to assess reliability. Statistical analyses of reliability may be based on the McNemar’s test (for further reading, see Reference 7). It should be noted that reliability is different from correlation. Correlation coefficients, like Pearson’s correlation coefficient for continuous data and Spearman’s rank correlation coefficient for ordinal and continuous observations, measure the linear and monotonic relationship of two methods, respectively, not the agreement. As shown in Fig. 2, even an optimal linear correlation does not imply a moderate or substantial agreement, but may result in a poor agreement. To estimate the agreement of multiple methods, it is clearly not sufficient to only calculate the correlation. Shoukri and Edge (5) and Bland and Altman (8) provide introductory explanations in more detail on reliability. Advanced readers are referred to Shoukri (9) and Fleiss (7, 10).

Method 1

1.0

Method 1

1.0

optimal correlation, poor agreement

optimal correlation, optimal agreement 0

0 0

1.0 Method 2

0

1.0 Method 2

Figure 2. Example for the correlation and agreement of two methods

DIAGNOSTIC STUDIES

3.1 Reliability Parameters/Measurements of Reliability 3.1.1 Kappa. In 1960, Cohen (11) proposed kappa for the measurement of the agreement of dichotomous outcomes from two methods and is based on a 2 × 2 contingency table. It should be noted that neither method is a standard of reference. Kappa compares the observed proportion of agreement in the diagonal cells (PO ) and the expected proportion of agreement based on chance alone (PE ). Both proportions can be calculated based on the counts of the 2 × 2 table. These counts are shown in Table 1. In addition, the agreement rates for the outcome ‘‘disease present’’ (Pdiseased ) and for the outcome ‘‘disease absent’’ (Pnon diseased ) can be calculated. The average of these rates is then the overall agreement rate (Pall ). The

formulas for kappa and the associated parameters can be found in Table 2. In Fleiss (7), the standard error and confidence interval is presented along with a description of the kappa statistic. For illustration, the calculation of kappa and the associated proportions is shown by an example. In a hypothetical study, 169 ultrasound images were assessed by two different readers, evaluating the presence or absence of cirrhosis in the liver. The results of the evaluation are shown in Table 3 with the absolute counts and percentages. From this 2 × 2 table, the observed and expected proportions as well as the proportions of diseased and nondiseased patients then are: PO = 0.70, PE = 0.62, Pdiseased = 0.40, and Pnon diseased = 0.80. Then, kappa = (0.7–0.62)/(1 − 0.62) = 0.21. After the

Table 1. 2 × 2 Table of Counts for Two Methods with Two Outcomes Method 1

Method 2 disease present disease absent

disease present disease absent

a c a+c

b d b+d

a+b c+d a+b+c+d=n

Table 2. Formulas for Kappa and Associated Parameters (7) Kappa: κ=

PO − PE , 1 − PE

PO =

a+d = observed agreement rate, n

PE =

[(a + c) · (a + b)] + [(b + d) · (c + d)] = expected agreement rate by chance, n2

a, b, c, d and n according to Table 1. Overall Proportion of Agreement: Pall = 0.5 · (Pdiseased + Pnot Pdiseased =

diseased ),

2a = proportion with outcome ‘‘disease present’’, 2a + b + c

Pnot diseased =

2d = proportion with outcome ‘‘disease absent’’, 2d + b + c

a, b, c, d and n according to Table 1.

5

6

DIAGNOSTIC STUDIES Table 3. 2 × 2 Table of an Ultrasound Evaluation of Two Readers Reader 2 Reader 1 cirrhosis present cirrhosis absent

cirrhosis present

cirrhosis absent

17 (10%) 17 (10%) 34 (20%)

34 (20%) 101 (60%) 135 (80%)

Table 4. Interpretation Rules for Kappa (12) <0.01 0.01–0.20 0.21–0.40 0.41–0.60 0.61–0.80 0.81–1.00

chance of agreement slight agreement fair agreement moderate agreement substantial agreement almost perfect agreement

calculation of kappa, the question of how to interpret the results develops. In 1977, Landis and Koch (12) proposed rules for the interpretation of kappa, which are shown in Table 4. For the ultrasound example, the two readers would be regarded as in fair agreement. With kappa = 1, an optimal agreement of two methods is concluded; for kappa > 0, the observed agreement is higher than the agreement rate by chance alone; for kappa = 0, the observed and the expected agreement rates are the same. When a kappa < 0 is found, then the observed agreement is worse than the agreement by chance. The expected agreement rate is dependent on the prevalence of the outcome (i.e., with a prevalence near 0 or 1, kappa can be low even when the observed agreement rate is high). Therefore, kappa may change with different prevalences of the disease, even when the observed agreement rate stays the same. This effect is illustrated by a hypothetical example with different prevalences (Tables 5 and 6). In both examples, the observed proportion of agreement is the same (i.e., PO = 0.870). With a prevalence of 65% in example 1 and 85% in example 2, the expected agreement rates of PE = 0.554 and PE = 0.766, respectively, differ, which leads to different estimates for kappa (i.e., kappa = 0.709 for example 1 and kappa = 0.444 for example 2). Kappa can also be calculated when more than two outcomes for each method are present

51 (30%) 118 (70%) 169 (100%)

(e.g., a pain score with 5 outcomes from ‘‘no pain’’ to ‘‘severe pain’’). Then the overall kappa is estimated as the weighted mean of individual kappas, which are based on the 2 × 2 cross tables for each pair of outcomes (for further reading, see References 7 and 9). 3.1.2 The Intraclass Correlation Coefficient. The intraclass correlation coefficient (ICC) was introduced by Bloch and Kraemer in 1989 alternatively to Cohen’s kappa and is a commonly used measurement of agreement for quantitative assessments. The ICC and kappa are equivalent and differ only by a term with the factor 1/n, where n is the number of patients under investigation, but the ICC can also be used for the analysis of continuous data. The ICC can be obtained by applying an analysis of variance. For the introduction of the ICC, the one-factorial linear model is used. Let Y be a single outcome for a patient and X be the underlying mean score of many repeated measurements for this patient (i.e., the expected or ‘‘true’’ score). When an experiment is conducted, Y can be found to be different from X for several reasons like misinterpretations of images by the reader or imperfect calibration of the test procedure. The difference of the observed score Y and the expected score X is expressed as error e in the classical linear model Y = X + e. The score X will vary across a population of patients with mean mX and variance s2 X . The mean of the error e will vary around zero with variance s2 e . The distributions of e and X are assumed to be independent, so that the variances of e and X are also independent. The variance of Y can then be expressed as the sum of variance components of X and e (i.e., s2 Y = s2 X + s2 e ). Then the ICC1 is the proportion of the variance component of X compared with the overall variance. With a decreasing error variance, the denominator increases, and so the agreement rate increases to a

DIAGNOSTIC STUDIES

7

Table 5. Example 1: Kappa Coefficient in Case of a Medium Prevalence of the Disease Method 2 Method 1 disease present disease absent

disease present

disease absent

60 5 65

8 27 35

68 32 100

Table 6. Example 2: Kappa Coefficient in Case of a High Prevalence of the Disease Method 2 Method 1 disease present disease absent

disease present

disease absent

80 5 85

8 7 15

maximum of 1, whereas an increasing error variance would lead to a decreased agreement rate to a minimum of 0. The ICC can therefore be interpreted as the proportion of variance, caused by the patient-to-patient variability and corrected for random error. In Table 7, the formulas are given along with the confidence interval for ICC1 (7, 13). For a sample size of at least 20 patients, the ICC2 can be equivalently expressed in terms of the within patient mean square (WMS) and the between patient mean square (BMS) obtained by an analysis of variance as shown in Table 8. Despite the name, the ICC measures the agreement of multiple methods, not the correlation in terms of some monotonic relationship between the methods, as it also takes into account, whether the methods assess on the same scale. As rule for interpretation, an ICC of at least 0.75 is suggested to be meaningful and of good reliability (14). Taking into account the sampling error, the ICC should be called meaningful when the lower limit of the 95% confidence interval for the ICC is at least 0.75. A shortcoming of the ICC is its dependency on the degree of variability within the sample as it is calculated on this variation. In Fleiss (10), further introductory explanations can be found; advanced readers are referred to Fleiss (7). 3.1.3 Lin’s Coefficient of Concordance. As an alternative to the ICC, Lin’s coefficient of concordance rL may be used, which measures

88 12 100

the degree of concordance between two methods M1 and M2 taking into account not only the variance of the two methods but also the means. In Table 9 (15), the formulas are shown for the coefficient and its variance. A rL = 1 indicates a perfect positive agreement, rL = 0 indicates no agreement, and a rL = − 1 indicates a perfect negative agreement (15). Like the ICC, Lin’s coefficient is dependent on the degree of variability within the sample. 3.2 Analysis of Reliability 3.2.1 Bland-Altman Plot. For the graphical investigation of agreement, the BlandAltman plot may be used (16). In the graph, the difference of the values of method 1 and method 2 is plotted against the average of method 1 and method 2. An example is shown in Fig. 3. The limits of agreement are defined as two times the standard deviation of the mean difference. The plot should be checked for any systematic pattern (e.g., when method 1 gives systematically three times higher values than method 2). If the mean difference is close to 0 and the single differences do not exceed the agreement limits, good agreement can be concluded. As a shortcoming, the Bland-Altman plot does not provide any quantitative measure of agreement, but provides a good graphical insight into the data. To overcome this shortcoming, the BradleyBlackwood procedure may be used, which is extensively described in Bartko (17).

8

DIAGNOSTIC STUDIES Table 7. Formulas for ICC1 and Associated Parameters (7, 13) ICC1 by Variance Components: Let Y = X + e be a classical linear model with observations Y, factor X and error e, sX 2 = variance component of factor X, se 2 = variance component of error term e. => ICC1 =

s2X s2X

+ s2e

.

95% Confidence Interval:
95% CI(ICC) = F=

 F/FL − 1 F/FU − 1 ; , with n + F/FU − 1 n + F/FL − 1

MSX , MSE MSX = mean square of X, MSE = mean square of e,

FU and FL = upper and lower points of the F-distribution,

method 1 - method 2

a−1 P(FL ≤ Fa(n−1) ≤ FU ) = 1 − α.

+ 2*SD

Mean

− 2*SD

Average of method 1 and method 2 Figure 3. Example for a Bland-Altman plot

3.2.2 McNemar’s Test. For the analysis of reliability in terms of hypothesis testing, McNemar’s test may be used. For dichotomous outcomes, McNemar’s test determines whether the probability of a positive test result is the same for two methods. McNemar’s test is based on the determination of the symmetry in the discordant pairs of observations (i.e., where the two methods provide different outcomes). The hypotheses and test statistic are shown in Table 10. In case

of clustered data (i.e., where several observations are obtained from the same subject like the detection of focal liver lesions in the same patient), the cluster-adjusted McNemar test (4, 5) should be used. For illustration, McNemar’s test is calculated for the ultrasound example (Table 3). Let X be reader 1 and Y reader 2, then PX = 34/169 = 20%, PY = 51/169 = 30%, and the SE(PX −PY ) = 0.0423. The test statistic then is χ 2 = (34 − 17 − 1)2 /51 = 5.02 and leads to

DIAGNOSTIC STUDIES

9

Table 8. Formulas for ICC2 and Associated Parameters (10) ICC2 by Mean Squares: Let BMS =

n 1  (xi − mi · p)2 be the mean square between methods · n mi i=1

and WMS =

n  xi · (mi − xi )2 1 · be the mean square within the methods n(m − 1) mi i=1

with mi = number of ratings on the ith patient with mean m =

n 1  mi , · n i=1

n 

m0 = m −

p=

(mi − m)2

i=1

n · (n − 1) · m

, xi = number of positive ratings on the ith patient,

n 1  · xi = mean proportion of positive ratings and nm i=1

n = number of patients, = > ICC2 =

BMS − WMS . BMS + (m0 − 1) · WMS

statistical significance as compared with the critical value of χ 1,0.05 2 = 3.841. 4

VALIDITY

The validity of a diagnostic procedure is substantial for its performance. It is defined as the accuracy or the grade of truthfulness of the test result, but can only be evaluated when the true disease status is known. The main question regarding the validity of a procedure is, therefore, whether it measures what it is intended to measure. Often, the observations for estimating the validity are on binary scale (i.e., with two outcomes) or dichotomized (i.e., reduced to two outcomes) to estimate validity parameters like sensitivity and specificity. When the observations are on ordinal scale (i.e., with more than two ordered outcomes like rating scores on 5-scale) or when the data are continuous (e.g., laboratory values of a PSA test as marker for prostate cancer), the area under the Receiver Operator Characteristic curve (ROC) and likelihood ratios are commonly used to analyze the validity of diagnostic

procedures. Most often, the primary efficacy objective in diagnostic studies is based on the validity (i.e., the diagnostic accuracy and the ability of the diagnostic procedure to discriminate the disease status from the nondisease status). The advantage of the new diagnostic procedure (i.e., the add-on value) is often based on an increased validity compared with some control. 4.1 Validity Parameters Basis for the definition of validity parameters is the 2 × 2-table defining true positive, false positive, false negative, and true negative outcomes (please refer to section ‘‘True Positives, True Negatives, False Positives, False Negatives’’). Then the prevalence of a disease is defined as the number of patients with a disease divided by all patients in the target population at a given and fixed point in time. The prevalence is called the prior or pretest probability for the proportion of patients with the disease. Validity parameters are sensitivity, specificity, and accuracy. The sensitivity is defined as the number of diseased patients

10

DIAGNOSTIC STUDIES Table 9. Formulas for Lin’s Concordance Coefficient and Associated Parameters (15) Lin’s Concordance Coefficient: rLin =

2sM1,M2 , s2M1 + s2M2 + (µM1 − µM2 )2 M1 = method 1, M2 = method 2, S2M1 of method M1, S2M2 of method M2, sM1,M2 = covariance of M1 and M2, µM1 = mean for method M1, µM2 = mean for method M2

Variance: Var(rLin ) =

1 n−2



 4 · r3Lin · (1 − rLin ) · u2 (1 − r2 ) · rLin · (1 − r2Lin ) 2 · r4Lin · u4 + , − r2 r r2

µM1 − µM2 r = Pearson’s correlation coefficient, u = √ sM1 · sM2

Table 10. Formulas for McNemar’s Test and Associated Parameters (10) McNemar’s Test: Hypotheses: Let M1 = method 1 and M2 = method 2, then H 0 : PM1 = PM2 vs. H 1 : PM1 = PM2 ⇔ H 0 : PM1 − PM2 = 0 vs. H 1 : PM1 − PM2 = 0, PM1 =

a+b , n

PM2 =

a+c , a, b, c, d and n according to Table 1. n

b−c = difference in proportions n √ b+c . with standard error SE(PM1 − PM2 ) = n

= > PM1 − PM2 =

Test Statistic:
χ12 =

|PM1 − PM2 | − 1/n SE(PM1 − PM2 )

2 =

(|b − c| − 1)2 with 1 degree of freedom. b+c

with positive test result divided by the overall number of patients with the disease. In statistical terms, the sensitivity can also be expressed as the probability for a positive test result under the condition that the disease is present. Therefore, the sensitivity is exclusively based on patients with the disease.

The specificity, on the other hand, is based on nondiseased subjects only. It is defined as the number of nondiseased patients with a negative test result divided by the overall number of patients without the disease. In statistical terms, the specificity is the probability of a negative test result under the

DIAGNOSTIC STUDIES

condition that the disease is absent. A combination of the sensitivity and specificity is the accuracy, where the information of both diseased patients and nondiseased patients is used. The accuracy is defined as the number of patients with either true positive or true negative test result divided by the overall number of patients under investigation. The disadvantage of the accuracy is its dependence on the prevalence of the disease: When the prevalence is low, the accuracy is mainly driven by the specificity (i.e., the number of true negative test results). When the prevalence is high, the accuracy is mainly influenced by the sensitivity (i.e., the number of true positive test results). As a consequence, a low specificity at a high prevalence and a low sensitivity at a low prevalence may be masked and missed. Other possibilities of combining sensitivity and specificity are the prevalence-independent receiver operating characteristic curve and the likelihood ratio, which are described later. Other validity parameters are the positive and negative predictive value. The positive predictive value (PPV) is the number of true positive test results divided by the overall number of positive test results, both true results and false results. It expresses the probability that a positive test result is correct, which is equivalent to the post-test probability for the disease when a positive test result is found. The negative predictive value (NPV) is the number of correct negative test results divided by the number of negative test results overall, which also is true only in cases where the prevalence in the sample is equal to the prevalence in the target population. In this case, the NPV is the probability for a negative test result being correct. The NPV is equivalent to the post-test probability that a negative test result is correct. Both PPV and NPV are measurements of how well the diagnostic procedure performs and can also be expressed in terms of sensitivity and specificity. In Zhou et al. (18), further descriptions can be found for situations in which the prevalences of the sample and the target population differ. In Table 11 (19), the formulas of all validation parameters and the associated parameters like the prevalence are given for a 2 × 2 table.

11

To demonstrate the calculation of the validation parameters, assume a study with 100 patients with suspected prostate cancer to test the performance of a new laboratory test to discriminate diseased and nondiseased patients. The test measures the PSA level in blood. A cut-off value is chosen to discriminate the diseased and nondiseased patients. Biopsy is used as standard of reference to obtain information on the true disease status. In Table 12, hypothetical results of such a study are in terms of a 2 × 2 table. The validation parameters and associated parameters can be calculated to investigate the performance of the new laboratory test on PSA levels. The results are: Se = 55/70 = 78.6%, Sp = 20/30 = 66.7%, Acc = 75/100 = 75%, PPV = 55/65 = 84.6%, and NPV = 20/35 = 57.1%. For further reading, see Panzer (20) as an introductory text; Zhou et al. (18) for advanced readers regarding validation parameters, their variances, and confidence intervals; and Agresti (21) for categorical data in general. 4.2 Analysis of Validity 4.2.1 Likelihood Ratio. As a measurement of the discrimination of a diagnostic procedure, the likelihood ratio combines information on the sensitivity and specificity simultaneously. For dichotomous data, the likelihood ratio for diseased patients (LR+) is the probability for a positive test result in diseased patients (i.e., the sensitivity) compared with nondiseased patients (i.e., 1-specificity). Analogously, the likelihood ratio for nondiseased patients is the probability for a negative test result in diseased patients compared with the patients without the disease. In Table 13, the likelihood ratios are given for dichotomous but also ordinal data with more than two categories along with the 95% confidence interval. With a likelihood ratio of 1, the diagnostic procedure adds no information, with increasing LR+ and decreasing LR−, the ability of the diagnostic procedure to discriminate between diseased and nondiseased patients increases. With the data from the PSA example (Table 12), the likelihood ratios for the PSA test would lead to LR+ = 0.786/(1 − 0.667) = 2.360 with 95% CI = [1.402; 3.975] and

12

DIAGNOSTIC STUDIES

LR− = (1 − 0.786)/0.667 = 0.321 with 95% CI = [0.192; 0.538]. In Knottnerus (3), more introductory explanations can be found.

4.2.2 ROC. A standard approach to evaluate the sensitivity and specificity of diagnostic procedures simultaneously is the Receiver Operating Characteristic (ROC) curve. For ordinal and continuous outcomes, a curve

is fitted to describe the inherent tradeoff between sensitivity and specificity of a diagnostic procedure. In difference to the accuracy, the ROC is independent from the prevalence of the disease. Each point on the ROC curve is associated with a specific diagnostic criterion (i.e., a specific cut-off value to classify the patient with regard to the presence/absence of the disease). The ROC was

Table 11. Formulas for the Validation Parameters and Associated Parameters for a 2 × 2 Table (19) True positives, true negatives, false positives, false negatives: see section ‘‘True Positives, False Positives, True Negatives, False Negatives’’. Prevalence: Pr =

TP + FN N

Sensitivity: Se = P(T + | D+) =  Standard Error: SE(Se) =

TP , where T + = test positive, D + = disease present. TP + FN Se · (1 − Se) (TP + FN)

95% Confidence Interval: 95% CI(Se) = Se ± 1.96 · SE(Se) Specificity: Sp = P(T − | D−) =  Standard Error: SE(Sp) =

TN , where T- = test negative, D- = disease absent. FP + TN Sp · (1 − Sp) (TN + FP)

95% Confidence Interval: 95% CI(Sp) = Sp ± 1.96 · SE(Sp) Accuracy: Acc =

TP + TN TP + FP + FN + TN

Positive Predictive Value: PPV = P(D + | T+) =

TP Pr ·Se = TP + FP Pr ·Se + (1 − Pr) · (1 − Sp)

Negative Predictive Value: NPV = P(D − | T−) =

TN (1 − Pr) · Sp = TN + FN (1 − Pr) · Sp + Pr ·(1 − Se)

Table 12. An 2 × 2 Table for an Hypothetical Clinical Study in Patients with Suspected Prostate Cancer

Test result T + (test positive) T − (test negative)

Standard of reference D+ D− (disease present) (disease absent) 55 15 70

10 20 30

65 35 100

DIAGNOSTIC STUDIES

13

Table 13. Formulas for the Likelihood Ratio and Associated Parameters (3) Likelihood Ratio for data with X categories (X > 2): likelihood ratio for outcome X: LRX =

P(X| D+) , P(X| D−)

P(X— D+) = probability for outcome X in diseased patients, P(X— D−) = probability for outcome X in non-diseased patients. Likelihood Ratios for dichotomous data: likelihood ratio for diseased patients: LR+ =

P(T + | D+) Se = , P(T + | D−) 1 − Sp

likelihood ratio for non-diseased patients: LR− =

1 − Se P(T − | D+) = , P(T − | D−) Sp

P(T + — D +) = probability for a positive test result in diseased patients, P(T + — D −) = probability for a positive test result in non-diseased patients, P(T − — D +) = probability for a negative test result in diseased patients, P(T − — D −) = probability for a negative test result in non-diseased patients, Se = sensitivity, Sp = specificity. 95% Confidence Interval for likelihood ratio:  95%CI(LRX ) = exp ln

 p1 ± 1.96 · p0

 1 − p1 1 − p0 + , p1 · n1 p0 · n0

p1 = P(X—D +) based on sample size n1 of diseased patients, p0 = P(X—D −) based on sample size n0 of non-diseased patients.

developed in the signal detection theory during World War II for the analysis of radar images and was adopted to medical statistics in the 1960s and 1970s for the analysis and interpretation of diagnostic procedures where signals on images or screens were to be evaluated. For the estimation of a ROC curve, first the sample is divided into the groups truly diseased and truly not diseased patients. These groups have different distributions of the outcomes of the diagnostic procedure (see Fig. 4). Assuming that higher test results indicate the disease, the test procedure is set to ‘‘positive’’ when the outcome is equal or

higher than the cut-off value and to ‘‘negative’’ when the result is lower than the cut-off value. Each possible cut-off value c represents an inherent tradeoff between sensitivity and specificity and results in a 2 × 2 table. For the ROC analysis, the outcome of a diagnostic procedure has to be at least ordinal (see Table 14). In the example, the image quality is measured on a score from 1 to 10, the cut-off value dichotomizes the outcome so that, for example, scores 1 to 5 would lead to the outcome ‘‘poor image’’ and scores 6 to 10 would lead to the outcome ‘‘good image.’’ The discriminatory accuracy of a diagnostic procedure can be represented graphically

14

DIAGNOSTIC STUDIES

Cut-off value c Test negative

Test positive

TN

TP

not diseased patients

diseased patients FN

FP

score of diagnostic procedure

Figure 4. Distribution of patients with and without disease with cut-off value Table 14. Outcome Categories and True Disease Status in a 2 × C Table Truth Test result categories

D + (D = 1): disease present

D − (D = 0): disease absent

n11 n12 ... n1K n1

n01 n02 ... n0K n0

1 2 ... K

Sensitivity

1.0

0 0

1.0 1-Specificity

Figure 5. Example of a ROC curve

by a ROC curve (see Fig. 5), where the sensitivity (i.e., the rate of true positive test results) is displayed on the vertical axis and 1-specificity (i.e., the rate of false positive results) on the horizontal axis. If the ROC hits the upper-left corner, an optimal differentiation between diseased and not diseased patients (i.e., the true positive rate = 1 and

n.1 n.2 ... n.K N

the false negative rate = 0) is achieved. At the bisector of the graph, the discrimination of the diagnostic procedure is equal to chance. For c = + ∞ (plus infinity), all patients are set as ‘‘not diseased’’; for c = − ∞ (minus infinity), all patients are set as ‘‘diseased.’’ The advantages of the ROC analysis are that the curve is invariant with regard to the scale of the diagnostic procedure itself and is invariant to monotonic transformations. These advantages enable scientists to compare diagnostic procedures with different scales based on ROC curves. ROC curves are, in addition, independent from the prevalence of the disease. The likelihood ratio can be extracted from the ROC curve as the slope of the tangent line at a given cut-off value. As a measurement of the validity, the area under the ROC curve (AUC) summarizes the sensitivity and specificity over the whole range of cut-off values. An AUC of 0.5 indicates that the procedure is equal to chance alone, whereas a value of 1 indicates a perfect discrimination of ‘‘diseased’’ and ‘‘not diseased’’ patients. The AUC is limited to the interval [0, 1]. In practice, the AUC should be at least 0.5. Values below 0.5 lead to a diagnostic procedure that would be worse than chance

DIAGNOSTIC STUDIES

1.0

15

Sensitivity

Sensitivity

1.0

AUC = 100%

AUC = 50%

0

0 0

1.0

1.0

0

1-Specificity

1-Specificity

Figure 6. Examples of AUCs

alone. In general, the AUC can be interpreted as the average sensitivity across all values of the specificity and vice versa. For the assessment of a diagnostic procedure, in addition to the AUC, the ROC curve should also be provided and taken into consideration to check whether the curves of the test and control procedure cross or whether one procedure is consistently superior to the other (see Fig. 6). For further reading, see Dorfman (22), Hanley and McNeil (23), and especially Zweig and Campbell (24) as an introductory text and Zhou et al. (18) for advanced readers. REFERENCES 1. CPMP, CPMP points to consider: points to consider on the evaluation of diagnostic agents, Committee for Proprietary Medicinal Products (CPMP) of the European Agency for the Evaluation of Medicinal Products (EMEA), CPMP/EWP/1119/98, 2001. 2. FDA Draft Guideline, Developing Medical Imaging Drugs and Biological Products. Washington, DC: FDA, 2000 3. A. J. Knottnerus, The Evidence Base of Clinical Diagnosis. London: BMJ Books, 2002. 4. M. Eliasziw and A. Donner, Application of the McNemar test to non-independent matched pair data. Stat. Med. 1991; 10: 1981–1991. 5. M. M. Shoukri and V. Edge, Statistical Methods for the Health Sciences. Boca Raton, FL: CRC Press, 1996. 6. A. Donner, Statistical Methods in Ophthalmology: An Adjusted Chi-Square Approach. Biometrics, 1989; 45(2): 605–611. 7. Fleiss1981.

8. J. M. Bland and D. G. Altman, Measuring agreement in method comparison studies. Stat. Meth. Med. Res. 1999; 8: 135–160. 9. M. M. Shoukri, Measures of Interobserver Agreement. Boca Raton, FL: CRC Press, 2003. 10. Fleiss1999. 11. J. Cohen, A coefficient of agreement for nominal scales. Educ. Psycholog. Measur. 1960; 20: 37–46. 12. J. Landis and G. Koch, The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174. 13. Searle1992. 14. L. G. Portney and M. P. Watkins, Foundations of Clinical Research: Applications to Practice. Norwalk, CT: Appleton & Lange, 1993. 15. L. I. Lin, A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989; 45: 255–268. 16. J. M. Bland and D. G. Altman, Statistical method for assessing agreement between two methods of clinical measurement. Lancet 1986; i: 307–310. 17. J. J. Bartko, General methodology II. Measures of agreement: a single procedure. Stat. Med. 1994; 13: 737–745. 18. X-H. Zhou, N. Obuchowski, and D. K. McClish, Statistical Methods in Diagnostic Medicine. New York: John Wiley and Sons, 2002. 19. L. Edler and C. Ittrich, Biostatistical methods for the validation of alternative methods for in vitro toxicity testing. ATLA 2003; 31(Suppl 1): S5–S41. 20. R. J. Panzer, E. R. Black, and P. F. Griner, Diagnostic Strategies for Common Medical Problems. Philadelphia, PA: American College of Physicians, 1991.

16

DIAGNOSTIC STUDIES

21. A. Agresti, An Introduction to Categorical Data Analysis. New York: John Wiley & Sons, 1996. 22. D. D. Dorfman, Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals rating method data. J. Math. Psychol. 1969; 6: 487–496. 23. J. A. Hanley and B. J. McNeil, The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143: 29–36. 24. M. H. Zweig and G. Campbell, Receiveroperating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin. Chem. 1993; 39/4: 561–577.

FURTHER READING P. Armitage and G. Berry, Statistical Methods in Medical Research. 3rd ed. Oxford: Blackwell Science, 1994.

DISCRIMINANT ANALYSIS

Another important task in clinical medicine is the prediction of a diagnosis for a certain patient given a set of p variables. In medical practice, situations often exist where the definitive diagnosis can only be made after a large number of clinical investigations, not seldom only after autopsy. What is needed here is a statistically-supported decision, taking into consideration available features x = (x1 , x2 , . . . , xp ) such as clinical or laboratory-type observations that can be accessed without too much inconvenience to the patient and without high cost. In a first (training) step of constructing a rule of prediction, one knows the class membership of patients (even from autopsy) and tries to ‘‘repeat’’ the diagnostic process by means of the trained prediction rule using only the available features x mentioned. If successful, further patients with unknown class membership can then be allocated (diagnosed) on the basis of the rule obtained using only the p available features x1 , . . . , xp (prediction step).

KLAUS-D. WERNECKE The Humboldt University of Berlin Charit´e —University Medicine Berlin Berlin, Germany

1

INTRODUCTION

Classification problems occur in a wide range of medical applications in which n objects (e.g., patients) are to be assigned to K certain classes (e.g., diagnoses, tumor classes, lesion types) on the basis of p observations x = (x1 , x2 , . . . , xp ) (e.g., symptoms, complaints, intraocular pressure, blood pressure, X-ray, histology, grading) on these patients. As a result of a set of p ≥ 1 observations (features or variables) for every patient, the corresponding statistical methods are designated as multivariate ones. Given a set of data from n patients in p variables, one task is the identification of groups, classes, or clusters of patients that are ‘‘similar’’ (mostly in terms of a distance) with respect to the variables x1 , . . . , xp within the classes and different between the classes. The question here is to give a description of (diagnostic) patient groups on the basis of the observed variables x1 , . . . , xp .

Example 2: Diagnosis of neuroborreliosis burgdorferi The correct diagnosis of neuroborreliosis burgdorferi is known to be extremely difficult in children. In the university hospital of Graz, Austria, various clinical outcomes were examined for their qualification to predict this diagnosis. Two patient groups with a known diagnosis served as a starting point of the investigations:

Example 1: Clustering with parameters from blood tests The features x1 : pH, x2 : bicarbonate, x3 : pO2 , x4 : pCO2 , x5 : lactate, x6 : glucose, x7 : sodium, x8 : potassium, x9 : chloride, x10 : carbamide, x11 : creatinine, x12 : content of serum iron, and x13 : content of alkaline phosphatase should be used for a description of diagnostic classes such as hepatitis, liver cirrhosis, obstructive jaundice, or healthy persons (1). Figure 1 shows a scatter-plot with 20 (numbered) patients using the two variables, x12 and x13 , respectively. Groups of patients, ‘‘similar’’ in x12 , x13 , and corresponding to the diagnoses in question, are bordered (H: hepatitis, LC: liver cirrhosis, OI: obstructive jaundice, HP: healthy persons).

• Group 1 (LNB): 20 patients suffering

from liquor-manifest neuroborreliosis (positive B. burgdorferi—CSF-titer— definitive neuroborreliosis) • Group 2 (NNB): 41 patients without neuroborreliosis (control group) (CSF: cerebro spinal fluid). The following features were examined: • Cephalea, Stiff Neck, Paresis, CSF cells, CSFigg, CSFalb, CSFprot, SBigg (SB: serum binding, igg: immune globulin g, alb: albumin, prot: protein).

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

DISCRIMINANT ANALYSIS

4.0 17

3.5

8

3.0

Content of serum iron [mg/l]

2.5

LC

5 18

2.0 1.5

H

16 7 10 14

3 HP

20

6 1.0 .5

OI

2

4 9

11

19 1 15

13

12

0.0 0 1 2 3 Content of alkaline phosphatase [U/I]

It was unknown, to which extend certain features and combinations of features are important for the diagnosis and which variables may be omitted for further diagnostic purposes (2). Following the remarks above, two principal tasks exist in classification (3): clustering (also known as unsupervised learning, classification, cluster analysis, class discovery, and unsupervised pattern recognition) and discrimination (also designated as supervised learning, classification, discriminant analysis, class prediction, and supervised pattern recognition). In the first case, the classes are unknown a priori and need to be discovered from the given observations. Then, the procedure consists in the estimation of the number K of classes (or clusters) and the determination of the classes in connection with the assignment of the given patients to these classes. This type of problem will not be considered in the following. However, in the second case, the existence of K distinct populations, groups, or classes is known a priori and a random sample has been drawn from each of these classes (for reasons of simplification, it is restricted to K = 2 classes). The given data are used to construct a classification rule (or classifier), which allows an allocation or prediction to

4

5

Figure 1. Visualization of the patients in a scatter-plot using blood tests

the predefined classes on the basis of the given observations x1j = (x1j1 , x1j2 , . . . , x1jp ), j = 1, . . . , n1 from class C1 , and x2j = (x2j1 , x2j2 , . . . , x2jp ), j = 1, . . . , n2 from class C2 , respectively. In this procedure, the given observations (with known class membership)— the socalled training sample {x} = {x11 , . . . , x2n2 }— serve for the construction of the classifier (training step) that forms the basis for a later allocation of patients with unknown class membership (prediction step). A close relationship exists between the two classification problems. On the one hand, the classes discovered from clustering can be used as a starting point for discrimination. On the other hand, in discriminant analysis, an evaluation of the homogeneity of the predefined classes can be achieved by a clustering of the data. 1.1 Exploratory Data Analysis Before applying a statistical procedure like discriminant analysis, it is always important to check the data with respect to the distributions of the given variables x1 , . . . , xp or to look for outliers (particularly when using procedures with serious preconditions on the data, for example, see Section 4).

DISCRIMINANT ANALYSIS

3

3000 NNB LNB

2000

1000

Figure 2. Data on SBigg and CSFprot for patients from LNB and NNB, respectively

Sbigg [mg/l]

0

−1000

0 20 40 CSFprot [mg/l]

Example 2 (continuation): Diagnosis of neuroborreliosis burgdorferi The next step should be a visualization of the data in order to judge whether a discrimination is worthwhile, which can be done by means of two-dimensional scatter-plots using some appropriately selected variables from the given set of features. Figure 2 shows the scatter-plot of the observations in the two classes LNB and NNB with respect to SBigg and CSFprot. In case of many given variables, a scatter-plot matrix can be helpful, but it may be principally difficult to select features of possible importance for discrimination (the general problem of feature selection will be considered in Section 4.2). One possibility of visualizing the data as a test of their suitability for discrimination is a transformation [under the criterion of minimal loss of information such as principal component analysis; see, among others, Ripley, (4)] of the given observation vectors x = (x1 , x2 , . . . , xp ) into vectors with lower dimension and their visualization in a two- or three-dimensional space. Figure 3 shows a scatter-plot with the first two principal components of the neuroborreliosis data. Third, one might test for a difference between the two groups of patients by a multivariate analysis of variance (MANOVA)

60

80

100

120

140

160

using the Lawley-Hotelling test statistics [e.g., Morrison (5)]. Whereas discriminant analysis allows an individual allocation for every single patient to one of the given classes, the MANOVA proves the hypothesis of differences in the middle between the classes. The analysis results in a significant difference between the classes LNB and NNB with
ni respect to the mean vectors xi = ( n1 j=1 xij1 , i
ni . . . , n1 x ); i = 1, 2. Although this outj=1 ijp i come may be interesting regarding the problem of the neuroborreliosis diagnostics from a scientific point of view, an affected patient is certainly not much interested in a statement for a group of patients than in the definitive diagnosis for himself or herself. In other words, one is looking for a decision rule that allows for an individual allocation of a given patient to exactly one of the classes (diagnoses) in question.

1.2 Fisher’s Approach R. A. Fisher (6) proposed a procedure for finding a linear combination of the variables x1 , x2 , . . . , xp that gives a separation ‘‘as good as possible’’ between the two classes C1 , C2 . Moreover, Fisher (6) suggested that such a

4

DISCRIMINANT ANALYSIS

3 2

Second Principal Component

1 0 −1 −2

NNB LNB

−3 −4 −2

−1

0

1

2

3

First Principal Component

linear combination can be obtained by a multiple regression approach zj = λ0 +

p 

λl xjl + j ,

j = 1, . . . , n,

l=1

n = n1 + n2 where  j is the jth error term that denotes the difference between the actual observation zj and the average value of the response z. λ0 , and λ1 , . . . , λp are unknown parameters that have to be estimated. Fisher (6) took zj = (−1)i ni /n if xj ∈ Ci , i = 1, 2 as indicators for the group membership. With the least-squares estimations of λ0 , and λ1 , . . . , λp from the equation above, one can calculate a corresponding predicted value zˆ j for every observation vecz1 = tor x
j = (xj1 , . . . , xjp ), which means that
1/n1 j zˆ j for all xj ∈C1 and z2 = 1/n2 j zˆ j for all xj ∈C2 , respectively. A plausible rule for an (individual) prediction of a given observation xj to one of the two given classes is then the allocation to that class Cm with the minimal distance to the corresponding mean: zˆ j − zm = mini [ˆzj − zi ](i = 1, 2).

4

Figure 3. Dimension reduction with principal component analysis. The first two principal components for patients from the LNB and NNB group are shown

Table 1. Classification Table for the Neuroborreliosis Data Using Multiple Regression Resubsitution Observed (true) class LNB NNB Total

Predicted class LNB NNB 14 0 14

6 41 47

Total 20 41 61

Example 2 (continuation): Diagnosis of neuroborreliosis burgdorferi Using this approach, the following results are obtained for the neuroborreliosis data, arranged in a so-called classification table (Table 1), in which for every observation the prediction has been compared with the (observed) true class of origin (that means a resubstitution of every observation). Thus, altogether 6 wrong predictions were made, which stands for an error rate for this decision rule of 9.84%.

2 ALLOCATION RULES AND THEIR ERROR RATE As already mentioned, the example is restricted to K = 2 classes to simplify matters.

DISCRIMINANT ANALYSIS

Nevertheless, all the methods to be described can be generalized to K > 2 easily. In the neuroborreliosis example, a decision rule is sought that allows for an (individual) prediction or allocation of a given patient to one of the two diagnoses in question. Generally, R(x) is defined as a prediction rule or an allocation rule on the basis of p observations x = (x1 , x2 , . . . , xp ) of a patient, where R(x) = i implies that the patient is to be assigned to the ith class Ci (i = 1, 2). The rule R(x) divides the features space into two disjunctive regions R1 and R2 , and, if x falls in the region Ri , the patient is allocated to class Ci . The probability of a wrong allocation from class C1 to class C2 for any patient (from class C1 ) is given by P12 (R(x)) = P(R(x) = 2|x ∈ C1 ) (which is the probability of an allocation to class C2 provided the patient belongs to class C1 ). The probability of a wrong allocation from class C2 to class C1 is defined analogously by P21 (R(x)) = P(R(x) = 1|x ∈ C2 ). Supposing patients in class Ci have feature vectors x = (x1 , . . . , xp ) with a class-specific p-dimensional distribution F i (x) and a corresponding density function f i (x)(i = 1, 2), then P12 (R(x)) and P21 (R(x)) can be expressed by  P12 (R(x)) = f1 (x)dx and P21 (R(x)) R2

 =

f2 (x)dx

respectively.

R1

As Fig. 4 illustrates for p = 1 and normal distributions F i (x) with densities f i (x),  integral R f1 (x)dx, for example, defines the 2 probability of patients from class 1 to have observed values x∈R2 (i.e., within the region R2 ), and it is nothing more than the area under the curve f 1 (x) over R2 (hatched areas in Fig. 4). The overall probability of misallocations, also known as (overall) misallocation rate, or (overall) error rate (7), can then be calculated to E(R(x)) = P12 (R(x)) + P21 (R(x))   = f1 (x)dx + f2 (x)dx R2

R1

5

Figure 4 shows the overall probability of misallocations as the sum of the two hatched areas in case of two normal distributions f i (x)(i = 1, 2) and p = 1. 2.1 Estimations of Error Rates The practical use of a discriminator depends on the quality by which newly appearing patients of unknown class membership can be allocated on the basis of the allocation rule R(x). Suppose a training sample {x} such as introduced in Section 1, estimate the overall probability of misallocations or the overall error rate connected with R(x). Among a lot of proposals for estimating that error rate [for a review, see Ambroise and McLachlan (8)] the leaving-one-out method (9) and the so-called π -error estimation (10), are frequently used (Fig. 5). In the leaving-one-out procedure, the available data are repeatedly reduced by one patient. The allocation rule is formed from the remaining subset and then assessed to the replaced patient. As a result of its known class membership, one can decide for a wrong or correct allocation. The procedure estimates the so-called true error rate, namely, the error rate with an allocation rule constructed on the basis of a training sample and referring to an individual patient not belonging to the training set (in the later application of the classifier, referring to patients with unknown class membership). The training sample {x} = {x11 , . . . , x2n2 }, which is customarily ordered by classes, has to be randomly mixed before starting the algorithm in order to secure a representative sample {x}rem . The leave-one-out error estimation and the wellknown resubstitution method (discriminator is trained and tested by the whole training sample, such as in Section 1.2 for the neuroborreliosis data) will be almost identical if the sample size n is large enough (relative to the number of features). Generally, the resubstitution method proves to be too small, thus it is overly optimistic (11). Another rather often used method of estimating the error rate is the bootstrap method (12). Again, one generates new samples (bootstrap samples) by drawing randomly one (or more) observation(s) from the training sample {x} and repeating this sampling several times (drawing with replacement). The error

6

DISCRIMINANT ANALYSIS

class 1 [f1(x )]

class 2 [f2(x )]

x R2

R1

wrongly all. (from 1 into 2) R2 f1(x ) dx

wrongly all. (from 2 into 1) R1 f2(x ) dx

Figure 4. Misallocation in case of two normal distributions f 1 (x) and f2 (x)(p = 1)

rate is then calculated on the basis of the bootstrap samples. 2.2 Error Rates in Diagnostic Tests The results of the error estimation will be summarized in a classification table. Table 2 shows the corresponding quantities for diagnostic tests, where C1 denotes the population of diseased, say D+, and C2 the population of non-diseased, say D−. Prediction to class C1 means a positive test result, say T+, and prediction to class C2 a negative test result, say T−. h11 , h12 , h21 , and h22 are the obtained frequencies of allocation, tp [tn] denotes the frequency of true positive [negative] decisions, and fp [fn] the frequency of false positive [negative] decisions. The probability P(T + |D+) of true positive decisions (prediction to D+ for patients from D+) is known as sensitivity; the probability P(T − |D−) of true negative (prediction to D− for patients from D−) is known

as specificity. For clinical practice, it is even more important to know what the probability of a true positive decision is, referring to all positive decisions (i.e., the probability of membership to D+ provided that the patient has been allocated to D+). That is called positive predictive value (PPV) with probability P(D + |T+). Analogously, the probability of membership to D−, provided that the patient has been allocated to D−, is denoted by P(D − |T−) [negative predictive value (NPV)]. The calculation of the above-mentioned statistical quantities in diagnostic tests are summarized in Table 3. Another calculation for the predictive values goes back to a very famous mathematical theorem, already established in 1763 by Thomas Bayes (13)—the so-called Bayes theorem. The Bayes theorem combines prior assessments (pre-test probabilities) with the eventual test results to obtain a posterior

Table 2. Classification Table in Diagnostic Tests True class (disease) D+

D− Total

Prediction (test result) T+ T− h11 (tp) sensitivity pos. pred. value h21 (fp) 1-spec. 1-PPV h11 + h21

h12 (fn) 1-sensit. 1-NPV h22 (tn) specificity neg. pred. value h12 + h22

Total h11 + h12

h21 + h22 n = h11 + h12 + h21 + h22

DISCRIMINANT ANALYSIS

7

k=1 Remove a subset {x}k = {xk1, ..., xkm} from {x} {x}rem = {x} − {x}k k=k+1

Train the classifier on {x}rem Test the classifier with {x}k pk =

m

j =1

pkj with pkj =

n

0, if xkj correctly classified 1, if xkj wrongly classified

k < n/m ?

π− error−estimation: E(p) =

y

1 n

n/m k =1

Table 3. Statistical Quantities in Diagnostic Tests Probability

Description

P(T + |D+)

Sensitivity

P(T − |D−)

Specificity

P(D + |T+)

Positive predictive value PPV

tp tp + fn tn tn + fp tp tp + fp

P(D − |T−)

Negative predictive value NPV

tn tn + fn

Estimation

assessment of the diagnosis (post-test probability). Taking the pre-test probabilities P(D + ) as prevalence of the disease, and P(D − ) as probability for no disease, positive predictive value P(D + |T+) can be calculated according to Bayes theorem to: P(D + |T+) = =

Figure 5. π -error-estimation (m = 1: leaving-one-out method)

pk

Example 2 (continuation): Diagnosis of neuroborreliosis burgdorferi For reasons of simplification, only the resubsitution method is applied for estimating the error rate in Section 1.2. To complete the calculations, the results are now presented using Fisher’s approach as allocation rule and the leaving-one-out method for error estimation, supplemented with the statistical quantities mentioned above (Table 4). The overall error rate of 16.39% is connected with a sensitivity of only 60.00% but a high specificity of 95.12% (i.e., the prediction of patients from class NNB is better than those from LNB). Positive and negative predictive values are sufficiently high (85.71% and 82.98%, respectively), referring to the prevalence from the given sample (this prevalence is not necessarily representative for the common population).

P(T + |D+)P(D+) P(T+) P(T + |D+)P(D+) P(T + |D+)P(D+)+ P(T + |D−)P(D−)

(P(T + |D+): sensitivity, P(T + |D−): 1-specificity), which shows the dependence of P(D + |T+) on the prevalence. Analogously, one calculates NPV.

3

BAYES ALLOCATION RULES

The Bayes theorem, mentioned in the previous section, based the (posterior) probability of belonging to a certain class on a prior estimate of that probability. The prior probability that a patient x belongs to class C1 may be denoted by π 1 = P(x∈C1 ) (e.g., the patient

8

DISCRIMINANT ANALYSIS

Table 4. Classification Table for the Neuroborreliosis Data Row [%]: sensitivity and specificity; Col. [%]: positive predictive value and negative predictive value leave-one-out Row [%] Col. [%] True class LNB

NNB Total

Fisher’s approach

3.1 Prediction A Bayes rule RI (x) is defined by RI (x) = m,

if

πm fm (x) ≥ πi fi (x) or

P(Cm |x) = max P(Ci |x); Predicted to class LNB NNB 12 60.00% 85.71% 2 4.88% 14.29% 14

8 40.00% 17.02% 39 95.12% 82.98% 47

i

Total 20

41

61

i = 1, 2

after which a patient will be assigned to that class Cm with the maximum posterior probability P(Cm | x). This rule is also called prediction or identification, respectively. Figure 6 illustrates the identification rule in case of two normal distributions and equal priors π 1 = π 2. 3.2 Action-Oriented Discrimination

has the disease, known as prior probability of the disease or prevalence). Analogously, membership to class C2 is assigned to π 2 = P(x∈C2 ) (e.g., prior probability of no disease), supposing π 1 + π 2 = 1. Furthermore, it is known a priori the distribution function F i (x) of feature vectors (x1 , . . . , xp ) from class Ci and their corresponding densities f i (x)(i = 1, 2). The posterior probability that a patient with unknown class membership and given observation vector x belongs to class Ci is given by

The allocation rule RI (x) assesses every misallocation with the same probability 1 − P(Ci | x); i = 1, 2 (i.e., equally for all classes), which may be disadvantageous in medical applications, where certain diagnosis have to be predicted with particular safety. Therefore, losses or costs lij of allocation are introduced when an patient from Ci is allocated to Cj , with lij = 0 for i = j (i.e., zero costs for a correct prediction). A Bayes’ rule RA (x), given a patient x, is then defined as RA (x) = m,

= min[CP1 (R(x)), CP2 (R(x))] with

P(patient ∈ Ci |x) = P(Ci |x) = πi fi (x)/[π1 f1 (x) + π2 f2 (x)];

i = 1, 2

according to Bayes’ theorem (13). In other words, given a patient with vector of observations x and unknown class membership, are asks for the (posterior) probability that the patient belongs to class Ci , given the (prior) probabilities of class membership π i , and the class-specific probability functions P( x| x ∈ Ci ) = fi ( x) of class Ci (i = 1, 2). In decision theory terms, any specific allocation rule that minimizes the overall probability of misallocations or the so-called optimal error rate (14) for known π i and f i (x) is said to be a Bayes rule.

if CPm (R(x))

CP1 (R(x)) = π2 l21 f2 (x) CP2 (R(x)) = π1 l12 f1 (x)

after which a patient will be assigned to that class Cm with the minimum misallocation loss CPm (R(x)), conditional on x. This rule is also called the action-oriented classification rule (15). It can easily be seen that the rule RI (x) follows immediately from RA (x) for equal losses l12 = l21 . As a result of the rather arbitrary nature of assigning losses of misallocation in practice, they are often taken as equal. According to McLachlan (11) this assignment is not as arbitrary as it may appear at first sight. For Bayes’ rule, only the ratio of l12 and l21 is relevant; therefore, the losses can be scaled by π 1 l12 + π 2 l21 . For an example, given two classes where C1 are patients suffering from

DISCRIMINANT ANALYSIS

class 2 [f2(x )]

Figure 6. Allocation according to identification for equal priors π1 = π2

9

class 1 [f1(x )]

x x ⇒ class 2

x ⇒ class 1

[f2(x ) ≥ f1(x )]

[f1(x ) ≥ f2(x )]

a rare disease and C2 are not, then, although π 1 and π 2 are substantially different, the cost of misallocating a patient from C1 may be much greater than the cost of misclassifying a healthy individual. Consequently, π 1 l12 and π 2 l21 may be comparable and the assumption of equal priors with unit costs of misallocation is not very far from real situations. 3.3 Maximum-Likelihood Discrimination Rules In the real world, class priors π i and class conditional densities f i (x) are unknown. Then, the frequentist analogue of Bayes’ rule is the maximum likelihood discriminant rule, in which the unknown quantities have to be estimated appropriately. For known class conditional densities f i (x), the maximum likelihood (ML) discriminator allocates an observation with vector x to the class with the greatest likelihood: RML (x) = m, if P(Cm |x) = maxi f i (x). In the case of equal class priors π i , it means maximizing the posterior probabilities [i.e., the optimal or Bayes rule RI (x)]. Otherwise, the rule RML (x) is not optimal (i.e., it does not minimize the overall error rate). Approaches of estimating the class conditional densities f i (x) have been developed by various authors, among others so-called plug-in rules (see Section 4.1), or nonparametric methods like kernel- or nearest neighbor-methods (see Sections 5 and 7, respectively).

4 DISCRIMINATION USING NORMAL MODELS The precondition of known priors is usually not fulfilled in practice. In medical diagnostics, the prevalence rates can be taken as estimations of the priors if available. If the training sample {x} = {x11 , . . . , x2n2 } has been obtained by a mixture C of the supposed classes C1 and C2 , then the prior π i can be estimated by its maximum likelihood estimator πˆ i = ni /n (i = 1, 2). Even more problematic is the estimation of the usually unknown class conditional densities f i (x) from the given training sample {x}. With the precondition of p-dimensional normally distributed observation vectors (x1 , . . . , xp ) [i.e., x ∼ N(µi , i )], with corresponding class densities f i (x) as normal (the parameters µ1 , µ2 denote the (expected) class means and  1 ,  2 the corresponding covariance matrices in the classes), two special decision rules from Bayes’ formula are obtained: linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA), respectively. These two decision rules are widely used classification tools and implemented in most of the known software packages for statistics. Another approach to discrimination in case of normally distributed observations x is the already mentioned Fisher’s discriminant analysis (6) that uses a multiple linear regression approach with the class indicator variable as response and the observations x = (x1 , . . . , xp ) as regressors (16).

10

DISCRIMINANT ANALYSIS

4.1 Linear Discriminant Analysis Supposing equal covariance matrices in all classes (i.e.,  1 =  2 ), the optimal or Bayes’ decision rule (i = 1, 2) is obtained RO (x) = m, (x − µm )

if

−1

(x − µm )T − ln πm

= min[(x − µi ) −1 (x − µi )T − ln πi ] i

diagnostic process often consists in the investigation of the quantitative scaled data. For that reason, two groups of patients: were investigated 58 (n1 ) patients (class 1: glaucoma) and 41 (n2 ) patients (class 2: ocular hypertension) using the following features: • daily intraocular pressure (IOP), bra-

chialis blood pressure (BP), • ophthalmica blood pressure (OP), perfu-

sions pressure (difference BP − OP),

Supposing a training sample {x}, the corresponding maximum likelihood estimations from {x} are used for the unknown parameters and a so-called plug-in rule will be obtained LDA(x) = m, (x − xm )S

−1

if

(x − xm )T − ln πˆ m

= min[(x − xi )S−1 (x − xi )T − ln πˆ i ] i

ˆ where S = n/(n − 2) is the so-called biascorrected matrix of the estimated covariance ˆ Therefore, a patient is allocated matrix . with observation vector x0 to class Cm for which the Mahalanobis distance Mi ( x0 )(i = 1, 2) Mm (x0 ) = min Mi (x0 ) i

= min(x0 − xi )S−1 (x0 − xi )T i

will be minimal (except for − ln πˆ i ). Figure 7 shows the geometrical visualization [for  = I (I identity matrix), Mahalanobis distance equals to Euclidian distance]. In Fig. 7, the patient with observation vector x0 will be allocated to class C1 . Example 3: Differential diagnosis of glaucoma and ocular hypertension A difficult problem in ophthalmological diagnostics is the differentiation between glaucoma and ocular hypertension. In order to achieve the diagnosis, various clinical investigations (such as measurements of intraocular pressure, fields of vision, different anamnestic findings, clinical history, etc.) are accomplished, resulting in both quantitative and categorical scaled data. The first step of the

• treatment—test (IOP after treatment),

and others (2). Table 5 shows the results obtained using linear discriminant analysis [i.e., rule LDA(x)] supposing equal priors with unit losses. The true error rates have been estimated according to the leave-one-out method, including the first four features (IOP, BP, OP, BP-OP), which resulted in altogether 16 wrong allocations, which means an overall error estimation of 16.16%. As to be expected, the rule LDA(x) has a very good quality for patients suffering from ocular hypertension (only one patient wrongly allocated: 2.44%), whereas 15 patients from 58 (25.86%) were wrongly diagnosed in the glaucoma class. On the other hand, the positive predictive value of glaucoma is rather large with 97.73% (i.e., the probability of a glaucoma is high when predicted). 4.2 Feature Selection Particularly in medical applications, the number p of features may be very large relative to the sample size. Then, consideration

Table 5. Classification Table for the Glaucoma Data leave-one-out LDA Row [%] Col. [%] Allocated to class True class glaucoma hypertension glaucoma

hyper -tension Total

43 74.14% 97.73% 1 2.44% 2.27% 44

15 25.86% 27.27% 40 97.56% 72.73% 55

Total 58

41

99

DISCRIMINANT ANALYSIS

11

x2

M2(x 0)

x02 x12

M1(x 0) x22

x11

x01

x21

x1

Figure 7. Geometrical illustration of linear discriminant analysis (patients from class C1 with mean x1 are marked by crosses, class C2 with mean x2 marked by circles)

might be given to methods for reducing the number of features because too many features may produce problems with the performance of the sample discriminant function. On the other hand, each feature selection is an intervention into the given data and always causes a loss of information (17). Nevertheless, for practical applications in medicine, it is often interesting to separate important variables from redundant ones. Principally, if one wants to know which set of variables is effective, one must examine all possible subsets of x1 , x2 , . . . , xp . As many subsets may exist, a common strategy is to accomplish the selection in a stepwise procedure, forward (including, step-by-step, the most important variable) or backward (excluding redundant variables). As a measure for the ‘‘importance’’ of a variable, the amount of the overall error rate as selection criterion is preferred, which means a measure of class separation over one or a few linear combinations of the features (4). Furthermore, a cross-validated feature selection is always performed. Thereby, the given sample is reduced repeatedly by a certain number of observations and, with the remaining, the feature selection process is carried out. In every step, the removed patients are allocated using the decision rule obtained and the wrong allocations are counted (crossvalidated error rate). In the end, those features that appear most frequently in the

validation steps are selected for a discrimination rule (Fig. 8). If certain features show up again and again in the validation steps, they can be considered as particularly stable. These most frequent features were recommended to the medical doctors, and for those features, a new error estimation has been accomplished (recommended error rate). Example 3 (continuation): Diagnosis of glaucoma and ocular hypertension The discrimination result of an overall error rate of 16.16% was reached for the four features IOP, BP, OP, BP-OP. In addition, it is always interesting to estimate the true error rate for certain features univariately in order to get an overview of the performance of the features separately. On the other hand, it may be of interest to judge the relationships between features and their impact on the classification results when combined. Table 6 gives an overview about the error estimations (leaving-one-out estimation) separately and in building-up steps of feature selection (without cross-validation). This procedure selects in each step those features that have the best error estimation, accordingly. As a result, IOP is the best-predicting feature among the four given characteristics.

12

DISCRIMINANT ANALYSIS

Table 6. Error Rates of Separate Features and Consecutive Combinations Feature

Error rate (separately)

Error rate (in combination)

IOP BP OP BP-OP

16.16% 38.38% 19.19% 34.34%

16.16% 15.15% 18.18% 16.16%

vectors (x1 , x2 , . . . , xp ), another decision rule, the so-called quadratic discriminant analysis - QDA, is obtained written in the form of a plug-in rule, as QDA(x) = m,

if Qm (x) = min Qi (x) and

Qi (x) = 1/2(x −

i

xi )S−1 i (x

− xi )T

+ 1/2 ln |Si | − ln πˆ i ; i = 1, 2 Example 2 (continuation): Diagnosis of neuroborreliosis burgdorferi For the diagnosis mentioned, the doctors want to separate important from redundant variables. When selecting features, the obtained set of variables will certainly depend on the arbitrariness of the given training sample. Therefore, a serious feature selection must always include a cross-validation in order to avoid that randomness (crossvalidated feature selection). Table 7 gives the results of the crossvalidation process for the neuroborreliosis data. A set of three features was mostly chosen (five times), and the variables 5 (SBigg), 8 (CSF-prot), and 3 (Paresis) were selected with the frequencies 11, 11, 10, respectively. Thus, the feature set with 3 variables including 5 (SBigg), 8 (CSF-prot), and 3 (Paresis) was recommended to the medical doctors. The recommended error estimation (leave-one-out) for this feature set resulted in 11.48%. For reasons of brevity, the corresponding classification table is not presented here. 4.3 Quadratic Discriminant Analysis In case of unequal covariance matrices,  1   2 , but with normally distributed feature = Table 7. Results of a Cross-Validation in 11 Steps Results of cross-validation selected numbers of features corresponding frequencies selected features corresponding frequencies

3

4

5

5

4

2

5 11

8 11

3 10

1 6

2 3

where the estimations of the covariance ˆ i are replaced by the bias-corrected matrices  ˆ i /(ni − 1). This rule is estimations Si = ni  also implemented in most of the commercial software packages, although it is rather seldom worth applying, because of non essential improvements of the results in the majority of applications. It could be shown in numerous simulation experiments that quadratic discriminant analysis should be applied only in case of ‘‘huge’’ differences between the covariance matrices (18). 5 NONPARAMETRIC DISCRIMINATION The methods of discrimination described so far suppose some model [for the class conditional densities f i (x)] on the basis of which certain parameters have to be estimated. As an example, refer to Section 4 for normally distributed densities f i (x). Such approaches are called parametric. On the other side, procedures that have been developed without any postulation of a certain model are called nonparametric. The focus here is on kernelmethods that dominate the nonparametric density estimation literature. Given a training sample {x} = {x11 , . . . , x2n2 }, in the nonparametric kernel estimation of the class conditional density f i (x), a ‘‘kernel’’ is laid around each observation xij of the training sample. The kernel function K (i) (x, xij ) may have any shape, but has to fulfil the conditions of a probability density function. A convenient kernel that is very often used is a multivariate normal density (i = 1, 2) ˆ i |−1/2 K (i) (x, xij ) = (2π )−p/2 | ˆ −1 (x − xij )T } × exp{− 12 (x − xij ) i

DISCRIMINANT ANALYSIS

13

k=1 Remove a subset {x}k = {xk1,...,xkm} from {x} {x}rem = {x} − {x}k Train the classifier on {x}rem and select a feature set [Mk]

k=k+1

Test the classifier with {x}k 0, if xkj correctly classified 1, if xkj wrongly classified

pkj =

m ·k < n ?

n

y

Selection of the feature set [Mko ], which was most frequent in the validations and estimation of the corresponding error on the basis of the whole sample Cross-validated error: F (cv ) =

1 n

m

Sk

k =1 j =1

pkj

Recommended error: F (rec) =

1 n

n j =1

pko j

Figure 8. Cross-validated feature selection (compare also Fig. 5)

ˆ i = ςi i with the (diagonal) covariance ( matrix i = diag{s2i1 , . . . , s2ip } of the sample standard deviations s2il = 1/(ni − 1)
ni 2 (l = 1, . . . , p) and elements j=1 (xijl − xil ) ςi —so-called smoothing parameters—which have to be estimated from the training sample). The kernel estimator of f i (x) in class Ci is then defined as the arithmetic mean of all K (i) (x,xij ) in the ith class (i = 1, 2) ni 1  K (i) (x, xij ) fˆi (x) = ni j=1

A graphical representation of a kernel estimation in class Ci gives Fig. 9 with 7 observations and p = 1 (15). With the estimated class conditional densities fˆi (x), decision rule RI (x) or RA (x), can be applied for discrimination, respectively.

Example 4: The IIBM (International Investigation of Breast MRI) study Accuracy data about contrast-enhanced MRI in the literature vary, and concern exists that a widespread use of contrast-enhanced MRI combined with insufficient standards might cause unsatisfying results. Therefore, an international multicenter study was established (19) in order to find out whether— based on MRI enhancement characteristics— a statistically founded distinction between benign and malignant lesions is possible. The study resulted in findings of more than 500 patients. The analyses were performed with the MRI signal intensities SIGs to six consecutive sample points (s = 1, . . . , 6). The differentiation between ‘‘malignant’’ and ‘‘benign’’ was made on the basis of histological classes, resulting in 132 malignant and 63 benign lesions (one per patient) with complete datasets. For each lesion, a total of 12 parameters were calculated from the original MRI signal intensities. From the training sample, it was concluded that the

14

DISCRIMINANT ANALYSIS

fˆi (x)

x

Figure 9. Kernel estimation fˆi (x) of the class conditional density f i (x)

Table 8. Classification with Kernel Discriminator and Two Different Variants of Priors leave-o-out Row [%] Col. [%] True class malignant

benign Total

πˆ 1 = πˆ 2 = 0.5 ζˆ1 = 0.31, ζˆ2 = 0.24 Allocated to class malignant benign 120 90.91% 86.33% 19 30.16% 13.67% 139

12 9.09% 21.43% 44 69.84% 78.57% 56

Total 132

108 81.82% 87.80% 15 23.81% 12.20% 123

malignant 63 benign 195

assumption of normally distributed densities f i (x) was not fulfilled. Thus, a nonparametric kernel discriminator was applied and achieved after a (cross-validated) feature selection and with equal priors πˆ 1 = πˆ 2 = 0.5 a best π -error-estimation of 15.90%. Table 8 (left side) presents the results obtained, showing both high sensitivity (90.91%) and high positive predictive value (86.33%). But the study aimed at a discrimination with a possible high specificity. For this reason, the original priors πˆ 1 were replaced by πˆ 12 = πˆ 1 l12 , and πˆ 2 by πˆ 21 = πˆ 1 l21 , respectively, in order to realize the influence of different losses lij (compare Section 3.2). With the priors πˆ 12 = 0.3 and πˆ 21 = 0.7, the specificity has been increased to 76.19% with a slightly improved positive predictive value of 87.80%, but at the expense of a higher overall error rate of 20.00% (Table 8, right side). 6

πˆ 12 = 0.3, πˆ 21 = 0.7 ζˆ1 = 0.31, ζˆ2 = 0.24 Allocated to class malignant benign

leave-o-out Row [%] Col. [%] True class

LOGISTIC DISCRIMINANT ANALYSIS

According to J. A. Anderson (20), logistic discrimination is broadly applicable in a wide variety of distributions, including multivariate normal distributions with equal

Total

24 18.18% 33.33% 48 76.19% 66.67% 72

Total 132

63

195

covariance matrices, multivariate discrete distributions following the log-linear model with equal interaction terms, joint distributions of continuous and discrete random variables, and therefore, particularly advantageous in applications with mixed data. The logistic discriminant analysis is called semi-parametric because the class conditional densities f i (x) are not modeled for themselves but for the ratio between them. With given priors π 1 and π 2 , the model log

π1 f1 (x) = β0 + βxT π2 f2 (x)

is postulated where β 0 and β = (β 1 , . . . ,β p ) are unknown parameters that have to be estimated. Remembering P(Ci | x) = πi fi ( x) (without the denominator π1 f1 ( x) + π2 f2 ( x)) from rule RI (x)) leads to P(C1 | x)/[1 − P(C1 | x)] = exp(β0 + β xT ), from which one obtains

P(C1 |x) =

exp(β0 + βxT ) 1 + exp(β0 + βxT )

DISCRIMINANT ANALYSIS

and

P(C2 |x) =

1 1 + exp(β0 + βxT )

analogously as posterior probabilities for the corresponding allocations. After estimating the unknown parameters β 0 ,β 1 , . . . β p according to the maximum likelihood principle, it is again decided for the class with the greatest posterior probability [i.e., on the basis of decision rule RLOG (x) = m, if P(Cm |x) = maxi P(Ci |x) (as maximum likelihood discrimination rule)]. The software packages mostly contain an equivalent decision: allocation of the patient with observation vector x to the class C1 if P(C1 |x) > 0.5, otherwise to C2 . Example 3 (continuation): Diagnosis of glaucoma and ocular hypertension Beside the already mentioned metrically scaled measurements, categorical observations from various clinical investigations have been raised: • fields of vision according to Goldmann,

Tuebinger, Octopus • different anamnestic findings. The whole set of metrical variables in combination with two categorical features (fields of vision according to Tuebinger and Octopus) has been included in a logistic regression analysis in the sense of logistic discrimination as described above. With the estimated parameters βˆ0 , βˆ1 , . . . βˆp , one is able to determine the corresponding posterior probabilities and to decide for the corresponding allocations (predicted allocation). Together with the observed allocations (true classes), one can calculate a corresponding classification table (Table 9), which resulted in an overall error rate of 10.10%. As error estimations according to the π -method are not offered in commercial software for logistic discrimination, the numbers are shown with the resubstitution method, unfortunately. A comparison with previous results is therefore only possible by using the same error estimation. The LDA resulted in an overall error of 11.11%, only using the four features IOP, BP, OP, and BP— OP and resubsitution

15

Table 9. Classification Table for the Glaucoma Mixed Data Resubstitution Row [%] Col. [%] Observed glaucoma

hypertension Total

Logistic discrimination Predicted glaucoma hypertension 52 89.66% 92.86% 4 9.76% 7.14% 56

6 10.34% 13.95% 37 90.24% 86.05% 43

Total 58

41

99

as error estimation (2). Generally, it can be expected that the inclusion of clinically relevant features will improve the performance of a prediction rule. The problem of discriminant analysis for mixed data has been particularly treated by Wernecke (21), who gives a powerful algorithm for coupling the results of various discriminators, each adequate for the different scaling of data. Practical examples from medicine demonstrate the improvement of discrimination by using the full dataset of both quantitative and categorical features. 7 FURTHER METHODS OF DISCRIMINATION Beside the methods of discrimination mentioned, a number of further procedures for discrimination exist. A short overview is provided of some methods of particular interest for medical applications. In many medical applications, the data are categorically scaled. For those data, a special discrimination with categorical features can be applied. Instead of the original realizations of the given features x1 , . . . , xp , one most regard here the frequencies h1 (x1 , . . . , xp ), . . . , hK (x1 , . . . , xp ) by which the feature combinations appear in the classes C1 , . . . , CK (where for every (x1 , . . . , xp ) all combinations of category levels have to be inserted). Such frequencies are usually arranged in contingency tables. As an example, consider a study with patients suffering from migraine. Table 10 shows a corresponding contingency table in

16

DISCRIMINANT ANALYSIS

Table 10. Two-Dimensional Contingency Table for Class C1 with p = 2 Variables variable x1 (nausea)

cat. 1 (low)

cat. 1 (low) cat. 2 (strong)

h1 (x11 , x21 ) = 16 h1 (x12 , x21 ) = 10

six cells with two features (p = 2) in three respective two categories, each, for class C1 . Accordingly, there are 16 patients with low headache and nausea complaints, 28 patients with moderate headache and low nausea complaints, and 22 patients with strong headache and low nausea complaints, respectively overall, 66 patients exist with low nausea, 38 patients exist with strong headache complaints, and so on, and altogether, n1 = 116 patients in the class C1 . After estimating the unknown cell probabilities pi (x1 , . . . , xp ) with the help of the corresponding frequencies hi (x1 , . . . , xp ) (e.g., by
using pˆ i (x1 , . . . , xp ) = hi (x1 , . . . , xp )/ni , ni = hi (x1 , . . . , xp )—so-called actuarial model) (2), a patient with observation vector x will be assigned to the class Cm for which RC (x) = m,

if πm pˆ m (x)

= max πi pˆ i (x); i

i = 1, 2, . . . , K

[Bayes optimal for known π i , pi (x)—Linhart (22)]. Friedman (23) proposed a regularized discriminant analysis as a compromise between normal-based linear and quadratic discriminant analysis. In this approach, two parameters are introduced that control shrinkage of ˆ i (i = 1, . . . , the heteroscedastic estimations  ˆ K) toward a common estimate  and shrinkage toward a series of prespecified covariance matrices. Nearest neighbor classifiers are based on a pairwise distance function dj (x) = d(x,xj ) [known from cluster analysis; see, for example, Everitt (24)] between an observation x with unknown class membership and some vector of observations xj from the given training sample. Denoting the sorted distances by d(1) ( x) ≤ d(2) ( x) ≤ · · · ≤ d(n) ( x), class posterior probability estimates are obtained for class Ci as the fraction of class Ci observations among the k nearest neighbors to x (25)

variable x2 (headache) cat. 2 (moderate) h1 (x11 , x22 ) = 28 h1 (x12 , x22 ) = 24

cat. 3 (strong) h1 (x11 , x23 ) = 22 h1 (x12 , x23 ) = 16

n  ˆ i |x) = 1 [dj (x) ≤ d(k) (x)] P(C k j=1

for all xj ∈ Ci ˆ m |x) = and allocate x to the class Cm with P(C ˆ maxi P(Ci |x); i = 1, . . . , K. The number k of nearest neighbors is often taken as k = 1. Another approach is a cross-validated selection, where for given k = 1, 3, 5, . . . a leaveone-out error estimation (see Section 2.1) is applied to the training sample and that k with the minimal error rate will be chosen. A discriminant analysis for longitudinal data (repeated measurements) has been introduced by Tomasko et al. (26) and extended by Wernecke et al. (27). The procedure is a modification of the linear discriminant analysis using the mixed-model MANOVA for the estimation of fixed effects and for a determination of various structures of covariance matrices, including unstructured, compound symmetry, and autoregressive of order 1. Among the nonparametric statistical classification methods, Classification and Regression Trees (CART) play an important role, especially in medical applications (28). Given a set of risk factors x1 , . . . , xp that influence a response variable y, the data is split step-bystep into subgroups, which should be internally as homogeneous and externally as heterogeneous as possible, measured on a function F(y|xl ); l = 1, . . . , p (supposing dichotomous variables, the function F(y|xl ) will be the maximal measure of dependence between y and x1 from a χ 2 -statistics). Every split corresponds to a so-called node and every observation will be localized at exactly one terminal node in the end. The obtained tree defines a classification rule by assigning every terminal node to a class Ci by comparing the risks at the beginning and at the terminal nodes (29).

DISCRIMINANT ANALYSIS

Support vector machines were introduced by Vapnik (30) and refer to K = 2 classes. The discrimination is defined by the ‘‘best’’ hyperplane separating the two classes, where ‘‘best’’ means that hyperplane that minimizes the sum of distances from the hyperplane to the closest correctly classified observations while penalizing for the number of misclassifications (25).

8 JUDGING THE RELIABILITY OF THE ESTIMATED POSTERIOR PROBABILITY In the previous sections, the author dealt with discriminant rules on the basis of posterior probabilities P(patient ∈ Ci | x) =
P(Ci | x) = πi fi ( x)/[ k πk fk ( x)](i = 1, . . . , K), which could be estimated from a training sample
{ x} = { x1 , . . . , xn } in different ways (n = i ni , xj = (xj1 , . . . , xjp ), j = 1, . . . , n). The overall performance of the discrimination rules described can generally be judged by an appropriate estimation of the error rate (See Section 2.1). Nevertheless, one problem is left and should be mentioned at least: How can one judge the reliability of an allocation on the basis of the estimated posteriors, conditional on the observed patient x? As emphasized by McLachlan (11), even in case of low error rates, there may still be patients about them where great uncertainty as to their class of origin. Conversely, assuming high error rates, it may still be possible to allocate some patients with a great certainty. The error rate (as an average criterion) addresses the global performance of the decision rule and judges the quality of the allocation of some chosen patient with unknown class membership. Mainly in clinical medicine, it is often more appropriate to proceed conditionally on the observed individual patient x with its particular realizations of the variables x1 , . . . , xp . One way to assess the mentioned reliability is the calculation of standard errors and also interval estimates (95% confidence ˆ i |x), intervals) for the point estimations P(C conditional on x. For reasons of brevity, corresponding results are not presented here, but the reader is referred to the literature [see, among others, McLachlan (11)].

17

9 SOFTWARE PACKAGES FOR DISCRIMINATION

Software packages for discrimination are numerous. Disregarding the permanent changes of program packages and the development of new software, some remarks may be worthwhile, concentrating on the most important statistical packages. R. Goldstein (31) gives an excellent overview over commercial software in biostatistics, generally. Judgements for classification software (discrimination, clustering, and neural networks) were given in some detail by D. Michie et al. (3). In this critical overview, different program packages have been described and recommendations for their use given. For applications in clinical trials, not so many discrimination methods are worth considering. The applications of discriminant analysis in medical studies might be rather often confined to the methods described in Sections 4 and 6, respectively, including error estimations (Section 2.1) and feature selection (Section 4.2) (unfortunately usually not validated). Therefore, it seems to be sufficient to refer to the well-known commercial software packages SAS (SAS Institute, Inc., Cary, NC, USA), and SPSS (SPSS, Inc., Chicago, Il, USA). SPSS offers both linear and quadratic discrimination as well as logistic discriminant analysis with the mentioned π -error estimations (not for logistic regression) and stepwise feature selection (not validated). Moreover, various diagnostics, such as tests for equality of the covariance matrices  i , can be established. SAS provides a broader variety of discrimination methods including linear and quadratic discriminant analysis, logistic discrimination, nonparametric kernel discrimination, and nearest neighbor methods. Error estimations, (partly) according to Section 2.1, and stepwise feature selection (not validated) are also implemented. Users with ambitions to develop certain routines for themselves may be referred to S-PLUS 2000 (Data Analysis Products Division, MathSoft, Seattle, WA, USA). The

18

DISCRIMINANT ANALYSIS

above-mentioned examples have been calculated using mostly SAS and SPSS, respectively, and special routines (such as crossvalidated feature selection or discrimination for categorical features) developed in S-PLUS (available in the public domain on request for everybody).

of Statistical Science. New York: John Wiley and Sons, 1982, pp. 389–397. 15. J. Hermans, J. D. F. Habbema, T. K. D. Kasanmoentalib, and J. W. Raatgever, Manual for the alloc80 discriminant analysis program. The Netherlands: Department of Medical Statistics, University of Leiden, 1982. 16. B. Flury, A First Course in Multivariate Statistics. New York: Springer, 1997.

REFERENCES 1. H-J. Deichsel and G. Trampisch, Clusteranalyse und Diskriminanzanalyse. Stuttgart: Gustav Fischer Verlag, 1985. 2. K-D. Wernecke, Angewandte Statistik fur ¨ die Praxis. Bonn, Germany: Addison-Wesley, 1996. 3. D. Michie, D. J. Spiegelhalter, and C. C. Taylor, Machine Learning, Neural and Statistical Classification. Chichester: Ellis Horwood, 1994. 4. B. D. Ripley, Pattern Recognition and Neural Networks. Cambridge: Cambridge Universitiy Press, 2004. 5. D. F. Morrison, Multivariate analysis of variance. In: P. Armitage and T. Colton (eds.), Encyclopedia of Biostatistics. New York: John Wiley and Sons, 1998, pp. 2820–2825. 6. R. A. Fisher, The use of multiple measurements in taxonomic problems. Ann. Eugen. 1936; 7: 179–188. 7. P. A. Lachenbruch, Discriminant Analysis. New York: Hafner Press, 1975. 8. C. Ambroise and G. J. McLachlan, Selection bias in gene extraction on the basis of microarray gene-expression data. Proc. Natl. Acad. Sci. 2002; 99(10): 6562–6566. 9. P. A. Lachenbruch and M. R. Mickey, Estimation of error rates in discriminant analysis. Technometrics 1968; 10: 1–11. 10. G. T. Toussaint, Bibliography on estimation of misclassification. IEEE Trans. Inform. Theory 1974; 20: 472. 11. G. J. McLachlan, Discriminant Analysis and Statistical Pattern Recognition. New York: John Wiley and Sons, 1992. 12. B. Efron, The Jackknife, the Bootstrap and Other Resampling Plans. Philadelphia: SIAM, 1982. 13. T. Bayes, An essay towards solving a problem in the doctrine of chance. Communicated by Mr. Price, in a letter to John Canton, AM., F.R.S., December 1763. 14. P. A. Lachenbruch, Discriminant analysis. In: S. Kotz and N. L. Johnson (eds.), Encyclopedia

¨ 17. J. Lauter, Stabile Multivariate Verfahren (Diskriminanzanalyse, Regressionsanalyse, Faktoranalyse). Berlin: Akademie Verlag, 1992. 18. S. Marks and O. J. Dunn, Discriminant function when covariance matrices are unequal. JASA 1974; 69: 555–559. 19. S. H. Heywang-K¨obrunner, U. Bick, W. G. Bradley, B. Bon´e, J. Casselmanand, A. ¨ Coulthard, U. Fischer, M. Muller-Schimpfle, H. Oellinger R. Patt, J. Teubner, M. Friedrich, G. Newstead, R. Holland, A. Schauer, E. A. Sickles, L. Tabar, J. Waisman, and KD. Wernecke, International investigation of breast MRI: results of a multicentre study (11 sites) concerning diagnostic parameters for contrast-enhanced MRI based on 519 histopathologically correlated lesions. Eur. Radiol. 2001; 11: 531–546. 20. J. A. Anderson, Logistic discrimination. In: P. R. Krishnaiah and L. Kanal (eds.), Handbook of Statistics, 2: Amsterdam: North Holland, 1982, pp. 169–191. 21. K-D. Wernecke, A coupling procedure for the discrimination of mixed data. Biometrics 1992; 48(2): 497–506. 22. H. Linhart, Techniques for discriminant analysis with discrete variables. Metrica 1959; 2: 138–149. 23. J. H. Friedman, Regularized discriminant analysis. J. Amer. Statist. Assoc 1989; 84: 165–175. 24. B. S. Everitt, Cluster Analysis. London: Halstead Press, 1980. 25. S. Dudoit and J. Fridlyand, Classification in microarray experiments. In: T. Speed (ed.), Statistical Analysis of Gene Expression Microarray Data. Boca Raton, FL: Chapman and Hall/CRC, 2003. 26. L. Tomasko, R. W. Helms, and S. M. Snapinn, A discriminant analysis extension to mixed models. Stat. Med. 1999; 18: 1249–1260. 27. K-D. Wernecke, G. Kalb, T. Schink, and B. Wegner, A mixed model approach to discriminant analysis with longitudinal data. Biom. J. 2004; 46(2): 246–254.

DISCRIMINANT ANALYSIS 28. L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and Regression Trees. Monterey, CA: Wadsworth, 1984. 29. K-D. Wernecke, K. Possinger, G. Kalb, and J. Stein, Validating classification trees. Biom. J. 1998; 40(8): 993–1005. 30. V. Vapnik, Statistical Learning Theory. New York: Wiley and Sons, 1998. 31. R. Goldstein, Software, biostatistical. In: P. Armitage and T. Colton (eds.), Encyclopedia of Biostatistics. New York: John Wiley and Sons, 1998, pp. 2820–2825.

19

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS

filled teeth or % sites bleeding on probing). It has become accepted to average ordinal scores from gingival and plaque indices. Statistical properties of such measures have been examined by Sullivan and D’Agostino (3). These can also be examined using the percent of sites, teeth, or surfaces exceeding a specified index level. Summaries over subsets of teeth or sites of particular interest, such as pockets harboring specific bacterial species at baseline, or tooth surfaces diagnosed with ‘‘white spots’’ can provide more efficient and focused comparisons. In clinical trials, summary measures for each subject for primary outcomes of interest should be the basic unit of analysis for comparisons across treatment groups, not individual sites or teeth (4). This having been said, often great interest develops in understanding the dynamics of disease and treatment effects at specific sites or tooth surfaces. In such situations, it is critical that analyses of site-specific associations recognize and take into account that individual teeth and sites within subjects are not statistically independent units. Statistical methods for analyzing sitespecific association in the presence of intrasubject dependency are based on two general approaches. The first approach considers each subject as a stratum, computes a site-specific measure of association for each subject, evaluates the homogeneity of the individual measures of association across subjects, and, if appropriate, generates a summary measure of association. Examples are the use of Mantel–Haenszel procedures for categorical outcomes (5) or the incorporation of subject effect as a fixed effect indicator variable in a regression analysis of site-specific covariables (6). The second approach uses statistical methods explicitly incorporating intrasubject correlations among sites into the statistical model. As an example, generalized estimating equation procedures, described with applications to multiple site data in Reference 7, can accommodate logistic and standard multiple regression models, subject and site level covariates, unequal numbers of sites per subject, explicitly specified or empirically estimated correlation structures, and model-based or robust

MAX GOODSON RALPH KENT The Forsyth Institute—Clinical Research Boston, Massachusetts

Disease trials for dental drug products do not greatly differ from disease trials for medical drug products and should adhere to standard principles for the design, conduct, and evaluation of clinical trials (1). Oral research is, however, associated with certain important simplifications and some complications. One simplifying factor is that dentistry is principally concerned with treatment of two diseases: periodontal diseases and dental caries. In addition, however, a surprising number of dental drug products are designed for prevention, improved esthetics, and reduced pain. A principal complication is the multiplicity created by teeth. A unique attribute of statistical development in oral research is the 30-year presence of a statistical ‘‘think tank,’’ the Task Force on Design and Analysis (2). This organization has been the source of many American Dental Association Guidelines, professional input to the FDA, and has sponsored meetings and conferences to discuss methodological issues in oral research. Throughout this era, it has provided a forum for debate on substantial statistical issues connected with evaluation of drugs used in treatment of oral diseases. 1 COMMON FEATURES OF CLINICAL TRIALS FOR TESTING DENTAL DRUG PRODUCTS 1.1 Data Summary In oral disease research, evaluations are made for numerous teeth, tooth surfaces, or sites for each subject. Most commonly, observations are quantitative, dichotomous, or, as in the case of widely used indices for plaque and gingivitis, ordinal. Summary values for each subject are obtained in various ways, such as means (e.g., mean pocket depth) or percent or counts of sites or teeth that exhibit an attribute (e.g., # of decayed, missing or

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS

estimates of model parameters and standard errors. 1.2 Training and Calibration The subjectivity and variability of many assessments makes it critical to use examiner training and calibration (8,9), assignment of specific examiners to follow each subject throughout the study, blinding of examiners, and balanced assignment of examiners across experimental groups. 1.3 Adjustment for Baseline Severity Baseline levels of outcome measures for a subject are frequently highly correlated with subsequent posttreatment observations. Stratification or matching by baseline level, evaluation of change from baseline, and analysis of covariance are commonly used to reduce the effects of baseline variation across subjects for primary outcome measures. For example, the variability of an assay is largely reduced by selecting periodontal sites in a relatively narrow baseline pocket depth range (10). 2 TESTING DRUGS IN ORAL HYGIENE STUDIES Dental plaque is the soft, cheesy white material that accumulates on teeth in the absence of oral hygiene. It is composed of bacterial cells in a polysaccharide polymer created by the bacteria. It is considered by most individuals to be the precursor for most oral diseases. Among the more common clinical trials conducted to test dental products are those associated with toothpastes, mouth rinses, and toothbrushes used for suppression of dental plaque formation or removal. From the FDA viewpoint, dental plaque is not considered to be a disease but a cosmetic issue, so that these products are largely unregulated in this area. Studies of dental plaque removal or suppression depend largely on clinical indices designed to measure the extent or magnitude of plaque accumulation. The Turesky modification of the Quigley–Hein Index for plaque (11) is the most commonly used plaque index. By this method, stained plaque tooth coverage is

evaluated by assigned a value between 0 (none) and 5 (> = 2/3 coverage) for each tooth surface evaluated. When plaque becomes calcified, the hard, adherent mass is called dental calculus or tartar. Suppression of formation or removal of calculus is also most often measured by indices. In this case, the Volpe–Manhold index (12) is the index most commonly used. By this method, calculus accretion is evaluated by measuring the extent of tooth coverage at three locations on the tooth using a periodontal probe.

3 TESTING DRUGS IN GINGIVITIS STUDIES The most common periodontal disease is gingivitis, a mild inflammation of the periodontal tissues with no evidence of connective tissue destruction (13). This condition exists at some time in virtually everyone’s mouth (14). Many outcome variables have been defined for measurement of drugs used to treat gingivitis (15). All outcome variables depend on evaluating either gingival redness and/or gingival bleeding. Drugs most commonly tested for treatment of gingivitis are antibacterial mouth rinses and toothpaste. It is generally difficult if not impossible to demonstrate product efficacy in treatment of gingivitis by starting with patients having natural accumulations of plaque or tartar (calcified plaque, calculus) (16). For this reason, most studies are initiated with a thorough tooth cleaning and efficacy is demonstrated by the ability of a test product to reduce the rate of reaccumulation of plaque or tartar (17). The standard trial for treatment of gingivitis uses the protocol recommended by the American Dental Association Acceptance program (18) and evaluates gingival index (19) as the primary outcome variable. There are issues concerning use of qualitative ordinal indices as outcome variables. Difficulty in interpreting mean index values and the underlying assumptions of combining observations such as redness and bleeding into a single index have lead some researchers to consider statistical analysis of dichotomized index values (20).

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS

4 TESTING DRUGS IN PERIODONTITIS STUDIES Several periodontal diseases currently are differentiated by their clinical characteristics. Periodontitis is a complicated multifactorial chronic inflammatory disease condition that involves specific bacterial pathogens, host responses, and biochemical mediators that create inflammation in the tissues that support the teeth and may result in their ultimate destruction (21). The critical difference between periodontitis and gingivitis is that loss of supporting bone and periodontal ligament occurs in the former but not the latter. This loss of alveolar bone may be seen using radiographs, but because of the biohazard of gamma radiation Institutional Review Boards will seldom approve radiography as a clinical research measurement tool. For this reason, periodontal disease is most often evaluated indirectly by periodontal probing. Periodontal probing is used as a clinical measure of periodontal disease progression. Pocket depth (Fig. 1) is the distance from the free margin of the gingiva to the depth of the pocket formed by the disease process. In normal healthy individuals, a periodontal sulcus measures 2–3 mm in depth. In the early stages of periodontitis, a periodontal pocket forms adjacent to the tooth and can progressively deepen up to the length of the tooth root (12–15 mm). Periodontal pocket depth (PD), a continuous measure, is the most common outcome Periodontal Probe Measurements

variable used to evaluate periodontal disease. The periodontal probe is a cylindrical, thin (usually 0.5 mm diameter), hand-held measurement tool calibrated in millimeters and designed to be placed in the periodontal pocket. Skilled clinicians can measure pocket depth paired differences with a standard deviation of 0.6 mm. Several variants of this measurement are also used, notably, attachment level (AL or, more precisely, clinical attachment level), which estimates the distance from a fixed anatomical tooth landmark (the cementoenamel junction) to the base of the periodontal pocket (Fig. 1). This measure is generally considered to be the best indirect estimate of bone loss (22) because it corrects for gingival swelling (hypertrophy) or shrinkage (recession). Because AL is the difference between the probe measurements of PD and recession (B) (23), reproducibility is less than with PD (skilled clinicians can measure AL paired differences with a standard deviation of 0.8 mm). As one would expect, AL is highly correlated with PD. Drugs most commonly tested to treat periodontitis are antibacterial agents or antibiotics, occasionally anti-inflammatory agents. Subject selection should include cognizance of possible confounding with age, smoking (24), obesity, and diabetes (25). Proposed guidelines for periodontal therapy trials have been published (26). Anatomical Landmarks Tooth Enamel

Pocket Depth (PD)

“B” measure Attachment Level (AL)

AL = PD-B

3

Cemento-enamel Junction (CEJ) Gingiva Periodontal Pocket Tooth root (cementum-covered) Bone

Figure 1. Periodontal probe measurements and anatomical landmarks. Pocket depth (PD) is a single measurement most commonly recorded in clinical practice. The anatomically adjusted measurement of attachment level (AL) requires two measurements, PD and recession (B), and is associated with increased variability. The value of B carries a positive sign for sites with gingival hypertrophy and a negative sign for sites with gingival recession.

4

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS

4.1 Statistical Issues in Periodontitis Studies The appropriate unit of statistical analysis has been the subject of considerable controversy. The concept of periodontal sites was introduced into the literature as an analytical feature defining 4 to 6 locations around the tooth that were identified by anatomical landmarks and evaluated longitudinally to preserve information concerning the local site-specific changes in PD, AL, and bacteria potentially related to disease (27). Subsequent microbiological evaluations indicated that individual sites within the same mouth can have a completely different microbial ecology (28). Because bacterial diseases are generally defined by the infecting agent(s), the hypothesis was advanced that different periodontal diseases may coexist in the same mouth and that it would be inappropriate to pool periodontal sites simply because they had the same clinical signs of periodontitis. Some investigators interpreted these fundamental observations related to defining the pathogenic process as an effort to justify summarizing statistical data on periodontal sites and thereby inflating statistical significance (each subject evaluated at 6 sites/tooth may contribute as many as 192 sites). The choice of the unit of statistical analysis depends on the questions being asked. Most often, the subject is the appropriate unit of statistical summary. If sites, tooth surfaces, teeth or quadrants within subjects are to be used as the unit of analysis, however, then it is important to use procedures, such as described in Section 1.1, appropriate to the evaluation of correlated observations. The measurement of numerous bacterial species, host factors, clinical parameters, and genetic markers has created serious issues of multiple testing. For limited numbers of variables, standard multiple comparisons and multiple testing procedures can be used to control the Type I or ‘‘false positive’’ error rate. Loss of statistical power, however, can be severe for large numbers of variables. The application of groupings derived from data mining procedures such as cluster analysis is an alternate approach that has been useful in microbiological studies (29). For example, a clinical trial using ‘‘red complex’’ bacteria derived from cluster analysis as a primary

outcome variable successfully evaluated the antibacterial response of a locally applied antibiotic in the presence of conventional therapy (30). Because a high level of symmetry is found in the mouth, many investigators have suggested that test and control therapies in some instances can be conducted in a single subject with an assumed reduction in response variability. Potential problems introduced by these designs are considerable (31,32). Because all teeth are nourished by the same vascular system, one can never be certain that treatment effects are truly isolated. Evidence suggests that even toothcleaning on one side of the mouth may affect the other side (33). Carry-across effects can reduce differences between treatment and control responses. For example, statistically significant differences observed in single-site local antibiotic therapy (34) could not be repeated when multiple diseased sites were treated, presumably because antibiotic from multiple locally treated sites in one quadrant affected the response of control sites in other quadrants (35). Although it is true that the most likely error in split mouth designs is an underestimation of the therapeutic effectiveness of the test product, the magnitude of the consequences may not be acceptable. Without question, parallel designs are more dependable in evaluating therapeutic responses in periodontal disease therapy. 5 TESTING DRUGS FOR TREATMENT OF DENTAL CARIES Dental caries or dental decay occurs in more than 90% of our population (36). Although much may be said about dental caries as a disease process, from the clinical testing point of view, it can be viewed as the result of bacteria that partially ferment carbohydrates and produce organic acids that dissolve the calcium hydroxyapatite from teeth to form cavities. The disease process starts with a ‘‘white spot’’ that represents decalcification of the tooth enamel and proceeds to a cavity that if not treated can result in tooth loss. Outcome variables for the measurement of dental caries are discrete measurement variables that represent the number of holes in

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS

teeth, white spots in teeth, missing teeth, filled teeth, and so forth. The classic measure of decay in permanent teeth is DMFT (37): the number of decayed, missing, or filled permanent teeth (or, DMFS for surfaces). Criteria for evaluating dental caries using visual criteria or detection with a sharp probe vary greatly between investigators. A review of criteria that have been used to score dental decay found 29 unique systems (38). The NIDCR criteria (39) have been used for largescale epidemiologic surveys. The widespread acceptance of fluoride toothpaste as an effective decay prevention product often demands that trials be conducted using noninferiority designs (40). Guidelines for the conduct of clinical trials for agents used to treat dental caries have been published (41) and are widely accepted as a standard. Characteristically, clinical trials for prevention of dental caries measure net conversion from sound to cavitated tooth surfaces or teeth over an appropriate time interval (incidence). Reliance on classification rather than measurement results in inefficient clinical trials that require many subjects. In addition, these methods are not easily adapted to measuring treatments designed to reverse demineralization of tooth surfaces. Newer methods have been developed to address this problem (42), analytic methods have been described that can be adapted to this measurement system (43), and clinical trials using these methods have been conducted (36). At this time, it is fair to say that these methods will require more research before being generally accepted as dependable. 6 TESTING DRUGS FOR LOCAL ANESTHETICS AND ANALGESICS By far, the most commonly used drugs in dental therapy are local anesthetics and analgesics. The standard for analgesic efficacy, the third molar extraction model, has become a standard used in medicine as well. Third molar extraction as a standard pain stimulus is generally evaluated using a visual analog scale (VAS). This scale is a 10-cm line on which the subject marks a perceived pain level between the two extremes of no pain

5

and worst pain. Mean centimeter differences between treatments and controls are then evaluated (44). The onset and duration of local anesthesia is often evaluated by electrical pulp testing (45). Guidelines for local anesthetic testing have been published (46). 7 TESTING DRUGS FOR TOOTH WHITENING Tooth whitening has become an extremely popular product area. Research in this area is particularly demanding because of the intensely subjective nature of the response. The most common clinical measure is visual comparison by a professional examiner with a tooth shade guide. Shade guides are not linear and contain different levels of several pigments. Results using this approach are somewhat arbitrary and highly subjective. One may also solicit a response rating from the subject but this rating can be associated with a high placebo response. Objective measurement by reflectance spectrophotometers (chromameters) or photographic equivalents calibrated in the LAB color domain have also been used. These devices produce results that are somewhat insensitive and often difficult to interpret. Published guidelines for testing tooth whitening products (47) suggest that studies be conducted using both clinical and quantitative measures. Studies that find concordance between objective and clinical evaluations tend to provide the most convincing evidence from clinical testing (48). REFERENCES 1. R. B. D’Agostino, and J. M. Massaro, New developments in medical clinical trials. J. Dent. Res. 2004; 83 (Spec No C): C18–24. 2. A. Kingman, P. B. Imrey, B. L. Pihlstrom, and S. O. Zimmerman, Chilton, Fertig, Fleiss, and the Task Force on Design and Analysis in Dental and Oral Research. J. Dent. Res. 1997; 76: 1239–1243. 3. L. M. Sullivan, and R. B. D’Agostino, Robustness of the t test applied to data distorted from normality by floor effects. J. Dent. Res. 1992; 71: 1938–1943. 4. J. L. Fleiss, S. Wallenstein, N. W. Chilton, and J. M. Goodson, A re-examination of withinmouth correlations of attachment level and of

6

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS change in attachment level. J. Clin. Periodontol. 1988; 15: 411–414.

and gingivitis. J. Am. Dent. Assoc. 1986; 112: 529–532.

5. P. P. Hujoel, W. J. Loesche, and T. A. DeRouen, Assessment of relationships between site-specific variables. J. Periodontol. 1990; 61: 368–372.

19. H. Loe and J Silness, Periodontal disease in pregnancy. I. Prevalence and severity. Acta Odontol. Scand. 1963; 21: 533–551.

6. T. A. DeRouen, Statistical Models for Assessing Risk of Periodontal Disease. In: J. D. Bader (ed.), Risk Assessment in Dentistry. Chapel Hill, NC: University of North Carolina Dental Ecology, 1990. 7. T. A. DeRouen, L. Mancl, and P. Hujoel, Measurement of associations in periodontal diseases using statistical methods for dependent data. J. Periodontal. Res. 1991; 26: 218–229.

20. T. M. Marthaler, Discussion: Current status of indices of plaque. J. Clin. Periodontol. 1986; 13: 379–380. 21. M. A. Listgarten, Pathogenesis of periodontitis. J. Clin. Periodontol. 1986; 13: 418–430. 22. G. P. Kelly, R. J. Cain, J. W. Knowles, R. R. Nissle, F. G. Burgett, R. A. Shick, et al., Radiographs in clinical periodontal trials. J. Periodontol. 1975; 46: 381–386.

8. A. M. Polson, The research team, calibration, and quality assurance in clinical trials in periodontics. Ann. Periodontol. 1997; 2: 75–82.

23. S. P. Ramfjord, J. W. Knowles, R. R. Nissle, F. G. Burgett, and R. A. Shick, Results following three modalities of periodontal therapy. J. Periodontol. 1975; 46: 522–526.

9. E. G. Hill, E. H. Slate, R. E. Wiegand, S. G. Grossi, and C. F. Salinas, Study design for calibration of clinical examiners measuring periodontal parameters. J. Periodontol. 2006; 77: 1129–1141.

24. J. Haber, J. Wattles, M. Crowley, R. Mandell, K. Joshipura, and R. L. Kent, Evidence for cigarette smoking as a major risk factor for periodontitis. J. Periodontol. 1993; 64: 16–23.

10. J. M. Goodson, M. A. Cugini, R. L. Kent, G. C. Armitage, C. M. Cobb, D. Fine, et al., Multicenter evaluation of tetracycline fiber therapy: I. Experimental design, methods, and baseline data. J. Periodontal. Res. 1991; 26: 361–370. 11. S. L. Fischman, Current status of indices of plaque. J. Clin. Periodontol. 1986; 13: 371–380. 12. A. R. Volpe, J. H. Manhold, and S. P. Hazen, In Vivo Calculus Assessment. I. A method and its examiner reproducibility. J. Periodontol. 1965; 36: 292–298. 13. R. C. Page, Gingivitis. J. Clin. Periodontol. 1986; 13: 345–359. 14. J. W. Stamm, Epidemiology of gingivitis. J. Clin. Periodontol. 1986; 13: 360–370. 15. S. G. Ciancio, Current status of indices of gingivitis. J. Clin. Periodontol. 1986; 13: 375–378. 16. E. F. Corbet, J. O. Tam, K. Y. Zee, M. C. Wong, E. C. Lo, A. W. Mombelli, et al., Therapeutic effects of supervised chlorhexidine mouthrinses on untreated gingivitis. Oral Dis. 1997; 3: 9–18. 17. J. C. Gunsolley, A meta-analysis of sixmonth studies of antiplaque and antigingivitis agents. J. Am. Dent. Assoc. 2006; 137: 1649–1657. 18. Council on Dental Therapeutics, Guidelines for acceptance of chemotherapeutic products for the control of supragingival dental plaque

25. R. J. Genco, S. G. Grossi, A. Ho, F. Nishimura, and Y. Murayama, A proposed model linking inflammation to obesity, diabetes, and periodontal infections. J. Periodontol. 2005; 76: 2075–2084. 26. P. B. Imrey, N. W. Chilton, B. L. Pihlstrom, H. M. Proskin, A. Kingman, M. A. Listgarten, et al., Proposed guidelines for American Dental Association acceptance of products for professional, non-surgical treatment of adult periodontitis. Task Force on Design and Analysis in Dental and Oral Research. J. Periodontal. Res. 1994; 29: 348–360. 27. J. M. Goodson, A. C. Tanner, A. D. Haffajee, G. C. Sornberger, and S. S. Socransky, Patterns of progression and regression of advanced destructive periodontal disease. J. Clin. Periodontol. 1982; 9: 472–481. 28. S. S. Socransky, A. C. Tanner, J. M. Goodson, A. D. Haffajee, C. B. Walker, J. L. Ebersole, et al., An approach to the definition of periodontal disease syndromes by cluster analysis. J. Clin. Periodontol. 1982; 9: 460–471. 29. S. S. Socransky, A. D. Haffajee, M. A. Cugini, C. Smith, and R. L. Kent, Jr., Microbial complexes in subgingival plaque. J. Clin. Periodontol. 1998; 25: 134–144. 30. J. M. Goodson, J. C. Gunsolley, S. G. Grossi, P. S. Bland, J. Otomo-Corgel, F. Doherty, et al., Minocycline HCl microspheres reduce red-complex bacteria in periodontal disease therapy. J. Periodontol. 2007; 78: 1568–1579.

DISEASE TRIALS FOR DENTAL DRUG PRODUCTS 31. P. B. Imrey, Considerations in the statistical analysis of clinical trials in periodontitis. J. Clin. Periodontol. 1986; 13: 517–532. 32. P. P. Hujoel and T. A. DeRouen, Validity issues in split-mouth trials. J. Clin. Periodontol. 1992; 19: 625–627. 33. A. P. Pawlowski, A. Chen, B. M. Hacker, L. A. Mancl, R. C. Page, and F. A. Roberts, Clinical effects of scaling and root planing on untreated teeth. J. Clin. Periodontol. 2005; 32: 21–28. 34. J. M. Goodson, M. A. Cugini, R L. Kent, G. C. Armitage, C. M. Cobb, D. Fine, et al., Multicenter evaluation of tetracycline fiber therapy: II. Clinical response. J. Periodontal. Res. 1991; 26: 371–379. 35. C. L. Drisko, C. M. Cobb, W. J. Killoy, B. S. Michalowicz, B. L. Pihlstrom, R. A. Lowenguth, et al., Evaluation of periodontal treatments using controlled-release tetracycline fibers: clinical response. J. Periodontol. 1995; 66: 692–699. 36. J. D. Bader, D. A. Shugars, and A. J. Bonito, Systematic reviews of selected dental caries diagnostic and management methods. J. Dent. Educ. 2001; 53: 960–968. 37. H. Klein, C. E. Palmer, and J. W. Knutson, Studies on Dental Caries. Public Health Reports 1938; 53: 751–765. 38. A. I. Ismail, Visual and visuo-tactile detection of dental caries. J. Dent. Res. 2004; 83 Spec No C:C56–C66. 39. 39 Oral Health Surveys of the National Institute of Dental Research. Diagnostic Criteria and Procedures. NIH publication No. 91-2870 1991; 1–99. 40. R. B. D’Agostino, Sr., J. M. Massaro, and L. M. Sullivan, Non-inferiority trials: design concepts and issues—the encounters of academic consultants in statistics. Stat. Med. 2003; 22: 169–186. 41. 41 Guidelines for the acceptance of fluoridecontaining dentifrices. Council on Dental Therapeutics. J. Am. Dent. Assoc. 1985; 110: 545–547. 42. A. Hall and J. M. Girkin, A review of potential new diagnostic modalities for caries lesions. J. Dent. Res. 2004; 83 Spec No C:C89–C94. 43. P. B. Imrey and A. Kingman. Analysis of clinical trials involving non-cavitated caries lesions. J. Dent. Res. 2004; 83 Spec No C:C103–C108. 44. K. M. Hargreaves and K. Keiser, Development of new pain management strategies. J. Dent. Educ. 2002; 66: 113–121.

7

45. P. A. Moore, S. G. Boynes, E. V. Hersh, S. S. DeRossi, T. P. Sollecito, J. M. Goodson, et al., The anesthetic efficacy of 4 percent articaine 1:200,000 epinephrine: two controlled clinical trials. J. Am. Dent. Assoc. 2006; 137: 1572–1581. 46. 46 Guideline on appropriate use of local anesthesia for pediatric dental patients. Pediatr. Dent. 2005; 27 (7 Suppl):101–106. 47. 47 Guidelines for the acceptance of peroxidecontaining oral hygiene products. J. Am. Dent. Assoc. 1994; 125: 1140–1142. 48. M. Tavares, J. Stultz, M. Newman, V. Smith, R. Kent, E. Carpino, et al., Light augments tooth whitening with peroxide. J. Am. Dent. Assoc. 2003; 134: 167–175.

FURTHER READING J. L. Fleiss, The Design and Analysis of Clinical Experiments. New York: John Wiley & Sons, 1986.

CROSS-REFERENCES Surrogate Endpoints Data Mining Generalized Estimating Equations Analysis of Covariance (ANCOVA) Multiple Comparisons Data Mining Paired T-Test Two-sample T-Test Nonparametrics

DISEASE TRIALS ON REPRODUCTIVE DISEASES

creases the trust of the public in reproductive medicine and the science on which it relies. Fortunately, several moves have been employed to improve research in the area of reproductive health. The action by distinguished researchers has identified research standards and guides researchers in the development of translational and clinical research studies. Multiple issues have been identified that impair reproductive health science: a welldefined objective; diagnostic accuracy of the disease; appropriate study population; appropriate sample size; appropriate sampling and data collection; specifically developed materials and methods; appropriate randomization of treatment groups; avoidance of selection bias, test reliability and reproducibility; and appropriate index test and/or reference standards. In addition, for infertility and treatment, potential bias can exist because of previous failed or successful treatment and the use of crossover trials. Even when the ‘‘gold standard’’ of clinical trials (the prospective, RCT) is selected, serious limitations have been observed when examining treatment of infertility (4). To estimate the limitations of reproductive medicine studies, researchers tested the accuracy of studies published in two distinguished journals: Human Reproduction and Fertility and Sterility. Using the Standards for Reporting of Diagnostic Accuracy checklist (STARD), which is a checklist to test for 25 measures of diagnostic accuracy, Coppus et al. (5) assessed publications in 1999 and 2004. They found that less than one half of the studies reported at least 50% of the items demonstrating lack of accuracy with no improvement between 1999 and 2004. Unfortunately, the reporting of individual items varied widely between studies; no consistent problem with diagnostic accuracy was observed in the articles reviewed. Clearly, researchers need to consider these issues when developing their studies, and journals need to ask that all items of diagnostic accuracy be addressed in the publication. Using the Consolidated Standards for Reporting Trials (CONSORT), some improvement has occurred in the past 12 years in the

JULIA V. JOHNSON University of Vermont College of Medicine Burlington, Vermont

1

INTRODUCTION

Reproductive health issues are common for men and women, but clinical research in this area has been limited in study number and quality. The importance of these health issues is clear. To use infertility as an example of reproductive health issues, this disease affects 15% of women aged 15–44 years (1). Indeed, infertility is one of the most common medical disorders that affect reproductive aged men and women. Although highquality prospective, randomized controlled trials (RCTs) are required to determine the ideal methods for diagnosis and treatment of this common disease, the quality of research is historically poor (2). The limitation of federal funding for reproductive health issues has led to programs in the NICHD in the past 10 years (3). In the interim, most reproductive research has been funded by pharmaceutical companies and academic centers without set standards for these clinical trials. The limited research funding impairs scientific analysis of the available data, and it adversely affects provider’s ability to care for patients. This article will review the limitation of the current research in reproductive medicine and recommend the standards for researchers and clinicians to consider when reviewing the literature. Also, the current NICHD programs to optimize research in reproductive medicine will be discussed. 2

LIMITATIONS OF CURRENT STUDIES

As in all medical fields, evidence-based medicine is critical to allow effective decision making for providers. The quality of studies in the reproductive sciences has not been consistent, which limits providers’ ability to make decisions with their patients and de-

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

DISEASE TRIALS ON REPRODUCTIVE DISEASES Table 1. Use of CONSORT in Reproductive Medicine RTCs 1990 (% of Studies)

1996

2002

16% 11% 67%

40% 20% 77%

48% 37% 95%

Randomized Trial Blinded Allocation Parallel Design

reporting of RCTs published in 1990, 1996, and 2002 (4). Collecting studies from the Cochrane Menstrual Disorder and Subfertility Group (MDSG), the researcher found that ideal methods of treatment allocation (randomization), concealment of allocation (blinding), and study design (parallel design) improved (see Table 1). However, many studies did not reach the characteristics suggested by CONSORT. In addition, this study demonstrated that 40% of studies had errors in patient flow (number of subjects randomized vs. number analyzed). Unit-of-analysis errors decreased over the 12 years but continued to occur. As is commonly the case with reproductive research, the data on pregnancy outcome was still lacking. Although miscarriages were reported in 62% of studies in 2002, only 28% reported ectopic pregnancies and 44% reported multiple gestations. An excellent example of a serious study design flaw is the use of crossover trials to assess infertility treatment. As described by Khan et al. (6), crossover trials overestimate the effectiveness of treatment, compared to parallel trials, by an odds ratio of 74%. Although it is tempting to use crossover trials to the lower sample size, the outcome (pregnancy) prevents participation in the second arm of the trial. Fortunately, the crossover study design was recognized as flawed; this information presumably led to the increase in parallel trials from 1990 (67%) to 2002 (95%) (4). 3

IDEAL REPRODUCTIVE STUDY DESIGN

Excellent reviews have been written to summarize the ideal clinical study in reproductive medicine (7,8). The first consideration, as with any research project, is to identify the objective clearly and determine whether the study will be effective in testing the hypothesis. In reproductive medicine, the sample size

is critical to consider. Because the sample size significantly increases if the difference with treatment is small, multicenter trials may be required for adequate subject recruitment. The target population must also be considered carefully. In fertility trials, for example, successful treatment is based on multiple factors, which include women’s age, sperm quality, previous pregnancy or failed therapy, gynecologic disorders, and the multiple causes of infertility. The inclusion and exclusion criteria need to be established early in the design of the study to assure that the appropriate population can be recruited. Once the patient population is established, a randomized controlled trial requires an appropriate study design. The CONSORT statement should be reviewed prior to beginning the study, not just for publication of the trial. Studies demonstrate that the use of CONSORT improves the quality of studies (9). Appropriate randomization, performed immediately before treatment to minimize postrandomization withdrawals, must be blinded to the researcher and subject to prevent selection and ascertainment biases. Subjects are followed closely to assure no unintended influences, or cointervention, that may alter results. Once subjects are randomized, those who discontinue treatment or are lost to follow-up must be included in the intent-to-treat analysis of the data. Also, as noted in the example described above, parallel trials are necessary for RCTs examining the treatment of infertility. The ideal study has a fully considered objective with an appropriate sample size to test the hypothesis as well as a lack of bias through randomization of a parallel trial and an analysis of all subjects. Maximal possible blinding is advised, as possible, for subjects, investigators, and health care providers, as well as laboratory personnel.

DISEASE TRIALS ON REPRODUCTIVE DISEASES

4 IMPROVING REPRODUCTIVE MEDICINE RESEARCH The importance of reproductive health issues and the demands of high-quality clinical research are recognized. The large sample size required for these studies has led to networks, such as the Reproductive Medicine Network (3). This network allows multicenter trials that involve eight centers and a datacoordinating center. Recently, the Reproductive Medicine Network completed a landmark study that demonstrated the most effective ovulation induction agent for women with polycystic ovarian syndrome (10). The unexpected results of this well-designed study altered medical care for women with this common disorder. Additional programs include the Specialized Cooperative Center Program in Reproduction Research (SCCPRR), which was developed to increase translational research in reproductive sciences. This program encourages collaboration with other centers within the institution and with outside entities. Four SCCPRR focus groups include endometrial function, ovarian physiology, male reproduction, and neuroendocrine function. The National Cooperative Program for Infertility Research (NCPIR) has two sites that involve patient-oriented research. Currently, the NCPIR is emphasizing the genetic basis of polycystic ovarian syndrome. The addition of the Cochrane MDSG has improved the interpretation of reproductive medicine trials (11). With more than 2000 randomized controlled trials in fertility, this group allows assessment of the quality and content of the published trials. 5

CONCLUSIONS

Providers rely on high-quality clinical research to guide their practice; researchers rely on standards that optimize study design and ensure appropriate analysis of results. Most importantly, patients rely on research to determine the most effective diagnostic tests and reliable treatment options. Classically, reproductive medicine had limited funding, which resulted in suboptimal clinical studies. The factors that complicate these

3

clinical trials are now recognized, and appropriate patient selection and study design has been clarified. The use of standards such as CONSORT and STARD will improve the research development and publication significantly. In addition, the efforts from the NICHD to increase multicenter and collaborative trials will set the standard for highquality research in reproductive medicine. REFERENCES 1. J. C. Abama, A. Chandra, W. D. Mosher, et al., Fertility, family planning, and women’s health: new data from the 1995 National Survey of Family Growth. Vital Health Stat. 1997; 19: 1–114. 2. S. Daya, Methodological issues in infertility research. Best Prac. Res. Clin. Obstet. Gynecol. 2006; 20: 779–797. 3. L. V. DePaolo and P. C. Leppert, Providing research training infrastructures for clinical research in the reproductive sciences. Am. J. Obstet. Gynecol. 2002; 187: 1087–1090. 4. A. Dias, R. McNamee, and A. Vail, Evidence of improving quality of reporting of randomized controlled trials in subfertility. Hum. Reprod. 2006; 21: 2617–2627. 5. S. F. P. J. Coppus, F. van der Venn, P. M. M. Bossuyt, and B. W. J. Mol, Quality of reporting of test accuracy studies in reproductive medicine: impact of the Standards for Reporting of Diagnostic Accuracy (STARD). Fertil. Steril. 2006; 86: 1321–1329. 6. K. S. Khan, S. Daya, JA. Collins, and S. D. Walter, Empirical evidence of bias in infertility research: overestimation of treatment effect in crossover trials using pregnancy as the outcome measure. Fertil. Steril. 1996; 65: 939–945. 7. J-C. Acre, A. Nyboe Anderson, and J. Collins, Resolving methodological and clinical issues in the design of efficacy trials in assisted reproductive technologies: a mini-review. Hum. Reprod. 2005; 20: 1751–1771. 8. S. Daya, Pitfalls in the design and analysis of efficacy trials in subfertility. Hum. Reprod. 2003: 18; 1005–1009. 9. D. Moher, K. F. Schulz, and D. Altman for the CONSORT Group, The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA 2001; 285: 197–191. 10. R. S. Legro, H. X. Barnhart, W. D. Sclaff, B. R. Carr, et al., for the Cooperative Multicenter

4

DISEASE TRIALS ON REPRODUCTIVE DISEASES

Reproductive Medicine Network, Clomiphene, metformin, or both for infertility in the polycystic ovarian syndrome. N. Engl. J. Med. 2007; 356: 551–566. 11. C. M. Farquhar, A. Prentice, D. H. Barlow, J. L. H. Evers, P. Vandekerckhove, and A. Vail, Effective treatment of subfertility: introducing the Cochrane menstrual disorders and subfertility group. Hum. Reprod. 1999; 14: 1678–1683.

FURTHER READING E. G. Hughes, Randomized clinical trials: The meeting place of medical practice and clinical research. Semin. Reprod. Med. 2003; 21: 55–64. G. Piaggio and A. P. Y. Pinol, Use of the equivalence approach in reproductive health clinical trials. Stats. Med. 2001; 20: 3571–3587. K. F. Schultz, I. Chalmers, R. J. Hayes, et al., Empirical evidence of bias: dimensions of methodological quality associated with treatment effects in controlled trials. JAMA 1995; 273: 408–412. L. Roenbert, Physician-scientists: endangered and essential. Science 1999; 288: 331–332. A. Vail and E. Gardener, Common statistical error in the design and analysis of subfertility trials. Hum. Reprod. 2003; 18: 1000–1004.

DISEASE TRIALS ON PEDIATRIC PATIENTS

Researchers face unique obstacles when conducing RCTs in children, including the limited number of children with major medical diseases, issues related to ethics, and finally, measurable objective outcomes. Unlike adults, most children in the United States are healthy. Of the estimated 1,400,000 patients newly diagnosed each year with cancer, only about 10,000 are in children 0–14 years of age (2). The most common chronic disease in childhood is asthma, which affects approximately 10% of the 75,000,000 children 0 to 18 years of age. However, the vast majority of even these children have mild to moderate disease. Few seek care in emergency rooms, and even fewer are hospitalized. These examples are two examples of important childhood diseases, but they affect such limited numbers of children that conducting RCTs is difficult. Investigators in child health, therefore, frequently have to resort to multicenter trials. A good example is the Children’s Oncology Group (COG), which enrolls children around the United States in clinical trials and has been conducting multisite trials for decades (3). Virtually every child in the United States with cancer can enroll in a clinical trial. More recent examples include the Pediatric Research in Office Settings group (PROS)(4), a pediatric emergency network in the United States (5), and a clinical trials unit in the United Kingdom (6). PROS is a practice-based network, established in 1986, which includes over 700 practices in the United States, Puerto Rico, and Canada. It was established by the American Academy of Pediatrics (AAP). A research staff at the AAP helps coordinate projects and assists with identifying funding and data analysis. Research ideas are vetted through numerous committees, which include biostatisticians, epidemiologists, and clinicians. The Pediatric Emergency Care Applied Research Network (PECARN) is the first federally funded collaborative network for research in pediatric emergency medicine. The group recently published the results of a large RCT that assessed the role of dexamethasone for infants diagnosed with bronchiolitis in the emergency

HOWARD BAUCHNER Boston University School of Medicine Boston Medical Center Boston, Massachusetts

Conducting clinical trials in which children are the subjects is important. Appropriately powered randomized clinical trials (RCTs) remain the most influential study design. Some clinical trials change practice on their own; others contribute through metaanalyses. Both RCTs and meta-analyses are considered the cornerstone for the Cochrane Collaboration and the primary basis for recommendations of the United States Preventive Services Task Force. In a recent analysis, we found a large gap between high-quality study designs in children and adults. We assessed all research articles published in the New England Journal of Medicine, Journal of the American Medical Association, Annals of Internal Medicine, Pediatrics, Archives of Internal Medicine and Archives of Adolescent and Pediatric Medicine during the first 3 months of 2005 (1). 405 original research reports were reviewed, of which 189 included only adults as subjects and 181 included only children as subjects (total N = 370). Both RCTs and systematic reviews were significantly more common in studies that involved adults compared with those that involved children. Of the 370 trials, 32.6% were RCTs, 23.8% involving adults and 8.8% involving children. Of the 12.3% of studies that were systematic reviews, 10.6% involved adults and 1.7% involved children. Cross-sectional studies (considered less robust designs) were twice as common in the pediatric literature (38.1% vs. 17.7%). Unlike large RCTs, in which both known and unknown biases are usually equally distributed between groups, in cross-sectional designs, confounding is of major concern. This survey highlights the concern about lack of high-quality study designs—RCTs and systematic reviews—in child health research but begs the question why this gap exists.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

DISEASE TRIALS ON PEDIATRIC PATIENTS

room setting (5). Twenty emergency rooms were involved in this study. The United Kingdom network, Medicines for Children Research Network (MCRN), was created in 2005 because of the concern about lack of high-quality clinical trials in children. The unit is based in Liverpool, but six local research networks are established, covering most of England, and nine clinical studies groups exist, too, including anaesthesia, diabetes, neonatal, neurosciences, and respiratory. The goal is to develop and conduct RCTs that will enroll children from around the United Kingdom. Research consortia like COG, PROS, PECARN, and MCRN are critical to the future health of children. However, unlike single-site trials, multisite trials are far more difficult to conduct because of issues related to expense, reliability of data, and ethics committee approval. Ethical issues are an important consideration when conducting research with children (7). According to federal regulation, children are considered a vulnerable population and, as such, are afforded additional protections by Institutional Review Boards. For example, children can participate in research with greater than minimal risk only as long as they are likely to benefit from research. The risk/benefit ratio must be perceived at least as favorable as available alternatives. The issue of when children, rather than their parents, should give consent is also complicated. When considering whether children are capable of consenting, IRBs consider age, maturity, and psychological state of the children. The adolescent years pose another potential problem for researchers. Determining when adolescents become ‘‘adults’’ and can consent to participate in research without their parents being present or contacted is another complicated issue that can vary from state to state. A final consideration in conducting trials in children is appropriate health outcomes. Often trials with adults use as objective outcomes, death, hospitalization, or major medical problems, such as myocardial infarction or stroke. These outcomes are quite rare in children, so the outcomes used in many pediatric trials relate to functional measures of performance, including cognitive outcomes and physical activity. These outcomes are

affected by many variables, so ensuring adequate sample sizes in RCTs that involve children is critical. In addition, many other variables can modify the results (effect modifiers). In addition, many outcomes that are important to child health researchers occur in the adolescent and adult years. For example, researchers in childhood cancer are particularly interested in rates of secondary cancer. Many of these occur 20–30 years after initial treatment, hence the need for longterm follow-up studies. The same is true for Type 1 diabetes and cystic fibrosis, cases in which investigators are interested in complications and/or other outcomes that occur in the adult years. Many problems in child health necessitate long-term follow-up, which is both expensive and logistically difficult. Clinical trials improve health care. However, the number of clinical trials that involve children is limited, with researchers facing a unique set of obstacles. However, I remain optimistic that the development of research networks and the growing recognition that we need to conduct more clinical trials that involve children will lead to a growing number of RCTs. REFERENCES 1. C. Martinez-Castaldi, M. Silverstein, and H. Bauchner, Child versus adult research: The gap in high quality study design. Abstract, Pediatric Academic Societies Meeting, May 2006. Available: http://www.abstracts2view. com/pasall/search.php?query=Bauchner&where []=authors&intMaxHits=10&search=do. 2. http://www.cancer.org/downloads/STT/CAFF 2007PWSecured.pdf. 3. http://www.curesearch.org/. 4. http://www.aap.org/pros/abtpros.htm. 5. H. M. Corneli et al., A multicentre, randomized, controlled trial of dexamethasone for bronchiolitis. NEJM. 2007; 357: 331–339. Available: http://ctuprod.liv.ac.uk/mcrn/library/docs/ MCRNCC%202005-06.doc. 6. http://www.mcrn/org.uk. 7. http://www.bu.edu/crtimes/.

DNA BANK

research applications. Because DNA collected at a particular time can survive for a long time, the scope of use of the specimen may be far broader than specified at the time of collection.

THERU A SIVAKUMARAN Department of Epidemiology and Biostatistics, Case Western Reserve University, Cleveland, Ohio

1.1 Common or Complex Diseases Modest-sized genetic studies have been successful in the identification of single genes that, when mutated, cause rare, highly heritable mendelian disorders. However, studies that are aimed at identifying genes responsible for predisposition to common diseases, such as heart disease, stroke, diabetes, agerelated macular degeneration, cancer, and so on, are challenging. Common diseases are thought to be caused by many modest-sized, often additive effects, which represent the outcome of genetic predisposition, as well as lifestyle factors such as diet and smoking habits, and the environment in which we live and work. Locating these complex interactions between genes and the environment, and their role in disease depends on the collection of well-documented epidemiological clinical information, as well as biological specimens. The concept of obtaining and maintaining prospective cohorts to understand health and disease has a precedent in epidemiological research, with the investment in studies such as the Framingham Heart Study (3), but this paradigm has only recently been broadened to genetic epidemiology investigations. In response to the large sample sizes needed to obtain statistical power to detect the predisposing genes, as well as the recent technological developments in identifying the genetic variants, DNA biobanks are growing from small, local biological repositories to large population-based collections.

SUDHA K IYENGAR Departments of Epidemiology and Biostatistics, Genetics, and Ophthalmology, Case Western Reserve University, Cleveland, Ohio

1 DEFINITION AND OBJECTIVES OF DNA BIOBANKS DNA bank, also known as DNA biobank or Biobank, is an organized collection of DNA or biological samples, such as blood plasma, and so on, that are used to isolate DNA. These warehouses also contain information about the donor of the material, such as demographic characteristics, the type of disease associated with the sample, the outcome of the disease, treatment, and so on. According the American Society of Human Genetics (ASHG) policy statement (1), a DNA bank is defined as a facility to store DNA for future analysis. The American National Bioethics Advisory commission defined a DNA bank as a facility that stores extracted DNA, transformed cell lines, frozen blood or other tissue or biological materials for future DNA analysis (2). Although many types of DNA banks are available (described below), this article focuses mainly on the DNA banks that are meant for human genome research (i.e., academic laboratory DNA banks, populationbased public DNA biobanks, and commercial biobanks). Population-based public or national DNA biobanks represent a new paradigm for biomedical research and differ from traditional academic laboratory DNA banks in several aspects. The public or commercial DNA biobanks that obtain samples may not be engaged in research but may be only intermediary brokers who supply specimens to other researchers. They act as repositories that can be used for many

1.2 Pharmacogenomics Medications prescribed by a physician can have associated side effects called adverse drug reactions in some individuals. Approximately 100,000 people die each year from adverse reactions to drugs (4), and millions of people must bear uncomfortable or even dangerous side effects. Currently, no simple

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

DNA BANK

method is available to determine whether people will respond well, badly, or not at all to a particular drug. Therefore, all pharmaceutical companies follow a ‘‘one size fits all’’ system, which allows for development of drugs to which the ‘‘average’’ patient will respond. Once a drug is administered, it is absorbed and distributed to its site of action, where it interacts with numerous proteins, such as carrier proteins, transporters, metabolizing enzymes, and multiple types of receptors. These proteins determine drug absorption, distribution, excretion, target site of action, and pharmacological response. Moreover, drugs can also trigger downstream secondary events that may vary among patients, for example, seldane (terfenadine), which is an antihistamine used to treat allergies, hives (urticaria), and other allergic inflammatory conditions, was withdrawn because it caused increased risk for long QT syndrome and other heart rhythm abnormalities. Similarly, Vioxx (Merck & Co., Whitehouse Station, NJ), which is a drug prescribed to relieve signs and symptoms of arthritis and acute pain in adults, as well as painful menstrual cycles, has also been removed from the market because of an increased risk for heart attack and stroke. Thus, the way a person responds to a drug is a complex trait that is influenced by many different genes that code for these proteins. Without knowing all of the genes involved in drug response, scientists have found it difficult to develop genetic tests that could predict a person’s response to a particular drug. Pharmacogenomics is a science that examines the inherited variations in genes that dictate drug response and explores the ways these variations can be used to predict whether a patient will have a good response to a drug, a bad response to a drug, or no response at all. The concept of pharmacogenetics, which is a discipline that assesses the genetic basis of drug response and toxicity, originated from the clinical observation that many patients had very high or very low plasma or urinary drug concentrations, followed by the realization that the biochemical traits that lead to this variation were inherited. The studies conducted in early 1950s examine the drug metabolizing enzyme variants in

the cytochrome P450 family. The cytochrome P450 monooxygenase system is responsible for a major portion of drug metabolism in humans. This large family of genes has been intensely studied, and among the numerous subtypes, CYP2D6, 3A4/3A5, 1A2, 2E1, 2C9, and 2C19 play particularly critical roles in genetically determined responses to a broad spectrum of drugs (5). Because drugs that act on the central nervous system penetrate the blood-brain barrier, renal excretion is minimal for these compounds, and cytochrome P450 metabolism, particularly CYP2D6 and CYP2C19, provides the only means of effective drug elimination. The activity of the CYP2D6 enzyme is extremely variable because of more than 50 genetic variants, and patients who are homozygous for the CYP2D6 null alleles have impaired degradation and excretion of many drugs, which include debrisoquine, metoprolol, nortriptyline, and propafone (6). These patients are termed ‘‘poor metabolizers’’ for CYP2D6 selective drugs, and they are more likely to exhibit adverse drug reactions. Similarly, patients who are homozygous for the ‘‘null’’ allele of the P450 isoform CYP2C19 are highly sensitive to omeprazole, diazepam, propranolol, mephenytoin, amitriptyline, hexobarbital, and other drugs. Today, clinical trial researchers use genetic tests for variations in cytochrome P450 genes to screen and monitor patients prior to drug administration. 1.3 Finding-Causative Genetic Factors The process of identifying genes that are responsible for common disease as well as drug response begins with scanning the genetic variations in the entire DNA sequence of many individuals with the disease, or those who respond to a particular drug, and contrasting this information with that from individuals without the disease, or who do not respond to the drug. The most commonly found human genetic variations between two individuals are variants that involve a single nucleotide, named single nucleotide polymorphisms (SNPs). In comparing two haploid genomes, a SNP was observed to occur on average every 1331 bases. When more than two haploid genomes are surveyed, a SNP

DNA BANK

is expected to occur on average every 240 bases (7). In the general population, the collective human genome is expected to contain about 10 million SNPs, and testing all of them would be very expensive. Systematic studies that identify these common genetic variants are facilitated by the fact that it is not necessary to interrogate every possible variant. The International HapMap Project (8) showed that a particular SNP allele at one site often carries information about specific alleles at other nearby variant sites; this association is known as linkage disequilibrium (LD). Thus, genetic variants that are located close to each other tend to be inherited together; these variants are often labeled as a haplotype or LD block. Because little or no recombination is observed within this haplotype block, a small subset of SNPs, called tag SNPs, are sufficient to distinguish each pair of patterns in the block and this reduces the necessity of querying all 10 million SNPs. Approximately 1 million common SNPs carry most information on common variants. Examination of these SNPs by traditional methods, such as sequencing, is very slow and expensive, but DNA microarray technology has made it possible to genotype large numbers of SNPs in a single array/chip. In the past few years, the number of SNPs available on a single microarray chip has grown from 10,000 to 1,000,000. Once a particular area of the genome is associated with disease is identified through scanning thousands of tag SNPs, it should be possible to zero in on specific genes involved in the disease process much more quickly. This process occurs because when a new causal variant develops through mutation, it is initially tethered to a unique chromosomal region that is marked by a haplotype block. If a disease is associated with a tag SNP within that particular haplotype block, then variants that contribute to the disease might be located somewhere within or near that haplotype block. In summary, it is feasible to use surrogate markers to locate disease genes or to find genes that affect the action of pharmaceutical compounds. Biobanks play a significant role in this discovery process.

2

3

TYPES OF DNA BIOBANKS

Several kinds of DNA banks are based on usage and location. 2.1 Academic DNA Banks Repositories are housed in the laboratories of researchers who study one or more genetic disorders. This collection typically contains DNA samples obtained from families at risk for the disease of interest or from cases with certain common disease, such as heart disease, diabetes, cancer, and so on, as well as samples from healthy controls. One of the first academic biobanks was the Centre d’etude du Polymorphism Humaine (CEPH) (9), which facilitated human gene mapping and was the nexus for the Human Genome Project and the International HapMap Project. Unlike most other collections, the CEPH collection was initially assembled with no information on disease. 2.2 Population-Based DNA Biobanks These centers are large private/public repositories of donated human DNA, with health and other relevant information collected from volunteers with and without the disease. The aim of public initiatives, which are supported by local governments, is to provide specimens to the scientific community for research geared toward identifying genes that contribute to common disease as well as to drug response and toxicity. The volunteers for these biobanks are approached mainly through their health care providers. Most pharmaceutical companies have their own large collections of DNA samples acquired through research collaborations or from subjects enrolled in clinical trials (10). 2.3 Commercial Biobanks Several commercial biobanks have been established in the last few years with the goal of storing personal genetic information and providing this information to researchers, as well as to pharmaceutical companies, for-a-fee. Some commercial biobanks in the United States are Genomics Collaborative, Inc., Ardais Corporation, and DNA Sciences Inc. (11). Genomics Collaborative Inc. claims

4

DNA BANK

to have a repository that contains about 500,000 biological samples and also health data from 120,000 people from all over the world. Ardais Corporation, in agreement with several physicians, recruits patients to donate samples, and the repository contains more than 220,000 tissue samples collected from over 15,000 donors. DNA Sciences Inc. has a collection of over 18,000 DNA samples in its repository (11). 2.4 State Forensic DNA Data Banks DNA profiling and databases have become common in criminal investigation and prosecution. Forensic DNA data banks serve as repositories for long-term storage of DNA collected from certain defined criminal offenders and the profiles derived from analysis of the DNA. Most states in the United States permit DNA profile databasing of offenders, of missing persons and their relatives, and of DNA profiles from criminal-case evidence in which the depositor is unknown. The national DNA database of the United Kingdom also stores DNA samples and other specimens collected from crime scenes as well as samples taken from individuals in police custody. 2.5 Military DNA Data Banks Military DNA banks assist in the identification of human remains. The U.S. military uses DNA profiles in place of traditional means of identification such as ‘‘dog tags,’’ and new recruits routinely supply blood and saliva samples used to identify them in case they are killed in action. 2.6 Nonhuman DNA Banks DNA banking has become an important resource in worldwide efforts to address the biodiversity crisis, manage the world’s genetic resources, and maximize their potential. Several plant and animal DNA banks are located around the world. A Japan-based National Institute of Agrobiological sciences’ DNA Bank was established in 1994 as a section of Ministry of Agriculture, Forestry, and Fisheries. It maintains the DNA materials and genomic information that were collected as part of rice, animal, and silkworm genome projects (12). Royal Botanic

Gardens (RBG) at Kew established a DNA bank database with the aim of extracting DNA from all plant species that are grown at RBG. Currently, it has genomic DNA from over 22,000 plants in its facility, which are stored at −80◦ C, and these samples are sent to collaborators all over the world for research purposes (13). The Australian plant DNA bank has a comprehensive collection of DNA from both Australian native and important crop plant species in its repository. It provides DNA samples and the associated data to the research community (14). Recently, Cornell University’s College of veterinary medicine has established a DNA bank called Cornell Medical Genetic Archive to understand the genetic basis of disease in many species such as dogs, cats, horses, cows, and exotic animals. The blood samples taken from these species, with the owner’s written consent, will be used for DNA extraction, and these samples will be used by the researchers at Cornell (15). In summary, DNA banks can be speciesspecific, and their use is tailored to particular needs. In general, human biobanks have been established to (1) study a specific genetic disease under collaborative guidelines; (2) to accrue information from populations of entire countries (see below) for current surveillance or for future use; (3) in pharmacogenetic studies, as part of clinical trials; and (4) in hospital based settings to be used for current and future research.

3 TYPES OF SAMPLES STORED

A variety of biological materials is now banked for isolating genomic DNA for genetic research. All nucleated cells, which include cells from the hair follicle, buccal swab, saliva, and urine, are suitable for isolating DNA. Many large biobanks obtain whole blood, as it provides amounts of DNA necessary for current applications. Whole blood is generally stored in three different forms in the biobank (i.e., genomic DNA, dried blood spots, and Epstein-Barr virus (EBV)transformed cell lines).

DNA BANK

3.1 Genomic DNA Blood is most often collected using ethylenediaminetetraacetic acid (EDTA) or anticoagulants that include heparin and acid citrate dextrose and genomic DNA is isolated using homebrewn methods or commercially available kits. The purified genomic DNA is stored at 4◦ C or −20◦ C. Most samples stored in the biobanks are isolated genomic DNA (16). Often, an EDTA biospecimen can also serve as a source of material for other bioassays, as this type of specimen is suitable for a variety of biochemical, hematologic, and other types of tests (17).

5

EBV-transformed lymphoid cells are stored at even lower temperatures. Some factors that generally affect the quality or integrity of DNA samples are the following: 1. Improper storage conditions, such as temperature, evaporation, frequent freeze, and thaw, may lead to the degradation of samples. 2. Possibility of cross contamination of neighboring samples either by sample carry over during handling or by sample switches during processing. 3. Improper handling of DNA samples (i.e., improper mixing and pipetting).

3.2 Dried Blood Spots Dried blood spots, which are also known as Guthrie cards, are generally used in newborn screening programs to quantify the analytes that include metabolic products, enzymes, and hormones (18). Blood spots from military personnel are also stored to serve as biologic ‘‘dog tags’’ for identification purposes. Blood spots may yield enough DNA to genotype several variants (19,20), although with current genotyping technology it is feasible to genotype thousands of markers. The United Kingdom’s national biobank is planning to spot whole blood onto filter paper in a 384well configuration for storage (10). 3.3 EBV Transformed Cell Lines EBV transformed lymphocytes provide an unlimited source of DNA, to be used for whole genome scans. These cells are also used for functional studies. (See Cell Line for additional details). 4 QUALITY ASSURANCE AND QUALITY CONTROL IN DNA BIOBANKS The quality of the DNA and other biological specimens in biobanks is of primary importance, as they are the resource for the downstream processes such as genotyping, sequencing, and other bioassays. The storage of these specimens varies according the type of samples, that is, DNA samples are generally stored at 4◦ C and −20◦ C; blood and other tissue samples used for DNA extraction are stored in −80◦ C; and viable cells such as

The ASHG policy statement on DNA banking (1) proposed minimal standards for quality assurance, which include: (1) A DNA bank should occupy space equipped with an alarm system that is separate from other functions; (2) the bank should maintain a manual of procedures and train personnel in meticulous technique; (3) samples should be coded and very few individuals have access to the identity; (4) records should be maintained for the receipt, disposition, and storage of each sample; (5) samples should be divided and stored in more than once place; and (6) control samples should analyzed before deposit and at periodic intervals to demonstrate that DNA profiles are unaffected by storage. To ensure the quality and integrity of biological specimens stored over a long term, biobanks follow standard procedures at every step, such as collection, transport, storage, and handling. Some procedures that are followed to ensure the quality of samples include: • Proper labeling of samples and the asso-

ciated data at the time of collection and during downstream processes. • All samples are barcoded and scanned at every step during procedures. Storage units are highly organized and compartmentalized, in which every sample occupies a unique and computer-tracked location. • Measures are taken to ensure the safety of the samples as well as to prevent damage caused by disasters, such as

6

DNA BANK

•

•

•

•

availability of backup power and backup cooling systems in case of power failure. Conducting DNA profiling of each sample and generating an extensive database of signature profiles prior to storage for later identification. Storing samples in conditions that protect them from degradation and following procedures to maintain their integrity caused by frequent freeze and thaw. Performing periodic maintenance checks to ensure sample traceability and integrity. This process is usually done by DNA profiling samples and comparing the results with the original profiles stored in a database. Finally, the clinical information that accompanies the samples and other relevant information is stored in computer systems that are run on complex database management and analysis software.

With proliferation in the number of samples stored in biobanks and the increased need to access more samples quickly and accurately, researchers started using automated systems for sample storage and retrieval (10). These automated sample management systems dramatically increase the throughput and eliminate manual errors. Some automated biological management systems are the DNA archive and reformatting (DART) system that is used by AstraZeneca, the Polar system developed by The Automated Partnership, and the RTS Life Science’s automated sample management systems. The sample storage and use of DART system at AztraZeneca is controlled by a Laboratory Information Management System. The DART system can store over 400,000 tubes and can handle over 650 tubes per hour. A complete description of the automation of sample management can be observed elsewhere (10,21). 5

ETHICAL ISSUES

Biobanks cause concerns about several technical, ethical, and legal challenges. In the

realm of ethics, these concerns revolve primarily around how donors of biological material or data will be assured that their privacy and interests will be protected, given the increasing number of large for-profit companies and entirely private DNA biobanks. 5.1 Informed Consent Informed consent is a key ethical standard required for clinical research, and its underlying principle is respect for the autonomy of individuals. The codes of ethical practice, such as the Declaration of Helsinki (22), state that a research participant must be provided the opportunity for voluntary informed consent before a study begins. In the United States, the rights and welfare of the human subjects in genetic research is protected by institutional review boards (IRB). The IRB is an independent ethics committee that is governed by principles established by the Office of Human Research Protections in the Department of Health and Human Services (23), and it is located at the institution where the research is conducted. IRBs are also run by commercial enterprises to assist governmental and industrial bodies in regulating research. In general, informed consent is requested using a special form approved by the IRB. All aspects relative to the handling of the preserved material, as well as the data, must be written clearly in the consent form. Elements of a traditional model of informed consent include the following: • The document should explain the pro-

posed research, its purpose, as well as the duration, procedures of the study, and description of potential risks. • The document should state that the sample will be used exclusively for diagnostic or research purpose and never for direct profit. If profit is the goal, then the beneficiaries should be clearly listed. • The potential benefits derived from the use of these samples to the individual and/or the entire community should be explained in lay terms. • The document should clarify the procedure for handling the data to ensure anonymity as well as the protection of

DNA BANK

confidentiality of the sample and its related investigations. • The informed consent document should state whom to contact for questions about the subjects’ rights or about studyspecific questions. • Finally, information on how to withdraw consent should be outlined, with provisions for immediate withdrawal of samples and participation. Although the principle of informed consent is recognized, its translation into the biobanks that store samples and data for long-term use, and provide this data to any entity with appropriate permissions (such as an IRB) on a fee-for-service basis, leads to many practical difficulties (24–26). Some of these difficulties include: (1) potential participants can only be given information about the sort of research that will be performed with the present knowledge, but it would be difficult to describe all the types of research that would be feasible at a later date; (2) at the time of recruitment, it is not possible to describe the information that will be subsequently collected from the volunteers because they would not know which disease they are going to develop (27); (3) it is not possible to give information about the research or research teams who will be using the samples; and (4) it is not possible to give information back to the participant on their risk profile if the research is being conducted by a third party. Because of these difficulties, no clear international guidelines on informed consent for biobanks are available. Several models of informed consent are found in the literature (27–29), and the common recommendation proposed by the United Nations Educational, Scientific and Cultural Organization (UNESCO) International Bioethics Committee is called ‘‘Blanket consent.’’ Herein, research participants can consent once about the future use their biological samples (30). The U.K. Biobank and the Estonian Biobank use the blanket approach (29,27), whereas the Icelandic Biobank uses two different consent forms: Form ‘‘A’’ authorizes the use of samples for specific research, and form ‘‘B’’ authorizes the use of samples for specific research and for

7

additional similar research if approved by National Bioethics Committee (31) without further contact with the participant. 5.2 Confidentiality The term confidentiality is defined as the safety of identifiable information that an individual discloses with the expectation that it will not be provided to third parties (23). The possibility of tracing the person from whom sample and data were derived varies according to how samples are linked to their donor identity. In general, the labeling of a sample can range from irreversible anonymity to complete identification based on the choices expressed in the written informed consent form. 5.3 Anonymous Samples are identified only by a code from the start, and all identifying links to the donors are destroyed at the time of collection. Therefore, it is not possible to link the sample to a given person, and this mechanism offers the most protection of confidentiality. 5.4 Anonymized The patient’s personal data are removed after a code is assigned, after which it is no longer possible to connect the two. 5.5 Double-Coding A anonymous number is assigned to the participants as well as to the samples, and a code that links the anonymous number and the personal information is placed is secure location, which is accessible to the person in charge of the biobank and his/her immediate colleagues. 5.6 Identifiable The identity of individuals is directly linked to the samples or data. This option is only possible on explicit request by the interested party and in any case only for exclusive personal and family use.

8

6

DNA BANK

CURRENT BIOBANK INITIATIVES

Centralized population-based biobanks have been established in at least eight countries, which include Iceland, the United Kingdom, Canada, Japan, Sweden, Latvia, Singapore, and Estonia. Population homogeneity and a strong interest in genealogy made for the establishment of the world’s first populationbased biobank in Iceland. deCODE Genetics, which is a private company, successfully partnered with the Icelandic parliament in 1998 to create and operate a centralized database of nonidentifiable health data (32). They aimed to enroll most of the country’s 270,000 citizens and currently have genetic samples from more than 100,000 volunteers (10,33). deCODE links data from the Iceland Health Database as well as public genealogical records. In Canada, three population-based biobank initiatives are in various stages of development. One project is CARTaGENE, which is a publicly funded biobank in Quebec. In the first phase, it aims to recruit a random sample of about 20,000 adults between 45 to 69 years, which represents about 1% of its citizens of this age group in the selected regions of Quebec. This resource will be available to researchers who are interested in conducting population-based genomic research (34). Two other population-based genetic biobank projects planned in Canada are the national birth control cohort and a longitudinal study on aging. The U.K. Biobank was started in August 2006 with the goal to study the separate and combined effects of genetic and nongenetic risk factors in the development of multifactorial diseases of adult life. This program plans to involve 0.5 million people 45–69 years of age and intends to collect environmental and lifestyle data, as well as information from medical records, along with biological samples. The center will also coordinate activities of the six scientific regional collaborating centers; each is a consortium of academic and research institutions responsible locally for recruitment and collection of data and samples. This biobank will serve as a major resource that can support a diverse range of research intended to improve the prevention, diagnosis, and treatment of illness and the promotion of health throughout society (35).

A genome project set up by the tiny European country of Estonia, founded in 2001, is accelerating plans to take blood samples from three quarters (i.e., 1 million of its 1.4 million) of its population and promises to be the biggest such initiative. This project aims not only to enable research on the genetic and nongenetic components of common disease, but also to create a biological description of a large and representative sample of the Estonian population (36). The Latvian Genome Project, which was launched in 2002, is a unified national network of genetic information and data processing to collect representative amounts of genetic material for genotyping of the Latvian population and to compare genomic data with clinical and pedigree information. This project is planned for a period of 10 years, with an expected sample size of 60,000 in its pilot phase (37). The Swedish National Biobank is estimated to house about 50–100 million human samples, which increases at the rate of 3–4 million samples per year (38). GenomEUtwin is a project that aims to analyze twin and general cohorts to determine the influence of genetic and nongenetic factors that cause predisposition to obesity, stature, coronary heart disease, stroke, and longevity, and to create synergies in genetic epidemiology. The implementation is coordinated by the Finland National Public Health Institute and University of Helsinki and it builds on existing twin cohorts from a few other European countries (39). The Biobank Japan project, which was established in 2003 with the support of the Japanese government, plans to collect DNA, sera, and clinical information from 300,000 patients in participating hospitals (40). Genetic Repositories Australia is a central national facility for establishing, distributing, and maintaining the long-term secure storage of human genetic samples. They provide Australian medical researchers with DNA and cell lines, as well as associated clinical information, collected from patients and controls (41). China has also launched its first biobank program, called ‘‘The Guangzhou Biobank Cohort Study,’’ with the aim of creating profiles on about 11,000 Guangzhou people aged above 50 years in the first phase (42).

DNA BANK

Several biobanks are located in the U.S., and one of the largest populationbased biobanks is the personalized medicine research project of the Marshfield Medical Clinic in Wisconsin. The investigators of this project are planning to enroll at least 100,000 people who live in northern and central Wisconsin and make the samples, as well as data, available to other researchers. Currently, this repository contains information from 40,000 participants (43,44). The Center for Genetic Medicine of Northwestern University also initiated the first hospital DNA biobank, and it plans to collect DNA samples with associated clinical healthcare information from 100,000 volunteers who receive their healthcare at Northwesternaffiliated hospitals and clinics (45). The DNA bank and tissue repository at the Center for Human Genetics at Duke University is one of the oldest academic DNA banks in the United States, and it contains samples of more than 127,500 individuals (46). Howard University is also planning to establish a large DNA and health database on individuals of African American decent. They aim to enroll about 25,000 volunteers over 5 years and use the data to study the genetic and lifestyle factors that contribute to common diseases (47). Children’s Hospital of Philadelphia has launched a biobank of Children’s DNA with the aim to collect DNA from 100,000 children and use it to study common childhood diseases such as asthma, diabetes, obesity, and so on. They plan to create a database of children’s genetic profiles, which hospital researchers can use to develop diagnostic tests and drugs (48). The U.S. Department of Veterans Affairs has also proposed a national gene bank that would link DNA donated by up to 7 million veterans and their family members with anonymous medical records (49). Other similar initiatives are being implemented in many parts of the United States, but no centralized federally mandated repository is active. 7

CONCLUSIONS

Biobanking initiatives have been globally embraced because the value of these enterprises in solving the problem of emerging chronic diseases has been recognized.

9

However, translating the basic knowledge obtained from specific gene variants to the clinic remains the current challenge. The current pipeline to pharmaceutical drug discovery is very inefficient, and in general, the yield has been disappointingly low. Although, it was initially hoped that the Human Genome Project and the International HapMap Project would accelerate drug discovery, this phenomenon did not follow the hypothesized pace because discovery of variants in genes and drug targets do not follow parallel paths. The first has been the domain of academic scientists interested in the biological mechanisms of disease, whereas the latter, drug discovery and chemical screening, has been the province of the pharmaceutical industry, each with its respective governance and pipeline. In the past decade, there has been a proliferation in the size of biobanks and the clinical data collected by academic and industry researchers alike, all with the hope that the superior sample sizes, and revolutions in technology will enable them to develop better models for disease and health. REFERENCES 1. ASHG Ad Hoc Committee on DNA technology, DNA banking and DNA analysis—points to consider. Am. J. Hum. Genet. 1988; 42: 781–783. 2. National Bioethics Advisory Commission, Research Involving Human Biological Materials: Ethical Issues and Policy Guidance. 1999. National Bioethics Advisory Commission, Rockville, MD. 3. Framingham Heart Study. Available at: http://www.framinghamheartstudy.org/. 4. J. Lazarou, B. H. Pomeranz, and P. N. Corey, Incidence of adverse drug reactions in hospitalized patients—a meta-analysis of prospective studies. JAMA. 1998; 279: 1200–1205. 5. W. W. Weber, Pharmacogenetics. New York: Oxford University Press, 1997. 6. F. Broly, A. Gaedigk, M. Heim, M. Eichelbaum, K. Morike, and U. A. Meyer, Debrisoquine sparteine hydroxylation genotype and phenotype— analysis of common mutations and alleles of Cyp2D6 in a european population. DNA Cell. Biol. 1991; 10: 545–558. 7. The International SNP Map Working Group. Nature 2001; 409: 928–933.

10

DNA BANK

8. The International HapMap Consortium. The International HapMap Project. Nature 2003; 426: 789–796. 9. Centre d’etude du Polymorphism Humaine (CEPH). Available at http://www.cephb.fr/. 10. Thornton, M., Gladwin, A., Payne, R., Moore, R., Cresswell, C., McKechnie, D., Kelly, S., and March, R. Automation and validation of DNA-banking systems. Drug Discov. Today 2005; 10: 1369–1375. 11. J. Kaiser, Biobanks—Private biobanks spark ethical concerns. Science 2002; 298: 1160. 12. NIAS DNA Bank. Available at: http://www. dna.affrc.go.jp/about/. 13. Royal Botanic Gardens, Kew: Plant DNA Bank database. Available at: http://www. kew.org/data/dnaBank/homepage.html. 14. The Australian Plant DNA Bank. Available at: https://www.dnabank.com.au/. 15. The Cornell Medical Genetic Archive. Available at; http://www.vet.cornell.edu/research/ DNABank/intro.htm. 16. I. Hirtzlin, C. Dubreuil, N. Preaubert, J. Duchier, B. Jansen, J. Simon, P. L. de Faria, A. Perez-Lezaun, B. Visser, G. D. Williams, and A. Cambon-Thomsen, An empirical survey on biobanking of human genetic material and data in six EU countries. Eur. J. Hum. Genet. 2003; 11: 475–488. 17. S. Clark, L. D. Youngman, A. Palmer, S. Parish, R. Peto, and R. Collins, Stability of plasma analytes after delayed separation of whole blood: implications for epidemiological studies. Int. J. Epidemiol. 2003; 32: 125–130. 18. E. R. B. Mccabe, S. Z. Huang, W. K. Seltzer, and M. L. Law, DNA microextraction from dried blood spots on filter-paper blotters—potential applications to newborn screening. Hum. Genet. 1987; 75: 213–216. 19. J. V. Mei, J. R. Alexander, B. W. Adam, and W. H. Hannon, Use of filter paper for the collection and analysis of human whole blood specimens. J. Nutr. 2001; 131: 1631S–1636S. 20. K. Steinberg, J. Beck, D. Nickerson, M. Garcia-Closas, M. Gallagher, M. Caggana, Y. Reid, M. Cosentino, J. Ji, D. Johnson, R. B. Hayes, M. Earley, F. Lorey, H. Hannon, M. J. Khoury, and E. Sampson, DNA banking for epidemiologic studies: A review of current practices. Epidemiology 2002; 13: 246–254. 21. S. Mahan, K. G. Ardlie, K. F. Krenitsky, G. Walsh, and G. Clough, Collaborative design for automated DNA storage that allows for rapid, accurate, large-scale studies. Assay Drug Devel. Technol. 2004; 2: 683–689.

22. World Medical Association, Declaration of Helsinki: Ethical principles for medical research involving human subjects. 2000. http://www.wma.net/e/policy/pdf/17c.pdf. 23. IRB Guidebook, Office of Human Research Protections, Department of Health and Human Services. Available at: http://www. hhs.gov/ohrp/irb/irb guidebook.htm. 24. A. Cambon-Thomsen, Science and society—The social and ethical issues of post-genomic human biobanks. Nat. Rev. Genet. 2004; 5: 866–873. 25. M. Deschenes, G. Cardinal, B. M. Knoppers, and K. C. Glass, Human genetic research, DNA banking and consent: a question of ‘form’? Clin. Genet. 2001; 59: 221–239. 26. B. Godard, J. Schmidtke, J. J. Cassiman, and S. Ayme, Data storage and DNA banking for biomedical research: informed consent, confidentiality, quality issues, ownership, return of benefits. A professional perspective. Eur. J. Hum. Genet. 2003; 11: S88–S122. 27. D. Shickle, The consent problem within DNA biobanks. Stud. Hist. Phil. Biol & Biomed. Sci. 2006; 37: 503–519. 28. M. G. Hansson, J. Dillner, C. R. Bartrarn, J. A. Carlson, and G. Helgesson, Should donors be allowed to give broad consent to future biobank research? Lancet Oncol. 2006; 7: 266–269. 29. K. J. Maschke, Navigating an ethical patchwork—human gene banks. Nat. Biotechnol. 2005; 23: 539–545. 30. UNESCO. Human genetic data: Preliminary study by the IBC on its collection, processing, storage and use. 2002. http://portal.unesco. org/shs/en/files/2138/10563744931Rapfinal gendata en.pdf/Rapfinal gendata en.pdf. 31. V. Arnason, Coding and consent: moral challenges of the database project in Iceland. Bioethics 2004; 18: 27–49. 32. M.A. Austin, S. Harding, and C. McElroy, Genebanks: a comparison of eight proposed international genetic databases. Community Genet. 2007; 6: 500–502. 33. deCODE Genetics. Available at: http://www. decode.com/biorepository/index.php. 34. CARTaGENE project. Available at http:// www.cartagene.qc.ca/accueil/index.asp. 35. The UK Biobank. Available at http://www. ukbiobank.ac.uk/about/what.php. 36. The Estonian Genome Project. Available at http://www.geenivaramu.ee/index.php?lang= eng&sub=58.

DNA BANK 37. The Latvian Genome Project. Available at http://bmc.biomed.lu.lv/gene/. 38. Swedish National Biobank Program. Available at http://www.biobanks.se/. 39. GenomeEUtwin. Available at http://www. genomeutwin.org/index.htm. 40. Nakamura, Y. Biobank Japan Project: Towards personalised medicine. 2007. http:// hgm2007.hugo-international.org/Abstracts/ Publish/Plenaries/Plenary01/hgm02.html. 41. Genetic Repositories Australia. Available at http://www.powmri.edu.au/GRA.htm. 42. C. Q. Jiang, G. N. Thomas, T. H. Lam, C. M. Schooling, W. S. Zhang, X. Q. Lao, R. Adab, B. Liu, G. M. Leung, and K. K. Cheng, Cohort profile: the Guangzhou Biobank Cohort Study, a Guangzhou-Hong Kong-Birmingham collaboration. Int. J. Epidemiol. 2006; 35: 844–852. 43. C. A. McCarty, R. A. Wilke, P. F. Giampietro, S. D. Wesbrook, and M. D. Caldwell, Marshfield Clinic Personalized Medicine Research Project (PMRP): design, methods and recruitment for a large population-based biobank. Personalized Med. 2005; 2: 49–79. 44. H. Swede, C. L. Stone, and A. R. Norwood, National population-based biobanks for genetic research. Genet. Med. 2007; 9: 141–149. 45. NUgene Project. Available at: http://www. nugene.org/. 46. DNA Bank and Tissue Repository, Center for Human Genetics, Duke University. Available at http://www.chg.duke.edu/ research/dnabank.html. 47. J. Kaiser, Genomic medicine—AfricanAmerican population biobank proposed. Science 2003; 300: 1485–1485. 48. J. Kaiser, Genetics—US hospital launches large biobank of children’s DNA. Science 2006; 312: 1584–1585. 49. J. Couzin, Veterans Affairs—Gene bank proposal draws support—and a competitor. Science 2005; 309: 684–685.

CROSS-REFERENCES Genetic Association Analysis Pharmacogenomics Microarray Repository Two-Stage Genetic Association Studies

11

ESCALATION AND UP-AND-DOWN DESIGNS

at a time to target a wide range of quantiles
. Storer (10) and Korn et al. (11) used decision rules of group designs to suggest several designs for dose finding. Among the designs studied in Korn et al. (11) and Shih and Lin (12) were versions of the traditional or 3 + 3 design widely used in dose-finding trails in oncology. Lin and Shih (13) generalized the 3 + 3 by introducing A + B designs.

ANASTASIA IVANOVA University of North Carolina at Chapel Hill Chapel Hill, North Carolina

Up-and-down designs are widely used in both preclinical (animal) and clinical dose-finding trials. These designs operate on prespecified set of doses of the investigational product. The dose for the next subject (or a cohort of subjects) is repeated, increased, or decreased according to the outcome of the subject (or a cohort of subjects) having received the immediate prior dose. The decision rules used in up-and-down designs are very simple and intuitive. The goal is to find the maximally tolerated dose (MTD). The MTD is sometimes defined as the dose just below the lowest dose level with unacceptable toxicity rate; or the MTD can be defined as the dose with the probability of toxicity closest to a prespecified rate
. The underlying model assumption is that the probability of toxicity is a nondecreasing function of dose. Upand-down designs do not require any other assumptions on the dose–toxicity relationship.

2

BIASED COIN DESIGNS

In biased coin designs, subjects are assigned to a dose level of a drug one at a time. The biased coin design from Durham and Flournoy (4), developed for the case of
≤ 0.5, uses a biased coin with the probability of heads equal to b =
/(1−
), 0 ≤ b ≤ 0.5. If the outcome of the most recent subject is toxicity, the dose is decreased. If the outcome of the most recent subject is no toxicity, the dose is increased if the biased coin’s toss results in heads, and repeated if the toss results in tails. This process is continued until prespecified number of subjects is assigned. To determine the next assignment, biased coin designs use the outcome of a single (most recent) subject only and hence are not efficient when data from more than one subject at a dose are available.

1 HISTORY OF UP-AND-DOWN DESIGNS FOR DOSE FINDING

3 Von B´ek´esy (1) and Dixon and Mood (2) described an up-and-down design where the dose level increased after a nontoxic response and decreased if toxicity is observed. This approach clusters the treatment distribution around the dose for which the probability of toxicity is equal to
= 0.5. To target any quantile
, Derman (3) modified the decision rule of the design using a biased coin with the probability of heads computed as a function of the target quantile
. Durham and Flournoy 4, 5 considered two biased coin designs in the spirit of Derman. Giovagnoli and Pintacuda (6) later obtained interesting theoretical results on biased coin designs. Wetherill (7) and Tsutakawa (8, 9) proposed assigning subjects in groups rather than one

GROUP UP-AND-DOWN DESIGNS

Group up-and-down designs (7–9) are the building blocks for many dose-finding designs (14). A group up-and-down design induces a Markov chain on the set of doses. Hence, some statistical properties of a group design can be obtained using the Markov chain theory. Gezmu and Flournoy (15) studied small sample properties of group designs. Ivanova et al. (16) studied large sample properties. Let d1 < . . . < dK be dose levels selected for the study, and p1 < . . . < pK be corresponding probabilities of toxicity at these doses. In a group up-and-down design, subjects are assigned to treatment in cohorts of size s starting with the lowest dose. Let X be the number of toxicities in the most recent

Wiley Encyclopedia of Clinical Trials, Copyright © 2007, John Wiley & Sons, Inc.

1

2

ESCALATION AND UP-AND-DOWN DESIGNS

cohort assigned to dose dj , X ∼ Bin(s,pj ), where Bin(s,p) denotes the binomial random variable with parameters s and p. Let cL and cU be two integers such that 0 ≤ cL < cU ≤ s. Then, 1. if X ≤ cL , the next cohort of s subjects is assigned to dose dj+1 . 2. if cL < X < cU , the dose is repeated for the next cohort of s subjects. 3. if X ≥ cU , the next cohort of s subjects is assigned to dose dj−1 . Appropriate adjustments are made at the lowest and highest dose so that the assignments stay within d1 , . . . , dK . The process is continued until a prespecified number of patients are assigned. Assignments in group up-and-down design for large sample sizes are clustered around the dose with toxicity rate
s , where
s is the solution of Pr{Bin(s,
s ) ≤ cL } = Pr{Bin(s,
s ) ≥ cU }. That is, if there is a dose dk such that
s = pk , the assignments are clustered around dk . If pk – 1 <
s < pk , the assignments are clustered around dose k − 1 or k (16). To find
s for given parameters s, cL , and cU , one needs to write the equation above using formulas for binomial probabilities. For example, for the group design with group size s, cL = 0 and cU = 1, the equation has the form (1−
s )s = 1 − (1−
s )s with the solution
s = 1 − (0.5)1/s . For most of the group up-and-down designs, closed form solutions of this equation do not exist, but the equation can be easily solved numerically. For most practical applications of group designs, approximation
s ≈ (cU /s − cL /s)/2 can be used. 4

ESCALATION DESIGNS

Escalation designs are widely used in firsttime-in-human phase I studies in a variety of therapeutic areas. Such trials usually investigate three to eight dose levels, and patients are typically assigned to cohorts with a size of six to eight patients, some among whom are receiving placebo (instead of the investigational agent). All patients receiving investigational product in a cohort receive the same

dose. Doses are increased by one level for each subsequent cohort. The trial is stopped when an unacceptable number of adverse events or an unacceptable type of adverse event is observed, when the highest dose level is reached, or for other reasons as well. The ‘‘target dose,’’ the dose recommended for future trials, is usually determined on the basis of the rates or types of adverse events (or both) at the dose levels studied, in addition often to pharmacokinetic parameters and considerations. If only the rate of adverse events is taken into account with the goal of finding the target dose, the dose with adverse events rate of
, the escalation design can be more formally defined as follows. Patients are assigned to treatment in cohorts of size m starting with the lowest dose. Let design parameter CU be an integer such that 0 ≤ CU < m. Assume that the most recent cohort of patients was assigned to dose level dj , j = 1, . . . , K − 1. Let X be the number of adverse events in a cohort assigned to dj . Then if X ≤ CU , the next cohort of m patients is assigned to dose dj+1 ; otherwise, the trial is stopped. The dose one level below the dose where > CU adverse events were observed is the estimated target dose. If the escalation was not stopped at any of the doses, the highest dose dK is recommended. The obvious choice of CU is CU /m ≤
< (CU + 1)/m. The frequency of stopping escalation at a certain dose level depends on the adverse event rate at this dose as well as rates at all lower dose levels. Ivanova (14) studied how cohort size and the choice of dose levels affect the precision of the escalation design. Clearly, designs with large cohort size have better statistical precision, hence values m < 6 are not recommended. From a safety standpoint, especially in early phase studies, it might be risky for all m patients in a cohort to receive the investigational product at the same time. In such trials, the cohort of m can be split, for example, into two subcohorts. Patients are assigned in the first subcohort and then to the second, if treatment in the first cohort was well tolerated. A similar strategy is used in the A + B designs described in the next section.

ESCALATION AND UP-AND-DOWN DESIGNS

5

A + B DESIGNS

The A + B designs (13) include a stopping rule as in escalation designs but save resources at lower doses. In general terms, A + B design without de-escalation is defined as follows. Let A and B be positive integers. Let cL , cU , and CU be integers such that 0 ≤ cL < cU ≤ A, cU − cL ≥ 2, and cL ≤ CU < A + B. Patients are assigned to doses in cohorts of size A starting with the lowest dose. Assume that the most recent cohort was a cohort of A patients assigned to receive dose dj , j = 1, . . . , K − 1. Let X A be the number of toxicities in a cohort of size A assigned to dose dj , and let X A+B be the number of toxicities in a cohort of size A + B. Then,

Table 1. Examples of A + B designs.

1. 0 ≤ cL < cU ≤ A, cU − cL ≥ 2, and cL ≤ CU < A + B. 2. (cU /A − cL /A)/2 ≈
or slightly exceeds
.

A = B = 5, cL A = B = 3, cL A = B = 4, cL A = B = 4, cL A = B = 3, cL

0.1 0.2 0.3 0.4 0.5

= 0, cU = 0, cU = 0, cU = 1, cU = 0, cU

= 2, CU = 2, CU = 3, CU = 3, CU = 3, CU

=1 =1 =2 =3 =3

3. CU /(A + B) <
< (CU + 0.5) (A + B). The choice of cohort sizes A and B so that A ≤ B yields more effective designs on average. Several A + B designs that satisfy the rules above are presented in Table (1). The values of
in Table (1) were computed from constraint [3] as the midinterval rounded to the nearest decimal. Designs described here are A + B designs without dose de-escalation. The description of A + B designs with dose de-escalation and of several modifications of the A + B designs can be found in Shih and Lin (12, 13).

Then, if in the combined cohort assigned to dj , X A+B ≤ CU , the next cohort of size A receives dose dj+1 ; otherwise, the trial is stopped:

The dose one level below the dose where an unacceptable number of toxicities were observed ( ≥ cU toxicities in a cohort of size A or > CU toxicities in a cohort of size A + B) is the estimated MTD. If the escalation was not stopped at any of the doses, the highest dose dK is recommended. The frequency of stopping escalation at a certain dose level depends on the toxicity rate at this dose as well as the rates at all lower dose levels. Hence, it is impossible to identify a quantile targeted by a certain A + B design. However, some practical guidelines on how to choose the design parameters can be formulated (14). If
is the target quantile, parameters A, B, cL , cU , and CU in the A + B design can be selected according to the following constraints:

Design parameters




1. if X A ≤ cL , the next cohort of A patients is assigned to dose dj+1 . 2. if cL < X A < cU , the cohort of B patients is assigned to dose dj .

1. if X A ≥ cU , the trial is stopped.

3

6

Traditional or 3 + 3 Design

The traditional or 3 + 3 design frequently used in dose finding in oncology (see the article on phase I oncology trials for a description of the 3 + 3 design) is a special case of an A + B design without de-escalation (13), as was described here with A = B = 3, cL = 0, cU = 2, and CU = 1. REFERENCES 1. G. von B´ek´esy, A new audiometer. Acta Otolaryngologica. 1947; 35: 411–422. 2. W. J. Dixon and A. M. Mood, A method for obtaining and analyzing sensitivity data. J Am Stat Assoc. 1954; 43: 109–126. 3. C. Derman, Nonparametric up and down experimentation. Ann Math Stat. 1957; 28: 795–798. 4. S. D. Durham and N. Flournoy, Random walks for quantile estimation. In: S. S. Gupta and J. O. Berger (eds.), Statistical Decision Theory and Related Topics V. New York: SpringerVerlag, 1994, pp. 467–476.

4

ESCALATION AND UP-AND-DOWN DESIGNS

5. S. D. Durham and N. Flournoy, Up-and-down designs I. Stationary treatment distributions. In: N. Flournoy and W. F. Rosenberger (eds.), Adaptive Designs. Hayward, CA: Institute of Mathematical Statistics, 1995, pp. 139–157. 6. A. Giovagnoli and N. Pintacuda, Properties of frequency distributions induced by general ‘‘up-and-down’’ methods for estimating quantiles. J Stat Plan Inference. 1998; 74: 51–63. 7. G. B. Wetherill, Sequential estimation of quantal response curves. J R Stat Soc Ser B Methodol. 1963; 25: 1–48. 8. R. K. Tsutakawa, Random walk design in bioassay. J Am Stat Assoc. 1967; 62: 842–856. 9. R. K. Tsutakawa, Asymptotic properties of the block up-and-down method in bio-assay. Ann Math Stat. 1967; 38: 1822–1828. 10. B. E. Storer, Design and analysis of phase I clinical trials. Biometrics. 1989; 45: 925–937. 11. L. Korn, D. Midthune, T. T. Chen, L. V. Rubinstein, M. C. Christian, and R. M. Simon, A comparison of two phase I trial designs. Stat Med. 1994; 13: 1799–1806. 12. W. J. Shih and Y. Lin, Traditional and modified algorithm-based designs for phase I cancer clinical trials. In: S. Chevret (ed.), Statistical Methods for Dose Finding. New York: Wiley, 2006, pp. 61–90.

13. Y. Lin and W. J. Shih, Statistical properties of the traditional algorithm-based designs for phase I cancer clinical trials. Biostatistics. 2001; 2: 203–215. 14. A. Ivanova, Escalation, group and A + B designs for dose-finding trials. Stat Med. 2006; 25: 3668–3678. 15. M. Gezmu and N. Flournoy, Group up-anddown designs for dose-finding. J Stat Plan Inference. 2006; 136: 1749–1764. 16. A. Ivanova, N. Flournoy, and Y. Chung, Cumulative cohort design for dose-finding. J Stat Plan Inference. 2007; 137: 2316–2327.

FURTHER READING S. Chevret, ed., Statistical Methods for Dose Finding. New York: Wiley, 2006. J. Crowley, ed., Handbook of Statistics in Clinical Oncology. New York/Basel: Marcel Dekker, 2006. N. Ting, ed., Dose Finding in Drug Development. New York: Springer-Verlag, 2006.

CROSS-REFERENCES Phase I trials in oncology

DOSE ESCALATION GUIDED BY GRADED TOXICITIES

in which level 7 is the correct level. Patients enter the study sequentially. The working dose-toxicity curve, which is taken from the CRM class (described below), is refitted after each inclusion. The curve is then inverted to identify which available level has an associated estimated probability as close as we can get to the targeted acceptable toxicity level. The next patient is then treated at this level. The cycle is continued until a fixed number of subjects has been treated or until we apply some stopping rule (1,2). The di , which is often multidimensional, describes the actual doses or combinations of doses being used. We assume monotonicity, and we take monotonicity to mean that the dose levels are equally well identified by their integer subscripts i (i = 1, . . . , k), which are ordered whereby the probability of toxicity at level i is greater than that at level i whenever i is greater than i . The monotonicity requirement or the assumption that we can so order our available dose levels is important. The dose for the jth entered patient, X j can be viewed as random taking values xj , most often discrete in which case xj
{d1 , . . . , dk } but possibly continuous where X j = x; x
R+ . In light of the remarks of the previous two paragraphs we can, if desired, entirely suppress the notion of dose and retain only information that pertains to dose level. This information is all we need, and we may prefer to write xj
{1, . . . , k}. Let Y j be a binary random variable (0, 1) where 1 denotes severe toxic response for the jth entered patient (j = 1, . . . , n). We model R(xj ), which is the true probability of toxic response at Xj = xj ; xj
{d1 , . . . , dk } or xj
{1, . . . , k} via

JOHN O’QUIGLEY NOLAN WAGES University of Virginia Charlottesville, Virginia

1

BACKGROUND

In phase 1 and phase 2 dose-finding studies, the endpoints of interest are typically the presence or absence of toxicity and/or the presence or absence of some indication of therapeutic effect. In numerical terms, these outcomes are represented as simple binary variables. Most protocols, however, will stipulate that intermediary degrees, or grades, of toxicity be recorded. Our purpose here is to consider how such intermediary information may be used to obtain a more accurate estimation of the maximum tolerated dose, both at the end of the study and for those patients being treated during the course of the study. For now, we limit our attention to phase 1 studies alone, in which toxicity is the focus of our interest. Working with two-stage continual reassessment method (CRM) designs, which are briefly described in the following section, we can observe that considerable use can be made of graded information both during the initial escalation period as well as during the second stage of a two-stage design. Here, we appeal to simple working models. 2

TWO-STAGE CRM DESIGNS

R(xj ) = Pr(Yj = 1|Xj = xj )

The purpose of the design is to identify a level, from among the k dose levels available d1 , . . . , dk , such that the probability of toxicity at that level is as close as possible to some value
. The value
is chosen by the investigator such that he or she considers probabilities of toxicity higher than
to be unacceptably high, whereas those lower than
are unacceptably low in that they indicate, indirectly, the likelihood of too weak an antitumor effect. Figure 1 illustrates typical behavior of a CRM design with a fixed sample

= E(Yj |xj ) = (xj , a) for some one-parameter working model (xj , a). For given fixed x, we require that (x, a) be strictly monotonic in a. For fixed a, we require that (x, a) be monotonic increasing in x or, in the usual case of discrete dose levels di , i = 1, . . . , k, that (di , a) > (dm , a) whenever i > m. The true probability of toxicity at x (i.e., whatever treatment

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

DOSE ESCALATION GUIDED BY GRADED TOXICITIES Trial History CRM toxicity

9 8

Dose Level

7 6 5 4 3 2

Figure 1. A typical trial history for a two-stage CRM design using accelerated early escalation based on grade. MTD corresponds to a toxicity rate of 25% and is found at level 7.

1 1

7

10

13

16

19

22

25

28

Subject No

combination has been coded by x) is given by R(x), and we require that, for the specific doses under study (d1 , . . . , dk ) values of a, say a1 , . . . , ak exist such that (di , ai ) = R(di ), (i = 1, . . . , k). In other words, our oneparameter working model has to be rich enough to model the true probability of toxicity at any given level. We call it a working model because we do not anticipate a single value of a to work precisely at every level, that is, we do not anticipate a1 = a2 = · · · = ak = a. Many choices are possible. Excellent results have been obtained with the simple choice: (di , a) = αia , (i = 1, . . . , k)

4

(1)

where 0 < α1 < · · · < αk < 1 and 0 < a < ∞. It can be sometimes advantageous to make use of the reparameterized model (di , a) = exp(a) so that no constraints are placed on αi the parameter a. Of course, likelihood estimates of are unchanged. Once a model has been chosen and we have data in the form of the set j = {y1 , x1 , . . . , yj , xj }, the outcomes of ˆ i ), the first j experiments obtain estimates R(d (i = 1, . . . , k) of the true unknown probabilities R(di ), (i = 1, . . . , k) at the k dose levels. The target dose level is that level having associated with it a probability of toxicity as close as we can get to
. The dose or dose level xj assigned to the jth included patient is

such that ˆ j ) −
| < |R(d ˆ i ) −
|, (i = 1, . . . , k; xj = dj ) |R(x Thus xj is the closest level to the target level in the above precise sense. Other choices of closeness could be made by incorporating cost or other considerations. We could also weight ˆ j )− the distance, for example multiply |R(x
| by some constant greater than 1 when ˆ j ) >
. This method would favor conserR(x vatism; such a design tends to experiment more often below the target than a design without weights. Similar ideas have been pursued by Babb et al. (3). After the inclusion of the first j patients, the log-likelihood can be written as: Lj (a) =

j


y log (x , a)

=1

+

j
(1 − y ) log(1 − (x , a)) (2) =1

and is maximized at a = aˆ j . Maximization of Lj (a) can easily be achieved with a Newton Raphson algorithm or by visual inspection using some software package such as Microsoft Excel (Microsoft Corporation, Redmond, WA). Once we have calculated aˆ j , we can next obtain an estimate of the probability of toxicity at each dose level di via: ˆ i ) = (di , aˆ j ) (i = 1, . . . , k) R(d

DOSE ESCALATION GUIDED BY GRADED TOXICITIES

We would not anticipate these estimates to be consistent at all dose levels, which would usually require a richer model than what we work with. However, under broad conditions, we will (4) obtain consistency at the recommended maximum tolerated dose (MTD). Based on this formula, the dose to be given to the (j + 1)th patient, xj+1 is determined. The experiment is considered as not being fully underway until we have some heterogeneity in the responses. These examples could develop in a variety of different ways, which include use of the standard Up and Down approach, use of an initial Bayesian CRM as outlined below, or use of a design believed to be more appropriate by the investigator. Once we have achieved heterogeneity, the model kicks in and we continue as prescribed above iterating between estimation and dose allocation. The design is then split into two stages: an initial exploratory escalation followed by a more refined homing in on the target. Storer (5) was the first to propose twostage designs in the context of the classic Up and Down schemes. His idea was to enable more rapid escalation in the early part of the trial where we may be far from a level at which treatment activity could be anticipated. Moller (6) was the first to use the idea in the context of CRM designs. Her idea was to allow the first stage to be based on some variant of the usual Up and Down procedures. In the context of sequential likelihood estimation, the necessity of an initial stage was pointed out by O’Quigley and Shen (7), because the likelihood equation fails to have a solution on the interior of the parameter space unless some heterogeneity in the responses has been observed. Their suggestion was to work with any initial scheme, such as Bayesian CRM or Up and Down. For any reasonable scheme, the operating characteristics seem relatively insensitive to this choice. However, something very natural and desirable is observed in two stage designs, and currently they could be taken as the designs of choice. The reason is the following: Early behavior of the method, in the absence of heterogeneity (i.e., lack of toxic response), seems to be rather arbitrary. A decision to escalate after inclusion of three patients who tolerated some level, or after a

3

single patient tolerated a level or according to some Bayesian prior, however constructed, is translating directly (although less directly for the Bayesian prescription) the simple desire to try a higher dose because we’ve encountered no toxicity thus far. We can make use of information on toxicity grade in either one of these two stages. In the first stage, no model is being used, and we use graded toxicities simply to escalate more rapidly when it seems we are far below any level likely to result in doselimiting toxicities. Once we begin to observe some intermediary toxicity, then we slow the escalation down. The ideas are straightforward and appeal mostly to common sense arguments. Nonetheless, it can be observed that use of graded toxicity information in the first stage alone can make an important contribution to increased efficiency. Use of graded toxicity information in the second stage requires an additional model to that already used to demonstrate the rate of toxicities. We consider these two different situations in the following two sections. 3 USING GRADED INFORMATION IN THE FIRST STAGE Consider the following example of a twostage design that has been used in practice. Many dose levels were used, and the first included patient was treated at a low level. As long as we observe very low-grade toxicities, then we escalate quickly, which includes only a single patient at each level. As soon as we encounter more serious toxicities, escalation is slowed down. Ultimately, we encounter dose-limiting toxicities at which time the second stage, based on fitting a CRM model, comes fully into play. This method is done by integrating this information and that obtained on all the earlier non-dose-limiting toxicities to estimate the most appropriate dose level. We can use information on low-grade toxicities in the first stage of a two-stage design to allow rapid initial escalation, because it may be the case that we be far below the target level. Specifically, we define a grade severity variable S(i) to be the average toxicity severity observed at dose level i (i.e., the sum of the

4

DOSE ESCALATION GUIDED BY GRADED TOXICITIES

severities at that level divided by the number of patients treated at that level). The rule is to escalate providing S(i) is less than 2. Furthermore, once we have included three patients at some level, escalation to higher levels only occurs if each cohort of three patients does not experience dose-limiting toxicity. This scheme means that, in practice, as long as we observe only toxicities of severities coded 0 or 1, we escalate. Only a single patient is necessary (for whom little or no evidence of any side effects is observed) to decide to escalate. The first severity coded 2 necessitates another inclusion at this same level and, anything other than a 0 severity for this inclusion would require yet another inclusion and a non-dose-limiting toxicity before being able to escalate. This design also has the advantage that, should we be slowed down by a severe (severity 3), albeit non-doselimiting toxicity, we retain the capability of picking up speed (in escalation) should subsequent toxicities be of low degree (0 or 1). This method can be helpful in avoiding being handicapped by an outlier or an unanticipated and possibly not drug-related toxicity. Many variants on this particular escalation scheme and use of graded severity are available. It is for the investigator to decide which scheme is suitable for the given circumstance and which scheme seems to provide the best balance between rapid escalation and caution in not moving so quickly as to overshoot the region where we will begin to encounter dose-limiting toxicites. Once a dose-limiting toxicity has been encountered, this phase of the study (the initial escalation scheme) ends, and we proceed to the second stage based on a CRM modelbased recommendation. Although the initial phase is closed, the information obtained on both dose-limiting and non-dose-limiting toxicities is used in the second stage.

a statistician is to consider that the response variable, which is toxicity, has been simplified when going from five levels to two and that it may help to employ models accommodating multilevel responses. In fact, we do not believe that progress is to be made using these methods. The issue is not that of modeling a response (toxicity) at 5 levels but of controlling for dose-limiting toxicity, mostly grade 4 but possibly also certain kinds of grade 3. Lower grades are helpful in that their occurrence indicates that we are approaching a zone in which the probability of encountering a dose-limiting toxicity is becoming large enough to be of concern. This idea is used implicitly in the two-stage designs described in the section entitled, ‘‘Using graded information in the first stage.’’ Hopefully, if we proceed more formally and extract yet more information from the observations, then we need models to relate the occurrence of dose-limiting toxicities to the occurrence of lower-grade toxicities. By modeling the ratio of the probabilities of the different types of toxicity, we can make striking gains in efficiency because the more frequently observed lower grade toxicities carry a great deal of information on the potential occurrence of dose-limiting toxicities. Such a situation would also allow gains in safety because, at least hypothetically, it would be possible to predict at some level the rate of occurrence of dose-limiting toxicities without necessarily having observed very many, the prediction leaning largely on the model. At the opposite end of the model/hypothesis spectrum, we might decide we know nothing about the relative rates of occurrence of the different toxicity types and simply allow the accumulating observations to provide the necessary estimates. In this case, it turns out that we neither lose nor

4 USE OF GRADED TOXICITIES IN THE SECOND STAGE

Table 1. Toxicity ‘‘Grades’’ (Severities) for Trial

Although we refer to dose-limiting toxicities as a binary (0,1) variable, most studies record information on the degree of toxicity, from 0, complete absence of side effects, to 4, lifethreatening toxicity. The natural reaction for

0 1 2 3 4

No toxicity 1 Mild toxicity (non-dose-limiting) Non-mild toxicity (non-dose-limiting) Severe toxicity (non-dose-limiting) Dose-limiting toxicity

DOSE ESCALATION GUIDED BY GRADED TOXICITIES

5

Table 2. Compared Frequency of Final Recommendations of a Standard CRM and a Design Using Known Information on Graded Toxicities

Rk Standard Using grades

1

2

3

4

5

6

0.05 0.04 0.00

0.11 0.22 0.09

0.22 0.54 0.60

0.35 0.14 0.29

0.45 0.06 0.02

0.60 0.00 0.00

gain efficiency, and the method behaves identically to one in which the only information we obtain is whether the toxicity is dose limiting. These two situations suggest a middle road might exist, using a Bayesian prescription, in which very careful modeling can lead to efficiency improvements, if only moderate, without making strong assumptions. To make this model more precise, let us consider the case of three toxicity levels, the highest being dose limiting. Let Y j denote the toxic response for subject j, (j = 1, . . . , n). The variable Y j can assume three levels: 1, 2, and 3. The goal of the trial is to identify a level of dose whose probability of severe toxicity is closest to a given percentile of the dosetoxicity curve. Supposing, for patient j, that xj = di , then a working model for the CRM could be: exp(a)

Pr(Yj = 3) = 1 (xj , a) = αi Pr(Yj = 2 or Yj = 3) = (xj , a, b) exp(a+b)

= αi

from which Pr(Yj = 1) = 1 − 2 (xj , a, b) and exp(a+b) exp(a) Pr(Yj = 2) = αi − αi . The contributions to the likelihood are: 1 − 2 (xj , a, b) when Y j = 1, (xj , a) when Y j = 3 and 2 (xj , a, b) − 1 (xj , a) when Y j = 2. With no prior information, and being able to maximize the likelihood, we obtain almost indistinguishable results to those obtained with the more usual one-parameter CRM, which is caused by near-parameter orthogonality. Therefore, no efficiency gain occurs, although there is the advantage of learning about the relationship between the different toxicity types. However, based on previous studies, we often have a very precise idea concerning the relative rates between certain toxicity grades. We can imagine that this relationship can be estimated with good precision. Suppose that the parameter b is known precisely.

The model need not be correctly specified, although b should maintain interpretation outside the model, for instance some simple function of the ratio of grade 3 to grade 2 toxicities. Efficiency gains can then be substantial. Table 2 provides a simple illustration of the order of magnitude of the gains we might anticipate when we are targeting a value of
around 0.25. The rate of lower grade toxicities is known to be twice this rate. A Bayesian framework would allow us to make weaker assumptions on the parameter b so that any errors in assumptions can then be overwritten by the data. More work is needed on this subject, but the early results are very promising. REFERENCES 1. J. O’Quigley and E. Reiner, A stopping rule for the continual reassessment method. Biometrika 1998; 85: 741–748. 2. J. O’Quigley, Estimating the probability of toxicity at the recommended dose following a Phase I clinical trial in cancer. Biometrics 1992; 48: 853–862. 3. Baab, J., Rogatko, A., and Zacks, S., (1998). Cancer phase I clinical trials: Efficient close escalation with overdose control. Statistics in Medicine 17. 1103–1120. 4. L. Z. Shen and J. O’Quigley, Consistency of continual reassessment method in dose finding studies. Biometrika 1996; 83: 395–406. 5. B. E. Storer, Phase I clinical trials. In: Encylopedia of Biostatistics. 1998. Wiley, New York. 6. S. Moller, An extension of the continual reassessment method using a preliminary up and down design in a dose finding study in cancer patients in order to investigate a greater number of dose levels. Stats. Med. 1995; 14: 911–922. 7. J. O’Quigley and L. Z. Shen, Continual reassessment method: a likelihood approach. Biometrics 1996; 52: 163–174.

6

DOSE ESCALATION GUIDED BY GRADED TOXICITIES

FURTHER READING C. Ahn, An evaluation of phase I cancer clinical trial designs. Stats. Med. 1998; 17: 1537–1549. S. Chevret, The continual reassessment method in cancer phase I clinical trials: a simulation study. Stats. Med. 1993; 12: 1093–1108. D. Faries, Practical modifications of the continual reassessment method for phase I cancer clinical trials. J. Biopharm. Stat. 1994; 4: 147–164. C. Gatsonis and J. B. Greenhouse, Bayesian methods for phase I clinical trials. Stats. Med. 1992; 11: 1377–1389. S. Goodman, M. L. Zahurak, and S. Piantadosi, Some practical improvements in the continual reassessment method for phase I studies. Stats. Med. 14: 1149–1161. J. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: a practical design for Phase I clinical trials in cancer. Biometrics 1990; 46: 33–48. J. O’Quigley and S. Chevret, Methods for dose finding studies in cancer clinical trials: a review and results of a Monte Carlo study. Stats. Med. 1991; 10: 1647–1664. J. O’Quigley, L. Shen, and A. Gamst, Two sample continual reassessment method. J. Biopharm. Statist. 1999; 9: 17–44. S. Piantadosi and G. Liu, Improved designs for dose escalation studies using pharmacokinetic measurements. Stats. Med. 1996; 15: 1605–1618. J. Whitehead and D. Williamson, Bayesian decision procedures based on logistic regression models for dose-finding studies. J. Biopharm. Statist. 1998; 8: 445–467.

DOSE-FINDING STUDIES

both efficacy and safety, as one of the key drivers of the high attrition rates currently plaguing late phase clinical trials across the pharmaceutical industry (7, 8). PhRMA has constituted a working group to evaluate and propose recommendations to address this specific issue (8). In light of these ongoing discussions and activities, this article reviews some of the key methodologies used in the dose-finding trials typically encountered in the late stage of drug development, that is, in phase II and/or phase III clinical trials. Out of scope for this overview are dose-finding studies in early development, which often take place under different constraints and use different methodologies, such as the traditional 3 + 3 designs, up-and-down designs, or continual reassessment methods (4, 5).

FRANK BRETZ Novartis Parma AG Basel, Switzerland

JOSE´ C. PINHEIRO Novartis Pharmaceuticals East Hanover, New Jersey

Understanding and adequately representing the dose–response profile of a compound, with respect to both efficacy and safety, is a fundamental objective of clinical drug development. An indication of its importance is the early publication of the International Conference on Harmonization E4 guideline on dose–response studies (1). The dose–response profile describes how the expected response—for example, a clinical endpoint of interest—varies in relation to the dose levels being administered. Proper understanding of this relationship is crucial for two critical decisions required during drug development: (1) whether there is an overall dose–response effect (proof of concept), and (2) if so, which dose level(s) should be selected for further development (dose finding). Selecting too low a dose decreases the chance of showing efficacy in later studies, whereas selecting too high a dose may result in tolerability or safety problems. Indeed, it may occur that only after having marketed a new drug at a specified dose does it become apparent that the level was set too high. This phenomenon has been documented by the U.S. Food and Drug Administration (FDA), who reported that approximately 10% of drugs approved between 1980 and 1989 have undergone dose changes—mostly decreases—of greater than 33% (2, 3). Over the past several years, an increase in interest and research activities in this area has taken place. Illustrating this trend, three books solely dedicated to dose finding in clinical drug development, from different perspectives, have recently been published (4–6). In addition, both the FDA and the Pharmaceutical Research and Manufacturers of America (PhRMA) have identified poor dose selection resulting from incorrect or incomplete knowledge of the dose–response relationship, for

1

MULTIPLE COMPARISON PROCEDURES

The analysis of dose-finding studies can be classified into two major strategies: modeling techniques (9, 10) and multiple comparison procedures (MCP) (11, 12). Modeling techniques assume a functional relationship between the dose (taken as a quantitative factor) and the response variable, according to a prespecified parametric model (defined in the study protocol). In this section we consider MCP, in which the dose is regarded as a qualitative factor and very few, if any, assumptions are made about the underlying dose–response model. MCP can be used either for detecting an overall dose-related signal by means of trend tests or for the estimation of target doses by stepwise testing strategies, while preserving the overall type I error rate at a prespecified level α. Such procedures are relatively robust to the underlying dose–response shape, but they are not designed for extrapolation of information beyond the observed dose levels. Inference is thus confined to the selection of the target dose among the dose levels under investigation. A classic method proposed by Dunnett (13) compares several treatments with a control. Because it relies on pairwise comparisons based on t-tests using the pooled variance estimate, structural information from the

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

DOSE-FINDING STUDIES

logical ordering of the dose levels is not incorporated. Trend tests exist that borrow strength from neighboring dose levels to increase the likelihood of successfully detecting a dose–response signal at the end of the study. The likelihood ratio test (14) is an example of a powerful test for detecting such a dose–response trend. However, because its critical values are difficult to compute, its application is reduced to balanced one-way layouts and other simple designs (15). Single contrast tests provide a popular alternative (16, 17), but these tests are potentially less powerful than competing methods if the true dose–response shape deviates substantially from the prespecified vector of contrast coefficients. Multiple contrast tests have been proposed instead, which take the maximum test statistic over several single contrast tests, properly adjusting it for multiplicity (18, 19). The advantage of such an approach is its greater robustness with regard to the uncertainty about the unknown true dose–response model, resulting from testing simultaneously different shapes. An additional appealing feature of multiple contrast tests is that they can be easily extended to general linear models incorporating covariates, factorial treatment structures, and random effects. Many standard trend tests are in fact special cases of (or at least closely related to) multiple contrast tests (20). Bretz et al. (21) provide more technical details on multiple contrast tests. As mentioned, the second major goal of dose-finding studies is to estimate target doses of interest, such as the minimum effective dose (MinED), which is the smallest dose showing a statistically significant and clinically relevant effect (1, 3); the maximum safe dose, which is the largest dose that is still safe; or the maximum effective dose, which is the smallest dose showing a maximum effect (1). If MCP are used, the dose levels are typically tested in a fixed order. Different possibilities exist for choosing the sequence of hypotheses to be tested and the appropriate test statistics (e.g., pairwise comparisons with the control group or any of the previously mentioned trend tests). Tamhane et al. (17, 22) and Strassburger et al. (23) provide more details.

2 MODELING TECHNIQUES Modeling approaches are commonly used to describe the functional dose–response relationship, and many different models have been proposed (24). One classification of such models is according to whether they are linear in their parameters (standard linear regression model, linear in log-dose model, etc.) or not (Emax , logistic, etc.). Pinheiro et al. (9) described several linear and nonlinear regression dose–response models commonly used in practice, including the clinical interpretations for the associated model parameters. Once a dose–response model is fitted (10), it can be used to test whether a doserelated effect is present. For example, one could test whether the slope parameter is different from 0 in a linear regression model. If a dose–response signal has been detected, the fitted dose response could then be used to estimate a target dose achieving a desired response. In contrast to MCP, dose estimation under modeling is not confined to the set of dose levels under investigation. Although such a modeling approach provides flexibility in investigating the effect of doses not used in the actual study, the validity of its conclusions depends highly on the correct choice of the dose–response model, which is typically unknown. We will describe a more detailed description of model-based approaches and potential solutions to overcome their disadvantages. A major pitfall when conducting statistical modeling is related to the inherent model uncertainty. The intrinsic problem is the introduction of a new source of variability by selecting a particular model M (for example) at any stage before the final analysis. Standard statistical analysis neglects this fact and reports the final outcomes without accounting for this extra variability. Typically, one
is interested in computing the variance var θˆ of a parameter estimate θˆ . In practice, how
 ever, the conditional variance var θˆ |M for a given model
M  is computed and stated as if it were var θˆ , ignoring the model uncertainty. In addition, substantial bias in estimating the parameters of interest can be introduced from the model selection process. Whereas it is admittedly a more difficult task

DOSE-FINDING STUDIES Set of candidate models

to compute unbiased estimates conditional on the selected model, ignoring completely the model uncertainty can lead to very undesirable effects (25, 26). A common approach to addressing the model selection problem is to use information criteria based on a reasonable discrepancy measure assessing the lack of fit. Many model selection criteria are available, and the discussion of which gives the best method is still ongoing (27, 28). Examples of such model selection criteria include the Akaike information criterion (AIC), the Bayesian information criterion (BIC), and the order restricted information criterion (29). It should be kept in mind, however, that the application of any of the model selection criteria will always lead to the selection of a model, irrespective of its goodness of fit to the observed data. Different approaches have thus been proposed to overcome the problem of conditional inference on a selected model. Such proposals include (1) weighting methods, which incorporate, rather than ignore, model uncertainty by computing estimates for quantities of interest which are defined for all models (such as the MinED) using a weighted average across the models (30–32); (2) computer-intensive simulationbased inferences, such as cross-validation techniques, which select the model with the best predictive ability across the replications (26, 33); and (3) considering model selection as a multiple hypotheses testing problem, where the selection of a specific model is done while controlling the overall type I error rate (34). 3 HYBRID APPROACHES COMBINING MCP AND MODELING Hybrid dose-finding methods combine principles of MCP with modeling techniques; see Tukey et al. (35) for an early reference. Bretz et al. (36) proposed a methodology for dose-finding studies that they called MCP-Mod, which provides the flexibility of modeling for dose estimation while preserving the robustness to model misspecification associated with MCP. The general flow of the MCP-Mod methodology, including its key steps, is depicted in Figure 1. Practical considerations regarding the implementation of

3

Optimum contrast coefficients Selection of significant models while controlling the overall type I error

Selection of a single model using max-T, AIC, ..., possibly combined with external data Dose estimation and selection (minimum effective dose, ...)

Figure 1. Combining multiple comparisons and modeling techniques in dose-finding studies

this methodology were discussed by Pinheiro et al. (37). Extensions to Bayesian methods estimating or selecting the dose–response curve from a sparse dose design have also been investigated (38, 39). The central idea of the MCP-Mod methodology is to use a set of candidate dose–response models to cover the possible shapes anticipated for the dose–response relationship. Multiple comparison procedures are applied to a set of test statistics, determined by optimal contrasts representing the models in the candidate set, to decide which shapes give statistically significant signals. If no candidate model is statistically significant, the procedure stops and declares that no dose–response relationship can be established from the observed data (i.e., no proof of concept). Otherwise, the maximum contrast test and possibly further contrast tests are statistically significant. Out of the statistically significant models in the candidate set, a best model is selected for dose estimation in the last stage of the procedure. The selection of the dose-estimation model can be based on the minimum P-value (of the model contrast tests) or some other relevant model selection criteria like the AIC or the BIC. The selected dose–response model is then employed to estimate target doses using inverse regression techniques and possibly incorporating information on clinically relevant effects. The precision of the estimated doses can be assessed using, for example, bootstrap methods. In contrast to a direct application of modelbased dose estimation, the MCP step accounts

4

DOSE-FINDING STUDIES

for possible model misspecification and includes the associated statistical uncertainty in a hypothesis-testing context. Note that different model selection criteria may lead to different dose estimates because of different sources of information and/or decision goals. Information criteria, such as the AIC or BIC, are statistical decision rules taking into account only the data from the study under consideration. Bayesian decision rules, on the other hand, may additionally include information external to the study, though still based on statistical reasoning. Finally, nonstatistical selection rules based on updated clinical knowledge, economic reasons, and so forth may also be used in the dose-selection process. Simulation results suggest that hybrid methods are as powerful as standard trend tests while allowing more precise estimation of target doses than MCP due to their modeling component (9, 36). It is worth pointing out that these methods can be seen as seamless designs that combine proof of concept (phase Ib/IIa) with dose finding (phase IIb) in one single study. 4

ADAPTIVE DOSE-FINDING DESIGNS

A fast emerging field of research is the area of adaptive dose-finding designs, which can be employed to extend the previously described methods. These designs offer efficient ways to learn about the dose response through repeated looks at the data being accrued during the conduct of a clinical trial. This interim information can be used to guide decision making on which dose to select for further development or whether to discontinue a program. It is both feasible and advantageous to design a proof-of-concept study as an adaptive dose-finding trial. The continuation of a dose-finding trial into a confirmatory stage through a seamless design is a further opportunity to increase information earlier in development on the correct dose, and thus reduce the total duration of the clinical development program. Accordingly, we briefly review (1) adaptive dose ranging studies as investigated by the related PhRMA working group (8), (2) flexible designs that strongly control the overall type I error rate, and (3) Bayesian adaptive designs.

The PhRMA working group on adaptive dose ranging studies has evaluated several existing dose-finding methods and provided recommendations on their use in clinical drug development (8). The methods considered comprise a representative cross-section of currently available dose-finding procedures, ranging from more traditional methods based on analysis of variance up to adaptive designs based on an advanced Bayesian model. Through an extensive simulation study based on a common set of scenarios (sample sizes, number of doses, etc.) for all procedures, the strengths and weaknesses of each method were investigated, in particular with respect to the ability of the procedures to learn from the data and adapt to emerging information. Flexible designs that strongly control the overall type I error rate of incorrectly rejecting any null hypothesis of no dose level effect are available for adaptive dose selection in multistage clinical trials. In the context of adaptive dose-finding trials, multiplicity concerns typically arise due to (1) the comparison of several doses with a control and (2) multiple interim looks at the data for decision making. Performing each hypothesis test at the nominal level α intended for the whole trial would inflate the overall type I error rate. Therefore, the significance levels of the individual tests have to be adjusted appropriately. Classic group sequential designs are a type of flexible design in which the planned sample size or, more generally, the planned amount of information may be updated as a result of the trial. In these trials, test statistics produced at interim analyses are compared with prespecified upper or lower stopping boundaries that ensure the overall type I error rate control (40–42). Stallard and Todd (43) extended classic group sequential procedures to multiarm clinical trials incorporating treatment selection at the interim analyses. Flexible designs, which may be regarded as an extension of classic group sequential designs, offer more flexibility for adaptation within the multistage framework. These methods offer a high level of flexibility for decision making during the trial, such as increasing the sample size based on the observed effect, modifying the target patient population, or selecting good treatments

DOSE-FINDING STUDIES

(44, 45). They require little prespecification of decision rules before the beginning of a trial; therefore, the total information available at each interim time point can be used in designing or adjusting the next stage. Bayesian adaptive dose-finding designs are an important alternative to flexible designs. Information can either be updated continuously as data are accrued in the trial, or in cohorts of patients. This makes this class of designs very appealing to sequential decision making and experimentation, including clinical studies. Bayesian approaches enable the calculation of predictive probabilities of future results for any particular design, which allows comparison of designs on the basis of probabilities of their consequences. Although control of the type I error rate is not an intrinsic property of a Bayesian design, simulations can be used to tailor a Bayesian adaptive trial such that it maintains this and other desirable frequentist operational characteristics. A potential downside to the Bayesian approach is the computational complexity coupled with the absence of commercial software packages to assist with study design and analysis. Berry et al. (46) and Krams et al. (47) have more methodological details and an example of a clinical study employing Bayesian dose-finding methods in practice. 5

CONCLUSION

Dose-finding studies play a key role in any drug development program and are often the gatekeeper for the large confirmatory studies in phase III. Many approaches exist for the proper design and analysis of these trials. The ultimate choice of the method to be applied depends on the particular settings and goals. Dose-finding studies should thus be tailored to best fit the needs of the particular drug development program under consideration. Methods are available, for example, to allow the conduct of seamless proof-ofconcept and dose-finding studies. Alternatively, if it is desired to extend dose-finding trials straight into a confirmatory phase III study, adaptive designs offer efficient possibilities to control the overall type I error rate at a prespecified level. We encourage the consideration and implementation of advanced

5

dose-finding methods, which efficiently make use of accumulating information during the drug development process. REFERENCES 1. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E4 Dose-Response Information to Support Drug Registration. Step 4 version, March 1994. Available at: http://www.ich.org/LOB/media/ MEDIA480.pdf. 2. FDC Reports from May 6, 1991. The Pink Sheet. 1991: 53(18): 14–15. 3. S. J. Ruberg, Dose response studies I. Some design considerations. J Biopharm Stat. 1995; 5: 1–14. 4. N. Ting (ed.), Dose Finding in Drug Development. New York: Springer, 2006. 5. S. Chevret (ed.), Statistical Methods for Dose Finding Experiments. New York: Wiley, 2006. 6. R. Krishna (ed.), Dose Optimization in Drug Development. New York: Informa Healthcare, 2006. 7. U.S. Food and Drug Administration, Department of Health and Human Services. Innovation/Stagnation: Challenge and Opportunity on the Critical Path to New Medical Products. March 2004. Available at: http://www.fda.gov/ oc/initiatives/criticalpath/whitepaper.html 8. B. Bornkamp, F. Bretz, A. Dmitrienko, G. Enas, B. Gaydos, et al., Innovative Approaches for Designing and Analyzing Adaptive DoseRanging Trials. White Paper of the PhRMA Working Group on Adaptive Dose-Ranging Studies. J Biopharm Stat. 2007, in press. Available at: http://www.biopharmnet.com/ doc/phrmaadrs white paper.pdf 9. J. Pinheiro, F. Bretz, M. Branson, Analysis of dose response studies: modeling approaches In: N. Ting, (ed.), Dose Finding in Drug Development. New York: Springer, 2006, pp. 146–171. 10. D. M. Bates and D. G. Watts, Nonlinear Regression Analysis and Its Applications. New York: Wiley, 1988. 11. Y. Hochberg, and A. C. Tamhane, Multiple Comparisons Procedures. New York: Wiley, 1987. 12. J. C. Hsu, Multiple Comparisons. New York: Chapman and Hall, 1996. 13. C. W. Dunnett, A multiple comparison procedure for comparing several treatments

6

14.

15.

16.

17.

18.

19.

20.

DOSE-FINDING STUDIES with a control. J Am Stat Assoc. 1955; 50: 1096–1121. D. J. Bartholomew, Ordered tests in the analysis of variance. Biometrika. 1961; 48: 325–332. T. Robertson, F. T. Wright, and R. L. Dykstra, Order Restricted Statistical Inference. New York: Wiley, 1988. S. J. Ruberg, Contrast for identifying the minimum effective dose. J Am Stat Assoc. 1989; 84: 816–822. A. C. Tamhane, C. W. Dunnett, and Y. Hochberg, Multiple test procedures for dose finding. Biometrics. 1996; 52: 21–37. ¨ L. A. Hothorn, M. Neuhauser, H. F. Koch, Analysis of randomized dose-finding studies: closure test modifications based on multiple contrast tests. Biom J. 1997; 39: 467–479. W. H. Stewart and S. J. Ruberg, Detecting dose response with contrasts. Stat Med. 2000; 19: 913–921. F. Bretz, An extension of the Williams trend test to general unbalanced linear models. Comput Stat Data Anal. 2006; 50: 1735–1748.

21. F. Bretz, J. Pinheiro, and A. C. Tamhane, Multiple testing and modeling in dose response problems In: A. Dmitrienko, A. C. Tamhane, and F. Bretz (eds.), Multiple Testing Problems in Pharmaceutical Statistics. New York: Taylor & Francis, 2009 (in press). 22. A. C. Tamhane, C. W. Dunnett, J. W. Green, and J. D. Wetherington, Multiple test procedures for identifying the maximum safe dose. J Am Stat Assoc. 2001; 96: 835–843. 23. K. Strassburger, F. Bretz, and H. Finner, Ordered multiple comparisons with the best and their applications to dose-response studies. Biometrics. 2007; May 8 (e-pub). 24. D. A. Ratkowsky, Handbook of Nonlinear Regression Models. New York: Marcel Dekker, 1989. 25. D. Draper, Assessment and propagation of model uncertainty. J R Stat Soc Ser B Methodol. 1995; 57: 45–97. 26. J. S. Hjorth, Computer Intensive Statistical Methods—Validation, Model Selection and Bootstrap. London: Chapman & Hall, 1994. 27. W. Zucchini, An introduction to model selection. J Math Psychol. 2000; 44: 41–61. 28. J. B. Kadane and N. A. Lazar, Methods and criteria for model selection. J Am Stat Assoc. 2004; 99: 279–290. 29. K. Anraku, An information criterion for parameters under a simple order restriction. Biometrika. 1999; 86: 141–152.

30. S. T. Buckland, K. P. Burnham, and N. H. Augustin, Model selection: an integral part of inference. Biometrics. 1997; 53: 603–618. 31. J. Hoeting, D. Madigan, A. E. Raftery, and C. T. Volinsky, Bayesian model averaging. Stat Sci. 1999; 14: 382–417. 32. K. H. Morales, J. G. Ibrahi, C. J. Chen, and L. M. Ryan, Bayesian model averaging with applications to benchmark dose estimation for arsenic in drinking water. J Am Stat Assoc. 2006; 101: 9–17. 33. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning. New York: Springer, 2001. 34. H. Shimodaira, An application of multiple comparison techniques to model selection. Ann Inst Stat Math. 1998; 50: 1–13. 35. J. W. Tukey, J. L. Ciminera, and J. F. Heyse, Testing the statistical certainty of a response to increasing doses of a drug. Biometrics. 1985; 41: 295–301. 36. F. Bretz, J. Pinheiro, and M. Branson, Combining multiple comparisons and modeling techniques in dose response studies. Biometrics. 2005; 61: 738–748. 37. J. Pinheiro, B. Bornkamp, and F. Bretz, Design and analysis of dose finding studies combining multiple comparisons and modeling procedures. J Biopharm Stat. 2006; 16: 639–656. 38. T. Neal, Hypothesis testing and Bayesian estimation using a sigmoid Emax model applied to sparse dose response designs. J Biopharm Stat. 2006; 16: 657–677. 39. A. Wakana, I. Yoshimura, and C. Hamada, A method for therapeutic dose selection in a phase II clinical trial using contrast statistics. Stat Med. 2007; 26: 498–511. 40. S. J. Pocock, Group sequential methods in the design and analysis of clinical trials. Biometrika. 1977; 64: 191–199. 41. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics. 1979; 35: 549–556. 42. C. Jennison, and B. W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. London: Chapman and Hall, 2000. 43. N. Stallard and S. Todd, Sequential designs for phase III clinical trials incorporating treatment selection. Stat Med. 2003; 22: 689–703. 44. G. Hommel, Adaptive modifications of hypotheses after an interim analysis. Biom J. 2001; 43: 581–589. 45. F. Bretz, H. Schmidli, F. K¨onig, A. Racine, and W. Maurer, Confirmatory seamless phase

DOSE-FINDING STUDIES II/III clinical trials with hypotheses selection at interim: general concepts (with discussion). Biom J. 2006; 48: 623–634. ¨ 46. D. A. Berry, P. Muller, A. P. Grieve, M. Smith, T. Parke, et al., Adaptive Bayesian designs for dose-ranging drug trials. In: C. Gatsonis, B. Carlin, and A. Carriquiry (eds.), Case Studies in Bayesian Statistics V. New York: Springer, New York, 2001, pp. 99–181. 47. M. Krams, K. R. Lees, W. Hacke, A. P. Grieve, J. M. Orgogozo, G. A. Ford, Acute stroke therapy by inhibition of neutrophils (ASTIN): an adaptive dose response study of UK-279,276 in acute ischemic stroke. Stroke. 2003; 34: 2543–2548.

CROSS-REFERENCES Hypothesis testing Minimum effective dose (MinED) Multiple comparisons

7

DOSE RANGING CROSS-OVER DESIGNS

doses. Too much anticoagulation and undesired bleeding may result, which threatens a patient’s well-being; too little anticoagulation could mean that thromboembolic events might not be treated or prevented (9). The choice of what series of doses of warfarin to use depends on many factors—age, nutritional status, gender, what other drugs the patient is taking, and so on. These factors make warfarin challenging to use in practice, and frequent laboratory monitoring of blood coagulation level is used to protect patients (8), although this seems not to be uniformly successful (9). Not all drugs, however, require such a high level of monitoring. Other drugs may be more benign in terms of maintaining safety over a range of doses. Early drug development focuses on exploration of the dose–response relationship (for both undesired and desired effects). The focus here (in what is commonly referred to as Phase I and II) is on ‘‘learning’’ (10) about the compound’s properties. Once a safe and effective range of doses is identified, confirmatory clinical trials are performed to provide regulators with sufficient evidence of benefit relative to risk to support access of a product at certain prescribed doses to patients in the marketplace. This article focuses on the use of crossover designs (11–13) in clinical drug development, which are used for dose-ranging to evaluate desired and undesired response. Those interested in the general topic of clinical doseresponse may find recent works in References 14–16 helpful in their research. These works tend to concentrate on application of dose-ranging in parallel group designs (most frequently, those used in oncology), and we cross-reference the findings of these works when appropriate to their application in crossover designs. Those interested in the application of techniques in nonclinical, toxicology drug research will find Hothorn and Hauschke’s work in Chapters 4 and 6 of Reference 17 of interest. After brief discussions of current international guidance and of clinical dose-ranging study design, we discuss two categories of dose ranging crossover designs. In the next

SCOTT D. PATTERSON Wyeth Research & Development, Collegeville, Pennsylvania

BYRON JONES Pfizer Pharmaceuticals, Sandwich, UK ´ NEVINE ZARIFFA

GlaxoSmithKline, Philadelphia, Pennsylvania

1

INTRODUCTION

Why do we care about dose? Many people seemingly are of the opinion that ‘‘if a little is good, more is probably better.’’ However, this opinion is not necessarily the case in drug treatment. It is more complicated than that. As illustrated by Sheiner and Steimer (1) and described in more detail in References 2 and 3, administration of a dose of drug results in exposure to drug at the site of action once drug enters the systemic circulation. At the site of action, drug molecules bind to receptors in the relevant organ, tissue, and so on, and hopefully cause a desired response, like lowering blood pressure, which improves how a patient feels, how long a patient lives, or some other desired clinical outcome. Of course, once in the circulatory system, the drug may go to other sites than that desired and cause an undesired response. For example, consider the fatal example of terfenadine (4). Doses of this drug resulted in cardiac arrythmia and death when given with a metabolic inhibitor. Experiences such as this have confirmed that a thorough understanding of how dose relates to response is critically important for a proper understanding of a drug’s usefulness (or lack thereof) (5–7). An example in which response to dose can be monitored is in the use of a drug called warfarin. Such a drug is dangerous to use, but its use is said to prevent ‘‘twenty strokes for every episode of bleeding’’ (8). The drug is titrated to the desired level of effective and safe anticoagulation in a series of daily

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

DOSE RANGING CROSS-OVER DESIGNS

section, titration designs and extension studies are discussed. Then, fully randomized crossover studies are considered. Each section includes references to examples of data, the statistical model, and methods of inference. A comprehensive exposition of dose-response modeling in crossover designs is beyond the scope of this article, and we concentrate on models which in our experience are most frequently applied, providing references for those interested in more details. 1.1 Objectives of Dose-Ranging and Summary of Current International Regulations The International Conference on Harmonization guidance ICH E4 (18) denotes choice of dose as an ‘‘integral part of drug development.’’ However, the guidance is complex and calls for determination of several factors: for example, the starting dose, the maximum tolerated dose (MTD), the minimum effective dose, the maximum effective dose, and a titration algorithm for physicians to use to determine which dose to use. Thus, a doseranging trial has several objectives that can be covered by statements as nebulous as, for example, ‘‘characterizing dose response’’ to as precise as confirming that ‘‘X mg is the MTD in a certain population.’’ In addition to choice of objective, the choice of endpoint is critical to the selection of design. Drug research contains examples of both acute, reversible responses (e.g., blood pressure), and chronic responses that change (generally for the worse) with time (e.g., survival time), which depend on the therapy area being investigated. Studies used to assess dose-response should be ‘‘well-controlled, using randomization and blinding’’ and ‘‘should be of adequate size’’ (18). In early phase development, ‘‘placebo controlled individual subject titration designs (18)’’ support later, larger trials in patients. ICH E4 recommends the use of parallel-group dose response studies; however, this method is not in keeping with regulatory trends toward the individualization of therapy (5) and it requires greater consideration of different regimens within the same patient. ICH E4 lists several potential study designs

1. Parallel dose-response designs— Subjects are randomized to fixed-dose groups and followed while on that dose to evaluate response. This type of dose ranging design is the most common in drug development but does not generally permit the evaluation of individual response (18). These designs are discussed in References 15 and 16 and are not discussed here in favor of discussion of more powerful (11) designs as follows. 2. Titration designs—Subjects (forced titration) or subjects not yet achieving a response (optional titration) receive predetermined increases in dose until the desired effect is observed, or until dosing is stopped because patients are not tolerating the regimen. Although this approach confounds time-related effects with dose and with cumulative dose response effects, it does permit the evaluation of individual dose-response under certain assumptions. Titration designs are discussed in the section entitled, ‘‘Titration designs and extension studies’’. 3. Crossover designs—Subjects are randomized to sequences of different doses with each period of treatment with a given dose separated by a wash-out period to enable the body’s function to return to basal levels (11, 12). This type of design is discussed in the section entitled, ‘‘Randomized designs.’’ FDA’s implementation of ICH E4 (18) may be found in Reference 19. This guideline enhances several aspects of ICH E4 by calling, in part, for (19): 1. Prospectively defined hypotheses/objectives, 2. Use of an appropriate control group, 3. Use of randomization to ensure comparability of treatment groups and to minimize bias. Both guidelines (18, 19) suggest that results of dose-response studies can serve as supporting evidence for confirmatory studies (conducted subsequently in drug development). This evidence may mitigate requirements for multiple confirmatory trials in

DOSE RANGING CROSS-OVER DESIGNS

some settings. Especially critical to this role as supporting evidence is the choice of endpoint. If the dose-response study or studies consider a clinically relevant endpoint related directly to dose with little to no time lag to response (e.g., blood pressure for a hypertensive agent), then use of many alternative designs (parallel, titration, or crossover) can be informative, and the outcome of the study may support registration directly. Where significant hysteresis (see Reference 3 for a definition) is present, more care in choice of design is warranted, and the relationship of dose ranging to registration is defined less well, see Fig. 1. The lists of designs in References 18 and 19 are by no means exhaustive, and other alternatives may be used. For example, another regulatory perspective may be found in Hemmings’ Chapter 2 of Reference 15, which discusses several alternative crossover study designs: 1. Randomized titration designs in nonresponders—Here, dose is titrated to effect in subjects randomly assigned to a dose or a control group. Those subjects who respond to the dose continue on that dose whereas nonresponders are randomized to continue on the dose (to evaluate whether prolonged exposure

Figure 1. Selected points to consider in different doseresponse study designs from FDA guidance (19).

3

will result in an effect) or to a higher dose. 2. Randomized withdrawal designs— Here, dose is titrated to effect in subjects randomly assigned to a dose of drug or placebo. Subjects randomly assigned to drug who respond to treatment are randomly assigned to switch to placebo or to continue on drug (to enable consideration of duration of effect once drug is discontinued). The latter type of study is known as Discontinuation designs (15), and it is a class of incomplete crossover designs (11). This design is sometimes called ‘‘Retreatment’’ designs (20) or ‘‘Enrichment’’ designs (21–24). We do not examine such studies further here as: 1. They are not generally analyzed (25, 26) in a manner consistent with crossover designs (11), and 2. Published examples examined for this article (23, 27, 28) were not dose-ranging [despite suggestions that such designs have use in this area (24)]. Thus, it can be observed that the objective of dose-response studies may be tactical {i.e., exploratory in the sense of deciding how

4

DOSE RANGING CROSS-OVER DESIGNS

to dose in subsequent studies (10)], strategic (i.e., decide how to dose in confirmatory trials for regulatory submission), and/or regulatory (i.e., provide data directly supporting submissions in place of a confirmatory trial). For the latter, ICH E5 (29) for example calls for collection of dose-response as a prerequisite for bridging findings from one ethnicity to another for registration. However, practical implementation of such an approach is region-specific (30–32). As a practical matter for those reviewing such trial designs (or their results), one can generally deduce what the true purpose of the design is by looking at the choice of endpoint and how many assumptions are made in design. If the endpoint is a biomarker (c.f., 33) and the design is titration (making the assumption that time related effects are small relative to the effect of dose), then the study is probably designed for tactical, exploratory purposes. If, however, the endpoint is a surrogate marker (33) or clinical endpoint (34), and the study employs a fully randomized control and sequence of doses [(e.g., for a crossover a William’s square (11)], then it is most likely for strategic or for regulatory purposes. Generally, the number of subjects in the study is not a good way to deduce the purpose of a study, as regulatory considerations for extent of exposure recommended for registration are more complex than this, see Reference 35. 1.2 Statistical Aspects of the Design of Dose Ranging Studies As in all protocols, the choice of population is of critical importance as are the controlled aspects of study procedures, but we will neglect those subjects here to consider statistical analysis for dose-ranging trials. Statistical consideration of such dose-ranging studies has been geared mainly toward provision of confirmatory hypothesis tests (generally used in the later stages of drug development referred to as Phase IIb-III). Modeling is the procedure that should most often be used in exploratory trials (in the early stages of drug development, referred to as Phase I and IIa). The analysis method chosen depends directly on choice of objective and endpoint.

The choice of endpoint generally determines whether one can consider a crossover (see the section entitled ‘‘Randomized designs’’) trial design in this setting (11, 13). The advantage of such a design is that one can compare the response at different doses of drug ‘‘within-subject,’’ which presumably leads to a more accurate and precise measurement (11). A potential disadvantage is that responses from the previous administration may carryover to the next treatment period, which complicates (and potentially confounds) a clear inference. If one is considering a response that is fairly ‘‘permanent’’ (i.e., survival time), then one is likely constrained by circumstance to use a parallel group or titration design (see the section entitled ‘‘Titration designs and extension studies’’). This latter design is useful here if one can assume that time related effects are small relative to the effects of dose. If one is looking at an endpoint that is reversible (i.e., generally goes back to where it started when one stops treatment), then crossover designs are recommended. The key question many statisticians face in design of dose ranging trials is, ‘‘how many subjects do we need?’’ This question is very complex to answer if one adopts a traditional approach to powering. Closed form solutions amenable to use in confirmatory trials can become very complicated as multiple doses, multiple looks, adaptive or group-sequential adjustment are involved (e.g., see Reference 36). We recommend the simpler procedure of simulation be employed. Simulated crossover data are generated very easily using a variety of commercial software packages (13), and we will not dwell on technical details here. For an example, see Chapter 6 of Reference 13. Simulation of data permits one to consider a variety of modeling and testing techniques at the design stage, as appropriate to the objective and endpoint of the trial, and we now consider two such approaches: modeling and hypothesis tests. Modeling of data from dose response trials follows the general principle 9 of regression and has been described by many authors, for example, References 1, 10, 14, and 37–42. These approaches examine response as a mathematical function of dose. These may

DOSE RANGING CROSS-OVER DESIGNS

be linear, nonlinear, or some step-function of dose as appropriate to the response involved. A well-known model used in this setting is the Power model (40) as follows: yik = (α + ξk ) + πi + β(ld) + γi−1 + εik

9 of Reference 16) is: E=

Emax(CN ) + E0 , EC50N + CN

where E is the effect being modeled, E0 is the effect observed without any drug present, C is the concentration of drug in the effect compartment, EC50 is the concentration needed to cause a 50% response, N is the slope factor (estimating ‘‘sensitivity of the response to the dose range’’), and Emax is the maximum effect that can occur with drug treatment. This example shows a nonlinear (in concentration) additive model. If drug concentration in the effect compartment is not related to effect, then Emax and EC50 would be zero. Note that instead of concentration C, one may use the dose or the log-dose in the expression if no effect compartment is being used. As a practical matter when modeling, we recommend that those who devleop such follow the principle of parsimony—Section 4.10.1 of Reference 44; the simplest model that accurately describes the data is best. The parameters estimated from such a model may be used for picking doses in subsequent studies as described in References 13, 45–47— see additional examples below. Assessment of model fit is done typically by inspection of residual values (i.e., predicted less observed), although more formal methods may sometimes be applied (48). Such

(1)

5

6

logAUC 7 8

9

10

where α is the overall mean pharmacokinetic response at a unit dose (logDose, ld = 0) known in statistics as the population intercept, ξk is the random intercept that accounts for each subject (k) as their own control, πi is the time-related effect of period i on the response, β is the slope parameter of interest regressed on logDose (parameter ld), γi−1 is any effect of the regimen from the previous period [known as a carry-over effect, (11)], and εij denotes within-subject error for each log-transformed response (yik , e.g., AUC or Cmax, see Reference 13). See for example Fig. 2. The authors have found this model particularly useful in the setting of pharmacokinetics (43). Modern computing technology has permitted the application of even more complex models in this setting (1). In brief, nonlinear mixed effect models for pharmacokinetics are described in an ‘‘effect compartment’’ (1) (which is a hypothesized part of the body where pharmacodynamic effect is thought to be induced by drug treatment); a function of this model is related to response using a statistical model (41). An Emax model (Chapter

Figure 2. Estimated logDose versus logAUC curve with individual data points [reproduced from Patterson and Jones (13), Example 7.2.1, with the permission of CRC Press)].

5

2

3

4 logDose

5

6

DOSE RANGING CROSS-OVER DESIGNS

model-based approaches may be extended to situations in which more than one endpoint is of importance (49). These approaches are also amenable to statistical optimization (50, 51) although such techniques are rarely applied in drug development. Bayesian approaches are not rare in oncology (52–54), and extensions to consider more than one endpoint are known (55). Although Bayesian extensions to the models described above may also be applied in exploratory studies in other therapy areas (43, 56–59), such use is not yet widespread (60). Results of models such as those applied later in this article may be used for confirmatory purposes in the testing of hypotheses. These testing procedures are geared toward provision of statistical confirmation of effect and are designed more to make a claim. As such, results must prove responses as desired beyond a reasonable doubt under a traditional hypothesis testing framework for consideration by regulators. ICH E4 (18) does not require this proof, but it is still often done for strategic and confirmatory purposes. One might test for example: H0 : µDi − µP ≤ 0

(2)

for all i where Di denotes dose i = 1, 2, 3, 4, , , relative to Placebo P. This hypothesis is to be tested versus the alternative hypothesis: H1 : µDi − µP > 0

(3)

for at least one Di in the study. Modifications of the traditional statistical hypothesis tests can be used for this purpose to adjust for the multiplicity inherent to such analyses in later phase (IIb–III) development. Many alternatives exist: the Fisher’s LSD test, the Bonferroni-Dunn test, the Tukey’s HSD test, the Newman-Keuls test, the Scheff´e test, the Dunnett test to name a few—see Chapter 24 of Reference 61, as well as References 62 and 63 for details. These alternative procedures are many and varied, and they are geared toward protection against a type 1 error (which is declaring a dose as providing a given response, incorrectly, relative to a null hypothesis that is true). The focus in this article is on human clinical trials, but the confirmatory

techniques discussed can be applied to nonhuman studies of the same purpose (and vice-versa). For readers interested in more on this topic, we recommend Hothorn and Hauschke’s work in Chapters 4 and 6 of Reference 17. Recent summaries of such testing procedures may be found in References 64 and 65. Use of such procedures can have an impact on study design, and it is recommended that those who employ such approaches see References 51 and 66 for discussion of relevant aspects of study design. Note that in such confirmatory trials, control of type 1 error can become complex because group-sequential or adaptive looks are commonly employed (67–71). In theory, one could also use such an approach to inference in a titration (see the next section) or discontinuation design, but given the assumptions necessary to perform such an analysis (i.e., the confounding of time with dose), it is rarely done in practice, and emphasis is placed on model-based interpretation. We now turn to such data. 2 TITRATION DESIGNS AND EXTENSION STUDIES Titration designs are used when one wishes to observe the safety and the efficacy results at one dose before choosing subsequent doses in a given experimental unit. These types of approaches are commonly used in real-world clinical practice (37) to titrate to effect under a forced (predetermined) or, most often, an optional escalation structure. Warfarin is an historical example of such an approach. Eligible patients are titrated at weekly doses of warfarin between 27.5 to 42.5 mg to achieve an international normalized ratio (INR, a measure of the blood’s ability to coagulate—the greater the value, the lesser the coagulation) of 2 to 3 (8). Weekly INR measurements are taken to determine how to adjust the dose upward or downward at the physician’s discretion. Warfarin is by no means the only drug used in this manner—see Reference 72 and Chapter 18 of Reference 3 for other examples. Clinical trials also use such approaches when the endpoint involved is reversible and/or concerns develop with tolerability at

DOSE RANGING CROSS-OVER DESIGNS

higher doses. In such cases, a titration design may be used. For example in first-time-inhuman trials, doses are generally chosen for inclusion in the study based on allometric scaling of findings from experiments in animals or in vitro studies (73, 74). Once this range of potentially safe doses is determined, testing in humans commences with small, well-controlled, data-intensive trials. See Chapter 7 of Reference 13 for more details. Exposure data from such a trial is shown in Table 1. In this case, it is determined that dose-escalation in this three-period, randomized (to placebo), single blind, crossover design would be halted when mean exposure neared 20 ng.h/mL. Dosing in this study began with the first subject at 100 mg and proceeded in cohorts of subjects until exposure levels neared the desired level. Once this was achieved, it was desired to determine what exposure levels were at lower levels (i.e., 50 mg), and several subjects were dosed at that level. Such data are easily analyzed using the power model of expression [Equation (1)] that accounts for each subject as their own control

7

as described in the section entitled, ‘‘Statistical aspects of the design of dose ranging studies’’ (see Chapter 7 of Reference 13 for more details). This procedure may be performed after the study (as illustrated here) or while the study is ongoing to aid in the choice of doses. Note that in this study, each study period was separated by a washout of at least seven days to ensure no effect of the previous period carried over to the next period. Study procedures, which include the assay, were standardized and systematic in all periods to minimize the potential for any time related effects. Therefore, we can assume that the effect of the previous dose does not carry over (i.e., in some manner explain or confound) the results at the next dose (i.e., that γi−1 = 0) and that any time-related effects are negligible (i.e., that πi = 0) relative to the effect of dose. Note also that placebo data are not included in this model, as their null values are challenging to account for in such a small sample size relative to nondetectable levels of the exposure assay. Different approaches to analysis may be taken if this is not the case (75).

Table 1. Example of Exposures (ng.h/mL) in an Optional Titration Cross-over Design Subject

Sequence

50 mg

100 mg

250 mg

500 mg

1000 mg

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 18 19

APA AAP PAA PAA AAP APA APA AAP PAA PAA PPA APA PAA APA PPA AAP APA PAA

· · · · · · · · · · · 1.857 ND 0.963 1.939 1.387 · 1.812

2.688 4.258 3.21 · 4.379 · · · · · · · · · · · 2.802 4.146

· 8.651 · 5.373 8.111 2.624 6.568 · · · · · · · · · 5.256 ·

13.255 · 19.508 5.436 5.436 7.931 · 7.84 8.069 · · · · · · · · ·

· · · · · · 9.257 · 8.664 7.07 14.196 · 15.054 · · · · ·

P: Placebo, A: Active Dose of Drug ND: Not Detectable by Assay

1500 mg · · · · · · · · · 8.595 20.917 17.418 · 11.496 36.783 13.516 · ·

8

DOSE RANGING CROSS-OVER DESIGNS

Thus, the model is a reasonable choice given the design, it looks to describe the data well, and the resulting estimates can be useful for decision making. However, we have not accounted for uncertainty in the choice of subsequent doses conditioned on the observed data, and implicitly, the physician and the medical staff will use such data to help determine their next choice of dose under protocol. If one wishes to take this sort of approach statistically, then methods are available to accommodate such models. This example shows an inductive reasoning approach of Bayesian analysis (13), but we will not discuss this approach here as application is extremely limited to date in drug development (60). In this case, after the study, we see that average exposure levels overlapped the desired maximum at 1500 mg as described in Table 2. Predicted mean responses may be compared using pairwise statistical comparisons as described in the first section, although this comparison is not often done in practice. When one uses the conservative Tukey’s approach (63), for example, one finds that exposure is significantly greater across the increasing doses used in this study until one reaches the 500–1500-mg doses. The average exposure does not seem to increase significantly with dose (adjusted P > 0.10).

Table 2. Predicted Mean (90% Confidence Interval, CI) Exposures (ng.h/mL) in an optional Titration Cross-over Design Dose (mg) 50 100 250 500 1000 1500

relative to a control. For example, a study might be designed in which n eligible subjects randomly receive one of two active doses or placebo in period 1, and treatment is double blind. At the end of this session of therapy, the data is assessed, and if the drug was effective, then every subject receives the higher of the two doses in period 2, see Table 3. If this method is planned from the start of the study [often done for the purposes of collecting additional safety data under (35)], and if the endpoint is not biased by the open-label extension nature of period 2 (i.e., one is assessing a hard endpoint like blood level where knowledge of treatment dose not influence outcome or a ‘‘softer’’ endpoint is assessed by a blinded adjudication committee), then one can regard this design as an incomplete Balaam’s design (11). Subjects are randomized de facto to sequence at study start, and data may be used for the purposes of treatment effect assessment (which one would expect would be more precise than the between-group assessment of period 1 data alone). One can assess continuity of effect with treatment by looking at data from the HH sequence and can confirm that the onset of treatment effects observed between groups in period 1 is similar to that observed withinsubjects in the PH sequence. One can also assess using the LH sequence data whether those ‘‘under-treated’’ initially benefit from later treatment with the higher dose. The models applied are similar to those described

Predicted Mean (90% CI) 1.7 (1.3-2.2) 2.8 (2.2-3.4) 5.2 (4.4-6.3) 8.5 (7.1-10.2) 13.7 (11.2-16.8) 18.2 (14.4-22.8)

We now turn to a related topic in drug development known as extension studies. Extension studies are nonrandomized crossover studies in which everyone receives a certain dose after the close of the double blind randomized portion of the study

Table 3. Example of a Random Allocation of Sequences to Subject in an Extension Design Subject

Sequence

Period 1

1 2 3 · · · n

LH HH PH · · · HH

L H P · · · H

L: Low Dose H: High Dose P: Placebo

Period 2 H H H · · · H

DOSE RANGING CROSS-OVER DESIGNS

in a titration design above or in nonrandomized crossover studies (see Chapter 7 of Reference 13) and so are omitted here. To illustrate, consider the following summary of such a simulated data set (n = 1000) based on a dose-response curve for lipid data found in Reference 76. See Table 4. As one would anticipate from the data of Table 4, the data of period 1 confirms that a dose response is present (Tukey adjusted P < 0.0001); however, when one takes into account each subject as their own control by inclusion of the period 2 data (accounting for sequence and dose in the model), standard errors for the comparisons of interest narrow by 20%, which is consistent with the increased degree of precision expected in accounting for within-subject controls. This analysis is additionally informative in confirming (P < 0.0001) that subjects differ in their response to dose depending on sequence assignment, which is consistent with the differential response observed by dose in period 1. Analysis by sequence confirms that the effect of treatment with 500 mg after the placebo treatment of period 1 is similar in magnitude to the effect between groups in period 1 [i.e., 28.0 (SE = 0.05) versus 27.7 (SE = 0.15)], which suggests that delay in active treatment is not detrimental to a subject’s later response. However, comparison of the data from the HH sequence suggests that continuation of treatment may result in a decreased average response over time of approximately 0.9 (SE = 0.1, P < 0.0001). Thus, we might expect to see treatment effects diminishing over time in long studies.

Table 4. Unadjusted Mean (SE) of Response Data in an Extension Design Sequence

n

Period 1

Period 2

LH HH PH

333 334 333

4.7 (0.1) 28.0 (0.1) 0.1 (0.1)

27.7 (0.1) 26.9 (0.1) 27.9 (0.1)

L: 100 mg H: 500 mg P: Placebo

9

Inclusion of the period 2 data in this setting seems to be informative. Of note, however, if such models are applied, then one must take care to assess what assumptions are being made to determine whether it is appropriate to properly evaluate the results that the length of treatment in period 2 is sufficient to preclude the need for a wash-out from period 1 (i.e., no carryover), and that disease state in the subjects does not change significantly, all else being equal, across the length of the trial (i.e., period effects are null). If either of these assumptions are of concern, perhaps the body acquires resistance to the drug for example or the body is sufficiently improved in period 1 to allow that disease state is modified by period 2, then one may wish to alter the design to better accommodate unbiased assessment of dose-response over time (i.e., randomize to more sequences including LL, etc.), which leads into the next topic: fully randomized, dose-ranging crossover designs. 3

RANDOMIZED DESIGNS

Situations often develop in which a range of safe doses are available, and it is necessary to determine either the dose-response relationship or to decide which doses are superior to placebo. For some medical conditions, it is possible to give subjects a sequence of doses and in this situation crossover designs offer large reductions in sample size compared with traditional parallel group designs. The simplest crossover design compares either a single dose against placebo or one active dose against another. For example, Jayaram (77) compared 10 mg of montelukast against placebo to determine the eosinophils-lowering effect of the dose in asthma patients. The plan of such a design is given in Table 5. In this table, it will be observed that subjects are randomized to receive one of two treatment sequences: Dose-Placebo or PlaceboDose, over two periods of time. In Jayaram (77), for example, the periods were 4 weeks long. Ideally, an equal number of subjects should be allocated to each sequence group. Often, a wash-out period is inserted between the two periods to ensure that the entire drug has been eliminated before the second period

10

DOSE RANGING CROSS-OVER DESIGNS

Table 5. Plan of Two-treatment, Two-period Cross-over Trial

Table 6. Latin Square Design for Four Treatments

Period Group

1

1 2

Dose Placebo

Period 2 Placebo Dose

begins. In situations in which no wash-out period occurs, or in which the wash-out period is of inadequate length, it is possible that the effect of the drug given in the first period carries over into the second period. If carryover effects cannot be removed by a wash-out period, then the simple two-period design is unsuitable and a design with either more than two periods or treatment sequences should be considered. See Reference 11 for a comprehensive description and discussion of crossover designs for two treatments. When more than one dose is to be compared with placebo or a dose-response relationship over three or more doses is to be determined, then crossover designs with three or more periods may be used. As with the two-treatment designs, the medical condition studied must be such that crossover designs are appropriate. With three or more doses and periods, a range of designs exist to choose from. It is important to make the correct choice as some designs require many more subjects than others. One advantage of multiperiod crossover designs is that carryover effects can be allowed for in the design and the analysis and so it is not essential to have wash-out periods between the treatment periods. However, where possible it is good practice to include wash-out periods and/or to ensure the length of the treatment periods is adequate to remove carryover effects. As an example of a design with four periods, we refer to Reference 78, who compared three doses of magnesium and placebo using 12 subjects in three Latin squares. In a single Latin square, as many different treatment sequences exist treatments. An example of such a design for four treatments is given in Table 6. Every subject gets each treatment over the four periods. To ensure the

Sequence

1

2

3

4

1 2 3 4

Placebo Dose1 Dose2 Dose3

Dose1 Dose2 Dose3 Placebo

Dose2 Dose3 Placebo Dose1

Dose3 Placebo Dose1 Dose2

design has adequate power to detect differences of given sizes between the treatments, it is usually necessary to assign more than one subject to each sequence (or to use multiple Latin squares, as done in Reference 78). We note that each treatment occurs an equal number of times in the design, and it is balanced in the sense that all estimated pairwise comparisons among the treatments have the same variance. The ordering of the treatments is of critical importance because a bad choice can lead to a very inefficient design if carryover effects are present. A useful measure of design efficiency is defined in Reference 11. This design compares the variance of an estimated treatment comparison to a theoretical lower bound for the variance. If the lower bound is achieved, then the design has an efficiency of 100%. The design in Table 6 has an efficiency of 18.18% if differential carry-over effects are present, so it is very inefficient. On the other hand, the design in Table 7 has an efficiency of 90.91%. Another way of expressing efficiency is to say, ‘‘the design in Table 6 will require about 90.91/18.18 = 5 times more subjects to achieve the same power as the design in Table 7.’’ In the absence of differential carryover effects, both designs have an efficiency of 100%. Therefore, it is important at the planning stage to decide whether differential carryover effects are likely. The design in Table 7 is an example of a Williams design for four treatments. Such designs exist for all numbers of treatments, but for an odd number of treatments, the number of sequences is twice the number of treatments. Some exceptions to this rule exist. For example, a balanced design for 9 treatments only requires 9 sequences, not 18. An example for three treatments is given in

DOSE RANGING CROSS-OVER DESIGNS Table 7. Balanced Latin Square Design for Four Treatments

11

Table 9. Incomplete Design for Three Treatments

Period

Period

Subject

1

2

3

1 2 3 4

Placebo Dose1 Dose2 Dose3

Dose3 Placebo Dose1 Dose2

Dose1 Dose2 Dose3 Placebo

4 Dose2 Dose3 Placebo Dose1

Table 8. Williams Design for Three Treatments Period Sequence

1

2

1 2 3 4 5 6

Placebo Dose1 Dose2 Placebo Dose1 Dose2

Dose1 Dose2 Placebo Dose2 Placebo Dose1

Sequence

1

2

1 2 3 4 5 6 7 8 9 10 11 12

Dose1 Dose2 Dose3 Dose4 Dose1 Dose2 Dose3 Dose4 Dose1 Dose2 Dose3 Dose4

Dose2 Dose1 Dose4 Dose3 Dose3 Dose4 Dose1 Dose2 Dose4 Dose3 Dose2 Dose1

3 Dose3 Dose4 Dose1 Dose2 Dose4 Dose3 Dose2 Dose1 Dose2 Dose1 Dose4 Dose3

3 Dose2 Placebo Dose1 Dose1 Dose2 Placebo

Table 8. The efficiency of this design is 80% in the presence of differential carryover effects. When many doses exist in a study, for example, six plus a placebo, it is unlikely that a design with as many periods as treatments will be practical or desirable. Fortunately, many designs are available to deal with this situation. Tables of designs for up to nine treatments are given in Reference 11 and cover all practical situations. Some designs have the same number of periods as treatments, other designs have fewer periods than treatments, and still others have more periods than treatments. An example of a design with more treatments than periods is given in Reference 79, in which the authors a crossover trial to assess the dose proportionality of rosuvastatin in healthy volunteers. Four doses were compared over three periods. Although several designs could be suggested, the design in Table 9 would be a suitable choice. It has an efficiency of 71.96% in the presence of differential carry-over effects and an efficiency of 88.89% if differential carryover effects are absent.

An important and more recent use of crossover designs is in safety trials to determine whether certain doses of a drug prolong the QTc interval, see Reference 80. The QTc interval is determined from an electrocardiogram of the heart. If the QTc interval is prolonged sufficiently in humans, then potentially fatal cardiac arrhythmias can result. New drugs, and potentially existing drugs, that seek new indications must study and rule out the potential for QTc interval. An example of such data may be found in Chapter 8 of Reference 13. Regimen E was a dose of an agent known to be a mild prolonger of the QTc interval and was included as a positive control. Regimen C was a therapeutic dose, and D was a supra-therapeutic (greater than that normally administered) dose of a moderate QTc prolonging agent. Forty-one normal, healthy volunteers are in the example data set, and QTc was measured in triplicate at baseline (time 0) and over the course of the day at set times after dosing. Triplicate measurements were averaged at each time of ECG sampling (i.e., 0, 0.5, 1, 1.5, 2.4, 4, etc.) for inclusion in the analysis, and only samples out to four hours post dose are included here. In Figure 3, mild (E) and moderate degrees of prolongation (C) relative to regimen F (placebo) are observed with slightly greater prolongation observed at the supratherapeutic dose of the drug being studied (D). Both mild and moderate prolongation

12

DOSE RANGING CROSS-OVER DESIGNS Table 10. Mean Changes (90% CI) between following a single Dose in (n = 41), reproduced from Patterson and Jones [13] Example 8.1 with the permission of CRC Press Comparison

Time

Difference

C-F

0.5 1 1.5 2.5 4 0.5 1 1.5 2.5 4 0.5 1 1.5 2.5 4

4.4923 8.1830 6.0120 3.7444 5.2944 6.6868 10.4591 7.4421 6.2212 5.7591 2.0069 7.5171 6.2216 6.9994 8.4446

D-F

E-F

90% CI (2.1997, 6.7848) (5.8904, 10.4755) (3.7195, 8.3045) (1.4518, 6.0369) (3.0018, 7.5869) (4.4035, 8.9701) (8.1758, 12.7425) (5.1588, 9.7255) (3.9379, 8.5046) (3.4757, 8.0424) (−0.2778, 4.2915) (5.2324, 9.8017) (3.9369, 8.5062) (4.7147, 9.2840) (6.1599, 10.7292)

C = Therapeutic Dose D = Supra-therapeutic Dose E = Dose of Positive Control F = Placebo

refer to effect sizes greater than zero but less than the ICH E14 (80) level of probable concern for causing arrhythmia of 20 msec (6). In the example, it is observed that moderate and statistically significant (note lower 90% confidence bounds exceed zero) QTc prolongation is observed in C and D within a half-hour of dosing, and it remains prolonged out to four hours post dosing. Significant prolongation for E is not observed until a 1/2 hour after dosing and returns to parity with F immediately after 4 hours post dose (data not shown). If a randomized crossover design can be employed under circumstances in which parallel group or titration designs are considered, then it probably should be used. In addition to the substantial gains in statistical efficiency and power (34) by the use of each subject as their own control, the use of these designs allows for evaluation of time-related and carryover effects and consideration of their impact on the study results. Within-subject inclusion of a positive control seemingly is very valuable to Regulator’s seeking to confirm the validity of a trial (80).

4 DISCUSSION AND CONCLUSION Dose finding studies are of critical importance in drug development. In fact, it is generally accepted that it is one of the ‘‘opportunities’’ (7) waiting to be realized in the development of new medicines. As such, it is vitally important to design, execute, and interpret dose ranging studies early in the development of a new medicine. Broadly speaking, three types of dose ranging studies exist. The first type is the very early studies conducted in Phase I to help ascertain the likely range of doses for additional study of a new molecule. These studies are primarily pharmacokinetic studies that aim to translate the animal exposure to the exposure observed in humans. These trials are often exploratory in terms of the statistical framework. An example of this type of early dose ranging trial was highlighted in the section entitled, ‘‘Titration designs and extension studies.’’ The results of these early trials often set the dosage range for studies conducted in early Phase IIa, with a biomarker or a surrogate endpoint. Although this second type of dose ranging trials can be

420

DOSE RANGING CROSS-OVER DESIGNS

13

Figure 3. Mild and moderate QTc prolongation (n = 41) [reproduced from Patterson and Jones (13), Example 8.1, with the permission of CRC Press].

360

Adjusted Mean QTo 380 400

C D E Placebo

performed in the target patient population and with the registration endpoint of interest [e.g., blood pressure or lipid endpoints in cardiovascular research], it is much more common that these studies are moderately sized trials in low-risk patient populations that aim to refine the understanding of mechanistic effects of the drug under study, see the section entitled, ‘‘Titration designs and extension studies.’’ The last and perhaps most important of the dose ranging trials are the Phase IIb trials conducted in the patient population itself (or a very closely related patient population) with the registration endpoint or a well-accepted clinical surrogate (e.g., QTc, see the section entitled, ‘‘Randomized designs’’). The duration of dosing in these trials is often lengthy, and the use of a crossover design implies a very meaningful lengthening of the trial duration, which makes these potentially impractical. The last type of dose ranging trials is often viewed as time consuming and has historically been an area of opportunity for enhancement in drug development. A key issue in this last category of trials includes the choice of a control arm. Not discussed at length in this article is the option to include an active agent for the purpose of calibrating the trial results with something more convincing than historic data from older agents. This decision is often difficult, but it can add a tremendous amount of value when interpreting a trial’s findings.

In summary, beyond the statistical intricacies of the design and analysis of dose finding studies that are reviewed in this article, the fact remains these trials are often the lynch pin of drug development and deserve the full attention of statisticians.

1

2

3

4

Time (h) following a Single Dose

REFERENCES 1. L. B. Sheiner and J-L. Steimer, Pharmacokinetic-pharmacodynamic modeling in drug development. Annu. Rev. Pharmacol. Toxicol. 2000; 40: 67–95. 2. M. Rowland and T.N. Tozer, Clinical Pharmacokinetics: Concepts and Applications. Philadelphia, PA: Lea and Febidger, 1980. 3. A. Atkinson, C. Daniels, R. Dedrick, C. Grudzinskas, and S. Markey, eds., Principles of Clinical Pharmacology. San Diego, CA: Academic Press, 2001. 4. C. Pratt, S. Ruberg, J. Morganroth, B. McNutt, J. Woodward, S. Harris, J. Ruskin, and L. Moye, Dose-response relation between terfenadine (Seldane) and the QTc interval on the scalar electrocardiogram: distinguishing drug effect from spontaneous variability. Am. Heart J. 1996; 131: 472–480. 5. R. Temple, Policy developments in regulatory approval. Stats. Med. 2002; 21: 2939–2948. 6. R. Temple, Overview of the concept paper, history of the QT/TdP concern; Regulatory implications of QT prolongation. Presentations at Drug Information Agency/FDA Workshop, 2003. Available: www.diahome.org.

14

DOSE RANGING CROSS-OVER DESIGNS

7. FDA Position Paper, Challenge and Opportunity on the Critical Path to New Medical Products. 2004. 8. J. Horton and B. Bushwick, Warfarin therapy: evolving strategies in anticoagulation. Am. Fam. Physician 1999; 59: 635–648.

24. 25.

9. M. Reynolds, K. Fahrbach, O. Hauch, G. Wygant, R. Estok, C. Cella, and L. Nalysnyk, Warfarin anticoagulation and outcomes in patients with atrial fibrillation. Chest 2004; 126: 1938–1945.

26.

10. L.B. Sheiner, Learning versus confirming in clinical drug development. Clin. Pharmacol. Therapeut. 1997; 61, 275–291.

27.

11. B. Jones and M. G. Kenward, Design and Analysis of Cross-over Trials, 2nd ed. London: Chapman and Hall, CRC Press, 2003. 12. S. Senn, Cross-over Trials in Clinical Research, 2nd ed. New York: John Wiley and Sons, 2002.

28.

13. S. Patterson and B. Jones, Bioequivalence and Statistics in Clinical Pharmacology. London: Chapman and Hall, CRC Press, 2006. 14. R. Tallarida, Drug Synergism and Dose-Effect Data Analysis. London: Chapman and Hall, CRC Press, 2000. 15. S. Chevret, ed., Statistical Methods for Dose Finding Experiments. West Sussex, UK: Wiley, 2006.

29.

30.

16. N. Ting, ed., Dose Finding in Drug Development. New York: Springer, 2006. 17. S. C. Chow and J. Liu, eds., Design and Analysis of Animal Studies in Pharmaceutical Development. New York: Marcel Dekker, 1998.

31.

18. International Conference on Harmonization, E4: Dose Response Information to Support Drug Registration. 1994. Available: http://www.fda.gov/cder/guidance/.

32.

19. FDA Guidance, Exposure-Response Relationships—Study Design, Data Analysis, and Regulatory Applications. 2003. Available: http://www.fda.gov/cder/guidance/. 20. C. Dunger-Baldouf, A. Racine, G. Koch, Retreatment studies: design and analysis. Drug Informat. J. 2006; 40: 209–217. 21. R. Temple, Enrichment designs: efficiency in development of cancer treatments. J. Clin. Oncol. 2005; 23: 4838–4839. 22. R. Temple, FDA perspective on trials with interim efficacy evaluations. Stats. Med. 2006; 25: 3245–3249. 23. R. Temple, Special study designs: early

33.

34.

35.

escape, enrichment, studies in nonresponders. Communicat. Stat. Theory Methods 1994; 23: 499–531. R. Temple, Government viewpoint of clinical trials. Drug Informat. J. 1982; 1: 10–17. B. Freidlin and R. Simon, Evaluation of randomized discontinuation design. J. Clin. Oncol. 2005; 23: 5094–5098. J. Kopec, M. Abrahamowicz, and J. Esdaile, Randomized discontinuation trials: utility and efficiency. J. Clin. Epidemiol. 1993; 46: 959–971. W. Stadler, G. Rosner, E. Small, D. Hollis, B. Rini, S. Zaentz, J. Mahoney, and M. Ratain, Successful implementation of the randomized discontinuation trial design: an application to the study of the putative antiangiogenic agent carboxyaminoimidazole in renal cell carcinoma -CALGB69901. J. Clin. Oncol. 2005; 23: 3726–3732. G. Rosner, W. Stadler, and M. Ratain, Randomized discontinuation design: application to cytostatic antineoplastic agents. J. Clin. Oncol. 2002; 20: 4478–4484. International Conference on Harmonization, E5: Guidance on Ethnic Factors in the Acceptability of Foreign Clinical Data. 1998. Available: http://www.fda.gov/cder/guidance/. R. Nagata, H. Fukase, and J. Rafizadeh-Kabe, East-West development: Understanding the usability and acceptance of foreign data in Japan. Internat. J. Clin. Pharmacol. Therapeut. 2000; 38: 87–92. S. Ono, C. Yoshioka, O. Asaka, K. Tamura, T. Shibata, and K. Saito, New drug approval times and clinical evidence in Japan. Contemp. Clin. Trials 2005; 26: 660–672. Y. Uyama, T. Shibata, N. Nagai, H. Hanaoka, S. Toyoshima, and K. Mori, Successful bridging strategy based on ICH E5 guideline for drugs approved in Japan. Clin. Pharmacol. Therapeut. 2005; 78: 102–113. Biomarker Definition Working Group, Biomarkers and surrogate endpoints: preferred definitions and conceptual framework. Clin. Pharmacol. Therapeut. 2001; 69: 89–95. L. Lesko and A. Atkinson, Use of biomarkers and surrogate markers in drug development. Annu. Rev. Pharmacol. Toxicol. 2001; 41: 347–366. International Conference on Harmonization, E1: The Extent of Population Exposure to Assess Clinical Safety for Drugs Intended for Long-term Treatment of NonLife-Threatening Conditions. 1995. Available: http://www.fda.gov/cder/guidance/.

DOSE RANGING CROSS-OVER DESIGNS 36. A. Tamhane, K. Shi, and K. Strassburger, Power and sample size determination for a step-wise test procedure for determining the maximum safe dose. J. Stat. Planning Infer. 2006; 36: 2163–2181. 37. L. B. Sheiner, S. L. Beal, and N. C. Sambol, Study designs for dose-ranging. Clin. Pharmacol. Therapeut. 1989; 46: 63–77. 38. L. B. Sheiner, Y. Hashimoto, and S. Beal, A simulation study comparing designs for dose ranging. Stats. Med. 1991; 10: 303–321. 39. L. B. Sheiner, Bioequivalence revisited. Stats. Med. 1992; 11: 1777–1788. 40. R. E. Walpole, R. H. Myers, and S. L. Myers, Probability and Statistics for Engineers and Scientists, 6th ed. Englewood Cliffs, NJ: Prentice Hall, 1998. 41. S. Machado, R. Miller, and C. Hu, A regulatory perspective on pharmacokinetic and pharmacodynamic modelling. Stats. Methods Med. Res. 1999; 8: 217–245. 42. W. Slob, Dose-response modeling of continuous endpoints. Toxicol. Sci. 2002; 66: 298–312. 43. S. Patterson, S. Francis, M. Ireson, D. Webber, and J. Whitehead, A novel Bayesian decision procedure for early-phase dose-finding studies. J. Biopharmaceut. Stats. 1999; 9: 583–598. 44. F. Harrell, Regression Modelling Strategies. New York: Springer, 2001. 45. T. G. Filloon, Estimating the minimum therapeutically effective dose of a compound via regression modeling and percentile estimation. Stats Med. 1995; 14: 925–932. 46. T. N. Johnson, Modelling approaches to dose estimation in children. Br. J. Clin. Pharmacol. 2005; 59: 663–669. 47. T. Johnson, J. Taylor, R. Haken, and A. Eisbruch, A Bayesian mixture model relating dose to critical organs and functional complication in 3D conformal radiation therapy. Biostatistics 2005; 6: 615–632. 48. P. Lupinacci, and D. Raghavarao, Designs for testing lack of fit for a nonlinear dose-response curve model. J. Biopharmaceut. Stats. 2000; 10: 43–53. 49. J. Whitehead, Y. Zhou, J. Stevens, G. Blakey, J. Price, and J. Leadbetter, Bayesian decision procedures for dose-escalation based on evidence of undesirable events and therapeutic benefit. Stats. Med. 2006a; 25: 37–53. 50. V. Fedorov and P. Hackl, Model-Oriented Design of Experiments. New York: Springer, 1997.

15

51. S. Biedermann, H. Dette, and W. Zhu, Optimal designs for dose-response models with restricted design spaces. J. Am. Stats. Assoc. 2006; 101: 747–759. 52. L. Desfrere, S. Zohar, P. Morville, A. Brunhes, S. Chevret, G. Pons, G. Moriette, E. Reyes, and J. M. Treluyers, Dose-finding study of ibuprofen in patent ductus arteriosus using the continual reassessment method. J. Clin. Pharm. Therapeut. 2005; 30: 121–132. 53. M. Gonen, A Bayesian evaluation of enrolling additional patients at the maximum tolerated dose in phase I trials. Contemp. Clin. Trials 2005; 26: 131–140. 54. Y. Loke, S. Tan, Y. Cai, and D. Machin, A Bayesian dose finding design for dual endpoint phase I trials. Stats. Med. 2006; 25: 3–22. 55. P. Thall and J. Cook, Dose-finding based on efficacy-toxicity trade-offs. Biometrics 2004; 60: 684–693. 56. J. Whitehead, Y. Zhou, S. Patterson, D. Webber, and S. Francis, Easy-to implement Bayesian methods for dose-escalation studies in healthy volunteers. Biostatistics 2001a; 2: 47–61. 57. J. Whitehead, Y. Zhou, N. Stallard, S. Todd, and A. Whitehead, Learning from previous responses in phase 1 dose escalation studies. Br. J. Clin. Pharmacol. 2001b; 52: 1–7. 58. J. Whitehead, Y. Zhou, A. Mander, S. Ritchie, A. Sabin, and A. Wright, An evaluation of Bayesian designs for dose-escalation studies in healthy volunteers. Stats. Med. 2006b; 25: 433–445. 59. Y. Zhou, J. Whitehead, E. Bonvini, and J. Stevens, Bayesian decision procedures for binary and continuous bivariate doseescalation studies. Pharmaceut. Stats. 2006; 5: 125–133. 60. C. Buoen, O. Bjerrum, and M. Thomsen, How first-time-in-humans studies are being performed: a survey of phase I dose-escalation trials in healthy volunteers published between 1995 and 2004. J. Clin. Pharmacol. 2005; 45: 1123–1136. 61. D. Sheshkin, Handbook of Parametric and Nonparametric Statistical Procedures. London: Chapman & Hall, CRC Press, 2000. 62. J. Hsu, Multiple Comparisons: Theory and Methods. London: Chapman & Hall, CRC Press, 1996. 63. J. W. Tukey, J. L. Ciminera, and J. F. Heyse, Testing the statistical certainty of a response to increasing doses of a drug. Biometrics 1985; 41: 295–301.

16

DOSE RANGING CROSS-OVER DESIGNS

64. P. Bauer, J. Rohmel, W. Maurer, and L. Hothorn, Testing strategies in multi-dose experiments including active control. Stats. Med. 1998; 17: 2133–2146. 65. A. Dmitrienko, W. Offen, O. Wang, and D. Xiao, Gate-keeping procedures in doseresponse clinical trials based on the Dunnett test. Pharmaceut. Stats. 2006; 5: 19–28. 66. Y. Cheung, Coherence principles in dosefinding studies. Biometrika 2005; 92: 863–873. 67. J. A. Robinson, Sequential choice of an optimal dose: a prediction intervals approach. Biometrika 1978; 65: 75–78. 68. C. Jennison and B.W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. New York: Chapman and Hall, 2000.

77. L. Jayaram, M. Duong, M. M. M. Pizzichini, E. Pizzichini, D. Kamada, A. Efthimiadis, and F. E. Hargreave, Failure of montelukast to reduce sputum eosinophilia in high-dose corticosteroid-dependent asthma. Euro. Respir. J. 2005; 25: 41–46. 78. M. M. Huycke, M. T. Naguib, M. M. Stroemmel, K. Blick, K. Monti, S. Martin-Munley, and C. Kaufman, A double-blind placebocontrolled crossover trial of intravenous magnesium sulfate for foscarnet-induced ionized hypocalcemia and hypomagnesemia in patients with AIDS and cytomegalovirus infection. Antimicrob. Agents Chemother. 2000; 44: 2143–2148. 79. P. D. Martin, M. J. Warwick, A. L. Dane, and M. V. Cantarini, A double-blind, randomized, incomplete crossover trial to assess the dose proportionality of rosuvastatin in healthy volunteers. Clin. Therapeut. 2003; 25: 2215–2224.

69. M. Krams, K. Lees, W. Hacke, A. Grieve, J.-M. Orgogozo, and G. Ford, for the ASTIN Study Investigators, Acute stroke therapy by inhibition of neutrophils: an adaptive dose-response study of UK-279276 in acute ischemic stroke. Stroke 2003; 34: 2543–2548.

80. International Conference on Harmonization, E14: The Clinical Evaluation of QT/QTc Interval Prolongation and Proarrythmic Potential for Non-Antiarrythmic Drugs. 2005. http://www.fda.gov/cder/guidance/.

70. V. Dragalin and V. Fedorov, Adaptive ModelBased Designs for Dose-Finding Studies. GSK BDS Technical Report 2004–02, 2004.

FURTHER READING

71. L. Kong, G. Koch, T. Liu, and H. Wang, Performance of some multiple testing procedures to compare three doses of a test drug and placebo. Pharmaceut. Stats. 2004; 4: 25–35. 72. H. Moldofsky, F. Lue, C. Mously, B. RothSchechter, and W. Reynolds, The effect of zolpidem in patients with fibromyalgia: a dose ranging, double blind, placebocontrolled, modified cross-over study. J. Rheumatol. 1996; 23: 529–533. 73. FDA Draft Guidance, Estimating the Safe Starting Dose in Clinical Trials for Therapeutics in Adult Healthy Volunteers. 2006. Available: http://www.fda.gov/cder/guidance/. 74. B. Reigner and K. Blesch, Estimating the starting dose for entry into humans: principles and practice. Euro. J. Clin. Pharmacol. 2001; 57: 835–845. 75. C. Chuang-Stein and W. Shih, A note on the analysis of titration studies. Stats. Med. 1991; 10: 323–328. 76. J. Mandema, D. Hermann, W. Wang, T. Sheiner, M. Milad, R. Bakker-Arkema, and D. Hartman, Model-based development of gemcabene, a new lipid-altering agent. AAPS J. 2005; 7: E513–E522.

International Conference on Harmonization, E9: Statistical Principles for Clinical Trials. 1998. Available: http://www.fda.gov/cder/guidance/.

DOUBLE-DUMMY

3

KENNETH F. SCHULZ

Perhaps most crucially, blinding aids in reducing differential assessment of outcomes (commonly termed information or ascertainment bias) prompted by knowledge of the group assignment of individuals being observed (3–5). For example, if unblinded outcome assessors believe a new intervention is better, they could record more ‘‘charitable’’ responses to that intervention. Indeed, in a placebo-controlled multiple sclerosis trial, the unblinded, but not the blinded, neurologists’ assessments showed an apparent benefit of the intervention (10). Subjective outcomes (e.g., pain scores) present greater opportunities for bias (9). Even some outcomes considered as objective, such as myocardial infarction, can be fraught with subjectivity. In general, blinding becomes less important to reduce information bias as the outcomes become less subjective. Objective (hard) outcomes, such as death, leave little opportunity for bias. Less understood, blinding also operationally improves compliance and retention of trial participants and reduces biased supplemental care or treatment (sometimes called co-intervention) (3–5). Many potential advantages emanate from participants, investigators, and outcome assessors not knowing the intervention group to which the participants have been assigned (Table 1) (11).

Family Health International Research Triangle Park, North Carolina

1

INTRODUCTION

Blinding in research began over two centuries ago (1). Most researchers and readers grasp the concept. Unfortunately, beyond that general comprehension lies confusion. Terms such as ‘‘single-blind,’’ ‘‘double-blind,’’ and ‘‘triple-blind’’ mean different things to different people (2). Although clinical trial textbooks (3–5), clinical trial dictionaries (6, 7), and a new edition of Last’s epidemiology dictionary (8) address blinding, they do not entirely clear the lexicographical fog (9). Investigators, textbooks, and published articles all vary greatly in their interpretations (2). In other words, terminological inconsistencies surround blinding, and doubledummy terms add a whole additional level of confusion. I will discuss blinding in general and then describe the relevance of doubledummy blinding 2

POTENTIAL IMPACTS OF BLINDING

‘‘DOUBLE-BLINDING’’ DEFINED

The terminology ‘‘double-blind’’ (or doublemask) usually means that trial participants, investigators (usually health care providers), and assessors (those collecting outcome data) all remain oblivious to the intervention assignments throughout the trial (9) so that they will not be influenced by that knowledge. Given that three groups are involved, ‘‘double-blind’’ appears misleading. In medical research, however, an investigator also frequently assesses, so, in that instance, the terminology accurately refers to two individuals. When I use ‘‘double-blind’’ or its derivatives in this article, I mean that steps have been taken to blind participants, investigators, and assessors. In reporting randomized control trials (RCTs), authors should explicitly state what steps were taken to keep whom blinded, as clearly stated in the CONSORT guidelines.

4

PLACEBOS AND BLINDING

Interventions (treatments) at times have no effect on the outcomes being studied (9). When an ineffective intervention is administered to participants in the context of a well-designed RCT, however, beneficial effects sometimes occur on participants attitudes, which in turn influences outcomes (3). Researchers refer to these phenomena as the ‘‘placebo effect.’’ A placebo refers to a pharmacologically inactive agent that investigators administer to participants in the control group in a trial (9). The use of a placebo control

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

DOUBLE-DUMMY Table 1. Potential Advantages of Successfully Blinding Participants, Investigators, and Assessors (11) If blinded?

Potential advantages

Participants

• • • •

Less likely to have biased responses to the interventions More likely to comply with trial treatments Less likely to seek supplementary interventions More likely to continue in the trial, providing outcome

Trial investigators

• • • • •

Less likely to transfer their preferences or viewpoints to participants Less likely to differentially dispense co-interventions Less likely to differentially alter dosage Less likely to differentially withdraw participants Less likely to differentially dissuade participants to continue in the trial

Assessors

• Less likely to have biases influence their outcome assessments

group balances the placebo effect in the treatment group allowing for the independent assessment of the treatment effect. Although placebos may have effects mediated through psychological mechanisms, they are administered to participants in a trial because they are otherwise ‘‘inactive.’’ Although the effect of placebos is contentious (12), when assessing the effects of a proposed new treatment for a condition for which no effective treatment already exists, the widespread view remains that placebos should be administered, whenever possible, to participants in the control group (3, 4). Placebos are generally used in trials of drugs, vaccines, and other medicinal interventions, but can sometimes also be used in trials of procedures, such as ultrasound, acupuncture, and occasionally, surgery. 5

DOUBLE-DUMMY BLINDING

As most double-blinding involves drugs, I confine my comments to such instances. When an effective standard treatment exists, it is usually used in the control group for comparison against a new treatment (9). Thus, trialists compare two active treatments. In that circumstance, they usually have three options to double-blind their trial. First, they could obtain the drugs in raw form and package the two treatments identically, such as in capsules or pills. Preferably, taste would also be equilibrated. Participants

would simply take one normal-sized capsule or pill. Frequently, however, this approach presents production problems. Pharmaceutical manufacturers seldom provide their drugs in any form other than the standard formulations. Moreover, even if they did, generating different formulations might raise objections with government regulatory bodies, such as the U.S. Food and Drug Administration (FDA), as to the equivalent bioavailability of the created formulations, even with drugs with approved formulations (13). Erasing those objections probably involves more research, more expense, and more time. This identical packaging approach frequently presents formidable production obstacles. Second, the standard formulation drugs could be encapsulated in a larger capsule, which alleviates some production concerns, but, again, may trigger equivalent bioavailability questions (13), which generates additional research expense and delays. Moreover, although the participants would only take one capsule, it may be so large as to discourage ingestion. Furthermore, some encapsulation systems allow the participant to open the capsules thereby breaking the blind (13). This encapsulation approach still involves production difficulties and also presents compliance impediments and unblinding opportunities. Third, they could conduct a double-dummy (double-placebo) design where participants receive the assigned active drug and the

DOUBLE-DUMMY

placebo matched to the comparison drug. The trial involves two active drugs and two matching placebos. For example, in comparing two agents, one in a blue capsule and the other in a red capsule, the investigators would acquire blue placebo capsules and red placebo capsules. Then both treatment groups would receive a blue and a red capsule, one active and one inactive. This option protects the double-blinding better than the second option and as well as the first. Unlike the other two options, it does not raise equivalent bioavailability questions nor does it involve production problems, delays, and costs. The only downside tends to be that participants take more pills, which could hurt enrollment or compliance. However, when examined, investigators found minimal impact on enrollment (13). Pragmatically, investigators should find fewer procedural problems with double-dummy blinding. No wonder that this approach appears the most frequently used of the three options. For simplicity of presentation, most of the discussions focused on drugs in capsule, pill, or tablet formulations. The concepts, however, extend easily to other formulations, such as intravenous fluids and ampoules administered through injections or intravenous drips (Panel 1). Where double-dummy blinding becomes difficult to implement are those situations in which blinding itself is difficult under any circumstances, such as in surgical trials. Unfortunately, some authors have disparaged placebos. Many of those efforts have been misguided. Placebos have served as the scapegoat for the problem of having an inactive treatment control group when ethically an active treatment control group is indicated because an effective treatment exists. Inappropriate control groups are the problem, not placebos. Thus, placebos emerge as critical to double-blinding in randomized trials. Inescapably, placebos are usually scientifically necessary for double-blinding, regardless of whether the control group is active or inactive. In trials with a comparison group receiving no active treatment, placebos have obvious importance. That manifestation fits the common paradigm of placebo

3

usage. However, as displayed above, placebos also frequently emerge as necessary if a trial compares two or more active treatments. If those treatments differ,\emdash for example, in shape, size, weight, taste, or color, the double-dummy technique (using two placebos) nearly always indispensably serves methodological and production concerns. 5.1 Panel 1: Examples of Descriptions of ‘‘Double-Dummy’’ Blinding ‘‘The patients were allocated to doubledummy treatment with dalteparin 100 IU/kg subcutaneously twice a day (Fragmin, Pharmacia and Upjohn, Stockholm, Sweden) and placebo tablets every day, or aspirin tablets 160 mg every day (Albyl-E, Nycomed Pharma, Oslo, Norway) and placebo ampoules subcutaneously twice a day’’ (14). ‘‘To maintain masking, each patient received two simultaneous infusions, one active and one placebo. The placebo infusion regimen was identical to its respective active counterpart’’ (15). REFERENCES 1. T. J. Kaptchuk, Intentional ignorance: a history of blind assessment and placebo controls in medicine. Bull. Hist. Med. 1998; 72: 389–433. 2. P. J. Devereaux, B. J. Manns, W. A. Ghali, H. Quan, C. Lacchetti, V. M. Montori et al., Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials. JAMA 2001; 285: 2000–2003. 3. S. J. Pocock, Clinical Trials: A Practical Approach. Chichester: Wiley, 1983. 4. C. L. Meinert, Clinical Trials: Design, Conduct, and Analysis. New York: Oxford University Press, 1986. 5. L. Friedman, C. Furberg, and D. DeMets, Fundamentals of Clinical Trials. St. Louis, MO: Mosby, 1996. 6. C. L. Meinert, Clinical Trials Dictionary. Baltimore, MD: The Johns Hopkins Center for Clinical Trials, 1996. 7. S. Day, Dictionary for Clinical Trials. Chichester: Wiley, 1999. 8. J. M. Last (ed.), A Dictionary of Epidemiology. Oxford: Oxford University Press, 2001.

4

DOUBLE-DUMMY

9. K. F. Schulz, I. Chalmers, and D. G. Altman, The landscape and lexicon of blinding in randomized trials. Ann. Intern. Med. 2002; 136: 254–259. 10. J. H. Noseworthy, G. C. Ebers, M. K. Vandervoort, R. E. Farquhar, E. Yetisir, and R. Roberts, The impact of blinding on the results of a randomized, placebo- controlled multiple sclerosis clinical trial. Neurology 1994; 44: 16–20. 11. K. F. Schulz and D. A. Grimes, Blinding in randomised trials: hiding who got what. Lancet 2002; 359: 696–700. 12. A. Hrobjartsson and P. C. Gøtzsche, Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment. N. Engl. J. Med. 2001; 344: 1594–1602. 13. B. K. Martin, C. L. Meinert, and J. C. Breitner, Double placebo design in a prevention trial for Alzheimer’s disease. Control Clin. Trials 2002; 23: 93–99. 14. E. Berge, M. Abdelnoor, P. H. Nakstad, and P. M. Sandset, Low molecular-weight heparin versus aspirin in patients with acute ischaemic stroke and atrial fibrillation: a double-blind randomised study. HAEST Study Group. Heparin in acute embolic stroke trial. Lancet 2000; 355: 1205–1210. 15. F. Follath, J. G. Cleland, H. Just, J. G. Papp, H. Scholz, K. Peuhkurinen et al., Efficacy and safety of intravenous levosimendan compared with dobutamine in severe low-output heart failure (the LIDO study): a randomised double-blind trial. Lancet 2002; 360: 196–202.

DRIFT (FOR INTERIM ANALYSES)

The solution can be found by numerical integration, and a variety of software for doing so is available (10–13). Slud and Wei (14) developed an equation similar to Equation (2). Lan and DeMets related the α k to time through an alpha spending function α*(t). In the spending function approach, an increasing function α*(t) is defined such that α*(0) = 0, α*(1) = α, the overall type 1 error rate, and α k = α* (tk ) – α*(tk −1). In this way, a certain proportion of the overall type 1 error is ‘‘spent’’ at each interim analysis. To relate the Brownian motion process to accumulating evidence in a clinical trial directly, we use an analogy with partial sums (5,15,16). Consider N independent normal variables X1 , X2 , . . . , XN with unknown mean δ and variance equal to one. The sum of the first nk , S(nk ), is distributed normally with mean nk δ and variance nk , and the Brownian motion process at tk is distributed normally with mean tk θ and variance tk . By equating the proportion of observations in the partial sum and the Brownian motion time scale, √ tk = nk /N, the relation √ of means is δ = θ / N so that B(tk ) and S(nk )/ N have the same distri√ bution. √ Also, the joint distribution of S1 / N, . . . , SK / N is multivariate normal with the same covariance described by Equation (1). The Z statistics used for testing at each interim analysis are also related to the Brownian motion and partial sums. If Z(tk ) is the standard normal summary statistic√at interim analysis k, then B(tk ) and Z(tk ) tk have the same distribution and covariance structure. Note that this covariance structure holds for a wide variety of trials with different study outcomes, including any for which the summary is a maximum likelihood estimate (17,18). For factorial or crossover designs or studies with multiple primary endpoints, the situation is more complex, but the analogy holds for any trial that can be summarized by a single Z statistic. Thus, there is a correspondence among a Brownian motion process B(t), interim test statistics, and partial sums representing accumulating data in the study. A technical point associated with this description is that the time t must be understood as ‘‘information time’’ (16,19,20). That is, although actual analyses occur in calen-

DAVID M. REBOUSSIN Wake Forest University School of Medicine, Division of Public Health Sciences, Winston-Salem, North Carolina

1

INTRODUCTION

An early step in the development of group sequential testing as it is commonly applied to phase III clinical trials was the investigation of repeated significance tests (1). This led to consideration of sequentially recruited groups of equal sized (2,3) and then into procedures that allow some flexibility in the number and timing of interim analyses as a trial progresses using the spending function approach (4–6). We will focus on the spending function approach, but similar development can be applied to other approaches to group sequential testing (7,8). Brownian motion is a stochastic process that starts at zero and has independent, normally distributed increments (9). Group sequential testing is implemented using computed probabilities associated with a standard Brownian motion process, B(t), with mean zero observed at times 0 < t1 , . . . , distribution of B(t) at t = tK ≤ 1. The joint
(t1 , ... tk ) is N(θ t, ), where  kl

= min(tk , tl )

(1)

The parameter θ denotes the drift or mean of the Brownian motion process over time. Figure 1 illustrates an example of Brownian motion over time with a positive drift. A set of (two-sided symmetric) bounds ±b1 , . . . , ±bK associated with the values B(t1 ), . . . , B(tK ) can be determined iteratively. B(t1 ) has a normal distribution with mean zero and variance t1 so that calculation of b1 is straightforward; however, determination of subsequent bounds must account for not having stopped earlier. For k > 1, bk solves the equation Pr|{B(tj )| ≤ bj , j = 1, . . . , k − 1; |B(tk )| > bk } = αk (2)

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

DRIFT (FOR INTERIM ANALYSES)

dar time—a certain number of days since the start of randomization—the time scale that best describes the unfolding evidence about a treatment effect is the proportion of the expected statistical information that has accumulated when the analysis is conducted. This idea can be extended to various designs (19), and in many situations, statistical information accrues more or less uniformly in calendar time, so that the distinction is not critical. However, being mindful of the difference between calendar time and information time can prevent confusion in certain situations, for example, in survival studies when the event rate is lower than anticipated and calendar time runs out before the expected number of events has occurred. When determining bounds, the computation is done assuming B(t) has drift zero as it does under the null hypothesis and that there is no treatment effect. However, the same equation can be used to determine probabilities associated with nonzero drift means given a fixed set of bounds: The drift corresponding to given cumulative exit probability can be determined. This is the basis for determining the effect of a specific monitoring plan on sample size requirements, estimation, and computing of the ‘‘conditional power,’’ all of which are detailed below. 2 SAMPLE SIZE DETERMINATION FOR TRIALS INVOLVING GROUP SEQUENTIAL TESTS With the background above, sample size determinations that take into account the group sequential design can be done by relating the test statistic summarizing the treatment effect to a Brownian motion (5,13,15). Consider the required nonsequential sample size per group for a comparison of means in two normal populations, which is Nfixed = (z1−α/2 + z1 −β )2 (2σ 2 /(µT − µC )2 ) When there is no sequential monitoring, the drift corresponding to power 1−β is θ fixed = z1−α/2 + z1 −β . For two-sided tests with alpha = 0.05 and 80%, 85%, and 90% power, θ fixed is approximately 2.8, 3, and 3.25, respectively. For a given set of interim analysis times

and bounds, the corresponding drift can be computed as described above and inserted in place of z1−α/2 + z1 −β so that 2 (2σ 2 /(µT − µC )2 ) Nseq = θseq

For example, the drift corresponding to five equally spaced, two-sided O’Brien-Flemingtype bounds for alpha = 0.05 and 90% power is 3.28, so if σ 2 = 1 and µT − µC = 0.5, Nseq = 3.282 × (2/0.25) = 86. Note that the sequential design requires a slightly larger sample size, and the increase is the square of the ratio of the sequential drift to the fixed drift. In fact the details of the relationship between design parameters and the sequential drift are not essential in order to adjust a fixed sample size to account for sequential monitoring. The drift associated with the same design monitored sequentially with given bounds and times can be computed using the equation above and available software (13). Then the required proportional increase in sample size is just the ratio of the drift under the planned sequential analyses to the drift for the fixed design. In the example above, the increase is (3.28/3.25)2 or about 2%. This small increase reflects the early conservatism of the O’Brien-Fleming strategy, which is often desirable. Minor deviations in the number and timing of interim analyses has little effect on the drift (5), so it is reasonable to design a study based on an anticipated monitoring plan and expect that the actual monitoring will have negligible effect on power. For sample size determination, then, understanding that the drift of an underlying Brownian motion is related to the expectation of interim Z statistics provides a straightforward way to adjust the sample size for an anticipated sequential monitoring plan. 3 ESTIMATION AFTER GROUP SEQUENTIAL TESTING There are a variety of approaches to estimation and bias reduction in the context of a group sequential test, and some refer to an underlying Brownian motion with nonzero drift (21–27). As an example that is closely related to the discussion of sample size in the

DRIFT (FOR INTERIM ANALYSES)

3

previous section, we can consider the median unbiased estimator (23). This is related to a confidence interval procedure that defines the 1 − γ lower confidence limit as the smallest value of the drift for which an outcome as least as extreme as the observed has probability of at least γ , and likewise for the upper limit (22). Equation (2) in this context is used to determine the value of the drift. The last bound bk is replaced with the final observed Z statistic, and α k is replaced with γ or 1 − γ . The median unbiased estimator sets γ = α k = 0.5, and for a 95% confidence interval, the upper limit sets α k = 0.025 and the lower α k = 0.975. Thus, some approaches to estimation make use of the drift of an underlying Brownian motion to form confidence intervals and point estimates.

assumptions for θ . Typically the assumptions of particular interest are θ = 0, θ = B(t)/t (the current best estimate of the drift), and θ = θ des (the value assumed for the study design). For example, consider a study for which t = 0.6 and Z(0.6) = 1.28. Then B(0.6) = 0.99 and U(t) has a mean of θ × 0.4 and a variance of 0.4. The current estimate of drift is B(0.6)/0.6 = 1.65, so under this assumption, U(t) has a mean of 0.66. Using the current estimate for drift, the conditional probability that the final Z statistic, Z(1), will exceed 1.96 given that √ Z(0.6) = 1.28 is Pr(Z > (1.96 – 1.28 – 0.66)/ 0.4) = Z0.49 = 0.31 or 31%. In this way, use of the underlying Brownian motion drift provides a simple way of assessing partway through the study the likelihood of a statistically significant final result.

4

5

STOPPING FOR FUTILITY

The drift of a Brownian motion process is also a useful model for assessing probabilities associated with the decision to terminate a trial when interim analyses show that the null hypothesis is unlikely to be rejected. Lachin (28) provides a detailed review of methods based on ‘‘conditional power’’ or the probability that the final study result will be statistically significant given the accumulated data and an assumption about future trends. This approach has also been described as stochastic curtailment (29–31). Lan and Wittes (32) describe the B-value, which is related to a Brownian motion process and provides a straightforward way to compute conditional power. The B-value is a translation of the interim Z statistic to a scale on which it can be assessed with respect to the drift of the Brownian motion underlying the study’s accumulating √ data. At time t, the B-value is B(t) = Z(t) t and the time remaining before the planned end of the study is 1 − t. The entire study can be thought of as decomposed into two pieces before and after t: B(t) and U(t) = B(1) – B(t). The distribution of U(t), representing the as-yet unobserved data, is normal and independent of B(t) with mean θ × (1 − t) and variance 1 − t. B(t) + U(t) equals B(1) and Z(1), the Z value at the end of the study. Given the observed B(t), the probability that B(t) + U(t) exceeds a critical value and can be computed under various

CONCLUSION

Calculation of bounds for a group sequential procedure and the associated testing can be done without reference to Brownian motion or drift. However, the connection between interim test statistics and Brownian motion drift is tractable and facilitates understanding of such useful elaborations as sample size determination, estimation, and stopping for futility. REFERENCES 1. P. Armitage, C. K. McPherson, and B. C. Rowe, Repeated significance tests on accumulating data. J. R. Stat. Soc. Series A (General) 1969; 132(2):235–244. 2. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics 1979 Sep; 35(3):549–556. 3. S. J. Pocock, Group sequential methods in design and analysis of clinical-trials. Biometrika 1977; 64(2):191–200. 4. K. K. G. Lan and D. L. DeMets, Discrete sequential boundaries for clinical-trials. Biometrika 1983; 70(3):659–663. 5. K. Kim and D. L. DeMets, Design and analysis of group sequential tests based on the type i error spending rate function. Biometrika 1987 Mar; 74(1):149–154. 6. D. L. DeMets and K. K. Lan, Interim analysis: the alpha spending function approach. Stat. Med. 1994 Jul 15; 13(13-14):1341–1352.

4

DRIFT (FOR INTERIM ANALYSES) 7. S. S. Emerson and T. R. Fleming, Symmetric group sequential test designs. Biometrics 1989 Sep; 45(3):905–923. 8. J. Whitehead and I. Stratton, Group sequential clinical trials with triangular continuation regions. Biometrics 1983 Mar; 39(1):227–236. 9. P. Billingsley, Probability and Measure, 3rd ed. New York: Wiley, 1995.

10. Cytel Inc., East v. 5: Advanced Clinical Trial Design, Simulation and Monitoring System. Cambridge, MA: Cytel Inc., 2007. 11. MSP Research Unit, PEST 4: Operating Manual. Reading, UK: The University of Reading, 2000. 12. Insightful Corp., S + SeqTrial. Seattle, WA: Insightful Corp., 2006. 13. D. M. Reboussin, D. L. DeMets, K. Kim, and K. K. G. Lan, Computations for group sequential boundaries using the Lan-DeMets spending function method. Controlled Clin. Trials 2000; 21(3):190–207. 14. E. Slud and L. J. Wei, Two-sample repeated significance tests based on the modified Wilcoxon statistic. J. Am. Stat. Assoc. 1982 Dec; 77(380):862–868. 15. K. Kim and D. L. DeMets, Sample size determination for group sequential clinical trials with immediate response. Stat. Med. 1992 Jul; 11(10):1391–1399. 16. K. K. Lan and D. M. Zucker, Sequential monitoring of clinical trials: the role of information and Brownian motion. Stat. Med. 1993 Apr 30; 12(8):753–765. 17. C. Jennison and B. W. Turnbull, Groupsequential analysis incorporating covariate information. J. Am. Stat. Assoc. 1997 Dec; 92(440):1330–1341. 18. D. O. Scharfstein, A. A. Tsiatis, and J. M. Robins, Semiparametric efficiency and its implication on the design and analysis of group-sequential studies. J. Am. Stat. Assoc. 1997 Dec; 92(440):1342–1350. 19. K. K. G. Lan, D. M. Reboussin, and D. L. DeMets, Information and information fractions for design and sequential monitoring of clinical-trials. Commun. Stat. Theory Methods 1994; 23(2):403–420. 20. K. Lan and D. L. DeMets, Group sequential procedures: Calendar versus information time. Stat. Med. 1989 Oct; 8(10):1191–1198. 21. C. Jennison and B. W. Turnbull, Interim analyses: The repeated confidence interval approach. J. R. Stat. Soc. Series B (Methodological) 1989; 51(3):305–361.

22. K. Kim and D. L. DeMets, Confidence intervals following group sequential tests in clinical trials. Biometrics 1987 Dec; 43(4):857–864. 23. K. Kim, Point estimation following group sequential tests. Biometrics 1989 Jun; 45(2):613–617. 24. M. LeBlanc and J. Crowley, Using the bootstrap for estimation in group sequential designs: An application to a clinical trial for nasopharyngeal cancer. Stat. Med. 1999 Oct 15; 18(19):2635–2644. 25. Z. Q. Li ZQ and D. L. DeMets, On the bias of estimation of a Brownian motion drift following group sequential tests. Statistica Sinica 1999 Oct; 9(4):923–937. 26. A. Y. Liu and W. J. Hall, Unbiased estimation following a group sequential test. Biometrika 1999 Mar; 86(1):71–78. 27. J. C. Pinheiro and D. L. DeMets, Estimating and reducing bias in group sequential designs with Gaussian independent increment structure. Biometrika 1997 Dec; 84(4):831–845. 28. J. M. Lachin, A review of methods for futility stopping based on conditional power. Stat. Med. 2005 Sep 30; 24(18):2747–2764. 29. M. Halperin, K. K. Lan, E. C. Wright, and M. A. Foulkes, Stochastic curtailing for comparison of slopes in longitudinal studies. Control Clin. Trials 1987 Dec; 8(4):315–326. 30. K. K. G. Lan, D. L. DeMets, and M. Halperin, more flexible sequential and nonsequential designs in long-term clinicaltrials. Commun. Stat. Theory Methods 1984; 13(19):2339–2353. 31. M. Halperin, K. K. Lan, J. H. Ware, N. J. Johnson, and D. L. DeMets, An aid to data monitoring in long-term clinical trials. Control Clin Trials 1982 Dec; 3(4):311–323. 32. K. K. Lan and J. Wittes, The B-value: a tool for monitoring data. Biometrics 1988 Jun; 44(2):579–585.

CROSS-REFERENCES Group Sequential Designs Interim Analyses Stopping Boundaries Alpha-spending function Conditional power

DRUG DEVELOPMENT

Pre-clinical research

Clinical studies E

Synthesis and purification

NDA review

Phase 1 E

Phase 2 Phase 3

Accelerated development/review Animal testing

E

Short -term

Treatment IND Parallel track

Long -term Institutional review boards Industry time FDA time IND submitted

NDA submitted Review decision Sponsor/FDA meetings encouraged Early access: Sponsor answers E subpart E Advisory committees any questions from review

Under the Food and Drug Administration (FDA) requirements, a sponsor first must submit data that shows that the drug is reasonably safe for use in initial, small-scale clinical studies. Depending on whether the compound has been studied or marketed previously, the sponsor may have several options for fulfilling this requirement: (1) compiling existing nonclinical data from past in vitro laboratory or animal studies on the compound, (2) compiling data from previous clinical testing or marketing of the drug in the United States or another country whose

population is relevant to the U.S. population, or (3) undertaking new preclinical studies designed to provide the evidence necessary to support the safety of administering the compound to humans. During preclinical drug development, a sponsor evaluates the toxic and pharmacologic effects of the drug through in vitro and in vivo laboratory animal testing. Genotoxicity screening is performed, as well as investigations on drug absorption and metabolism, the toxicity of the metabolites of the drug, and the speed with which the drug and its metabolites are excreted from the body. At the preclinical stage, the FDA generally will ask, at a minimum, that sponsors: (1) develop

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/develop.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

DRUG DEVELOPMENT

a pharmacologic profile of the drug, (2) determine the acute toxicity of the drug in at least two species of animals, and (3) conduct shortterm toxicity studies ranging from 2 weeks to 3 months, depending on the proposed duration of use of the substance in the proposed clinical studies. The research process is complicated, time consuming, and costly, and the end result is never guaranteed. Literally hundreds and sometimes thousands of chemical compounds must be made and tested in an effort to find one that can achieve a desirable result. The FDA estimates that it takes approximately 8.5 years to study and test a new drug before it can be approved for the general public. This estimate includes early laboratory and animal testing and later clinical trials that use human subjects. No standard route exists through which drugs are developed. A pharmaceutical company may decide to develop a new drug aimed at a specific disease or medical condition. Sometimes, scientists choose to pursue an interesting or promising line of research. In other cases, new findings from university, government, or other laboratories may point the way for drug companies to follow with their own research. New drug research starts with an understanding of how the body functions, both normally and abnormally, at its most basic levels. The questions raised by this research help determine a concept of how a drug might be used to prevent, cure, or treat a disease or medical condition. This concept provides the researcher with a target. Sometimes scientists find the right compound quickly, but usually hundreds or thousands must be screened. In a series of test tube experiments called assays, compounds are added one at a time to enzymes, cell cultures, or cellular substances grown in a laboratory. The goal is to find which additions show some effect. This process may require testing hundreds of compounds because some may not work but will indicate ways of changing the chemical structure of the compound to improve its performance. Computers can be used to simulate a chemical compound and design chemical structures that might work against the compound. Enzymes attach to the correct site on the

membrane of a cell, which causes the disease. A computer can show scientists what the receptor site looks like and how one might tailor a compound to block an enzyme from attaching there. Although computers give chemists clues as to which compounds to make, a substance still must be tested within a living being. Another approach involves testing compounds made naturally by microscopic organisms. Candidates include fungi, viruses, and molds, such as those that led to penicillin and other antibiotics. Scientists grow the microorganisms in what is known as a ‘‘fermentation broth,’’ with one type of organism per broth. Sometimes, 100,000 or more broths are tested to see whether any compound made by a microorganism has a desirable effect. In animal testing, drug companies make every effort to use as few animals as possible and to ensure their humane and proper care. Generally, two or more species (one rodent, one nonrodent) are tested because a drug may affect one species differently than another. Animal testing is used to measure how much of a drug is absorbed into the blood, how it is broken down chemically in the body, the toxicity of the drug and its breakdown products (metabolites), and how quickly the drug and its metabolites are excreted from the body. Short-term testing in animals ranges in duration from 2 weeks to 3 months, depending on the proposed use of the substance. Long-term testing in animals ranges in duration from a few weeks to several years. Some animal testing continues after human tests begin to learn whether long-term use of a drug may cause cancer or birth defects. Much of this information is submitted to FDA when a sponsor requests to proceed with human clinical trials. The FDA reviews the preclinical research data and then makes a decision as to whether to allow the clinical trials to proceed. The new drug application (NDA) is the vehicle through which drug sponsors formally propose that the FDA approve a new pharmaceutical for sale in the United States. To obtain this authorization, a drug manufacturer submits in an NDA nonclinical (animal) and clinical (human) test data and analyses,

DRUG DEVELOPMENT

drug information, and descriptions of manufacturing procedures. An NDA must provide sufficient information, data, and analyses to permit FDA reviewers to reach several key decisions, including • Whether the drug is safe and effective

for its proposed use(s), and whether the benefits of the drug outweigh its risks; • Whether the proposed labeling of the drug is appropriate, and, if not, what the label of the drug should contain; and • Whether the methods used in manufacturing the drug and the controls used to maintain the quality of the drug are adequate to preserve the identity, strength, quality, and purity of the drug. The purpose of preclinical work—animal pharmacology/toxicology testing— is to develop adequate data to undergird a decision that it is reasonably safe to proceed with human trials of the drug. Clinical trials represent the ultimate premarket testing ground for unapproved drugs. During these trials, an investigational compound is administered to humans and is evaluated for its safety and effectiveness in treating, preventing, or diagnosing a specific disease or condition. The results of this testing will comprise the single most important factor in the approval or disapproval of a new drug. Although the goal of clinical trials is to obtain safety and effectiveness data, the overriding consideration in these studies is the safety of the people in the trials. CDER monitors the study design and conduct of clinical trials to ensure that people in the trials are not exposed to unnecessary risks.

3

DRUG REGISTRATION AND LISTING SYSTEM (DRLS)

Biological Products) whether they enter interstate commerce. All domestic distributors and foreign firms that import drug products into the United States must obtain a labeler code and must list all of their products.

The Food and Drug Administration (FDA) attempted a comprehensive drug inventory for drug listings by establishing two voluntary programs. However, these two voluntary programs were unsuccessful. To make these efforts mandatory, the FDA instituted the Drug Listing Act of 1972; this regulatory policy is in the 21 Code of Federal Regulations (CFR) Part 207. The 21 CFR Part 207 addresses definitions, drug registration requirements, and drug listing requirements by FDA. This Act amended Section 510 of the Federal Food, Drug, and Cosmetic Act and defines the applicable following terms:

2

All firms, unless exempted by the Act, are requested to list their commercially marketed drug products with FDA within 5 days after the beginning of operation. They are required to list/update their drug products listing twice a year (June and December). The initial listing and updates of a product is completed on a form FDA 2657. Manufacturers are allowed to list the products for distributors on form FDA 2658. To assist the firms with the mandatory update in June, the Product Information Management Branch mails a Compliance Verification Report (CVR) to the firms. The CVR goes to all firms which have at least one prescription product listed with FDA. The firm is required to update the CVR and mail it back within 30 days.

• The term Firm refers to a company

engaged in the manufacture, preparation, propagation, compounding, or processing of a drug product. • The term Drug Products refers to human drugs, veterinary drugs, and medicated animal feed premixes that include biological products, but do not include blood and blood components. • The term Manufacturing and Processing refers to repackaging or otherwise changing the container, wrapper, or labeling of any drug product package in the distribution process from the original ‘‘maker’’ to the ultimate consumer. 1

LISTING REQUIREMENTS

3

REGISTRATION EXEMPTIONS

The following is a list of those parties exempt from registration: pharmacies, hospitals, and clinics that dispense drug products at retail; licensed physicians who use drug products solely for purposes related to their professional practice; and/or persons who use drug products solely for their professional needs and are not for sale.

REGISTRATION REQUIREMENTS

A firm must register all drug products (Domestic Manufacturers, Domestic Repackers, Domestic Labelers, and submissions for New Human Drug Application, New Animal Drug Application, Medicated Feed Application, Antibiotic Drug Application, and Establishment License Application to Manufacture

4

REGISTRATION PROCESS

Firms can register by obtaining a Registration of Drug Establishment Form, FDA 2656 within 5 days after the beginning of operation or submission of an application. Firms are required to re-register annually by returning an Annual Registration of Drug Establishment Form, FDA 2656E, within 30 days after receiving it from the Product Information Management Branch.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/druglist.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

DRUG PACKAGING The sponsor should ensure that the investigational product(s) (including active comparator(s) and placebo, if applicable) is characterized as appropriate to the stage of development of the product(s), is manufactured in accordance with any applicable Good Manufacturing Practice (GMP), and is coded and labeled in a manner that protects the blinding, if applicable. In addition, the labeling should comply with applicable regulatory requirement(s). The sponsor should determine, for the investigational product(s), acceptable storage temperatures, storage conditions (e.g., protection from light), storage times, reconstitution fluids and procedures, and devices for product infusion, if any. The sponsor should inform all involved parties (e.g., monitors, investigators, pharmacists, storage managers) of these determinations. The investigational product(s) should be packaged to prevent contamination and unacceptable deterioration during transport and storage. In blinded trials, the coding system for the investigational product(s) should include a mechanism that permits rapid identification of the product(s) in case of a medical emergency, but it does not permit undetectable breaks of the blinding. If significant formulation changes are made in the investigational or comparator product(s) during the course of clinical development, the results of any additional studies of the formulated product(s) (e.g., stability, dissolution rate, bioavailability) are needed to assess whether these changes would significantly alter the pharmacokinetic profile of the product should the product be available prior to the use of the new formulation in clinical trials.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

DRUG SUPPLY

The sponsor should: • Take steps to ensure that the investi-

gational product(s) are stable over the period of use. • Maintain sufficient quantities of the investigational product(s) used in the trials to reconfirm specifications, should it become necessary, and maintain records of batch sample analyses and characteristics. To the extent stability permits, samples should be retained either until the analyses of the trial data are complete or as required by the applicable regulatory requirement(s), whichever represents the longer retention period.

The sponsor is responsible for supplying the investigator(s)/institution(s) with the investigational product(s). The sponsor should not supply an investigator/institution with the investigational product(s) until the sponsor obtains all required documentation [e.g., approval/favorable opinion from IRB (Institutional Review Board)/IEC (Independent Ethics Committee) and regulatory authority(ies)]. The sponsor should ensure that written procedures include instructions that the investigator/institution should follow for the handling and storage of investigational product(s) for the trial and documentation thereof. The procedures should address adequate and safe receipt, handling, storage, dispensing, retrieval of unused product from subjects, and return of unused investigational product(s) to the sponsor [or alternative disposition if authorized by the sponsor and in compliance with the applicable regulatory requirement(s)]. The sponsor should: • Ensure timely delivery of investigational

product(s) to the investigator(s). • Maintain records that document ship-

ment, receipt, disposition, return, and destruction of the investigational product(s). • Maintain a system for retrieving investigational products and documenting this retrieval (e.g., for deficient product recall, reclaim after trial completion, expired product reclaim). • Maintain a system for the disposition of unused investigational product(s) and for the documentation of this disposition.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

EASTERN COOPERATIVE ONCOLOGY GROUP (ECOG)

review of tumor measurement data to determine response and progression. This article also gives an early version of the well-known five-point ECOG performance status scale for classifying overall patient status. It is notable that these early studies did not include predefined criteria for the amount of increase and decrease in tumor measurements needed to establish response or progression but instead relied on a vote of the investigators based on blinded review of the data. Standardized criteria were later developed, and the ECOG criteria used for many years are given in Reference 2. During the 1970s and 1980s the ECOG Statistical Center was led by Dr. Marvin Zelen, first at the State University of New York at Buffalo and later at the Dana-Farber Cancer Institute. During this period, the Statistical Center included many leading biostatisticians who made numerous important contributions to the statistical methodology for clinical trials, including basic work on methods for analyzing time-to-event data, for sample size and power calculations, on randomization algorithms, and on methods for monitoring studies.

ROBERT J. GRAY Dana-Farber Cancer Institute Boston, Massachusetts

The Eastern Cooperative Oncology Group (ECOG) is one of ten NCI-funded Cooperative Groups. ECOG has been continuously in operation since the Cooperative Group program was established in 1955. During the period 2004–2007, an average of 75 ECOGled protocols were open to accrual each year, of which 65 were therapeutic (17 Phase III), the others were variously prevention, symptom management, health practices, and laboratory studies. At any time, 50–60 studies are also closed to accrual but still in active follow-up before the primary analysis. Accrual to the ECOG-led therapeutic trials varied during this period from 2393 in 2005 to 5092 in 2007, and ECOG members also enrolled an average of 1580 therapeutic cases per year on studies coordinated by other groups. ECOG manages this large portfolio of studies with quite-limited grant funding. The major operational challenge for ECOG and the other cooperative groups has been conducting high-quality clinical trials as efficiently as possible. In this article, the history, structure, policies and procedures, and major accomplishments of ECOG are briefly described. 1

2

ORGANIZATION AND STRUCTURE

ECOG has a hierarchical membership structure with main institutions (usually academic centers) and affiliate institutions (usually community based). Also, separate community networks are funded through the Community Clinical Oncology Program (CCOP) of the NCI’s Division of Cancer Prevention. The principal investigators of the main institutions and CCOPs form the governing body of the Group. They elect the Group chair and approve the Group’s policies and procedures. The scientific program of ECOG is organized into disease-site committees (Breast, Gastrointestinal, Head and Neck, Lymphoma, Thoracic, Melanoma, Genitourinary, Leukemia, Myeloma, and a working group in Brain Tumors) and modality committees (such as surgery, radiation therapy, developmental therapeutics, and laboratory science).

HISTORY

ECOG was founded in 1955 (the original name was the Eastern Cooperative Group in Solid Tumor Chemotherapy) under the leadership of Dr. C. G. Zubrod. At the time, development of standardized methodology for conducting multicenter clinical trials in oncology and for measuring effects of chemotherapy was needed. Methodology developed for early ECOG trials is described in Reference 1, including use of sealed envelopes for treatment assignment; collection of detailed data on ‘‘flow sheets,’’ tumor measurement forms, and summary forms; and blinded central

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

EASTERN COOPERATIVE ONCOLOGY GROUP (ECOG)

Also, a separately funded program in prevention and cancer control research has been developed. The Group chair appoints the committee chairs, who then organize the scientific research program within each area and bring proposals for studies forward to the Group. An executive review committee reviews all study concepts and decides whether they are suitable for development in ECOG. All concepts and protocols must also be reviewed and approved by NCI through either the Cancer Therapy Evaluation Program or the Division of Cancer Prevention. A Group-wide meeting of the membership is held twice each year, at which each of the committees meets to discuss concepts for new studies and to review the progress of ongoing studies. ECOG has a Group Chair’s Office (in Philadelphia, PA), a combined Operations Office and Data Management Center (at Frontier Science and Technology Research Foundation in Brookline, MA) and a Statistical Office (at the Dana-Farber Cancer Institute). The head of the Statistical Center (the Group statistician) is appointed by the Group chair with the approval of the principal investigators. Approximately nine full-time-equivalent statisticians provide all statistical support for ECOG activities. These statisticians are assigned to work with the disease and modality committees. They collaborate actively with the investigators during the concept and protocol development process, and are responsible for developing the statistical design of the study and for reviewing all drafts of protocols. The statisticians also prepare the registration/randomization materials and collaborate with the data management staff on the development of the data collection instruments. After activation, statisticians monitor the progress of studies and adverse events and perform interim efficacy analyses when appropriate under the design. Reports are generated on all ongoing studies twice each year. Final reports are written by the statistician for each completed ECOG trial. Statisticians also work extensively on lab correlative studies conducted on the clinical trials and on investigator-initiated grants to support various aspects of ECOG activities (especially to provide support for lab studies).

3 PROCEDURES The Cancer Cooperative Groups have developed highly efficient systems for conducting clinical trials, but these systems also involve some compromises relative to standards in the pharmaceutical industry. A key difference is the ongoing relationship between ECOG and its member institutions; ECOG runs many studies simultaneously in the same network of centers, and ECOG studies are generally open to all member institutions. 3.1 Data Quality Control The focus in institutional monitoring in ECOG is on ensuring that the institution is following appropriate procedures and meeting acceptable data quality standards rather than on direct verification of all individual data items. Instead of relying on extensive (and expensive) on-site monitoring for each study to ensure complete accuracy of data, ECOG uses an audit system to verify that participating centers are performing well overall. All centers are audited at least once every 3 years, and an audit generally involves full review of at least 12% of the cases enrolled at a site as well as review of regulatory compliance and pharmacy procedures. Centers with deficiencies are required to submit written corrective action plans and may be reaudited more frequently or, in extreme cases, may be suspended. With only a limited proportion of cases audited at the institutions, ECOG also relies heavily on central review of data for internal consistency. Submitted data are checked for completeness and for invalid values, and extensive checks are made among submitted data items to ensure consistency of all data. Copies of limited source documentation, such as surgical pathology reports, are also collected and reviewed. For assessing response and progression, actual tumor measurements and lab results are required to provide documentation that protocol criteria have been met. A final review of the data coding is performed by the study chair. For the most part, ECOG study chairs are volunteers who are not paid for their time spent on ECOG studies. Available funding does not permit using paid independent clinical reviewers. Generally a blinded central radiology review is not

EASTERN COOPERATIVE ONCOLOGY GROUP (ECOG)

performed because of the time and expense required for collection and review of scans and other imaging studies. The extensive central review of reported measurements, together with the limited on-site auditing, has demonstrated acceptable results without central radiology review. 3.2 Adverse-Event Monitoring Timely monitoring of adverse events is a crucial part of any treatment trial. In ECOG, real-time monitoring occurs primarily through the expedited reporting of severe adverse events. Expedited reporting of certain classes of severe events is required by the NCI and FDA. Expedited reports are now submitted through an online system developed and maintained by NCI. The ECOG data management staff (and study chairs and statisticians) then can access the reports through the same system. A cumulative listing of expedited adverse-event reports is sent to the study chair, the study statistician, and the disease site toxicity monitor monthly if significant new events have been reported. They are each required to review the list and to determine whether any problems might require more review or other action. Adverse events are also routinely submitted on case report forms. Consistency between expedited reports and routine submission is checked as part of the data review process. Statisticians also prepare summary reports on the routine adverse-event data twice each year. These reports are made available to the study chairs, to the group membership, and, for Phase III studies, to the Data Monitoring Committee (DMC). For some studies (especially double-blind studies), more extensive information is also provided to the DMC. 3.3 Blinding of Treatment Until recent years, double-blind studies in oncology were rare. This rarity was partly because of the high level and sometimes characteristic nature of the toxicities and other side effects of many chemotherapy drugs. Also, many Phase III studies have had survival as their primary endpoint, and this endpoint is less subject to potential bias than other endpoints. With the advent of less-toxic

3

targeted therapies and increasing emphasis on endpoints such as progression-free survival, double-blind placebo-controlled studies have become more common. All registrations in ECOG now occur through a web-based registration system. On blinded studies, treatment assignments are prematched to drug identification numbers. At the time of randomization, the treatment assignment is determined using the randomization algorithm for the study, and then the assignment is matched to the drug identification. The treating center and the drug distribution center are only notified of the drug identification number. The links between the treatment codes and the drug identification numbers are kept in a database table that can only be accessed by senior database administrators and a few senior staff who need access for emergency unblinding and other reasons. All data review is performed blinded to treatment assignment. Statisticians obtain the treatment codes from the senior DBA when interim analyses are performed. Only pooled information on adverse events is made available to the group membership. Analyses by treatment arm are only presented to the DMC, until the DMC decides to release the information. Requests from institutions for unblinding treatment assignments are reviewed centrally by a committee of senior staff. Unblinding requires approval of a senior statistician and a senior clinician (usually the group statistician and the group executive officer, except in emergencies, if they are not available). Some studies permit routine unblinding of the treating physician at progression, if the information is generally needed for determining the next treatment (especially if the study involves crossover at progression). 3.4 Standardizing Terminology, Data Elements, and Criteria Standardization of data elements and terminology is important for efficient operations. As early as the mid-1980s, ECOG reached agreement with the Southwest Oncology Group (another NCI-funded Cooperative Group) on partial standardization of the data collected on breast cancer studies. Beginning in 1998, efforts at data standardization among

4

EASTERN COOPERATIVE ONCOLOGY GROUP (ECOG)

the Cooperative Groups were integrated into the NCI’s Common Data Elements (CDE) project. Efficient conduct of large studies requires that no unnecessary data items be collected; however, given the variety of diseases, treatments, modalities, endpoints, and ancillary scientific objectives, considerable variation in the data items is needed to address the objectives of a study. Consequently, the focus of the CDE project has been on improving efficiency through standardization of definitions and terminology. The CDE project has now become part of the NCI’s caBIG initiative, in the Vocabularies and Common Data Elements workspace (see https://cabig.nci.nih.gov). Standardization of evaluation criteria is another important issue for facilitating participation in studies led by different groups, for interpreting results across studies, and for combining data from multiple studies (e.g., in meta-analyses). Cooperative Groups have had a long history of standardizing criteria within their own studies (see, e.g., Reference 2), but they have not always considered standardization across groups. Recently, the NCI has taken a major role, coleading the development of the RECIST (3) solid tumor response criteria and developing the Common Terminology Criteria for Adverse Events (http://ctep.cancer.gov/reporting/ctc.html). Standardization of endpoints across studies and use of common terminology and definitions for different endpoints is another important issue for reporting of results. An important step in this direction for adjuvant breast cancer studies was recently taken by the Cooperative Group breast cancer committees (4). Although recognizing that different endpoints might be needed in different types of studies, they have given a common set of definitions and terminology for various possible endpoints. Whereas some issues are specific to breast cancer, such as the role of new ductal carcinoma in situ cancers in defining the primary endpoint for an adjuvant breast cancer study, some terminology could be applied much more broadly in studies of adjuvant therapy in other diseases. 3.5 Data Monitoring Committee ECOG typically has 20–30 Phase III studies being monitored at any time. Resource

limitations prevent setting up separate independent data monitoring committees for each study. Instead, ECOG has a single data monitoring committee (DMC) that monitors the entire portfolio of Phase III studies. This committee has nine voting members who are selected to have expertise in diverse areas of oncology and hematology. A majority of the voting members must have no affiliation with ECOG, and they must include an outside statistician and a consumer representative. The ECOG DMC meets regularly twice each year. Interim analyses are scheduled to occur in conjunction with the DMC meetings, using database cutoffs 8 weeks before the meeting date. Based on experience with Cooperative Group DMCs, it has been recommended that interim analyses of efficacy on Cooperative Group studies should be scheduled for every DMC meeting until the study reaches full information (5). Recent ECOG Phase III studies have often followed this recommendation, but it also requires considerable extra effort to have study databases cleaned for interim analyses twice each year. Another aspect of ECOG DMC operations is that the study statistician, rather than an independent outside statistician, performs the interim analysis. This aspect is contrary to the recommendations of the U.S. FDA (‘‘Establishment and Operation of Clinical Trial Data Monitoring Committees,’’ FDA Guidance Document, 2006, http://www.fda. gov/cber/gdlns/clintrialdmc.pdf). With the resource limitations of the Cooperative Groups, it is generally not feasible to have a second statistician funded to be sufficiently familiar with the study to perform meaningful analyses. The multiple levels of review required for protocol design changes within the disease committees, the Group’s executive committee, and the NCI, make manipulation of the design of the study on the basis of knowledge of interim results highly unlikely. However, the risk of errors in analysis and/or interpretation that could occur if the study statistician is not involved in the monitoring, because of lack of detailed understanding the protocol or of the study data collection and processing procedures, seems like the greater risk. This approach does require that the study statistician be very careful not to give any kind of hint of interim study results

EASTERN COOPERATIVE ONCOLOGY GROUP (ECOG)

to the study investigators, but in ECOG’s experience, this has not been a problem. 4

MAJOR ACCOMPLISHMENTS

ECOG has made many contributions advancing the care of cancer patients. A few key recent results are establishing the unfavorable risk–benefit profile of autologous bonemarrow transplant therapy in breast cancer (6,7); establishing the benefit of the antiCD20 monoclonal antibody rituximab in initial treatment of diffuse, aggressive nonHodgkin’s lymphoma (8); establishing the benefit of adding the anti-VEGF monoclonal antibody bevacizumab to chemotherapy in advanced lung cancer (9) and breast cancer (10) and in combination with oxaliplatinbased chemotherapy in advanced colorectal cancer (11); and establishing a new standard for initial treatment in multiple myeloma (12). Major studies currently accruing include Phase III studies of sorafenib in metastatic melanoma (E2603), of sorafenib or sunitinib for adjuvant renal cancer (E2805), of bevacizumab in combination with chemotherapy for adjuvant treatment of non-small cell lung cancer (E1505), of bevacizumab in combination with chemotherapy for adjuvant treatment of breast cancer (E5103), and a study of using the Oncotype DX genomic assay (13) to select patients for adjuvant chemotherapy in breast cancer (TAILORx). The NHL study E4494 of rituximab therapy (8) is an interesting case study of the complexity that can occur in monitoring and analyzing studies in oncology. While this study was underway, the GELA group in Europe released results of a similar study (14) with significantly better PFS and overall survival with rituximab. At that time, the ECOG DMC reviewed interim results from E4494 and the results from the GELA trial and decided that E4494 should continue, but the ECOG DMC also decided that they needed to review updated interim results every 6 months to reevaluate whether this recommendation continued to be appropriate, although only two interim analyses had been specified in the design. E4494 also involved an induction randomization between standard CHOP chemotherapy and

5

CHOP+rituximab and a maintenance therapy randomization of induction responders to rituximab versus observation, whereas the GELA trial only involved the induction randomization. The potential confounding of the effect of the maintenance treatment on the induction comparison was a significant problem for interpreting E4494. This problem led to the use of novel weighted analysis methods (see, e.g., Reference 15) for estimating the effect of induction rituximab in the absence of maintenance rituximab, although such analyses had not been specified prospectively. These analyses ultimately became part of the submission to the FDA for approval of rituximab for this indication. This study illustrates that when Phase III studies take an extended period to complete, plans developed at initiation can require significant modification. In such circumstances, it is important to have the flexibility to modify the original plans and to do so in accordance with sound statistical principles. The Cooperative Group Program is the premiere public system in the United States for conducting randomized comparative studies of cancer therapy. In this article, we have provided an overview of ECOG’s contributions to this program and its methods for achieving these results. REFERENCES 1. C. G. Zubrod, M. Schneiderman, E. Frei, C. Brindley, G. L. Gold, B. Shnider, R. Oviedo, J. Gorman J, R. Jones, Jr., U. Jonsson, J. Colsky, T. Chalmers, B. Ferguson, M. Dederick, J. Holland, O. Selawry, W. Regelson, L. Lasagna, A. H. Owens, Jr. Appraisal of methods for the study of chemotherapy of cancer in man: comparative therapeutic trial of nitrogen mustard and triethylene thiophosphoramide, J. Chron. Dis. 1960; 11:7–33 2. M. M. Oken, R. H. Creech, D. C. Tormey, J. Horton, T. E. Davis, E. T. McFadden, P. P. Carbone, Toxicity and response criteria of the Eastern Cooperative Oncology Group, Am. J. Clin. Oncol. 1982; 5:649–655. 3. P. Therasse, S. G. Arbuck, E. A. Eisenhauer, J. Wanders, R. S. Kaplan, L. Rubinstein, J. Verweij, M. V. Glabbeke, A. T. van Oosterom, M. C. Christian, and S. G. Gwyther. New guidelines to evaluate the response to treatment in solid tumors, J. Natl. Cancer Inst. 2000; 92:205–216.

6

EASTERN COOPERATIVE ONCOLOGY GROUP (ECOG)

4. C. A. Hudis, W. E. Barlow, J. P. Costantino, R. J. Gray, K. L. Pritchard, J. A. W. Chapman, J. A. Sparano, S. Hunsberger, R. A. Enos, R. D. Gelber, and J. Zujewski, Proposal for standardized definitions for efficacy end points in adjuvant breast cancer trials: the STEEP system, J. Clin. Oncol. 2007; 25:2127–2132. 5. B. Freidlin, E. L. Korn, and S. L. George, Data monitoring committees and interim monitoring guidelines, Controlled Clinical Trials. 1999; 20:395–40. 6. E. A. Stadtmauer, A. O’Neill, L. J. Goldstein, P. A. Crilley, K. F. Mangan, J. N. Ingle, I. Brodsky, S. Martino, H. M. Lazarus, J. K. Erban, C. Sickles, and J. H. Glick, Conventional-dose chemotherapy compared with high-dose chemotherapy plus autologous hematopoietic stem cell transplantation for metastatic breast cancer, N. Engl. J. Med. 2000; 342(15):1069–1076. 7. M. S. Tallman, R. J. Gray, N. J. Robert, C. F. LeMaistre, C. Osborne, W. P. Vaughan, W. J. Gradishar, T. M. Pisansky, J. H. Fetting, E. M. Paietta, H. M. Lazarus, Conventional adjuvant chemotherapy with or without high-dose chemotherapy and autologous stem cell transplantation in high-risk breast cancer, N. Engl. J. Med. 2003; 349(1):17–26. 8. T. M. Habermann, E. Weller, V. A. Morrison, R. Gascoyne, P. A. Cassileth, J. B. Cohn, S. R. Dakhil, B. Woda, R. I. Fisher, B. A. Peterson, and S. J. Horning, RituximabCHOP versus CHOP alone or with maintenance rituximab in older patients with diffuse large B-cell lymphoma, J. Clin. Oncol. 2006; 24(19)3121–3127. 9. A. B. Sandler, R. J. Gray, M. C. Perry, J. R. Brahmer, J H. Schiller, A. Dowlati, R. Lilenbaum, and D. H. Johnson, Paclitaxel plus Carboplatin with or without Bevacizumab in advanced non-squamous nonsmall cell lung cancer: a randomized study of the Eastern Coopertive Oncology Group, N. Engl. J. Med. 2006; 355(24):542–2550. 10. K. D. Miller, M. Wang, J. Gralow, M. Dickler, M. A. Cobleigh, E. A. Perez, T. Shenkier, D. F. Cella, and N. E. Davidson, Paclitaxel plus bevacizumab versus paclitaxel alone for metastatic breast cancer, N. Engl. J. Med. 2007; 357:2666–2676. 11. B. J. Giantonio, P. J. Catalano, N. J. Meropol, P. J. O’Dwyer, E. P. Mitchell, S. R. Alberts, M. A. Schwartz, and A. B. Benson III, Bevacizumab in combination with Oxaliplatin, Fluorouracil, and Leucovorin (FOLFOX4) for previously treated metastatic colorectal cancer: results from the Eastern Cooperative

Oncology Group Study E3200, J. Clin. Oncol. 2007; 25:1539–1544. 12. S. V. Rajkumar, S. J. Jacobus, N. S. Callander, R. Fonseca, D. H. Vesole, M. E. Williams, R. Abonour, D. S. Siegel, and P. R. Greipp, Phase III trial of lenalidomide plus highdose dexamethasone versus lenalidomide plus low-dose dexamethasone in newly diagnosed multiple myeloma (E4A03): a trial coordinated by the Eastern Cooperative Oncology Group [abstract]. Proc. ASCO 2007. Abstract LBA8025. 13. S. Paik, S. Shak, G. Tang, C. Kim, J. Baker, M. Cronin, F. L. Baehner, M. G. Walker, D. Watson, F. Park, W. Hiller, E. R. Fisher, D. L. Wickerham, J. Bryant, and N. Wolmark, A multigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer, N. Engl. J. Med. 2004; 351:2817–2826. 14. B. Coiffier, E. Lepage, J. Briere, R. Herbrecht, H. Tilly, R. Bouabdallah,. P. Morel, E. van den Neste, G. Salles, P. Gaulard, F. Reyes, and C. Gisselbrecht,. CHOP chemotherapy plus rituximab compared with CHOP alone in elderly patients with diffuse large-B-cell lymphoma, N. Engl. J. Med. 2002; 346:235–242. 15. J. K. Lunceford, M. Davidian, and A. A. Tsiatis, Estimation of survival distributions of treatment policies in two-stage randomization designs in clinical trials, Biometrics. 2002; 58:48–57.

CROSS-REFERENCES Adverse Event Clinical Data Management Data Monitoring Committee National Cancer Institute Southwest Oncology Group

ELIGIBILITY AND EXCLUSION CRITERIA

the eligibility criteria should be chosen to allow valid inferences to that population. For example, the Hypertension Prevention Trial (HPT) was aimed at normotensive individuals 25–49 years old with diastolic blood pressure between 78 mm Hg and 90 mm Hg, and these were the main eligibility criteria (3). Choosing the narrow eligibility criteria often appropriate for an explanatory trial can make it difficult to apply the results to a broader population (11). Yusuf (23), moreover, argues that a truly homogeneous cohort cannot be constituted because even apparently similar individuals can have very different outcomes. The consensus is that most Phase III randomized trials should be regarded as pragmatic.

MARY A. FOULKES Food & Drug Administration, Rockville, MD, USA

The choice of eligibility criteria in a clinical trial can increase or decrease the magnitude of between-patient variation, which will in turn decrease or increase the statistical power of the trial for a given sample size. Theoretically, the more homogeneous the trial population, the greater is the power of the trial, but the more limited is the ability to generalize the results to a broad population. Thus, the choice of eligibility criteria can profoundly influence both the results and the interpretation of the trial. Besides controlling variation, the Institute of Medicine (IOM) Committee on the Ethical and Legal Issues Relating to the Inclusion of Women in Clinical Studies (16) discusses four other issues related to the choice of trial population; namely, disease stage, clinical contraindications, regulatory or ethical restrictions, and compliance considerations. We will discuss these and the related issues of explanatory vs. pragmatic trials, screening and recruitment processes, and the impact of eligibility criteria on the generalizability of trial results. Other factors influencing the selection of patients, such as factors in the selection of institutions in multicenter studies and physician preferences are discussed elsewhere (2,22). 1

1.1 The Uncertainty Principle Byar et al. (4) describe the simplest possible form of eligibility criteria for a trial, in which patients are eligible provided the treating physician and the patient have ‘‘substantial uncertainty’’ as to which of the treatment options is better. This definition, known as the uncertainty principle, incorporates all factors that contraindicate one or more of the treatment options including stage of disease, co-existing disease, and patients’ preferences. However, it also largely devolves definition of eligibility to the individual physicians participating in the trial. The consequent lack of control and strict definition of the cohort of patients entering the trial has been unattractive to some investigators. 2 CONTROL OF VARIATION VS. EASE OF RECRUITMENT

EXPLANATORY VS. PRAGMATIC TRIALS

The objectives of a trial affect the appropriate eligibility criteria (20). If the trial is designed to estimate the biological effect of a treatment (explanatory trial), then the eligibility criteria should be chosen to minimize the impact of extraneous variation, as in early investigations of protease inhibitors against human immunodeficiency virus (HIV) infection (18). If, however, the trial is designed to estimate the effectiveness of a treatment in a target population (pragmatic trial), then

The debate over the uncertainty principle highlights the tension between two different ways of improving the precision of the estimated effect of treatment in a randomized trial. By using very strict eligibility criteria we seek to reduce between-patient variation in clinical outcomes, leading to improved precision of the treatment difference estimate. By using very flexible eligibility criteria (as with the uncertainty principle), we seek to allow a wider entry to the trial, thereby

1

2

ELIGIBILITY AND EXCLUSION CRITERIA

increasing the number of eligible patients and usually the precision of the treatment difference estimate. The question is, therefore, do we try to control variation and accept the smaller size of the sample, or do we try to increase the sample size and accept a wider between-patient variation? While this debate continues, the general consensus among clinical trial statisticians is that it is generally difficult to control between-patient variation successfully because often we do not know the important determinants of prognosis. Therefore, attempts to use very strict eligibility criteria are less successful than attempts to gain precision by entering very large numbers of patients into trials (23). However, if there are categories of patients who are considered very unlikely to benefit from the treatment, it is clearly conceivable to exclude them from the trial (see later sections on Stage of Disease and Clinical Contraindications). Begg (2) criticizes the common practice of introducing a long list of eligibility criteria in clinical trials, particularly in the treatment of cancer. Such an approach greatly increases the difficulty of recruiting patients in large numbers. In examining such lists it is often found that many of the criteria are of questionable importance and do not relate directly to the safety of the patient or to the lack of benefit to be derived from the treatment. 2.1 Issues in the Screening and Recruitment Process Establishing eligibility often involves a screening process. Examples include choosing individuals for a heart disease trial with ejection fraction between 0.35 and 0.8 and a specific number of ectopic beats, or choosing HIV-infected individuals with slowly rather than rapidly progressing disease (15). Some eligibility criteria may be implicit in this process. For example, the recruitment method may require the patients to be accessible by telephone contact or to be able to read and write in English, such as trials in which the initial contact is via a prepaid postal response card. Multiple ‘‘baseline’’ visits that are sometimes used in the screening process can provide multiple opportunities for exclusion, e.g. the Coronary Primary Prevention

Trial used five baseline visits and the HPT used three baseline visits. Thus, those ultimately enrolled may affect the recruitment and screening mechanisms and resources for multiple participant contacts as much as the protocol-specific eligibility criteria. The impact of eligibility criteria and of recruitment procedures on the overall cost of the trial has rarely been investigated. Borhani (3) indicated that the ordering of the application of eligibility criteria can substantially affect costs. These costs are also sensitive to the cutoffs applied to continuous responses, e.g. diastolic blood pressure, high density lipoprotein cholesterol, coronary ejection fraction, and T-cell lymphocyte counts. As mentioned earlier, eligibility criteria can have a strong impact on the ease of recruitment. For example, the need to enroll newly diagnosed or previously untreated patients can severely restrict the ability to recruit. The need to enroll rapidly after a stroke, myocardial infarction, head trauma, or exposure to infectious agent can lead to difficulties. If the condition renders the patient unconscious for some period of time, or the patient lives far from the treatment center, or is unaware that an infection, stroke, infarction, or other event has occurred, it is less likely that they will be available for enrollment. Similarly, Carew (5) suggests that recruitment be enhanced by broad eligibility criteria, allowing potentially more sites and more individuals to participate. 2.2 Stage of Disease Often the stage of disease strongly affects the outcome of treatment, and is a primary source of variation. Eligibility is often restricted to the stages of disease most appropriately managed by the treatment. For many diseases, classification or staging systems have been developed to aid clinical management. Eligibility criteria involving stage of disease are best defined using an established classification system that is in wide use. Examples of such classification systems include the coronary functional class (7), Dukes’ colon cancer staging system, and the World Health Organization staging system for HIV infection (26).

ELIGIBILITY AND EXCLUSION CRITERIA

3

2.3 Clinical Contraindications

2.6 Implementing the Eligibility Criteria

Exclusions arising because one of the treatments is clearly contraindicated are common (14). For example, 18% of those screened for the Beta Blocker Heart Attack Trial were excluded due to contraindications to the administration of propranolol (12). Since these prior conditions would preclude use of some of the treatments in a trial, the trial results could not apply to individuals with those conditions. Some argue that contraindications should be clearly delineated in the protocol to avoid investigator or regional differences in their use.

The characterization of the target population and baseline homogeneity can be subverted by deviations during the conduct of the trial from the protocol specified eligibility criteria. If extensive, these can adversely affect the assumptions underlying analyses and the interpretation of the results. Thus, monitoring the determination of eligibility criteria during the conduct of the trial is an important component of the implementation of the trial. Often, the office that conducts the randomized treatment assignment checks the eligibility criteria before enrolling the patient. Finkelstein & Green (9) discuss the exclusion from analysis of individuals found to be ineligible after enrollment in the trial.

2.4 Compliance Considerations A run-in (or qualification) period is sometimes built into the trial design so as to identify potential noncompliers and exclude them from enrollment. This reduces the dilution of treatment differences that noncompliance introduces. In some studies, this period can also be used to eliminate placebo responders. In these cases, the determination of noncompliance becomes one of the outcome measures of the trial. 2.5 Regulatory or Ethical Considerations Various demographically or otherwise defined populations have been excluded from clinical trials in the past. For example, in trials of heart disease prevention, women have been excluded as their incidence of heart disease is lower than in men and their inclusion would have required a larger sample size. Similarly, minority groups have sometimes had little or no representation because no special efforts had been made to include them. Recent changes in US regulations have required special justification for the exclusion of women, minorities, or the elderly from National Institutes of Health sponsored trials. The scientific argument for including these groups is that it provides a more solid basis for extrapolating the results of the trial to the general population (6,8,10,13,17,19,24,25). There will usually be inadequate statistical power for detecting different effects in subpopulations, but sometimes meta-analysis of several studies may be able to detect such differences.

2.7 Generalization of Results to Broader Populations Treatment trials (or prevention trials) are usually conducted on samples of convenience, enrolling participants who present at specific hospitals or clinical sites. Therefore, the population to whom the trial results apply is generally not well defined. External validity—the ability to generalize from the trial to some broader population—is the ultimate goal of any trial. Adequately randomized trials can be assumed to produce valid results for the specific group of individuals enrolled, i.e. internal validity; the difficulties arise in extending the inference beyond that limited cohort. Since complete enumeration of the target population is rarely possible, inferences from studies are based on substantive judgment. A strong argument that is often used is that treatment differences in outcome are generally less variable among different patient populations than the outcomes themselves (23). Following publication, critics questioned the generalizability of the results of the International Cooperative Trial of Extracranial– Intracranial (EC/IC) Arterial Anastomosis to evaluate the effect of the EC/IC procedure on the risk of ischemic stroke. The results showed a lack of benefit that surprised many in the surgical profession. It became clear that many of the eligible patients at the participating clinical sites did

4

ELIGIBILITY AND EXCLUSION CRITERIA

not enter the trial, while those enrolled in the trial were considered to have poorer risk and some argued that they were less likely to benefit from surgery (1,21). The ensuing controversy slowed acceptance of the trial results by the surgical community, although eventually they had a profound effect on the frequency with which EC/IC was performed.

3

CONCLUSIONS

The goals and objectives of the trial, the intended target population, and the anticipated inferences from the trial results should all be carefully specified from the outset. If that is done, then the appropriate choice of eligibility criteria usually becomes clearer. Experience has shown that simplifying eligibility criteria generally enhances recruitment, allows a wider participation, and gives greater justification for generalizing the results to a broader population.

REFERENCES 1. Barnett, H. J. M., Sackett, D., Taylor, D. W., Haynes, B., Peerless, S. J., Meissner, I., Hachinski, V. & Fox, A. (1987). Are the results of the extracranial–intracranial bypass trial generalizable?, New England Journal of Medicine 316, 820–824. 2. Begg, C. B. (1988). Selection of patients for clinical trials, Seminars in Oncology 15, 434–440. 3. Borhani, N. O., Tonascia, J., Schlundt, D. G., Prineas, R. J. & Jefferys, J. L. (1989). Recruitment in the Hypertension Prevention Trial, Controlled Clinical Trials 10, 30S–39S. 4. Byar, D. P., Schoenfeld, D. A. & Green, S. B. (1990). Design considerations for AIDS trials, New England Journal of Medicine 323, 1343–1348. 5. Carew, B. D., Ahn, S. A., Boichot, H. D., Diesenfeldt, B. J., Dolan, N. A., Edens, T. R., Weiner, D. H. & Probstfield, J. L. (1992). Recruitment strategies in the Studies of Left Ventricular Dysfunction (SOLVD), Controlled Clinical Trials 13, 325–338. 6. Cotton P. (1990). Is there still too much extrapolation from data on middle-aged white men?, Journal of the American Medical Association 263, 1049–1050.

7. Criteria Committee of NYHA (1964). Diseases of the Heart and Blood Vessels: Nomenclature and Criteria for Diagnosis, 6th Ed. Little, Brown & Company, Boston. 8. El-Sadr, W. & Capps, L. (1992). Special communication: the challenge of minority recruitment in clinical trials for AIDS, Journal of the American Medical Association 267, 954–957. 9. Finkelstein, D. M. & Green, S. B. (1995). Issues in analysis of AIDS clinical trials, in AIDS Clinical Trials, D. M. Finkelstein & D. A. Schonfeld, eds. Wiley–Liss, New York, pp. 243–256. 10. Freedman L. S., Simon, R., Foulkes, M. A., Friedman, L., Geller, N. L., Gordon, D. J. & Mowery, R. (1995). Inclusion of women and minorities in clinical trials and the NIH Revitalization Act of 1993—the perspective of NIH clinical trialists, Controlled Clinical Trials 16, 277–285. 11. Gail, M. H. (1985). Eligibility exclusions, losses to follow-up, removal of randomized patients, and uncounted events in cancer clinical trials, Cancer Treatment Reports 69, 1107–1112. 12. Goldstein, S., Byington, R. & the BHAT Research Group (1987). The Beta Blocker Heart Attack Trial: recruitment experience, Controlled Clinical Trials 8, 79 S–85 S. 13. Gurwitz, J. H., Col, N. F. & Avorn, J. (1992). The exclusion of the elderly and women from clinical trials in acute myocardial infarction, Journal of the American Medical Association 268, 1417–1422. 14. Harrison K., Veahov, D., Jones, K., Charron, K. & Clements, M. L. (1995). Medical eligibility, comprehension of the consent process, and retention of injection drug users recruited for an HIV vaccine trial, Journal of Acquired Immune Deficiency Syndrome 10, 386–390. 15. Haynes, B. F., Panteleo, G. & Fauci, A. S. (1996). Toward an understanding of the correlates of protective immunity to HIV infection, Science 271, 324–328. 16. IOM Committee on the Ethical and Legal Issues Relating to the Inclusion of Women in Clinical Studies, (1996). Women and Health Research: Ethical and Legal Issues of Including Women in Clinical Studies, A. C. Mastroianni, R. Faden & D. Federman, eds. National Academy Press, Washington. 17. Lagakos, S., Fischl, M. A., Stein, D. S., Lim, L. & Vollerding, P. (1991). Effects of zidovudine therapy in minority and other subpopulations with early HIV infection, Journal of the American Medical Association 266, 2709–2712.

ELIGIBILITY AND EXCLUSION CRITERIA 18. Markowitz, M., Mo, H., Kempf, D. J., Norbeck, D. W., Bhat, T. N., Erickson, J. W. & Ho, D. D. (1996). Triple therapy with AZT, 3TC, and ritonavir in 12 subjects newly infected with HIV-1, Eleventh International Conference on AIDS, Abstract Th.B. 933. 19. Patterson, W. B. & Emanuel, E. J. (1995). The eligibility of women for clinical research trials, Journal of Clinical Oncology 13, 293–299. 20. Schwartz, D., Flamant, R. & Lellouch, J. (1980). Clinical Trials. Academic Press, New York. 21. Sundt, T. M. (1987). Was the international randomized trial of extracranial–intracranial arterial bypass representative of the population at risk?, New England Journal of Medicine 316, 814–816. 22. Taylor, K. M., Margolese, R. G. & Soskolne, C. L. (1984). Physicians’ reasons for not entering eligible patients in a randomized clinical trial of surgery for breast cancer, New England Journal of Medicine 310, 1363–1367. 23. Yusuf, S., Held, P., Teo, K. K. & Toretsky, E. R. (1990). Selection of patients for randomized controlled trials: implications of wide or narrow eligibility criteria, Statistics in Medicine 9, 73–86. 24. Yusuf, S. & Furberg, C. D. (1991). Are we biased in our approach to treating elderly patients with heart disease?, American Journal of Cardiology 68, 954–956. 25. Wenger, N. K. (1992). Exclusion of the elderly and women from coronary trials: is their quality of care compromised?, Journal of the American Medical Association 268, 1460–1461. 26. World Health Organization (1990). Acquired immune deficiency syndrome (AIDS): interim proposal for a WHO staging system for HIV infection and disease, Weekly Epidemiology Record 65, 221–228.

CROSS-REFERENCES Intention to Treat Analysis

5

EMERGENCY USE INVESTIGATIONAL NEW DRUG (IND) The need for an investigational drug may develop in an emergency situation that does not allow time for submission of an Investigational New Drug (IND) Application in accordance with 21 CFR (Code of Federal Regulations) 312.23 or 21 CFR 312.34. In such a case, the Food and Drug Administration (FDA) may authorize shipment of the drug for a specified use in advance of submission of an IND. A request for such authorization may be transmitted to FDA by telephone or by other rapid communication means. For investigational biological drugs regulated by the Center for Biologics Evaluation and Research (CBER), the request should be directed to the Office of Communication, Training and Manufacturers Assistance (HFM–40), Center for Biologics Evaluation and Research. For all other investigational drugs, the request for authorization should be directed to the Division of Drug Information (HFD–240), Center for Drug Evaluation and Research. After normal working hours, Eastern Standard Time, the request should be directed to the FDA Office of Emergency Operations (HFA–615). Except in extraordinary circumstances, such authorization will be conditioned on the sponsor making an appropriate IND submission as soon as possible after receiving the authorization.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/regulatory/applications/ ind page 1.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

END-OF-PHASE II MEETING

more fully in the Food and Drug Administration (FDA) Staff Manual Guide 4850.7 that is publicly available under the FDA public information regulations in part 20. Arrangements for an End-of-Phase II meeting are to be made with the division in the FDA Center for Drug Evaluation and Research or the Center for Biologics Evaluation and Research that is responsible for review of the IND. The meeting will be scheduled by FDA at a time convenient to both FDA and the sponsor. Both the sponsor and FDA may bring consultants to the meeting. The meeting should be directed primarily at establishing agreement between FDA and the sponsor of the overall plan for Phase III and the objectives and design of particular studies. The adequacy of the technical information to support Phase III studies and/or a marketing application may also be discussed. FDA will also provide its best judgment, at that time, of the pediatric studies that will be required for the drug product and whether their submission will be deferred until after approval. Agreements reached at the meeting on these matters will be recorded in minutes of the conference that will be taken by FDA in accordance with Sec. 10.65 and provided to the sponsor. The minutes along with any other written material provided to the sponsor will serve as a permanent record of any agreements reached. Barring a significant scientific development that requires otherwise, studies conducted in accordance with the agreement shall be presumed to be sufficient in objective and design for the purpose of obtaining marketing approval for the drug.

The purpose of an End-of-Phase II Meeting is to determine the safety of proceeding to Phase III, to evaluate the Phase III plan and protocols and the adequacy of current studies and plans to assess pediatric safety and effectiveness, and to identify any additional information necessary to support a marketing application for the uses under investigation. Although the End-of-Phase II meeting is designed primarily for Investigational New Drugs (IND) that involve new molecular entities or major new uses of marketed drugs, a sponsor of any IND may request and obtain an End-of-Phase II meeting. To be most useful to the sponsor, Endof-Phase II meetings should be held before major commitments of effort and resources to specific Phase III tests are made. The scheduling of an End-of-Phase II meeting is not intended, however, to delay the transition of an investigation from Phase II to Phase III. 1

ADVANCE INFORMATION

At least 1 month in advance of an End-ofPhase II meeting, the sponsor should submit background information on the sponsor’s plan for Phase III, including summaries of the Phase I and II investigations, the specific protocols for Phase III clinical studies, plans for any additional nonclinical studies, plans for pediatric studies (including a time line for protocol finalization, enrollment, completion, and data analysis or information to support any planned request for waiver or deferral of pediatric studies), and, if available, tentative labeling for the drug. The recommended contents of such a submission are described This article was modified from the website of the United States Food and Drug Administration (http://frwebgate.access.gpo.gov/cgi-bin/getcfr.cgi?TITLE=21&PART=312&SECTION=47& YEAR=1999&TYPE=TEXT) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

END-OF-PHASE I MEETING When data from Phase I clinical testing are available, the sponsor again may request a meeting with Food and Drug Administration (FDA) reviewing officials. The primary purpose of this meeting is to review and reach agreement on the design of Phase II controlled clinical trials, with the goal that such testing will be adequate to provide sufficient data on the safety and effectiveness of the drug to support a decision on its approvability for marketing, and to discuss the need for, as well as the design and timing of, studies of the drug in pediatric patients. For drugs for life-threatening diseases, FDA will provide its best judgment, at that time, whether pediatric studies will be required and whether their submission will be deferred until after approval. The procedures outlined in Title 21 312.47(b)(1) with respect to End-of-Phase II conferences, including documentation of agreements reached, would also be used for End-of-Phase I meetings.

This article was modified from the website of the United States Food and Drug Administration (http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/ cfCFR/CFRSearch.cfm?CFRPart=312&showFR=1 &subpartNode=21:5.0.1.1.3.5) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

• Target

ENRICHMENT DESIGN

responder selection: Choosing the subpopulation that is more likely to respond or to experience an event than the general population, such as people who responded to a treatment at the initial stage, or those with a recurrent history of a certain event (10, 11).

VALERII V. FEDOROV Biomedical Data Sciences GlaxoSmithKline Pharmaceuticals Collegeville, Pennsylvania

TAO LIU These selection maneuvers are not mutually exclusive. An enrichment process can often achieve more than one of the above objectives. An effective enrichment can greatly increase the power of detecting the target treatment effect (if only for the selected subpopulation). However, a very strict, multistage enrichment process may lead to a small subpopulation size, and consequently to lower statistical precision or to a prolonged recruitment period. An ideal enrichment design should be based on a careful trade-off between these two aspects. There are many variants of enrichment designs. Figures 1, 2, and 3 show some relatively simple schemes. The randomized discontinuation trial (RDT) (Figure 1) was first proposed by Amery and Dony (4) as an alternative to the classic placebo (or comparator)-controlled, randomized clinical trial (RCT) to reduce the trial’s duration and the degree of the patients’ exposure to inert placebo. In this design, after all of the eligible population have provided informed consent for randomization, they are assigned to an experimental treatment at the first stage. This stage is called the open stage (4); at the end of the open stage, the individuals’ responses (often surrogate endpoints) are collected and evaluated by the study clinician. The individuals who had no response or showed serious adverse effects are excluded from the study. The rest (openstage responders) are then randomized to a placebo or to the experimental treatment (or a comparator) in a double-blind fashion. The first stage serves as a filter for removing those who are unlikely to respond to treatment. The rationale is that the nonresponders would contribute little information about the population for whom the treatment can be useful. The second stage serves

Department of Biostatistics & Epidemiology University of Pennsylvania School of Medicine Philadelphia, Pennsylvania

Enrichment designs for evaluating certain treatments or drugs had been used for decades before Hallstrom and Friedman (1), Temple (2), and Pablos-Mendez et al (3) gave them formal discussion and definition in 1990s. In such designs, a subpopulation is selected or screened out from the general population for an experimental study. The procedure for selection of such a subpopulation is called enrichment (2). The goal of the design is to enhance the signal of an external intervention in the enriched subpopulation and separate it from the interference of many other undesired factors. The discussion and employment of such designs can be traced back to the 1970s (4). Recent years have seen these designs gain great popularity in many different disciplines, particularly in the field of clinical oncology. The enrichment intentions can be roughly classified into the following categories (2). • Variance suppressing selection: Selec-

ting the most homogeneous subpopulation, such as those patients with the greatest tendency to adhere to the study protocol, those whose cholesterol level falls within a certain range, or those with a similar tumor size and health condition (5). • Response

enhanced selection: Identifying the subpopulation with the strongest potential magnitude of response, such as improvement in mental condition, extension of survival, or reduction in tumor growth rate (6–9).

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

ENRICHMENT DESIGN

Eligible population for a study

Experimental Treatment

Non-responders

Treatment responders Randomization

Stop follow-up Experimental Treatment

Placebo Figure 1. The diagram of a randomized discontinuation trial.

Statistical analysis

Eligible population

Placebo run-in: Complied?

Compliers

Non-compliers

Randomization

Stop follow-up

Experimental Treatment

Placebo

Statistical analysis Figure 2. The diagram of a placebo run-in trial.

to distinguish whether the treatment adds anything over the placebo effect. A commonly accepted assumption with an RDT is that ‘‘the treatment will not cure the condition during the open stage’’ (4). For this reason, an RDT is generally applied

under conditions that require sustained use of a therapy (6, 12, 13), such as stabilizing tumor growth or treating some chronic disease. Another often accepted assumption with this design is that the treatment effect at

ENRICHMENT DESIGN

3

Eligible subjects who agree to be randomized Active treatment (100-2a)% of sample

Responders leave the study

Nonresponders continue DB study and receive placebo

Placebo treatment a % of sample

Responders leave the study

Nonresponders continue DB study and receive active treatment

Placebo treatment a% of sample

Responders leave the study

Nonresponders continue DB study and receive placebo

Figure 3. Study design for a major depressive disorder trial. DB, double blind. (From Fava et al [11].)

the open stage will not carry over to the second stage. In oncology, this might mean that the tumor growth rate is uniquely defined by the treatment and is changed as the treatment is changed (6). Traditionally, the statistical analysis of an RDT uses only the outcomes from the second stage, treating it as an RCT rendered on the enriched subpopulation. Capra (14) compared the power of an RDT with that of an RCT when the primary endpoints are individuals’ survival times. Kopec et al (7) evaluated the utility and efficiency of the RDT when the endpoints are binary; they compared the relative sample size required for the desired power of the RDT versus the RCT under various scenarios and parameter settings. Fedorov and Liu (10) considered maximum likelihood estimation of the treatment effect for binary endpoints. With some moderate assumptions, they incorporated the information from the open stage into their inference. The placebo run-in trial (PRIT) is another often used enrichment design (15, 16). It is very similar to the RDT in its setup structure, except that the block ‘‘experimental treatment’’ is replaced by ‘‘placebo treatment’’ and ‘‘responders’’ by ‘‘compliers’’ (Figure 2). An assumption with the PRIT design is that the participants behave coherently throughout the trial. If a patient’s adherence to the protocol is poor during the placebo runin, then his adherence will be poor during the second stage, and vice versa. This design can be more efficient than a conventional RCT

when the compliance of the general population is known or expected to be poor (5) or ‘‘when poor adherence is associated with a substantial reduction of therapy’’ (17). Davis et al (18) examined the efficiency of the PRIT design through empirical evaluations, in the setting of evaluating a cholesterol-lowering drug for elderly patients. The analyses were carried out using the outcomes from the second stage only, as if it were a conventional RCT. Both RDT and PRIT designs are fairly simple schemes. In reality, researchers often employed these designs with certain modifications to meet each study’s requirements. Sometimes RDT and PRIT are even used in combination. For example, Fava et al (11) proposed a study design they named the ‘Sequential Parallel Comparison Design’ for their psychiatric disorder study (see Figure 3 for the design diagram). The first stage of the design consists of three double-blinded (DB) arms: two placebo arms and one treatment arm with unequal randomization (usually more patients are on the placebo arms). Only the nonresponders of the first stage continue to the second stage, and they are assigned in a double-blinded way to the active treatment or placebo, depending on the arm they were on at the first stage. The rationale behind the design is that ‘‘since patients on the second stage have already ‘failed placebo’, their placebo response will be reduced.’’ The data analysis of this design is similar to RDT and PRIT. Fava et al (11) have discussed the statistical model for the design and the design optimization.

4

1

ENRICHMENT DESIGN

parameters of interest can be µk∗ j , µk∗ j and πk∗ , where j and j denote two comparative treatments, and the other parameters can be viewed as nuisance. Often the parameter estimation is complemented or replaced by hypotheses testing. The population model (1) should be complemented by models that describe the evolution of the response to treatment, and the observation processes. For instance, the enrichment process of an RDT is often achieved using only surrogate endpoints at the end of the first stage, which are less accurate measures than the primary endpoints, and can lead to misclassification of treatment responders and nonresponders (Figure 1). Fedorov and Liu (10) proposed to model such an imperfect enrichment process through the introduction of false-positive and falsenegative detection rates. This model builds the connection between the outcomes at the first and the second stages, and hence makes it possible to use the observed information

MODELS

In most publications, it is (implicitly and explicitly) assumed that the general population consists of K subpopulations: ϕ(x, θ ) =

K


πk ϕ(x, θkj ),

j = 1, 2, . . . , J, (1)

k=1

false positive = 0.10, false negative = 0.10 0.5

0.25

0.30

where x is the endpoint of interest (which can be a collection of several variables), π k is a fraction of the k-th subpopulation, ϕ(x, θ kj ) is the distribution density of x in the k-th subpopulation under the j-th treatment, θ kj are unknown parameters defining the distribution of x in each subpopulation, ϕ(x, θ ) is the marginal distribution of x, and the vector θ comprises all π k and θ kj . The goal can be the estimation of all components of θ , or typically a subset of θ , such as the fraction of responders πk∗ and the parameters θk∗ j . In popular settings for continuous x, ϕ(x, θ kj ) is a normal density with θ 1kj = µkj = E(x | k) and θ2kj = σkj 2 = Var(x|k). In this case, the

0.15

0.2

0.10

0.20

0.3

0.15

0.0

0.05

0.1

0.0

0.2

0.4

0.6

0.8

Figure 4. The area below each curve corresponds to the rate for which a randomized discontinuation trial (RDT) is superior to a randomized controlled trail (RCT) given the fraction of patients randomized to placebo.

ENRICHMENT DESIGN

from both stages by constructing the complete data likelihood. In other settings (6), the outcome at the first stage is x(t1 ) (e.g., tumor size) at moment t1 , while at the final stage it is x(t1 + t2 ). Thus, a model describing the relationship between x(t1 ) and x(t1 + t2 ) is needed. In oncology, x(t) can be a tumor growth model. With this model in place, the optimal selection of t1 given t1 + t2 can be considered (6, 8). 2

DESIGN AND EFFICIENCY

In terms of design, the choice of randomization scheme, rates of randomization, and selection of the length of the first stage can be diversified in many ways. Let us consider a simple RDT design (Figure 1) with binary outcomes as an example. Figure 4 shows the efficiency comparison between an RDT design and an RCT with two equal arms (10). Suppose that, at the end of the open phase, 10% of the responders to an active treatment are misclassified as nonresponders, and that 10% of the nonresponders are misclassified as responders. The x-axis represents the population response rate to placebo, and the y-axis the increase in the response rate due to the treatment, the estimation of which is the primary interest. Numbers next to each curve indicate the fraction of patients randomized to the placebo arm in the second stage of an RDT (usually the smaller value is more ethical). In the region under each curve, the RDT dominates the RCT in terms of efficiency. This figure demonstrates that an RDT has better performance under some scenarios (particularly when the treatment effect is small). However this efficiency gain is not universal, even in this simple, idealized setting. For more realistic cases, careful selection of models, combined with ethical and logistical considerations, is essential. In some publications (6, 8, 9, 14), investigators use the Monte Carlo simulations to compare different designs. 3

APPLICATIONS

Majority of applications of enrichment designs are in the field of clinical oncology. Typical examples include a study with

5

enrichment screening based on preliminary data on erlotinib (19), a study of a cytostatic antineoplastic agent (9), and a study of the putative antiangiogenic agent carboxyaminoimoidazole in renal cell carcinoma (20). Temple (21) provided a review and discussion of enrichment designs in cancer treatments. For the early development of molecularly targeted anticancer agents, RDTs were employed to select sensitive subpopulations when an effect assay for such separation was not available (6, 8). Other applications of enrichment designs can be found in children’s health research (22, 23), clinical research in psychiatry (11, 24, 25), cardiac arrhythmia suppression study in cardiology (26), and a few other therapeutic areas (27–30). Similar enrichment strategies can also be found in some two-stage surveys (31). Freidlin and Simon (8) evaluated cytostatic drugs using a design in which they expected only certain patients to be sensitive to their target treatment. 4

DISCUSSION

Because the goal of enrichment is to separate certain subpopulations from the general population and randomize only the selected subpopulation in the trial, the enrichment design is usually only capable of detecting the efficacy rather than effectiveness. This type of efficacy distinction is often of main interest in oncology, which is the reason that RDT is frequently used for screening for a treatment activity. As certain assumptions are satisfied, the efficacy detection can be greatly enhanced. The work by Kopec et al (7) illustrated that, when compared with an RCT, the sample size required for an RDT can be reduced by more than 50%. Fedorov and Liu (10) showed that the increase in efficiency of efficacy detection can be even higher if additional assumptions are made and the information from the open stage can be seamlessly included. However, enrichment designs are not always superior to other designs, even when all of the ethical and logistical conditions are acceptable. For example, studies that have compared the relative efficiency between

6

ENRICHMENT DESIGN

RDTs and the classic RCT (7, 8, 10, 32) have found that the RCT can be more efficient under certain conditions, even when the separation of the subpopulation at the first stage is perfect (i.e., no misclassifications). Other limitations of enrichment designs include: 1. The benefits of using enrichment designs come at the expense of the applicability of the study results to the general population. The Coronary Drug Project Research Group study (22) illustrated the effect of compliance on the conclusions for the enriched subpopulations and the general population. 2. The recruitment process for an enrichment design could last much longer than for a conventional RCT. 3. The use of surrogate endpoints of RDT at the first stage can affect the performance of enrichment process. Fedorov and Liu (10) showed the consequences of misclassification on design efficiency in the first stage of an RDT. 4. The screening phases (the run-in phase for PRIT and the open phase for RDT) are not free; they cost the sponsors time and money and come with errors (33, 34). In general, an enrichment design should undergo a cost-benefit analysis (35) in which efficiency is not the only contributor to the utility function. The enrichment (selection) process is not universally applicable to all scenarios, and may prove ethically controversial (36–38), even though its rationale is well supported and it has been natural to apply to the most commonly reported applications. Researchers must scrutinize the validity of the assumptions and consider the possible ethical issues associated with enrichment before the design is carried out.

REFERENCES 1. A. P. Hallstrom and L. Friedman, Randomizing responders. Control Clin Trials. 1991; 12: 486–503.

2. R. J. Temple, Special study designs: early escape, enrichment, studies in nonresponders. Commun Stat Theory Methods. 1994; 23: 499–531. 3. A. Pablos-Mendez, A. G. Barr, and S. Shea, Run-in periods in randomized trials. JAMA. 1998; 279: 222–225. 4. W. Amery and J. Dony, Clinical trial design avoiding undue placebo treatment. J Clin Pharmacol. 1975; 15: 674–679. 5. E. Brittian and J. Wittes, The run-in period in clinical trials: the effect of misclassification on efficacy. Control Clin Trials. 1990; 11: 327–338. 6. G. L. Rosner, W. Stadler, and M. J. Ratain, Discontinuation design: application to cytostatic antineoplastic agents. J Clin Oncol. 2002; 20: 4478–4484. 7. J. Kopec, M. Abrahamowicz, and J. Esdaile, Randomize discontinuation trials: utility and efficiency. J Clin Epidemiol. 1993; 46: 959–971. 8. B. Freidlin and R. Simon, Evaluation of randomized discontinuation design. J Clin Oncol. 2005; 23: 5094–5098. 9. L. V. Rubinstein, E. L. Korn, B. Freidlin, S. Hunsberger, S. P. Ivy, and M. A. Smith, Design issues of randomized phase II trials and proposal for phase II screening trials. J Clin Oncol. 2005; 23: 7199–7206. 10. V. V. Fedorov and T. Liu, Randomized discontinuation trials: design and efficiency. GlaxoSmithKline Biomedical Data Science Technical Report, 2005-3. 11. M. Fava, A. E. Evins, D. J. Dorer, and D. A. Schoenfeld, The problem of the placebo response in clinical trials for psychiatric disorders: culprits, possible remedies, and a novel study design approach. Psychother Psychosom. 2003; 72: 115–127. 12. C. Chiron, O. Dulac, and L. Gram, Vigabatrin withdrawal randomized study in children. Epilepsy Res. 1996; 25: 209–215. 13. E. L. Korn, S. G. Arbuck, J. M. Pulda, R. Simon, R. S. Kaplan, and M. C. Christian, Clinical trial designs for cytostatic agents: are new approaches needed? J Clin Oncol. 2001; 19: 265–272. 14. W. B. Capra, Comparing the power of the discontinuation design to that of the classic randomized design on time-to-event endpoints. Control Clin Trials. 2004; 25: 168–177. 15. The SOLVD Investigators. Effect of enalapril on survival in patients with reduced left ventricular ejection fractions and congestive heart failure. N Engl J Med. 1991; 325: 293–302.

ENRICHMENT DESIGN 16. J. E. Buring and C. E. Hennekens, Cost and efficiency in clinical trials: the US Physicians’ Health Study. Stat Med. 1990; 9: 29–33. 17. K. B. Schechtman and M. E. Gordon, A comprehensive algorithm for determining whether a run-in strategy will be a costeffective design modification in a randomized clinical trial. Stat Med. 1993; 12: 111–128. 18. C. E. Davis, W. B. Applegate, D. J. Gordon, R. C. Curtis, and M. McCormick, An empirical evaluation of the placebo run-in. Control Clin Trials. 1995; 16: 41–50. 19. Tarceva (erlotinib). Tablet package Insert. Melville, NY: OSI Pharmaceuticals, December 2004. 20. W. M. Stadler, G. Rosner, E. Small, D. Hollis, B. Rini, S. D. Zaentz, and J. Mahoney, Successful implementation of the randomized discontinuation trial design: an application to the study of the putative antiangiogenic agent carboxyaminoimidazole in renal cell carcinoma—CALGB 69901. J Clin Oncol. 2005; 23: 3726–3732. 21. R. J. Temple, Enrichment designs: efficiency in development of cancer treatments. J Clin Oncol. 2005; 23: 4838–4839. 22. The Coronary Drug Project Research Group. Influence of adherence to treatment and response of cholesterol on mortality in the Coronary Drug Project. N Engl J Med. 1980; 303: 1038–1041. 23. P. Casaer, J. Aicardi, P. Curatolo, K. Dias, M. Maia, et al, Flunazirizine in alternating hemiplegia in childhood. An international study in 12 children. Neuropediatrics. 1987; 18: 191–195. 24. F. M. Quitkin and J. G. Rabkin, Methodological problems in studies of depressive disorder: utilities of the discontinuation design. J Clin Psychopharmacol. 1981; 1: 283–288. 25. D. D. Robinson, S. C. Lerfald, B. Bennett, D. Laux, E. Devereaux, et al, Continuation and maintenance treatment of major depression with the monoamine oxidase inhibitor phenelzine: a double blind placebo-controlled discontinuation study. Psychopharmacol Bull. 1991; 27: 31–39.

7

blocker with intrinsic sympathomimetic activity. Arch Intern Med. 1988; 148: 1725–1728. 28. D. S. Echt, P. R. Liebson, L. B. Mitchell, R. W. Peters, D. Obias-Manno, et al, and the CAST Investigators. Mortality and morbidity in patients receiving encainide, flecainide, or placebo: the Cardiac Arrhythmia Suppression Trial. N Engl J Med. 1991; 324: 781–788. 29. J. R. Evans, K. Pacht, P. Huss, D. V. Unverferth, T. M. Bashore, and C. V. Leier, Chronic oral amrinone therapy in congestive heart failure: a double-blind placebo-controlled withdrawal study. Int J Clin Pharmacol Res. 1984; 4: 9–18. 30. G. H. Guyatt, M. Townsend, S. Nogradi, S. O. Pugsley, J. L. Keller, and M. T. Newhouse, Acute response to bronchodilator. An imperfect guide for bronchodilator therapy in chronic airflow limitation. Arch Intern Med. 1988; 148: 1949–1952. 31. J. L. Vazquez-Barquero, J. F. Diez-Manrique, C. Pena, R. G. Quintanal, and M. Labrador Lopez, Two stage design in a community survey. Br J Psychiatry. 1986; 149: 88–897. 32. C. Mallinckrodt, C. Chuang-Stein, P. McSorley, J. Schwartz, D. G. Archibald, et al, A case study comparing a randomized withdrawal trial and a double-blind long term trial for assessing the long-term efficacy of an antidepressant. Pharm Stat. 2007; 6: 9–22. 33. E. Brittain and J. Wittes, The run-in period in clinical trials: the effect of misclassification on efficiency. Control Clin. Trials. 1990; 11: 327–338. 34. R. J. Glynn, J. E. Buring, C. H. Hennekens, Riley, D, T. J. Kaptchuk, et al, Concerns about run-in periods in randomized trials. JAMA. 1998; 279: 1526–1527. 35. K. B. Schechtman and M. E. Gordon, A comprehensive algorithm for determining whether a run-in strategy will be a costeffective design modification in a randomized clinical trial. Stat Med. 1993; 12: 111–128. 36. S. J. Senn, A personal view of some controversies in allocating treatment to patients in clinical trials. Stat Med. 1995; 14: 2661–2674.

26. D. S. Echt, P. R. Liebson, and L. B. Mitchell, Morality and morbidity in patients receiving encainide, flecainide, or placebo. N Engl J Med. 1991; 324: 781–788.

37. S. J. Senn, Ethical considerations concerning treatment allocation in drug development trials. Stat Methods Med Res. 2002; 11: 403–411.

27. T. D. Giles, G. E. Sander, L. Roffidal, M. G. Thomas, D. P. Mersch, et al, Remission of mild to moderate hypertension after treatment with carteolol, a beta-adrenoceptor

38. P. D. Leber and C. S. Davis, Threats to validity of clinical trials employing enrichment strategies for sample selection. Control Clin Trials. 1998; 19: 178–187.

8

ENRICHMENT DESIGN

CROSS-REFERENCES Clinical trial/study Adaptive design Randomization Sample size estimation Run-in period Estimation Inference Hypothesis testing Efficacy Effectiveness

ENVIRONMENTAL ASSESSMENTS (EAS) Under the National Environmental Policy Act of 1969 (NEPA), all Federal agencies are required to assess the environmental impact of their actions and to ensure that the interested and affected public is informed of the environmental analyses. The Center for Drug Evaluation and Research’s (CDER) Environmental Assessment of Human Drug and Biologics Applications (Issued 7/1998, Posted 7/24/98) provides detailed information on a variety of topics related to preparing and filing EAs. In CDER, adherence to NEPA is demonstrated by the EA portion of the drug application. This section focuses on the environmental implications of consumer use and disposal from use of the candidate drug. However, because approval of many drugs are unlikely to have significant environmental effects, CDER has provisions for submission of abbreviated EAs rather than full EAs under certain circumstances or has categorically excluded certain classes of actions. FDA has reevaluated its NEPA regulations found in 21 CFR (Code of Federal Regulations) Part 25 and has proposed to improve its efficiency in the implementation of NEPA and reduce the number of EAs by increasing the number of applicable categorical exclusions. The notice of proposed rule making was posted in the Federal Register on April 3, 1996.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/environ.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

EQUIVALENCE TRIALS AND EQUIVALENCE LIMITS1

noninferiority hypothesis, and δ is the noninferiority margin. On the other hand, in some applications, noninferiority testing may be intended to study some other objectives, particularly when a placebo is absent from the trial. In such cases, the noninferiority margin δ may have a special meaning; for instance, the noninferiority testing may be to infer that the test treatment would have beaten the placebo had a placebo been in the trial or that the test treatment would have retained a certain fraction of the control’s effect.

H. M. JAMES HUNG Division of Biometrics I, Office of Biostatistics Office of Translational Sciences, Center for Drug Evaluation and Research U.S. Food and Drug Administration Silver Spring, Maryland

In clinical trials, when a test treatment is compared with an active or positive control that is being used in medical practices, one possible study objective may be to show that the test treatment is ‘‘equivalent’’ to the active control with respect to therapeutic benefits or adverse effects. Traditionally, equivalence means that the difference between the test treatment and the control treatment is clinically insignificant. The threshold of clinical insignificance would need to be defined, and this threshold is the so-called equivalence limit or equivalence margin.

2

EQUIVALENCE LIMIT OR MARGIN

In general, there is only one equivalence margin that defines the acceptable degree of clinical difference between the test treatment and the control, particularly when two effective treatments are compared to show that either treatment is not inferior to the other. As already mentioned, for clinical endpoints, the unacceptable margin of the inferiority of the test treatment to the control may have a special definition; for example, it may be an explicit function of the postulated control’s effect. When such a noninferiority margin is determined to show that the test treatment is better than placebo or that the test treatment retains a certain fraction of the control’s effect, it may be too large to conclude that the test treatment is not unacceptably inferior to the control. This margin is certainly irrelevant for defining the degree of superiority of the test treatment over the control; thus, if equivalence testing is pursued in these applications, the equivalence hypothesis might have two limits, one for inferiority and one for superiority.

1 EQUIVALENCE VERSUS NONINFERIORITY In pharmaceutical applications, the concept of equivalence is mostly applied to assessment of a test formulation of a medical product relative to an established reference in so-called bioequivalence studies on a pharmacokinetic variable. In contrast, clinical trials are rarely aimed at showing equivalence; instead, the concept of noninferiority may be more applicable (1–18). The distinction between equivalence and noninferiority can be subtle. On one hand, a noninferiority hypothesis can be viewed as one side of an equivalence hypothesis. That is, if δ is the equivalence margin, then the equivalence hypothesis is that the test treatment and the control treatment differ by an extent smaller than δ. If the hypothesis at stake is that the test treatment is inferior to the control at most by a degree smaller than δ, it is a

3 DESIGN, ANALYSIS, AND INTERPRETATION OF EQUIVALENCE TRIALS The necessary considerations for design, analysis, and interpretation of a noninferiority trial as stipulated in the references are, in principle, applicable to equivalence testing. To accept the equivalence hypothesis defined by the equivalence limits, the confidence

1

The views presented in this paper are not necessarily those of the U.S. Food and Drug Administration.

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

EQUIVALENCE TRIALS AND EQUIVALENCE LIMITS

interval derived from the equivalence trial for the difference between the test treatment and the active control must completely fall within the equivalence range determined by the equivalence margins. This is in contrast to the noninferiority testing that requires that the confidence interval exclude only the relevant noninferiority margin. REFERENCES 1. P. Bauer and M. Kieser, A unifying approach for confidence intervals and testing of equivalence and difference. Biometrika. 1996; 83: 934–937. 2. Committee for Medicinal Products for Human Use (CHMP), European Medicines Agency. Guideline on the Choice of the Non-Inferiority Margin. London, UK, July 27, 2005. Available at: http://www.emea.europa.eu/pdfs/human/ ewp/215899en.pdf. 3. A. F. Ebbutt and L. Frith, Practical issues in equivalence trials. Stat Med. 1998; 17: 1691–1701. 4. S. S. Ellenberg and R. Temple, Placebocontrolled trials and active-control trials in the evaluation of new treatments. Part 2: Practical issues and specific cases. Ann Intern Med. 2000; 133: 464–470. 5. T. R. Fleming, Treatment evaluation in active control studies. Cancer Treat Reports. 1987; 71: 1061–1064. 6. T. R. Fleming, Design and interpretation of equivalence trials. Am Heart J. 2000; 139: S171–S176. 7. A. L. Gould, Another view of active-controlled trials. Control Clin Trials. 1991; 12: 474–485. 8. D. Hauschke, Choice of delta: a special case. Drug Inf J. 2001; 35: 875–879. 9. D. Hauschke and L. A. Hothorn, Letter to the Editor. Stat Med. 2007; 26: 230–233. 10. H. M. J. Hung, S. J. Wang, and R. O’Neill, Noninferiority trial. In: R. D’Agostino, L. Sullivan,

and J. Massaro (eds.), Encyclopedia of Clinical Trials. New York, Wiley, 2007. 11. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E9 Statistical Principles for Clinical Trials. Step 4 version, February 5, 1998. Available at: http://www.ich.org/LOB/media/ MEDIA485.pdf. 12. B. Jones, P. Jarvis, J. A. Lewis, and A. F. Ebbutt, Trials to assess equivalence: the importance of rigorous methods. BMJ. 1996; 313: 36–39. 13. T. H. Ng, Choice of delta in equivalence testing. Drug Inf J. 2001; 35: 1517–1527. 14. G. Pledger and D. B. Hall, Active control equivalence studies: do they address the efficacy issue? In: K. E. Peace (ed.), Statistical Issues in Drug Research and Development. New York: Marcel Dekker, 1990, pp. 226–238. 15. J. Rohmel, Therapeutic equivalence investigations: statistical considerations. Stat Med. 1998; 17: 1703–1714. 16. R. Temple, Problems in interpreting active control equivalence trials. Account Res. 1996; 4: 267–275. 17. R. Temple and S. S. Ellenberg, Placebocontrolled trials and active-control trials in the evaluation of new treatments. Part 1: Ethical and scientific issues. Ann Intern Med. 2000; 133: 455–463. 18. B. Wiens, Choosing an equivalence limit for non-inferiority or equivalence studies. Control Clin Trials. 2002; 23: 2–14.

CROSS-REFERENCES Non-inferiority Trial Non-inferiority Margin Bioequivalence

ESSENTIAL DOCUMENTS

Any or all documents addressed in this guideline may be subject to, and should be available for, audit by the sponsor’s auditor and inspection by the regulatory authority(ies).

Essential Documents are those documents that permit evaluation individually and collectively of the conduct of a trial and the quality of the data produced. These documents serve to demonstrate the compliance of the investigator, sponsor, and monitor with the standards of Good Clinical Practice (GCP) and with all applicable regulatory requirements. Essential documents also serve several other important purposes. Filing essential documents at the investigator/institution and sponsor sites in a timely manner can assist greatly in the successful management of a trial by the investigator, sponsor, and monitor. Also, these documents usually are audited by the sponsor’s independent audit function and are inspected by the regulatory authority(ies) as part of the process to confirm the validity of the trial conduct and the integrity of data collected. The minimum list of essential documents has been developed. The various documents are grouped in three sections according to the stage of the trial during which they will normally be generated: (1) Before the clinical phase of the trial commences, (2) during the clinical conduct of the trial, and (3) after completion or termination of the trial. A description is given of the purpose of each document, and whether it should be filed in either the investigator/institution or the sponsor files, or both. It is acceptable to combine some documents provided the individual elements are readily identifiable. Trial master files should be established at the beginning of the trial, both at the investigator/institution’s site and at the sponsor’s office. A final close-out of a trial can be performed only when the monitor has reviewed both investigator/institution and sponsor files and has confirmed that all necessary documents are in the appropriate files. This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

ETHICAL CHALLENGES POSED BY CLUSTER RANDOMIZATION

are discussed in Section 4. Section 5 concludes the article largely as a means of promoting debate, to provide recommendations for the reporting of ethical issues in cluster randomization trials. Readers interested in a more detailed discussion might wish to consult Donner and Klar (9), from which much of this article was abstracted.

NEIL KLAR Cancer Care Ontario Division of Preventive Oncology Toronto, Ontario, Canada

ALLAN DONNER The University of Western Ontario Department of Epidemiology and Biostatistics London, Ontario, Canada

1

2

EXAMPLES 1. A group of public health researchers in Montreal (10) conducted a household randomized trial to evaluate the risk of gastrointestinal disease due to consumption of drinking water. Participating households were randomly assigned to receive an in-home water filtration unit or were assigned to a control group that used tap water. Overall, 299 families (1206 individuals) were assigned to receive water filters and 308 families (1201 individuals) were in the control group. 2. The National Cancer Institute of the United States funded the Community Intervention Trial for Smoking Cessation (COMMIT), which investigated whether a community-level, 4-year intervention would increase quit rates of cigarette smokers (11) Communities were selected as the natural experimental unit because investigators assumed that interventions offered at this level would reach the greatest number of smokers and possibly change the overall environment, thus making smoking less socially acceptable. Overall 11 matched-pairs of communities were enrolled in this study with one community in each pair randomly assigned to the experimental intervention with the remaining community serving as a control. 3. Antenatal care in the developing world has attempted to mirror care that is offered in developed countries even though not all antenatal care interventions are known to be effective.

INTRODUCTION

Ethical guidelines for medical experimentation were first put forward in 1947 with publication of the Nuremberg Code (1, 2). Numerous national and international ethical codes have since been developed, written almost exclusively in the context of clinical trials in which a physician’s patients are individually assigned for the purpose of evaluating the effect of therapeutic interventions. In spite of some limited early examples (3), far less attention has been given to the distinct ethical challenges of cluster randomization (4–7), reflecting, perhaps, the recent growth of interest in this design. Experience has shown that cluster randomization is primarily adopted to evaluate nontherapeutic interventions, including lifestyle modification, educational programs, and innovations in the provision of health care. The limited attention given to the ethical aspects of this design may be more broadly related to the fact that trials of disease prevention have tended to be largely exempt from ethical constraints, possibly because of a perception that participation in such trials carries only minimal risk (8). In Section 2, the key ethical issues raised by cluster randomization trials will be examined by discussing them in the context of several recently published studies. The relative potential for harm and benefit to subjects participating in either individually randomized or cluster randomized trials is described in Section 3, whereas difficulties in obtaining informed consent before random assignment

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

ETHICAL CHALLENGES POSED BY CLUSTER RANDOMIZATION

The World Health Organization (WHO) antenatal care randomized trial (12) compared a new model of antenatal care that emphasized health-care interventions known to be effective with the standard model of antenatal care. The primary hypothesis in this equivalence trial was that the new model of antenatal health care would not increase the risk of having a low birthweight (<2500 g) child. Participating clinics, recruited from Argentina, Cuba, Saudi Arabia, and Thailand, were randomly assigned to an intervention group or control group separately within each country. Overall, 27 clinics (12,568 women) were randomly assigned to the experimental arm, whereas women attending 26 control group clinics (11,958 women) received standard antenatal care.

3

THE RISK OF HARM

The challenge of designing an ethical randomized trial requires balancing the potential benefits and risk of harm faced by individual participants with the potential long-term benefit to those subjects and to society at large. Indeed, it is concern for the risk of harm that has traditionally required that the use of random assignment be justified by the notion of ‘‘clinical equipoise,’’ whereby ‘‘experts are uncertain as to whether any of the proposed arms of a trial is superior to another’’ (2). Too often the potential risks of participating in health education or disease prevention trials have been largely neglected (8), which is unfortunate because not all such trials meet the criteria of minimal risk that, when achieved, may allow investigators to conduct research without first obtaining informed consent from study subjects (1). According to the U.S. Federal Government (13): Minimal risk means that the probability and magnitude of harm or discomfort anticipated in the research are not greater than those ordinarily encountered in daily life or during the

performance of routine physical or psychological examinations or tests.

Furthermore, as the benefits from participation in a trial decrease, so must the net risk of harm. Although risk could be considered low for subjects participating in each of the three trials described in Section 2, it is arguable that only COMMIT (11) meets the minimal risk criteria defined above. In this trial, participation was limited to the completion of standardized questionnaires and the intervention was comprised of various community-level smoking cessation programs. On the other hand, subject risk was perhaps greatest for participants in the WHO antenatal care trial, as women in the experimental group received a more limited range of antenatal care than women in the control group [although women at high risk for pregnancy complications in either group were offered whatever additional care was deemed appropriate (12)]. Attention to the risk/benefit ratio should also extend to control group subjects. Therefore, some investigators have attempted to ensure that these individuals can still benefit from participation by offering a minimal level of intervention or, alternatively, by offering all individuals the intervention by the technique of delaying its implementation in the control group. This strategy may have the added benefit of encouraging participation in the trial. Typically, however, control group subjects receive only usual care, as was the case, for example, with women participating in the WHO antenatal care trial. Investigators need to recognize that participation in a cluster randomization trial may entail considerable effort at both the individual and at the cluster level. At a minimum, it is important to communicate study findings both to trial participants and to decision makers. Investigators should also be sensitive to the consequences of a successful intervention so that the benefits enjoyed by study participants do not necessarily end when the trial is completed.

ETHICAL CHALLENGES POSED BY CLUSTER RANDOMIZATION

4

THE ROLE OF INFORMED CONSENT

According to the World Medical Association Declaration of Helsinki (14), consent must be obtained from each patient before random assignment. Such a requirement not only assures that the risks of experimentation are adequately communicated to patients, but also facilitates the process of random assignment, which may at times be seen to compromise the implicit contractual relationship between patient and physician. The situation is more complicated for cluster randomization trials, particularly when relatively large allocation units are randomized, such as schools, communities, or worksites. Then school principals, community leaders, or other key decision makers will usually provide permission for both random assignment and implementation of the intervention. For example, in the WHO antenatal care trial, women only became eligible to participate in the trial after becoming pregnant and so could not have been enrolled before random assignment (12). Individual study subjects may still be free to withhold their participation, although they may not even then be able to completely avoid the inherent risks of an intervention that is applied on a cluster-wide level (as, for example, in studies of water fluoridation or mass education). The identification of individuals mandated to provide agreement for random assignment may not be a simple task. Typically, it is elected or appointed officials who make such decisions. However, as Strasser et al. (7) point out, it is by no means certain when, or even if, securing the agreement of these officials is sufficient. Thus, as stated in a set of recently released guidelines for cluster randomized trials (15), ‘‘the roles of the guardians of the patients interests during the trial, the gatekeepers of access to patient groups, and sponsors of the research are even more important in cluster randomized trials where individuals may not have the opportunity to give informed consent to participation.’’ In the context of community-based trials, it may similarly be prudent for investigators to establish or collaborate with community advisory boards (16), a strategy used by COMMIT investigators (11, 17).

3

The practical difficulties of securing informed consent before random assignment may not necessarily occur when smaller clusters such as households or families are the unit of randomization. For instance, one of the eligibility criteria for the Montreal water filtration study (10) was a ‘‘willingness to participate in a longitudinal trial in which a random half of the households would have a filter installed.’’ The relative absence of ethical guidelines for cluster randomized trials appears to have created a research environment in which the choice of randomization unit may determine whether informed consent is deemed necessary before random assignment. This phenomenon can be seen, for example, in the several published trials of Vitamin A supplementation on childhood mortality. Informed consent was obtained from mothers before assigning children to either Vitamin A or placebo in the household randomization trial reported by Herrera et al. (18). However, in the community intervention trial of Vitamin A reported by the Ghana VAST Study Team (19), consent to participate was obtained only after random assignment, which may be seen as an example of the randomized consent design proposed by Zelen (20). It seems questionable, on both an ethical level and a methodological level, whether the unit of randomization should play such a critical role in deciding whether informed consent is required. 5

DISCUSSION

As a first step to developing a well-accepted set of ethical guidelines and norms for cluster randomization trials, editors could require investigators to report having IRB approval, and to indicate how issues of subject consent were dealt with. The greater challenge is to agree on ethical features of cluster randomization trials that should be reported (e.g., the timing of informed consen). When permission from key decision makers associated with each cluster is needed for assigning interventions, investigators should identify these decision makers and indicate how they were selected. Some information about the consent procedure administered

4

ETHICAL CHALLENGES POSED BY CLUSTER RANDOMIZATION

to individual study subjects should also be provided. For example, it would usually be helpful to know when consent was obtained and what opportunities, if any, existed for cluster members to avoid the inherent risks of intervention. Note that these suggestions may not require the development of novel ethical criteria. Very similar suggestions have been proposed earlier in the 1991 International Guidelines for Ethical Review of Epidemiological Studies put forward by CIOMS, the Council for International Organizations of Medical Sciences (1). To promote debate, the authors conclude by quoting from the Community Agreement section of these underutilized guidelines: When it is not possible to request informed consent from every individual to be studied, the agreement of a representative of a community or group may be sought, but the representative should be chosen according to the nature, traditions and political philosophy of the community or group. Approval given by a community representative should be consistent with general ethical principles. When investigators work with communities, they will consider communal rights and protection as they would individual rights and protection. For communities in which collective decision-making is customary, communal leaders can express the collective will. However, the refusal of individuals to participate in a study has to be respected: a leader may express agreement on behalf of a community, but an individual’s refusal of personal participation is binding (1).

REFERENCES 1. B. A. Brody, The Ethics of Biomedical Research. Oxford: Oxford University Press, 1998. 2. J. Sugarman, Ethics in the design and conduct of clinical trials. Epidemiologic Rev. 2002; 24: 54–58. 3. C. L. Schultze, Social programs and social experiments. In: A. M. Rivlin and P. M. Timpane (eds.), Ethical and Legal Issues of Social Experimentation. Washington, DC: The Brookings Institution, 1975, pp. 115–125. 4. K. Glanz, B. K. Rimer, and C. Lerman, Ethical issues in the design and conduct of community-based intervention studies. In:

S. S. Coughlin and T. L. Beauchamp (eds.), Ethics and Epidemiology. Oxford: Oxford University Press, 1996, pp. 156–177. 5. S. J. L. Edwards, D. A. Braunholtz, R. J. Lilford, and A. J. Stevens, Ethical issues in the design and conduct of cluster randomised controlled trials. Brit. Med. J. 1999; 318: 1407–1409. 6. J. L. Hutton, Are distintive ethical principles required for cluster randomized controlled trials? Stat. Med. 2001; 20: 473–488. 7. T. Strasser, O. Jeanneret, and L. Raymond, Ethical aspects of prevention trials. In: S. Doxiadis (ed.), Ethical Dilemmas in Health Promotion. New York: Wiley, 1987, pp. 183– 193. 8. P. Skrabanek, Why is preventive medicine exempted from ethical constraints? J. Med. Ethics 1990; 16: 187–190. 9. A. Donner and N. Klar, Design and Analysis of Cluster Randomization Trials in Health Research. London: Arnold, 2000. 10. P. Payment, L. Richardson, J. Siemiatycki, R. Dewar, M. Edwardes, and E. Franco, A randomized trial to evaluate the risk of gastrointestinal disease due to consumption of drinking water meeting microbiological standards. Amer. J. Public Health 1991; 81: 703–708. 11. COMMIT Research Group, Community Intervention Trial for Smoking Cessation (COMMIT): I. Cohort results from a four-year community intervention. Amer. J. Public Health 1995; 85: 183–192. 12. J. Villar et al. for the WHO Antenatal Care Trial Research Group, WHO antenatal care randomised trial for the evaluation of a new model of routine antenatal care. Lancet 2001; 357: 1551–1564. 13. Office for Protection from Research Risks, Protection of Human Subjects. Title 45, Code of Federal Regulations, Part 46. Bethesda, MD: Department of Health and Human Services N.I.H., 1994. 14. World Medical Association Declaration of Helsinki, Ethical principles for medical research involving human subjects. JAMA 2000; 284: 3043–3045. 15. Medical Research Council Clinical Trials Series, Cluster Randomized Trials: Methodological and Ethical Considerations. London: MRC, 2000. Available: www.MRC.ac.uk. 16. R. P. Strauss et al., The role of community advisory boards: involving communities in the informed consent process. Amer. J. Public Health 2001; 91: 1938–1943.

ETHICAL CHALLENGES POSED BY CLUSTER RANDOMIZATION 17. B. Thompson, G. Coronado, S. A. Snipes, and K. Puschel, Methodologic advances and ongoing challenges in designing community-based health promotion programs. Annu. Rev. Public Health 2003; 24: 315–340. 18. M. G. Herrera et al., Vitamin A supplementation and child survival. Lancet 1992; 340: 267–271. 19. Ghana VAST Study Team, Vitamin A supplementation in northern Ghana: effects on clinic attendances, hospital admissions, and child mortality. Lancet 1993; 342: 7–12. 20. M. Zelen, Randomized consent designs for clinical trials: an update. Stat. Med. 1990; 9: 645–656.

5

ETHICAL ISSUES IN INTERNATIONAL RESEARCH

concerns about the lack of new drugs in the pipeline. Pharmaceutical companies are looking for new ways to cut the costs of drug development and have shown increasing interest in outsourcing their clinical trials to low-cost countries. India and Eastern Europe have become attractive as new clinical trial sites because labor and infrastructure costs in those countries are cheaper, and there is an abundance of treatment-naive patients who are ideal for participation in trials of new candidate drugs. According to one estimate, 40% of all clinical trials are now carried out in countries such as Russia and India. It is estimated that patients can be recruited 10 times faster in Russia compared with Western countries, and drug development cost can be $200 million cheaper if conducted in India, a substantial saving of an estimated $800 million in normal drug development costs (2). One challenge common to all international collaborative research is how to deal with conflicting regulations between host and sponsor countries. This is particularly a challenge if host country regulations are territorial and cover research done in that country, whereas sponsor country regulations apply according to funding source, such as the U.S. Common Rule. If U.S. federally funded research is done in another country, both the U.S. regulations and the host country regulations apply to that research, so the question is what should be done if they are in conflict. If, for example, a host country research ethics committee takes the position that requiring a signature or a thumbprint on an informed consent form in an illiterate population is unethical, this decision would not be in accordance with the U.S. regulations. In the national legislation of some countries, research ethics committees do not have the authority to stop research; however, U.S. regulations require that institutional review boards (IRBs) have this authority. Fortunately, most of the time in cases of apparent conflict, the regulations are flexible enough to enable reasonable accommodations by the multiple ethics review boards involved in any one international trial.

REIDAR K. LIE Department of Bioethics National Institutes of Health Bethesda, Maryland

1 BACKGROUND AND RATIONALE FOR INTERNATIONAL RESEARCH Increasingly, clinical trials involve partners in two or more countries. Examples are multisite clinical trials recruiting patients in several countries, or clinical trials carried out in a host country different from the country of the sponsor. Such trials raise particularly challenging ethical issues, especially when there are substantial differences in wealth between host and sponsor countries. Sponsors can be public agencies or pharmaceutical companies. The portion of the U.S. National Institutes of Health (NIH) budget going to international sites, both directly and through U.S. principal investigators, has shown a steady increase over the past two decades, and is now over $500 million. The European Union, together with a number of national governments and private donors, has launched a high-profile initiative to fund research on interventions against poverty related diseases, primarily malaria, human immunodeficiency virus (HIV), and tuberculosis; these efforts through the European and Developing Countries Clinical Trials Partnership (EDCTP) have been primarily related to competence building in African countries (1). The mission of this initiative is to ‘‘accelerate the development of new clinical interventions to fight HIV/AIDS, malaria and tuberculosis in developing countries (DCs), particularly sub-Saharan Africa, and to improve generally the quality of research in relation to these diseases’’ (1). The pharmaceutical industry has traditionally been quite profitable; however, coupled with rising development costs, the risks associated with drug development have increased, and there have been recent

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

ETHICAL ISSUES IN INTERNATIONAL RESEARCH

Both publicly funded research on poverty related diseases and research sponsored by pharmaceutical companies in poor countries aimed at developing drugs for richer countries raise a number of ethical issues that have been discussed widely in the literature during the past few years. We will discuss four of the main concerns. 2

THE STANDARD OF CARE CONTROVERSY

Perhaps the most contentious issue in research ethics during the past few years has been whether there is an ethical obligation to provide participants in trials, irrespective of differences in economic circumstances, interventions that are at least as good as those that would be provided to participants in resource-rich countries. This debate started with the controversy over trials to test treatment regimens to prevent perinatal HIV transmission. Studies had demonstrated that an expensive preventive regimen of providing azidothymidine (AZT) for several weeks during pregnancy, intravenously during delivery, and to the newborn for some time after birth could dramatically reduce infection in newborns from around 30% to less than 10% (3). This regimen was considered too expensive and logistically impossible for resource-poor countries. There was therefore a need to identify more suitable interventions in these settings. Some argued that only by doing a trial where these new interventions were tested against placebo could one obtain results that would be useful for the host-countries (4). We have argued that this position is correct (5), although others have maintained that one could have obtained useful results by doing an equivalence trial (6). The critics of these placebo-controlled trials maintained that they are morally unacceptable because everyone in a clinical trial, regardless of location, is entitled to an intervention that is at least as good as that which is regarded to be the best treatment option for a particular condition (7). Irrespective of what one’s position is with regard to the perinatal HIV transmission controversy, the question remains as to whether it is at all permissible at least sometimes

to provide to participants a lesser standard of care than is available elsewhere, but not locally, to obtain useful results for that particular setting. The World Medical Association (WMA) has taken the position that it is not permissible (Declaration of Helsinki, paragraph 29): The benefits, risks, burdens and effectiveness of a new method should be tested against those of the best current prophylactic, diagnostic and therapeutic methods. This does not exclude the use of placebo, or no treatment, in studies where no proven, prophylactic, diagnostic or therapeutic method exists (8).

The clarification to this paragraph would also allow for placebo use in the cases where it is scientifically necessary when doing research on conditions where withholding effective interventions would only cause temporary and minor harm. It certainly would not allow withholding interventions in the case of lifesaving interventions such as antiretroviral agents to prevent HIV infection (Declaration of Helsinki, clarification to paragraph 29): The WMA hereby reaffirms its position that extreme care must be taken in making use of a placebo-controlled trial and that in general this methodology should only be used in the absence of existing proven therapy. However, a placebo-controlled trial may be ethically acceptable, even if proven therapy is available, under the following circumstances:

• Where for compelling and scientifically sound methodological reasons its use is necessary to determine the efficacy or safety of a prophylactic, diagnostic or therapeutic method; or • Where a prophylactic, diagnostic or therapeutic method is being investigated for a minor condition and the patients who receive placebo will not be subject to any additional risk of serious or irreversible harm (8).

We have argued that most other international guidance documents have disagreed with this position, and we accept what we have called the international consensus

ETHICAL ISSUES IN INTERNATIONAL RESEARCH

opinion (9). The Council for International Organizations of Medical Sciences (CIOMS) guidelines are more typical, and it is worth quoting at length from this document: An exception to the general rule [represented by the Declaration of Helsinki] is applicable in some studies designed to develop a therapeutic, preventive or diagnostic intervention for use in a country or community in which an established effective intervention is not available and unlikely in the foreseeable future to become available, usually for economic or logistic reasons. The purpose of such a study is to make available to the population of the country or community an effective alternative to an established effective intervention that is locally unavailable. Also, the scientific and ethical review committees must be satisfied that the established effective intervention cannot be used as comparator because its use would not yield scientifically reliable results that would be relevant to the health needs of the study population. In these circumstances an ethical review committee can approve a clinical trial in which the comparator is other than an established effective intervention, such as placebo or no treatment or a local remedy (10).

According to the CIOMS position, at least three conditions must be satisfied if one can allow an exception to the general rule of requiring participants to receive at least as good care as that which is regarded as state of the art care for the condition considered: 1. The state of the art intervention must not be available locally outside the research context. Research subjects must not be denied care that they would otherwise receive locally. 2. It is necessary to provide a different intervention than that which is regarded to be state of the art intervention for the subject’s condition for scientific reasons. One could not achieve the objectives of the trial in any other way. 3. The objective of the trial is to identify an intervention that can be useful in the local context. If we apply these criteria to the original perinatal HIV transmission controversy, it is unclear whether these trials would be justified because a case can be made that they

3

violated the third condition. This is the basis for arguments made by at least some of the critics of these trials (11). However, trials such as developing a new, simplified diagnostic tool, known to be less effective than viral load measurements, would be approved using this standard (12). 2.1 Obligations to Trial Participants to Provide Ancillary Care Researchers interacting with study participants in clinical research will at times identify needs for clinical care that would be beneficial to the participants, but that are not necessary to provide as part of the design of the study. For example, during a malaria vaccine trial, participants may be identified as needing treatment for HIV infection. Assuming that treatment of the HIV infection is not part of the protocol and would not affect the design of the research one way or another, the question is what obligation investigators have, if any, to address these types of health needs of their participants, assuming that their needs cannot be met by the regular health-care system because of lack of resources. Is there an obligation to provide ancillary care to study participants, defined as care that is not needed as part of the design of the study? At least one international guidance document, CIOMS, says that there is no obligation to provide ancillary care: ‘‘Although sponsors are, in general, not obliged to provide health-care services beyond that which is necessary to conduct research, it is morally praiseworthy to do so’’ (10). It is clear from discussions with researchers in resource-poor settings that, informally, researchers and sponsors do in fact provide a substantial amount of ancillary care to their trial participants because, following CIOMS, they think it is the right and decent thing to do. The unanswered question is whether we should take a position stronger than that and claim that certain types of ancillary care are not only morally praiseworthy but also obligatory. Recently, Richardson and Belsky (13, 14) have attempted to provide an argument for why there is on obligation to provide some limited form of ancillary care. Basically, they argue that obligations are created by the interactions between researchers

4

ETHICAL ISSUES IN INTERNATIONAL RESEARCH

and patients, and by the fact that the patients are providing researchers with privileged information that is useful for science. Information about a person’s health state that is gathered in the process of carrying out the protocol falls within the scope of the obligation. However, the strength of the claim may vary, depending on factors such as how important and urgent the health problem is, and how difficult it would be for the researcher to address the problem. In this model, if researchers need to test for HIV during a malaria vaccine trial because they are interested in seeing the influence of HIV status on the effectiveness of the vaccine, it would give them a prima facie obligation to ensure treatment access for the patients they identify as HIV positive. Whether they actually have such an obligation would in part depend on the cost of providing the care. However, in a pediatric trial of an HIV vaccine, it would not be an obligation for the researchers to ensure provision of treatment for malaria or acute life-threatening diarrhea, even though these health problems are arguably more urgent than the HIV infection in the malaria vaccine trial. For this reason, the Richardson/Belsky model has been criticized by Dickert et al. (15). The question of how to best analyze researcher and sponsor obligations to provide ancillary care has been relatively unexplored in the literature, so there is no clear consensus one way or another at the moment. 2.2 Obligations to Trial Participants to Continue to Provide Care after the Conclusion of the Trial Traditionally, researchers and sponsors have only been responsible for trial participants during the trial. Trial interventions have only been provided to participants for the time necessary to complete the study, and there is an expectation that the ordinary health-care system will begin to provide effective interventions identified during a trial. Although a single trial may not provide evidence that a particular intervention is effective, there certainly are cases where it is clear that participants receive benefits from whatever interventions they are provided as part of the

design of the study. The question is whether researchers have an obligation to ensure provision of effective interventions to their trial participants after the completion of the trial when there is no expectation that the ordinary health-care system can do so. According to the Declaration of Helsinki (paragraph 30), At the conclusion of the study, every patient entered into the study should be assured of access to the best proven prophylactic, diagnostic, and therapeutic methods identified by that study (8).

Thus, in a trial of two different drug combinations for HIV, for example, there is an obligation to ensure continuation of a new combination of drugs after trial completion if that combination has been identified as effective. After the revised version of the Declaration was accepted in 2000, this statement came under immediate criticism. It was pointed out that this would present an undue burden on research sponsors, whether public or private; this would divert scarce resources away from important research if these institutions were expected to budget for treatment provision after completion of a trial. There is also a regulatory issue with regard to the use of still unapproved drugs outside a clinical trial, if such use is necessary after trial completion but before regulatory approval. The World Medical Association considered these objections and issued the following note of clarification: The WMA hereby reaffirms its position that it is necessary during the study planning process to identify post-trial access by study participants to prophylactic, diagnostic and therapeutic procedures identified as beneficial in the study or access to other appropriate care. Post-trial access arrangements or other care must be described in the study protocol so the ethical review committee may consider such arrangements during its review (8).

In one sense, this clarification is weaker than the original paragraph. It now stated that ‘‘it is necessary . . . to identify post-trial access’’ and describe arrangements in the protocol. Although it no longer says one must ‘assure access’—and it certainly does not require

ETHICAL ISSUES IN INTERNATIONAL RESEARCH

funding access programs—what had actually been meant by assuring access in paragraph 30 was still left unspecified. Even if one agrees in principle that one cannot simply abandon one’s patients once the research is completed, there are a whole range of options open to sponsors and researchers, from discussing the options with patients to actually providing or paying for the interventions. Even if one agrees that one should do one’s best to ensure access to interventions once the trial is over, it is still not clear what follows in terms of arrangements that need to be in place before the trial starts. In particular, little guidance is provided regarding the criteria an IRB should use when rejecting a trial because insufficient arrangements have been made for post-trial access. An IRB would, for example, have to weigh the importance of the research and the benefits it can provide against the potential benefits to the participants of obtaining post-trial access. These types of trade-offs are illustrated by the NIH Guidance for Addressing the Provision of Antiretroviral Treatment for Trial Participants Following their Completion of NIH-Funded HIV Antiretroviral Treatment Trials in Developing Countries: For antiretroviral treatment trials conducted in developing countries, the NIH expects investigators/contractors to address the provision of antiretroviral treatment to trial participants after their completion of the trial . . . The NIH recommends investigators/contractors work with host countries’ authorities and other stakeholders to identify sources available, if any, in the country for the provision of antiretroviral treatment to antiretroviral treatment trial participants following their completion of the trial . . . Applicants are expected to provide NIH Program Staff for evaluation their plans that identify available sources, if any, for provision of antiretroviral treatment to research participants (16).

It is consistent with this policy that investigators have considered options but concluded that there is no possibility of continuing to provide post-trial access to antiretroviral agents used as part of the research design. Many would, of course, criticize this guidance as too weak.

5

However, the guidance also says that ‘‘Priority may be given to sites where sources are identified for provision of ARV treatment.’’ This part of the policy could be criticized as too strong. It could, for example, mean that research funded by NIH would shift to countries or sites that are relatively well off, where it is easier to guarantee post-trial access. This would deprive countries or sites that already are disadvantaged in terms of antiretroviral access the potential benefits of hosting an antiretroviral treatment trial. If IRBs are to address the continued provision of care to trial participants as part of the approval process, guidance is needed about how they should resolve this tension between ensuring access to continued, effective care while not denying participants and communities the benefits of research participation. However, at this time there is very little practical guidance about how to go about this. 2.3 Obligation to be Responsive to Host Country Health Needs International research ethics guidelines stress that research must be responsive to host country health needs. The Declaration of Helsinki, for example, states, ‘‘Medical research is only justified if there is a reasonable likelihood that the populations in which the research is carried out stand to benefit from the results of the research’’ (paragraph 19) (8). The CIOMS guidelines (10) state, ‘‘The health authorities of the host country, as well as a national or local ethical review committee, should ensure that the proposed research is responsive to the health needs and priorities of the host country’’ (see Guideline 3), and elsewhere maintain that research ‘‘should be responsive to their health needs and priorities in that any product developed is made reasonably available to them’’ (see General Ethical Principles). These guidelines suggest that there is a requirement to do research where the products developed in that research will meet some priority health needs of the host country. Outsourcing of drug development to host countries with the expectation that the product is primarily aimed at the sponsor country markets would therefore seem to be prohibited by these guidelines. It is unclear, however, whether

6

ETHICAL ISSUES IN INTERNATIONAL RESEARCH

the associated capacity building, infrastructure development, and training that could be used by the host country in the future would be allowed with the justification that the research ‘‘is responsive to the health needs and priorities of the host country.’’ Criticism has been leveled at the requirement that host country benefit should be understood as making the product or intervention developed in the research reasonably available to the host country population (17, 18). The essential part of this criticism is that there may be other, more important benefits for the host country. For example, healthcare infrastructure provided during a trial that can later be used to treat important health problems in the community may be more important than the ‘‘benefit’’ of guaranteed access to a product that may or may not turn out to be beneficial and, even if beneficial, may turn out to be inferior to comparable interventions tested in other countries. Or it may be that a vaccine trial for a product that is not of high priority for the host country may provide the crucial expertise to conduct locally sponsored trials of products that are still in development but target important health problems in the country; in such cases, a trial on a low priority product may be reasonably preferred over a low priority trial on a high priority product because of scientific doubts about its promise. Host countries are allowed to make these types of trade-offs in the fair benefits framework, but not according to the CIOMS guidelines. This framework has been criticized because it does not take into account the weak bargaining power of host countries when they negotiate benefit agreements. If whatever is negotiated during interactions with powerful sponsors, whether public or private, is accepted as a ‘‘fair transaction,’’ it is clear that developing countries will lose out because of existing lack of resources, which are in themselves unjust (19). But this is a misunderstanding of the fair benefits framework. There are two independent requirements in this framework. First, host countries are allowed to trade off health benefits to address current health problems against health benefits for future populations in terms of access to potential interventions. Second, the total amount of benefits received

will have to be fair, according to an acceptable theory of justice. What that level is—and what the appropriate theory of justice is—is not specified in the way this framework has been developed so far. That, of course, is a weakness of the present framework. Clearly, one’s position in this controversy would affect how one views the recent trends toward outsourcing of clinical research. If one takes the position that the product developed must be relevant to the health-care priorities of the host countries, one would probably be reluctant to approve studies outsourced to low cost countries for products that are primarily aimed at rich country markets. If, on the other hand, it is at least in principle justified to do studies in a country that are primarily intended for the health needs of other countries, then it becomes a matter of determining whether the host country benefit from the research is large enough and fair, by some account of justice. 3 CONCLUSION: LOCAL VERSUS UNIVERSAL All of the controversies we have examined seem to involve one underlying theme. Although all of us can agree that we have a responsibility to do something about the unjust and huge disparities in access to health care, there is disagreement about how one should go about doing something to change the current state of affairs. Some argue that, because we cannot do much about the underlying injustice, we should do the best we can within the current framework to produce some improvement. On the basis of this conviction, we could accept a local standard of care in a clinical trial to produce results for the common good, given current conditions. We might also accept a trial that may not yield a product that would be useful for the health priorities of the host country but would produce essential expertise not get easily obtained otherwise which in turn could be used to improve local conditions. Others, however, argue that we always have a direct responsibility to apply universal principles of justice; that we should always, in every action, try to change the underlying unjust structure, even if an alternative course of

ETHICAL ISSUES IN INTERNATIONAL RESEARCH

action would produce more benefits in the short term. In this view, we have an obligation to provide the best standard of care always, and we should only test interventions that can be directly useful for the population studied. Given the larger disagreements about underlying principles, it is not surprising that the ethical issues that arise in international research have generated so much controversy. 3.1.1 Acknowledgments. The opinions expressed are the authors’ own. They do not reflect any position or policy of the National Institutes of Health, U.S. Public Health Service, or Department of Health and Human Services. This research was supported by the Intramural Research Program of the NIH Clinical Center. REFERENCES 1. European and Developing Countries Clinical Trials Partnership Programme (EDCTP). Programme for Action: Accelerated Action on HIV/AIDS, Malaria and Tuberculosis in the Context of Poverty Reduction. June 16, 2003. Available at: http://ec.europa .eu/research/info/conferences/edctp/edctp en .html 2. S. Nundy and C. M. Gulhati, A new colonialism? Conducting clinical trials in India. N Engl J Med. 2005; 352: 1633–1636. 3. E. M. Conner, J. Sperling, and R. Gelber, Reduction of maternal-infant transmission of human immunodeficiency virus type 1 with zidovudine treatment. N Engl J Med. 1994; 331: 1173–1180. 4. H. Varmus and D. Satcher, Ethical complexities of conducting research in developing countries. N Engl J Med. 1997; 337; 1003–1005. 5. D. Wendler, E. J. Emanuel, and R. K. Lie, The standard of care debate: can research in developing countries be both ethical and responsive to those countries’ health needs? Am J Public Health. 2004; 94: 923–928. 6. P. Lurie and S. M. Wolf, Unethical trials of interventions to reduce perinatal transmission of the human immunodeficiency virus in developing countries. N Engl J Med. 1997; 337: 1377–1381.

7

7. M. Angell, The ethics of clinical research in the third world. N Engl J Med. 1997; 337: 847–849. 8. World Medical Association. Declaration of Helsinki: Ethical Principles for Medical Research Involving Human Subjects. September 10, 2004. Available at: http://www.wma .net/e/policy/b3.htm 9. R. K. Lie, E. J. Emanuel, C. Grady, and D. Wendler, The standard of care debate: the international consensus opinion versus the Declaration of Helsinki. J Med Ethics. 2004; 30: 190–193. 10. Council for International Organizations of Medical Sciences. International Ethical Guidelines for Biomedical Research Involving Human Subjects. Geneva: CIOMS. 2002. Available at: http://www.cioms.ch/frame guidelines nov 2002.htm 11. L. H. Glantz, G. J. Annas, M. A. Grodin, and W. K. Mariner, Research in developing countries. Taking ‘‘benefit’’ seriously. Hastings Center Report 1998; 28: 38–42. 12. J. Killen, C. Grady, G. K. Folkers, and A. C. Fauci, Ethics of clinical research in the developing world. Nat Rev Immunol. 2002; 2: 210–215. 13. L. Belsky and H. S. Richardson, Medical researchers’ ancillary clinical-care responsibilities. BMJ. 2004; 328: 1494–1496. 14. H. Richardson and L. Belsky, The ancillarycare responsibilities of medical researchers: an ethical framework for thinking about the clinical care that researchers owe their subjects. Hastings Center Report 2004; 34: 25–33. 15. N. Dickert, K. L. DeRiemer, P. Duffy, L. Garcia-Garcia, B. Sina, et al., Ancillary care responsibilities in observational research: two cases, two problems. Lancet. 2007; 369: 874–877. 16. Office of Extramural Research, National Institutes of Health. Guidance for Addressing the Provision of Antiretroviral Treatment for Trial Participants Following Their Completion of NIH-Funded HIV Antiretroviral Treatment Trials in Developing Countries. March 16, 2005. Available at: http://grants.nih.gov/ grants/policy/antiretroviral/index.htm 17. Participants in the 2001 Conference of Ethical Aspects of Research in Developing Countries. Fair benefits for research in developing countries. Science, 2002; 298: 2133–2134.

8

ETHICAL ISSUES IN INTERNATIONAL RESEARCH

18. Participants in the 2001 Conference of Ethical Aspects of Research in Developing Countries. Moving beyond reasonable availability to fair benefits for research in developing countries. Hastings Center Report 2004; 34: 17–27. 19. A. London, Justice and the human development approach to international research. Hastings Center Report 2005; 35: 24–37.

EUROPEAN MEDICINES AGENCY (EMEA)

reports suggest changes to the benefit–risk balance of a medicinal product. For veterinary medicinal products, the Agency has the responsibility to establish safe limits for medicinal residues in food of animal origin. The Agency also has a role to stimulate innovation and research in the pharmaceutical sector. The EMEA gives scientific advice and protocol assistance to companies for the development of new medicinal products. It publishes guidelines on quality, safety, and efficacy testing requirements. A dedicated office established in 2005 provides special assistance to small and medium-sized enterprises (SMEs). In 2001, the Committee for Orphan Medicinal Products (COMP) was established, charged with reviewing designation applications from persons or companies who intend to develop medicines for rare diseases (orphan drugs). The Committee on Herbal Medicinal Products (HMPC) was established in 2004 and provides scientific opinions on traditional herbal medicines. The Agency is also involved in referral procedures relating to medicinal products that are approved or under consideration by member states. The Agency brings together the scientific resources of over 40 national competent authorities in 30 EU and European Environment Agency’s (EEA) European Free Trade Association (EFTA) countries in a network of over 4000 European experts. It contributes to the EU’s international activities through its work with the European Pharmacopoeia, the World Health Organization, and the International Conference on Harmonization (ICH) and International Cooperation on Harmonization of Technical Requirements for Registration of Veterinary Medicinal Products (VICH) trilateral (the EU, Japan, and the U.S.) conferences on harmonization, among other international organizations and initiatives. The EMEA is headed by the executive director and has a secretariat of about 440 staff members in 2007. The management board is the supervisory body of the EMEA, which is responsible in particular for budgetary matters. The EMEA’s mission

The European Medicines Agency (EMEA) is a decentralized body of the European Union (EU) with headquarters in London. Its main responsibility is the protection and promotion of public and animal health through the evaluation and supervision of medicines for human and veterinary use. The EMEA is responsible for the scientific evaluation of applications for European marketing authorization for medicinal products. Under the centralized procedure, companies submit one single marketing authorization application to the EMEA. All medicinal products for human and animal use derived from biotechnology and other high-technology processes must be approved via the centralized procedure. The same applies to all human medicines intended for the treatment of human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS), cancer, diabetes, or neurodegenerative diseases and for all designated orphan medicines intended for the treatment of rare diseases. Similarly, all veterinary medicines intended for use as performance enhancers to promote the growth of treated animals or to increase yields from treated animals have to go through the centralized procedure. For medicinal products that do not fall under any of the above-mentioned categories companies can submit an application for a centralized marketing authorization to the EMEA, provided that the medicinal product constitutes a significant therapeutic, scientific, or technical innovation or the product is in any other respect in the interest of patient or animal health. The safety of medicines is monitored constantly by the Agency through a pharmacovigilance network. The EMEA takes appropriate actions if adverse drug reaction This article was modified from the website of the European Medicines Agency (http://www.emea. europa.eu/mission.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

EUROPEAN MEDICINES AGENCY (EMEA)

statement is, in the context of a continuing globalization, to protect and promote public and animal health by • Developing efficient and transparent

procedures to allow rapid access by users to safe and effective innovative medicines and to generic and nonprescription medicines through a single European marketing authorization. • Controlling the safety of medicines for humans and animals, in particular through a pharmacovigilance network and the establishment of safe limits for residues in food-producing animals. • Facilitating innovation and stimulating research and thereby contributing to the competitiveness of EU-based pharmaceutical industry. • Mobilizing and coordinating scientific resources from throughout the EU to provide high-quality evaluation of medicinal products; advise on research and development programs; perform inspections to ensure that fundamental Good Clinical Practice (GCP), Good Manufacturing Practice (GMP), and Good Laboratory Practice (GLP) provisions are consistently achieved; and provide useful and clear information to users and health-care professionals.

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC)

research achievements that have continued to improve the lives of cancer patients around the world. EORTC research has contributed to improvements in survival for testicular cancer (2,3), childhood leukemia and lymphoma (4–6), adult Hodgkin’s and non-Hodgkin’s lymphoma (7–9), as well as gastrointestinal stromal tumors (10,11). EORTC research has defined the standards of care for patients with glioblastoma (12,13), melanoma (14,15), breast (16–18), and colorectal (19,20) cancer. Organ preservation as well as minimal and nonmutilating surgeries is now considered safe and equally effective for patients with breast (21,22) and larynx (23) cancer. Improved treatments for patients with severe and life-threatening cancer therapy-related infections have been established (24–26). EORTC studies that evaluate high-precision 3-dimensional conformal radiation therapy techniques (27) and innovative multidisciplinary and complex therapeutic strategies have also impacted current treatment approaches. EORTC research has also led to the development of various diagnostic and treatment guidelines (28,29), as well as disease (30–32), treatment (33), and quality of life (34) prognostic calculators to aid clinicians in the treatment decision-making process. These achievements are the result of large, multinational clinical trials carried out by the EORTC network. Today, several thousands of patients are entered each year into EORTC studies and an additional 30,000 patients continue to be followed on a yearly basis (Fig. 1). The EORTC clinical study database, which is housed at the EORTC headquarters, now contains outcome data for over 160,000 cancer patients. The EORTC currently has 56 ongoing clinical trials actively enrolling patients. In the past 5 years, a greater shift toward clinico-genomic cancer research has led to a significant increase in the number of newly initiated EORTC studies that incorporate a complex translational research component. One such study that has the potential to alter the future approach to the treatment

MARTINE PICCART-GEBHARDT FRANCOISE MEUNIER EORTC, Brussels, Belgium

1 BACKGROUND 1.1 History of the EORTC The eminent Belgian cancer expert, Professor Henri Tagnon (1) along with other European pioneers, founded the European Organization for Research and Treatment of Cancer (EORTC) in 1962. This medical pioneer and former head of Medicine at the Institut Jules Bordet brought together leading specialists from major European cancer research institutes and hospitals to create the Groupe Europ´een de Chimioth´erapie Anticanc´ereuse (GECA). The GECA was established as a coordinated effort to conduct large-scale clinical studies, a need already recognized and beyond the capacity of any individual hospital or institution. Futuristic for the time, GECA quickly developed into a pan-European multidisciplinary cancer research organization with many early research successes; in 1968, GECA was renamed the European Organization for Research and Treatment of Cancer. Today, the EORTC is a leading international nonprofit academic cancer research organization duly registered under Belgian Law. The EORTC formal governing body leads an extensive pan-European network of collaborating cancer researchers, scientists, clinicians, affiliated institutions, and academic centers. 1.2 Landmark Research Over the past 45 years, the EORTC has continued to work in the spirit of its pioneering founders. Under the leadership of 14 consecutively elected presidents (Table 1), each of whom is a cancer research visionary in their own right, the EORTC has achieved multiple

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC) Table 1. EORTC Presidents EORTC Presidents 1962– 2009 ´ Georges MATHE, GARATTINI, Silvio VAN BEKKUM, Dirk TAGNON, Henri LATJA, Lazlo George SCHMIDT, Carl Gottfried VERONESI, Umberto DENIS, Louis VAN DER SCHUEREN, Emmanuel MCVIE, Gordon HORIOT, Jean-Claude VAN OOSTEROM, Allan T. EGGERMONT, Alexander M.M PICCART, Martine

Villejuif, Frances Milan, Italy Rijswijk, The Netherlands Brussels, Belgium Manchester, United Kingdom Essen, Germany Milan, Italy Antwerp, Belgium Beulven, Belgium London, United Kingdom Dijon, France Leuven, Belgium Rotterdam, The Netherlands Brussels, Belgium

1962–1965 1965–1968 1969–1975 1975–1978 1979–1981 1981–1984 1985–1988 1988–1991 1991–1994 1994–1997 1997–2000 2000–2003 2003–2006 2006–2009

Figure 1. EORTC clinical studies patient accrual from 2004 to 2006.

of breast cancer is the ‘‘microarray in node negative disease may avoid chemotherapy’’ or MINDACT study, which will evaluate 6000 women with early breast cancer (35). The primary objective of the MINDACT trial is to confirm that patients with a ‘‘low risk’’ molecular prognosis and ‘‘high risk’’ clinical prognosis can be safely spared chemotherapy without affecting distant metastases-free survival. The two other main objectives of the study address questions related to adjuvant treatment of breast cancer. MINDACT will compare

anthracycline-based chemotherapy regimens to a docetaxel-capecitabine regimen, which is possibly associated with increased efficacy and reduced long-term toxicities. MINDACT will also compare the efficacy and safety of 7 years single agent letrozole to those of the sequential strategy of 2 years of tamoxifen followed by 5 years of letrozole. The EORTC continues to grow and adapt innovative cancer research technologies, multidisciplinary strategies, and data management methodologies. Additional advances in the treatment of cancer patients will be

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC)

achieved primarily through translational research, efficient drug development, and the conduct of large, prospective randomized, multicenter clinical trials that aim to establish state-of-the-art treatment strategies and to impact daily clinical practice. After 45 years, the EORTC is ever more dedicated to improving and saving the lives of cancer patients. 1.3 The EORTC Mission The EORTC develops, conducts, coordinates, and stimulates high-quality translational and clinical research that will improve the standards of treatment for cancer patients in Europe. This result is achieved through the development of new drugs and other innovative approaches and through the testing of more effective therapeutic strategies, using currently approved drugs, surgery, and/or radiotherapy in clinical studies. The established EORTC research network facilitates and ensures the timely passage of experimental discoveries into state-of-the-art treatments by reducing the time for research breakthroughs to reach clinical use. 1.4 EORTC Funding The EORTC conducts cancer research that is primarily academic. Core independent funding is provided by the EORTC Charitable Trust, which is supported by several national cancer leagues. This organization includes cancer leagues from Denmark, Germany, United Kingdom, The Netherlands, France, Norway, Switzerland, Sweden, Belgium, Hong Kong, and Italy, as well as the ‘‘Fonds Cancer,’’ the U.S. National Cancer Institute (since 1972), private donations, and the Belgian National Lottery. The European Commission awards grants for specific research projects. The pharmaceutical industry provides mainly unrestricted educational grants and occasionally full sponsorship for EORTCled studies that develop and evaluate new agents. 1.4.1 The EORTC Charitable Trust. In 1976, the EORTC Foundation (now called The EORTC Charitable Trust) was established by Royal Decree under the laws of the

3

Kingdom of Belgium as an international association. The specific aim of the Trust is to provide financial support for EORTC activities, the peer-reviewed organizational structure, and independent academic research projects. National Cancer Charities are the major contributors to the Trust; all major European National Cancer Charities that support the EORTC through the Charitable Trust are represented in the Charitable Trust General Assembly, along with the Hong Kong Cancer Fund and several distinguished lay members. The Honorary President of the Trust is H.R.H. Princess Astrid of Belgium, Sir Christopher Mallaby serves as the Chairman, and Sir Ronald Grierson, Past-Chairman, is the Honorary Vice-President. 1.4.2 The Academic Research Fund. The Academic Fund, operational since 2005, supports selected clinical trials for which insufficient or a lack of funding exists. These trials must be of academic excellence, include a translational research component, provide for additional research networking, and have the potential to establish a new standard of care and thus change clinical practice. The EORTC Board oversees the selection process and the dispersing of funds. Five landmark studies with the potential to redefine current standards of cancer care have been approved for funding. All of these studies aim to advance the standards of cancer care by defining a ‘‘tailored’’ approach to the treatment of specific types and stages of different cancers. The SUPREMO breast cancer study (selective use of postoperative radiotherapy after mastectomy) is a United Kingdom-led intergroup clinical trial that also involves the EORTC (EORTC Study 22051-10052). It will evaluate the benefits and risks of radiotherapy after mastectomy in 3700 patients in an attempt to help clinicians better predict which patients require radiotherapy and which might be spared this treatment (36). The EORTC, together with the groupe d’etudes de lymphomes de l’adulte (GELA), are conducting the H10 EORTC-GELA early Hodgkin’s Lymphoma study (EORTC Study 20051). This study will evaluate the use of fludeoxglucose F 18 positron emission tomography scan-guided therapy in 1576 patients

4

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC)

to predict treatment response more accurately and therefore aid in selecting the most appropriate therapy for each patient (37). The Interfant-06 Study Group, which is an infant leukemia international collaborative group, is conducting a randomized clinical trial (EORTC 58051) to evaluate 445 infants under the age of 1 year with acute lymphoblastic or biphenotypic leukemia (38), whereas the EORTC 22043-30041 prostate cancer study will evaluate the use of postoperative hormone and radiation therapy compared with radiation therapy alone (39). The EORTC is collaborating with the Federation Nationale des Centres de Lutte contre le Cancer and the Intergroupe Francophone de Canc´erologie Thoracique in conducting the Lung Adjuvant RadioTherapy or LungART study (EORTC 22055-08053), which will evaluate the role of postoperative conformal thoracic radiotherapy after complete resection in 700 patients with lymph node positive non-small-cell lung cancer (40).

2 ORGANIZATIONAL STRUCTURE The EORTC General Assembly, Board, and Executive Committee provide leadership and strategic direction to the EORTC. The EORTC Network, which is composed of more than 2500 clinicians and scientists who are employed at over 200 institutions in 30 countries, carries out all EORTC laboratory and clinical research. Various committees provide oversight for the research activities of this complex network. The EORTC Headquarters ensures the overall coordination and daily management of all EORTC activities and related scientific, legal, and administrative issues (Fig. 2). 2.1 The General Assembly The General Assembly is the legislative body of the EORTC. It meets once a year to ratify policies, proposals, and other activities, and to delegate specific tasks to the EORTC Board, committees, or appointed persons. The General Assembly is composed of voting and non-voting EORTC members and includes

Figure 2. EORTC organizational structure.

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC)

the current and past 3 EORTC Presidents as well as the chairmen of all EORTC groups, task forces, committees, and divisions. Representatives from the top 15 clinical trialaccruing institutions have full general assembly membership. The General Assembly elects a new Board every 3 years.

2.2 The Board The Board is the steering and executive body that advises the General Assembly on all new activities and formulates proposals to be ratified by the General Assembly. The Board meets at least twice a year and consists of 21 elected voting members and several exofficio members. The voting members select from among themselves the EORTC President, Vice-President, Treasurer, and Secretary General.

2.3 The Executive Committee The Executive Committee, which was created in 1991, provides support to the EORTC President in the decision-making and strategyplanning process. It reports to the Board and consists of several voting members of

5

the Board plus the non-voting EORTC Director General. This committee meets approximately every 6 weeks and communicates by other means on a weekly basis. 2.4 The EORTC Research Network All EORTC research activities fall under the divisions of laboratory research and/or clinical research; scientists and clinicians, who participate on a voluntary basis in EORTC research, do so within expertise and diseasespecific EORTC groups or task forces that belong to these two main divisions (Table 2). This intricate research network provides an integrated structure for developing, evaluating and validating new cancer agents and multimodality therapeutic strategies through the full spectrum of early phase I studies to large-scale phase III confirmative clinical trials. The Laboratory Research Division focuses on the preclinical evaluation of new anticancer agents, tumor markers, and receptors, and supports translational research, pathology, pharmacology, and functional-imaging projects conducted within the EORTC. It fosters and strengthens direct interaction between basic cancer research and the EORTC clinical activities. A broader objective

Table 2. EORTC Groups and Tasks Forces EORTC Groups EORTC Brain Tumour Group EORTC Breast Cancer Group EORTC Children’s Leukaemia Group EORTC Gastro-intestinal Tract Cancer Group EORTC Genito-Urinary Tract Cancer Group EORTC Gynaecological Cancer Group EORTC Head and Neck Cancer Group EORTC Infectious Diseases Group EORTC Leukemia Group EORTC Lung Cancer Group EORTC Lymphoma Group EORTC Melanoma Group EORTC PathoBiology Group EORTC Pharmacology and Molecular Mechanisms Group EORTC Quality of Life Group EORTC Radiation Oncology Group EORTC Soft Tissue and Bone Sarcoma Group EORTC Task Forces EORTC Cancer in Elderly Task Force EORTC Cutaneous Lymphoma Task Force

6

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC)

of this division is to promote translational research projects and closer cooperation with the EORTC clinical groups. The Clinical Research Division (CRD) is responsible for proposing, designing, and conducting clinical trials to evaluate new therapeutic agents or novel treatment strategies through organized cooperative groups and task forces. Each of the 17 EORTC cooperative groups and the two EORTC task forces is disease-site specific or focuses on a cancerrelated discipline, such as radiotherapy or quality of life, and has an elected voting chairman on the General Assembly. 2.5 EORTC Committees Several committees oversee the independence of the EORTC, as well as the relevance and scientific value of all research efforts, which thereby ensures the highest quality of scientific work possible. Each committee chairman serves a 3-year elected term. The New Drug Advisory Committee (NDAC) was created to facilitate the drug acquisition process for EORTC research projects. It supports the CRD groups and provides recommendations on strategy and drug-development prioritization within the EORTC Network. NDAC members participate in advisory boards for the pharmaceutical industry, stimulate interaction between drug developers and the EORTC diseaseoriented groups, and advise the EORTC headquarters specialists on methodology issues that relate to innovative agents. This committee interacts with the early project optimization department (see below). The Translational Research Advisory Committee supports and provides expert advice on translational research projects conducted within the EORTC both from a scientific and practical perspective. This practice ensures the relevance and scientific soundness of translational research projects and the subsequent results. It increases the scientific visibility of the EORTC and guarantees, as an advisory committee, a permanent EORTC forum between the Clinical and Laboratory Research Divisions. This function serves to foster translational research interest within the Clinical Research Division Groups and also supports the inclusion of

translational research project ‘‘sub-studies’’ within EORTC clinical trial protocols. The Protocol Review Committee (PRC) reviews and approves all new clinical trial protocols submitted by the EORTC groups with respect to their scientific value and relevance within the EORTC strategy and framework. The PRC also assists the cooperative groups with any aspect of trial design or implementation when required. The Scientific Audit Committee gives independent advice to the EORTC Board on the activities, overall strategies, and achievements of all EORTC groups. These comments include recommendations on a series of criteria such as conformity with EORTC structure and policies, interaction between divisions and between groups, scientific output, and publications, in addition to overall priorities and strategies for future research. The Quality Assurance Committee develops quality assurance guidelines and guarantees that adequate quality-assurance control mechanisms operate in all EORTC divisions and groups in an effort to improve the quality of EORTC research and data. These activities include the supervision of quality-assurance activities within the EORTC headquarters and the advice it provides to the Board in the event of misconduct or suspicion of fraud. The Independent Data Monitoring Committee (IDMC) reviews the status of EORTC clinical trials and provides recommendations to the CRD groups concerning the continuation, modification, and/or publication of the trial. The IDMC is involved in all phase III clinical trials in which formal interim analyses and early stopping rules are anticipated. Ad hoc expert advice is provided to the committee on a specific per protocol basis. The Membership Committee reviews applications from potential new EORTC members and forwards them to the relevant Group Chairman for consideration. New memberships are approved on a continuous basis with endorsement by the EORTC Board occurring once a year. 2.6 The EORTC Headquarters The EORTC Headquarters, which is located in Brussels, Belgium, was established in 1974 to coordinate all EORTC scientific activities

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC)

7

Figure 3. EORTC headquarters.

and related legal and administrative issues. The Headquarters Director General implements all EORTC strategies and policies as defined by the Board and leads a staff of 160 professionals who represent 15 nationalities. The EORTC Headquarters team works within a matrix organization of cross-functional departments, offices, and units to provide full logistic and scientific support for all EORTC clinical trials and translational research projects (Fig. 3). The Headquarters staff consists of medical doctors, statisticians, data managers, quality of life specialists, healthcare professionals, computer specialists, research fellows, and administrative personnel. The Scientific Director oversees all medical and scientific activities related to prospective, newly proposed, and ongoing EORTC clinical trials. The Director of Methodology-Operations manages all departments and units involved in the operational aspects of clinical trial data acquisition, analysis, and quality assurance. This job duty includes ensuring that all EORTC trials are appropriately designed

from a methodological and statistical perspective. The EORTC staff carries out specific research projects in the fields of statistics, quality of life, quality assurance, and information technology. Between the years 2000 and 2006, the EORTC investigators and Headquarters team published a total of 472 peer-reviewed medical and scientific papers. The Medical Department clinical research physicians coordinate all research activities of the EORTC groups and act in a liaison function between the groups and the various Headquarters staff, departments, and units. One such unit, the Pharmacovigilance Unit, provides expertise for the reporting, analysis and handling of serious adverse events that occur during EORTC clinical trials. These safety-related advisory and regulatory activities are conducted according to GCP guidelines and current national and European regulations. The Translational Research Unit promotes and facilitates research activities that transform the latest discoveries in cancer molecular biology and cellular pathology into

8

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC)

clinical applications that benefit cancer patients. The future platform for such projects will be facilitated by the EORTC centralized tumor bio-bank, which is a collection of tissue samples, pathology, and radiology images from patients enrolled in EORTC clinical trials. This unit works in close cooperation with the Early Project Optimization Department (EPOD), which provides assistance to all EORTC Groups in the optimization, strategic development, and proposal process of new clinical-translational research projects. The EPOD provides the clinical groups with support to mature their proposals and/or strategies and coordinate their interaction with industry. This process aims to accelerate and increase the success rate of new cancer agent development by carefully analyzing promising discoveries and preferentially accelerating these into clinical evaluation. Additional specialized departments within the EORTC headquarters address specific areas and issues related to the development and conduct of cancer clinical trials. The Statistics Department not only performs all standard data analyses for EORTC clinical trials but also actively works in codeveloping new methods of data analyses applied to translational research and many other projects on methodology, meta-analyses, and prognostic factors. The Regulatory Affairs Unit fulfils all legal requirements for the initiation and conduct

of EORTC clinical trials according to the regulations of each participating country. The Regulatory Affairs and Pharmacovigilance Units act in close collaboration for expediting reports of serious adverse events to the competent authorities according to legal requirements. The Quality of Life Department works to stimulate, enhance, and coordinate quality of life research in cancer clinical trials by cooperating with the various EORTC study groups and the Quality of Life Group. It also manages the administrative aspects associated with the use of EORTC Qualityof-Life questionnaires in both EORTC and non-EORTC studies. This department has participated in the design to final analysis and publication of quality-of-life outcome results derived from more than 120 EORTC trials. The Protocol Development Department coordinates the efforts of the Headquarters study team and the EORTC Group Study Coordinator in the development and finalization of EORTC clinical trial protocols. The Study Coordinator is responsible for protocol development and writing of the medical sections as well as the Protocol Review Committee approval process. The Headquarters study team statistician, clinical research physician and data manager author all methodological and administrative sections of the protocol, perform final protocol review, and ensure that the process occurs within the shortest possible time.

Figure 4. EORTC headquarters core activities.

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC)

The Information Technology (IT) Department develops new technologies to facilitate and accelerate cancer clinical research. Inhouse designed specific computer software now allows for 24 hour/7 days a week on-line web-based patient registration and randomization for EORTC studies. Clinical trial data management with automatic consistency and crosschecks plus overdue information tracking is also now possible. Additional software was developed for the transfer and storage of pathology and radiology images and the transfer of information to EORTC partners via database export. Emerging technologies are revised regularly and integrated into the development strategy of the EORTC Headquarters. The IT system design reduces to a minimum any type of human error in the management of clinical trial data and is based on client/server architecture. ‘‘Vista RDC’’ (41) is the latest EORTC IT application that allows investigators to transmit data electronically. The EORTC Fellowship Programme, which was established in 1991, is offered to medical doctors, biostatisticians, computer analysts, and other scientists. The program promotes training in the methodology of highquality cancer clinical trials and fosters specific research projects on EORTC clinical trials, quality assurance, and quality of life. After completing their period of fellowship study and returning home, EORTC fellows serve as ambassadors by stimulating and maintaining new generations of European cancer researchers. All fellowship research is based on an international platform, which also helps expedite the diffusion of clinical trial study results. Scientific support is provided both by the EORTC Headquarters staff, and supervision is available from members of the EORTC disease/specialty-oriented groups. The EORTC Headquarters has awarded 99 research fellowships in the past 16 years to fellows from countries within the European Union, Argentina, Australia, Brazil, and Canada, which totals 150 years of cancer research. The EORTC Communications Office disseminates up-to-date EORTC information on a regular basis to the international scientific community, European Cancer Leagues, patients, and the public. It reports on the

9

initiation of major EORTC clinical trials as well as the results of EORTC research and its relevance to cancer patients. The Education Office provides organizational support for EORTC-sponsored courses (e.g., clinical trial methodology, statistics, data management, and quality of life), international EORTC conferences, and internal EORTC Network meetings. The European Union Programme Office, which was created in 2006, supports, optimizes, and ensures the participation, visibility, and cooperation of the EORTC in European Union (EU) institution activities. This office aids EORTC researchers in the preparation of proposals for EU project funding and subsequently provides support for the management of funded projects. The EORTC currently receives research funding under the European Commission sixth and seventh Research Framework Programs for translational research-related projects in the areas of breast cancer, melanoma, and sarcoma. The EORTC actively participates in the European Clinical Research Infrastructures Network and biotherapy facilities Preparation Phase for the Infrastructure project and the Impact in Clinical Research of European Legislation project. 3

SCIENTIFIC RESEARCH STRATEGY

The EORTC has undertaken organizational and strategic initiatives over the years to remain a world leader in academic cancer research. This flexibility allows the EORTC to meet the challenges presented by the changing environments of cancer research and technology, government policy, and public awareness. Researchers now work in the molecular biology ‘‘omic’’ era (e.g., genomics, proteomics, and metabolomics), that will increasingly enable physicians to predict treatment risk and responsiveness and therefore ‘‘tailor’’ or personalize cancer therapy. The recent groundbreaking discoveries in molecular biology, coupled with the expanding costs of clinical trials and increasing pressure from other cancer research groups have demanded additional refinement of the EORTC scientific strategy, project prioritization, and research funding. This refinement

10

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC)

will capitalize on the strengths of the organization, namely its vast range of expertise and cutting-edge research facility infrastructure. Despite these new challenges, the EORTC will continue to conduct large-scale clinical trials designed to improve the standards of cancer care, to increase the scientific understanding of cancer as a disease, and to address pivotal strategic therapeutic questions. The EORTC scientific and clinical research programs will be guided over the coming years by the establishment of new models for research collaboration in ‘‘niche’’ areas, the initiation of translational research-rich clinical trials for developing tailored targeted therapies, and the prioritization of highquality clinical trials that address specific scientific questions. 3.1 New Models of Collaboration The EORTC is seeking new ways to collaborate with national cancer research groups within Europe to address specific strategic research questions that involve multimodality cancer treatments (e.g., surgery, radiotherapy) and to ensure continued research of the less common types of cancer. These goals will be achieved through highly specialized, large phase III clinical trials conducted in ‘‘niche’’ areas of disease-specific and technical expertise. This approach involves the pooling of patient accrual at a pan-European level and allows for the study of rare cancers. It also permits the best use of nationally developed technical, scientific, and research expertise in nationally run clinical trials. 3.2 Translational Research-Rich Clinical Trials To meet the challenges of clinical cancer research in today’s molecular era, the EORTC established the Network of Core Institutions (NOCI) to promote, support, and conduct high-quality translational research-rich trials across the EORTC Groups. This strategy optimizes the transfer of basic research molecular discoveries to the development of targeted tailored cancer therapies in terms of time and quality by capitalizing on the broad research expertise and infrastructure

of large institutions located in various European countries. Innovative translational research concepts are prioritized and tested in early-phase multitumor line trials. Additional development proceeds based on early trial outcome results and prospectively streamlined for validation in disease-specific, large phase III confirmative EORTC trials. This method represents a new project-focused research strategy that is time, resource, and financially efficient while simultaneously providing a platform for collaboration with partners in the pharmaceutical industry. NOCI trials involve innovative complex study designs, elaborate translational research, and allow for the large-scale collection of blood and tissue samples or biospecimens. A centralized, independent EORTC tumor biobank facility provides for the storage and management of a representative selection of these biospecimens and is linked to an electronic patient treatment outcome database. The NOCI initiative represents a phenomenal potential and guarantees the future of independent academic clinico-genomic cancer research in Europe. 3.3 Prioritization of EORTC Clinical Trials The EORTC Board classifies clinical trials into three broad categories (Table 3). This classification system is used to determine the place of each new trial proposal within the overall EORTC scientific strategy. It also allows for the prioritization of resource and/or funding allocation. This categorization facilitates the selection and prospective development of early-phase multitumor clinico-genomic studies under the NOCI umbrella, as well as the development of ‘‘niche’’-specific and multimodality clinical trials, which thereby increases the success of discovering and developing new standards of care. The EORTC will stimulate this scientific strategy by awarding 1 million Euros in 2008 for academic and clinico-genomic trial research proposals that have the highest promise of defining new standards of cancer care. 4 INTERNATIONAL PARTNERSHIPS The EORTC collaborates with several international institutions, organizations, as-

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC)

11

Table 3. EORTC Clinical Trial Prioritization Category 1 1A Randomized phase III trials designed to answer a question which directly contributes to defining a new standard of care (e.g. Trials with a strong multidisciplinary component prepared and run jointly by different EORTC Groups). 1B Randomized phase III trial with a crucially important translational research component that may permit a fundamental advance in the understanding of a particular disease. Category 2 2A Intergroup randomized phase III trials is not lead by an EORTC group but corresponding to the criteria listed under category 1. Phase II randomized trials clearly designed as a preparatory work for a following randomized phase III. Registration trials with a clinically relevant question and a translational research component. 2B Phase Ib trials involving drugs with a novel mechanism of action and with commitment from the company for vertical drug development in the EORTC (including combination studies with radiatherapy and biological agents). Category 3 3A Randomized phase III trials that do not meet the requirements of the above mentioned categories. 3B Randomized phase II trials with no or a weak translational research component. Phase I and single arm phase II trials with drug having truly new mechanisms of action, but for which a plan for later development within the EORTC is lacking. 3C Other phase I and single arm phase II studies.

sociations, regulatory bodies, government policymakers, and public interest groups in relation to basic scientific and clinical research, educational activities, government policy initiatives, patient advocacy, and public health issues. 4.1 Research Partners The EORTC collaborates with the U.S. National Cancer Institute (NCI), the leading U.S. agency for cancer research and treatment on several research and educational projects. The NCI Liaison Office is located next to the EORTC Headquarters in Brussels. Agreements signed between the EORTC and the U.S. NCI facilitate collaboration in the development of new cancer treatments. Today, drugs are now developed to facilitate testing on either side of the Atlantic Ocean. As a result, common methods of compound acquisition, selection and screening, toxicology testing, and clinical evaluation are well established between the U.S. and Europe. The EORTC is actively involved in intergroup collaboration with both national and

international research groups. Transatlantic collaboration with U.S. and Canadian cooperative groups is ongoing. 4.2 Regulatory Agency Partners In 1998, the EORTC standard operating procedures were filed with the U.S. Food and Drug Administration (FDA) and received an EORTC Drug Master File Number (No. DMF 13059). This result places EORTC clinical trial data on par with data generated by the U.S. National Cancer Institute and its cooperative groups. EORTC research partners can therefore use the EORTC drug master file number as a reference when seeking U.S. approval for treatments tested in Europe. EORTC experts also serve as members of various European Medicines Evaluation Agency expert working groups to discuss and establish guidelines on the requirements for new anticancer agents submitted for market registration. The EORTC has both formal and informal contact with many national regulatory agencies for the submission of new

12

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC)

clinical studies and the exchange of researchrelated information. 4.3 Educational Partners International collaboration also takes the form of jointly organized research symposia and conferences. The most recent basic cancer research discoveries, technologies, and innovations are shared at the annual ‘‘Molecular Targets and Cancer Therapeutics’’ conference jointly organized by the U.S. NCI and the EORTC since 1986. In 2001, the American Association for Cancer Research joined the NCI–EORTC effort. Today, this conference is held annually and the location alternates between the U.S. and Europe. A second annual event, ‘‘Molecular Markers in Cancer’’ provides a forum for clinicians and scientists to discuss the clinical applications of the latest cancer research findings. As of 2007, it is jointly organized by the NCI, the EORTC, and the American Society of Clinical Oncology (ASCO). 4.4 European Journal of Cancer Collaboration The European Journal of Cancer (EJC) was created in 1965 by Professor H. Tagnon who served as editor-in-chief from 1963 until 1990. The EJC is the official journal of the EORTC, the European School of Oncology, the European Association for Cancer Research, the European Society of Mastology, and the European Cancer Organization. It is available in 18 countries within Europe and in 63 countries worldwide. The EJC publishes 18 journal issues annually, and several supplements are produced in the abstract books of major European cancer conferences. 5

FUTURE PERSPECTIVES

In recent years, numerous innovative agents have been discovered as a result of the tremendous development in the understanding of the molecular basis of cancer. More breakthroughs in the prevention, diagnosis, and treatment of cancer will be accomplished primarily through translational research projects, efficient drug development, and the conduct of translational research-guided, large prospective randomized multicenter

clinical trials. These trials will require a global cooperative approach coupled with an early and open exchange of research findings. Significant improvements in the survival of cancer patients will depend on the ability of researchers to discover new drugs and novel treatment strategies tailored to the individual patient and to specific characteristics of their disease. The EORTC has been a leader in cancer research for over 45 years and has undertaken major steps to create the infrastructure and organization needed for this continued challenge throughout the twenty-first century. REFERENCES 1. F. Meunier and A. T. van Osterom, 40 years of the EORTC: The evolution towards a unique network to develop new standards of cancer care. Eur. J. Cancer 2002; 38: 3–13. 2. R. De Wit, G. Stoter, S. B. Kaye, D. T. Steijffer, G. W. Jones, W. W. Ten Bokkel Huinink, L. A. Rea, L. Collette, and R. Sylvester, The importance of bleomycin in combination chemotherapy for good prognosis testicular non-seminoma: a randomized study of the EORTC Genitourinary Tract Cancer Cooperative Group. J. Clin. Oncol. 1997; 15: 1837–1843. 3. R. T. Oliver, M. D. Mason, G. M. Mead, H. von der Maase, G. J. Rustin, J. K. Joffe, R. de Wit, N. Aass, J. D. Graham, R. Coleman, S. J. Kirk, and S. P. Stenning, MRC TE19 collaborators and the EORTC 30982 collaborators. Radiotherapy versus single dose carboplatin in adjuvant treatment of stage I seminoma: a randomised trial. Lancet 2005; 366: 293–300. 4. F. Millot, S. Suciu, N. Philippe, Y. Benoit, F. Mazingue, A. Uyttebroeck, P. Lutz, F. Mechinaud, A. Robert, P. Boutard, et al., Children’s Leukemia Cooperative Group of the European Organization for Research and Treatment of Cancer, Value of High-Dose Cytarabine during interval therapy of a Berlin - Frankfurt - Munster based protocol in increased-risk Children with acute Lymphoblastic Lymphoma: Results of the European Organization for Research and Treatment of Cancer 58881 randomized phase III trial. J. Clin. Oncol. 2001; 19: 1935–1942. 5. N. Entz-Werle, S. Suciu, J. van der Werff ten Bosch, E. Vilmer, Y. Bertrand, Y. Benoit, G. Margueritte, E. Plouvier, P. Boutard,

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC) E. Vandecruys, et al., EORTC Children Leukemia Group, Results of 58872 and 58921 trials in acute myeloblastic leukemia and relative value of chemotherapy vs allogeneic bone marrow transplantation in first complete remission: the EORTC Children Leukemia Group report. Leukemia 2005; 19: 2072–2981. 6. J. van der Werff Ten Bosch, S. Suciu, A. Thyss, Y. Bertrand, L. Norton, F. Mazingue, A. Uyttebroeck, P. Lutz, A. Robert, P. Boutard, et al., Value of intravenous 6-mercaptopurine during continuation treatment in childhood acute lymphoblastic leukemia and non-Hodgkin’s lymphoma: final results of a randomized phase III trial (58881) of the EORTC CLG. Leukemia 2005; 19: 721–7226. 7. C. Ferm´e, H. Eghbali, J. H. Meerwaldt, C. Rieux, J. Bosq, F. Berger, T. Girinsky, P. Brice, M. B. van’t Veer, J. A. Walewski, et al. for the EORTC–GELA H8 Trial, Chemotherapy plus involved-field radiation in early-stage hodgkin’s disease. N. Engl. J. Med. 2007; 357: 1916–1927. 8. E. M. Noordijk, P. Carde, N. Dupouy, A. Hagenbeek, A. D. G. Krol, J. C. Kluin-Nelemans, U. Tirelli, M. Monconduit, J. Thomas, H. Eghbali, et al., Combinedmodality therapy for clinical stage I or II Hodgkin’s lymphoma: long-term results of the European Organisation for Research and Treatment of Cancer H7 Randomized controlled trials. J. Clin. Oncol. 2006; 24: 3128–3135. 9. M. H. van Oers, R. Klasa, R. E. Marcus, M. Wolf, E. Kimby, R. D. Gascoyne, A. Jack, M. Van’t Veer, A. Vranovsky, H. Holte, et al., Rituximab maintenance improves clinical outcome of relapsed/resistant follicular non-Hodgkin lymphoma in patients both with and without rituximab during induction: results of a prospective randomized phase 3 intergroup trial. Blood 2006; 108: 3295–3301. 10. J. Verweij, P. G. Casali, J. Zalcberg, A. LeCesne, P. Reichardt J. Y., Blay, R. Issels, A. van Oosterom, P. C. Hogendoorn, M. Van Glabbeke, et al., Progression-free survival in gastrointestinal stromal tumours with highdose imatinib: randomised trial. Lancet 2004; 364: 1127–1134. 11. M. Van Glabbeke, J. Verweij, P. G. Casali, J. Simes, A. Le Cesne, P. Reichardt, R. Issels, I. R. Judson, A. T. van Oosterom, J. Y. Blay Predicting toxicities for patients with advanced gastrointestinal stromal tumours treated with imatinib: a study of the European Organisation for Research and Treatment of Cancer, the Italian Sarcoma Group, and the

13

Australasian Gastro-Intestinal Trials Group (EORTC-ISG-AGITG). Eur. J. Cancer 2006; 42: 2277–2285. 12. R. Stupp, W. P. Mason, M. J. van den Bent, M. Weller, B. Fisher, M. J. Taphoorn, K. Belanger, A. A. Brandes, C. Marosi, U. Bogdahn, et al., European Organisation for Research and Treatment of Cancer Brain Tumor and Radiotherapy Groups, National Cancer Institute of Canada Clinical Trials Group. Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma. N. Engl. J. Med. 2005; 352: 987–996. 13. M. E. Hegi, A. C. Diserens, T. Gorlia, M. F. Hamou, N. de Tribolet, M. Weller, J. M. Kros, J. A. Hainfellner, W. Mason, L. Mariani, et al., MGMT gene silencing and benefit from temozolomide in glioblastoma. N. Engl. J. Med. 2005; 352: 997–1003. 14. A. M. Eggermont, S. Suciu, R. MacKie, W. Ruka, A. Testori, W. Kruit, C. J. Punt, M. Delauney, F. Sales, G. Groenewegen, et al., EORTC Melanoma Group, Post-surgery adjuvant therapy with intermediate doses of interferon alfa 2b versus observation in patients with stage IIb/III melanoma (EORTC 18952): randomised controlled trial. Lancet 2005; 366: 1189–1196. 15. V. Winnepenninckx, V. Lazar, S. Michiels, P. Dessen, M. Stas, S. R. Alonso, M. F. Avril, P. L. Ortiz Romero, T. Robert, O. Balacescu, et al., Melanoma Group of the European Organization for Research and Treatment of Cancer. Gene expression profiling of primary cutaneous melanoma and clinical outcome. J. Natl. Cancer Inst. 2006; 98: 472–482. 16. L. Morales, P. Canney, J. Dyczka, E. Rutgers, R. Coleman, T. Cufer, M. WelnickaJaskiewicz, J. Nortier, J. Bogaerts, P. Therasse, R. Paridaens, Postoperative adjuvant chemotherapy followed by adjuvant tamoxifen versus nil for patients with operable breast cancer: A randomised phase III trial of the EUROPEAN ORGANISATION FOR RESEARCH AND TREATMENT OF CANCER BREAST GROUP. Eur. J. Cancer 2007; 43: 331–340. 17. M. J. Piccart-Gebhart, M. Procter, B. LeylandJones, A. Goldhirsch, M. Untch, I. Smith, L. Gianni, J. Baselga, R. Bell, C. Jackisch, D. Cameron, et al., Herceptin Adjuvant (HERA) Trial Study Team. Trastuzumab after adjuvant chemotherapy in HER2-positive breast cancer. N. Engl. J. Med. 2005; 353: 1659–1672. 18. H. Bartelink, J.C. Horiot, P. M. Poortmans,

14

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC) H. Struikmans, W. Van den Bogaert, A. Fourquet, J. J. Jager, W. J. Hoogenraad, S. B. ´ am-Rodenhuis, ´ Oei, C. C. Warl M. Pierart, and L. Collette, Impact of a higher radiation dose on local control and survival in breast-conserving therapy of early breast cancer: 10-year results of the randomized boost versus no boost EORTC 22881-10882 trial. J. Clin. Oncol. 2007; 25: 3259–3265.

19. C. H. K¨ohne, E. van Cutsem, J. Wils, C. Bokemeyer, M. El-Serafi, M. P. Lutz, M. Lorenz, ¨ P. Reichardt, H. Ruckle-Lanz, N. Frickhofen, et al., European Organisation for Research and Treatment of Cancer Gastrointestinal Group. Phase III study of weekly high-dose infusional fluorouracil plus folinic acid with or without irinotecan in patients with metastatic colorectal cancer: European Organisation for Research and Treatment of Cancer Gastrointestinal Group Study 40986. J. Clin. Oncol. 2005; 23: 4856–4865. 20. B. Nordlinger, P. Rougier, J. P. Arnaud, M. Debois, J. Wils, J.C. Ollier, O. Grobost, P. Lasser, J. Wals, J. Lacourt, et al., Adjuvant regional chemotherapy and systemic chemotherapy versus systemic chemotherapy alone in patients with stage II-III colorectal cancer: a multicentre randomised controlled phase III trial. Lancet Oncol. 2005; 6: 459–468. 21. J. A. van Dongen, A. C. Voogd, I. S. Fentiman, C. Legrand, R. J. Sylvester, D. Tong, E. van der Schueren, P. A. Helle, K. van Zijl, and H. Bartelink, Long-term results of a randomized trial comparing breast-conserving therapy with mastectomy: European Organization for Research and Treatment of Cancer 10801 trial. J. Natl. Cancer Inst. 2000; 92: 1143–1150. 22. N. Bijker, P. Meijnen, J. L. Peterse, J. Bogaerts, I. Van Hoorebeeck, J. P. Julien, M. Gennaro, P. Rouanet, A. Avril, I. S. Fentiman, et al., Breast-conserving treatment with or without radiotherapy in ductal carcinomain-situ: ten-year results of European Organisation for Research and Treatment of Cancer randomized phase III trial 10853--a study by the EORTC Breast Cancer Cooperative Group and EORTC Radiotherapy Group. J. Clin. Oncol. 2006; 24: 3381–3387. 23. J. L. Lefebvre, D. Chevalier, B. Luboinski, A. Kirkpatrick, L. Collette, T. Sahmoud, Larynx preservation in pyriform sinus cancer: preliminary results of a European Organization for Research and Treatment of Cancer phase III trial. EORTC Head and Neck Cancer Cooperative Group. J. Natl. Cancer Inst.

1996; 88: 890–899. 24. R. Herbrecht, D. W. Denning, T. F. Patterson, J. E. Bennett, R. E. Greene, J. W. Oestmann, W. V. Kern, K. A. Marr, P. Ribaud, O. Lortholary, et al., Invasive Fungal Infections Group of the European Organisation for Research and Treatment of Cancer and the Global Aspergillus Study Group, Voriconazole versus amphotericin B for primary therapy of invasive aspergillosis. N. Engl. J. Med. 2002; 347: 408–415. 25. EORTC-International Antimicrobial Therapy Cooperative Group, Efficacy and toxicity of single daily doses of amikacin and ceftriaxone versus multiple daily doses of amikacin and ceftazidime for infection in patients with cancer and granulocytopenia. Ann. Intern. Med. 1993; 119: 584–593 26. C. Viscoli, A. Cometta, W. V. Kern, R. Bock, M. Paesmans, F. Crokaert, M. P. Glauser, T. Calandra, International Antimicrobial Therapy Group of the European Organization for Research and Treatment of Cancer, Piperacillin-tazobactam monotherapy in highrisk febrile and neutropenic cancer patients. Clin. Microbiol. Infect. 2006; 12: 212–216. 27. S. Senan, D. De Ruysscher, P. Giraud, R. Mirimanoff, and V. Budach, Radiotherapy Group of European Organization for Research and Treatment of Cancer, Literature-based recommendations for treatment planning and execution in high-dose radiotherapy for lung cancer. Radiother. Oncol. 2004; 71: 139–146. 28. A.P. van der Meijden, R. Sylvester, W. Oosterlinck, E. Solsona, A. Boehle, B. Lobel, and E. Rintala, for the EAU Working Party on Non Muscle Invasive Bladder Cancer, EAU guidelines on the diagnosis and treatment of urothelial carcinoma in situ. Eur. Urol. 2005; 48: 363–371. 29. M. S. Aapro, D. A. Cameron, R. Pettengell, J. Bohlius, J. Crawford, M. Ellis, N. Kearney, G. H. Lyman, V. C. Tjan-Heijnen, J. Walewski, et al., European Organisation for Research and Treatment of Cancer (EORTC) Granulocyte Colony-Stimulating Factor (G-CSF) Guidelines Working Party, EORTC guidelines for the use of granulocyte-colony stimulating factor to reduce the incidence of chemotherapy-induced febrile neutropenia in adult patients with lymphomas and solid tumours. Eur. J. Cancer 2006; 42: 2433–2453. 30. R. J. Sylvester, A. P. van der Meijden, W. Oosterlinck, J. A. Witjes, C. Bouffioux, L. Denis, D. W. Newling, and K. Kurth, Predicting recurrence and progression in individual patients with stage Ta T1 bladder cancer

EUROPEAN ORGANIZATION FOR RESEARCH AND TREATMENT OF CANCER (EORTC) using EORTC risk tables: a combined analysis of 2596 patients from seven EORTC trials. Eur. Urol. 2006; 49: 466–465; discussion 475–477. 31. M. Buyse, S. Loi, L. van’t Veer, G. Viale, M. Delorenzi, A. M. Glas, M. S. d’Assignies, J. Bergh, R. Lidereau, P. Ellis, et al., TRANSBIG Consortium, Validation and clinical utility of a 70-gene prognostic signature for women with node-negative breast cancer. J. Natl. Cancer Inst. 2006; 98: 1183–1192. 32. T. Gorlia, M. J. van den Bent, M. E. Hegi, R. O. Mirimanoff, M. Weller, J. G. Cairncross, E. Eisenhauer, K. Belanger, A. A. Brandes, A. Allgeier, et al., Nomograms for predicting survival of patients with newly diagnosed glioblastoma: prognostic factor analysis of EORTC and NCIC trial 26981-22981/CE.3. Lancet Oncol. 2008; 9: 29–38. 33. M. Van Glabbeke, J. Verweij, P. G. Casali, A. Le Cesne, P. Hohenberger, I. Ray-Coquard, M. Schlemmer, A. T. van Oosterom, D. Goldstein, R. Sciot, et al., Initial and late resistance to imatinib in advanced gastrointestinal stromal tumors are predicted by different prognostic factors: a European Organisation for Research and Treatment of Cancer-Italian

15

Sarcoma Group-Australasian Gastrointestinal Trials Group study. J. Clin. Oncol. 2005; 23: 5795–5804. 34. M. E. Mauer, M. J. Taphoorn, A. Bottomley, C. Coens, F. Efficace, M. Sanson, A. A. Brandes, C. C. van der Rijt, H. J. Bernsen, M. Fr´enay, et al., EORTC Brain Cancer Group, Prognostic value of health-related quality-oflife data in predicting survival in patients with anaplastic oligodendrogliomas, from a phase III EORTC brain cancer group study. J. Clin. Oncol. 2007; 25: 5731–5737. 35. MINDACT STUDY. Available: http://www. eortc.be/services/unit/mindact/MINDACT websiteii.asp. 36. SUPREMO STUDY. Available: http://www. supremo-trial.com. 37. H10 EORTC-GELA STUDY. Available: http://www.cancer.gov/clinicaltrials/EORTC20051. 38. INTERFANT 06 STUDY. Available: http:// www.trialregister.nl/trialreg/admin/rctview. asp?TC=695. 39. EORTC 22043-30041 PROSTATE STUDY. Available: http://groups.eortc.be/radio/future %20trials.htm. 40. LungART STUDY. Available: http://www. eortc.be/protoc/details.asp?protocol=22055. 41. VISTA RDC USER GUIDE. Available: http:// rdc.eortc.be/rdc/doc/VISTA-RDC UserGuide. pdf.

CROSS-REFERENCES Trials Oncology Drug development Human Genomics Quality of Life

Factor Analysis: Confirmatory Of primary import to factor analysis, in general, is the notion that some variables of theoretical interest cannot be observed directly; these unobserved variables are termed latent variables or factors. Although latent variables cannot be measured directly, information related to them can be obtained indirectly by noting their effects on observed variables that are believed to represent them. The oldest and best-known statistical procedure for investigating relations between sets of observed and latent variables is that of factor analysis. In using this approach to data analyses, researchers examine the covariation among a set of observed variables in order to gather information on the latent constructs (i.e., factors) that underlie them. In factor analysis models, each observed variable is hypothesized to be determined by two types of influences: (a) the latent variables (factors) and (b) unique variables (called either residual or error variables). The strength of the relation between a factor and an observed variable is usually termed the loading of the variable on the factor.

Exploratory versus Confirmatory Factor Analysis There are two basic types of factor analyses: exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). EFA is most appropriately used when the links between the observed variables and their underlying factors are unknown or uncertain. It is considered to be exploratory in the sense that the researcher has no prior knowledge that the observed variables do, indeed, measure the intended factors. Essentially, the researcher uses EFA to determine factor structure. In contrast, CFA is appropriately used when the researcher has some knowledge of the underlying latent variable structure. On the basis of theory and/or empirical research, he or she postulates relations between the observed measures and the underlying factors a priori, and then tests this hypothesized structure statistically. More specifically, the CFA approach examines the extent to which a highly constrained a priori factor structure is consistent with the sample data. In summarizing the primary distinction between the two methodologies, we can say

that whereas EFA operates inductively in allowing the observed data to determine the underlying factor structure a posteriori, CFA operates deductively in postulating the factor structure a priori [5]. Of the two factor analytic approaches, CFA is by far the more rigorous procedure. Indeed, it enables the researcher to overcome many limitations associated with the EFA model; these are as follows: First, whereas the EFA model assumes that all common factors are either correlated or uncorrelated, the CFA model makes no such assumptions. Rather, the researcher specifies, a priori, only those factor correlations that are considered to be substantively meaningful. Second, with the EFA model, all observed variables are directly influenced by all common factors. With CFA, each factor influences only those observed variables with which it is purported to be linked. Third, whereas in EFA, the unique factors are assumed to be uncorrelated, in CFA, specified covariation among particular uniquenesses can be tapped. Finally, provided with a malfitting model in EFA, there is no mechanism for identifying which areas of the model are contributing most to the misfit. In CFA, on the other hand, the researcher is guided to a more appropriately specified model via indices of misfit provided by the statistical program.

Hypothesizing a CFA Model Given the a priori knowledge of a factor structure and the testing of this factor structure based on the analysis of covariance structures, CFA belongs to a class of methodology known as structural equation modeling (SEM). The term structural equation modeling conveys two important notions: (a) that structural relations can be modeled pictorially to enable a clearer conceptualization of the theory under study, and (b) that the causal processes under study are represented by a series of structural (i.e., regression) equations. To assist the reader in conceptualizing a CFA model, I now describe the specification of a hypothesized CFA model in two ways; first, as a graphical representation of the hypothesized structure and, second, in terms of its structural equations.

Graphical Specification of the Model CFA models are schematically portrayed as path diagrams (see Path Analysis and Path Diagrams) through the incorporation of four geometric symbols: a circle (or ellipse) representing unobserved

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

2

Factor Analysis: Confirmatory

1.0

Physical SC (Appearance) F1

1.0 Physical SC (Ability) F2

1.0 Social SC (Peers) F3

1.0 Social SC (Parents) F4

Figure 1

Hypothesized CFA model

SDQ1

E1

SDQ8

E8

SDQ15

E15

SDQ22

E22

SDQ38

E38

SDQ46

E46

SDQ54

E54

SDQ62

E62

SDQ3

E3

SDQ10

E10

SDQ24

E24

SDQ32

E32

SDQ40

E40

SDQ48

E48

SDQ56

E56

SDQ64

E64

SDQ7

E7

SDQ14

E14

SDQ28

E28

SDQ36

E36

SDQ44

E44

SDQ52

E52

SDQ60

E60

SDQ69

E69

SDQ5

E5

SDQ19

E19

SDQ26

E26

SDQ34

E34

SDQ42

E42

SDQ50

E50

SDQ58

E58

SDQ66

E66

Factor Analysis: Confirmatory latent factors, a square (or rectangle) representing observed variables, a single-headed arrow (−>) representing the impact of one variable on another, and a double-headed arrow (<−>) representing covariance between pairs of variables. In building a CFA model, researchers use these symbols within the framework of three basic configurations, each of which represents an important component in the analytic process. We turn now to the CFA model presented in Figure 1, which represents the postulated four-factor structure of nonacademic self-concept (SC) as tapped by items comprising the Self Description QuestionnaireI (SDQ-I; [15]). As defined by the SDQ-I, nonacademic SC embraces the constructs of physical and social SCs. On the basis of the geometric configurations noted above, decomposition of this CFA model conveys the following information: (a) there are four factors, as indicated by the four ellipses labeled Physical SC (Appearance; F1), Physical SC (Ability; F2), Social SC (Peers; F3), and Social SC (Parents; F4); (b) the four factors are intercorrelated, as indicated by the six two-headed arrows; (c) there are 32 observed variables, as indicated by the 32 rectangles (SDQ1SDQ66); each represents one item from the SDQ-I; (d) the observed variables measure the factors in the following pattern: Items 1, 8, 15, 22, 38, 46, 54, and 62 measure Factor 1, Items 3, 10, 24, 32, 40, 48, 56, and 64 measure Factor 2, Items 7, 14, 28, 36, 44, 52, 60, and 69 measure Factor 3, and Items 5, 19, 26, 34, 42, 50, 58, and 66 measure Factor 4; (e) each observed variable measures one and only one factor; and (f) errors of measurement associated with each observed variable (E1-E66) are uncorrelated (i.e., there are no double-headed arrows connecting any two error terms. Although the error variables, technically speaking, are unobserved variables, and should have ellipses around them, common convention in such diagrams omits them in the interest of clarity. In summary, a more formal description of the CFA model in Figure 1 argues that: (a) responses to the SDQ-I are explained by four factors; (b) each item has a nonzero loading on the nonacademic SC factor it was designed to measure (termed target loadings), and zero loadings on all other factors (termed nontarget loadings); (c) the four factors are correlated; and (d) measurement error terms are uncorrelated.

3

Structural Equation Specification of the Model From a review of Figure 1, you will note that each observed variable is linked to its related factor by a single-headed arrow pointing from the factor to the observed variable. These arrows represent regression paths and, as such, imply the influence of each factor in predicting its set of observed variables. Take, for example, the arrow pointing from Physical SC (Ability) to SDQ1. This symbol conveys the notion that responses to Item 1 of the SDQ-I assessment measure are ‘caused’ by the underlying construct of physical SC, as it reflects one’s perception of his or her physical ability. In CFA, these symbolized regression paths represent factor loadings and, as with all factor analyses, their strength is of primary interest. Thus, specification of a hypothesized model focuses on the formulation of equations that represent these structural regression paths. Of secondary importance are any covariances between the factors and/or between the measurement errors. The building of these equations, in SEM, embraces two important notions: (a) that any variable in the model having an arrow pointing at it represents a dependent variable, and (b) dependent variables are always explained (i.e., accounted for) by other variables in the model. One relatively simple approach to formulating these structural equations, then, is first to note each dependent variable in the model and then to summarize all influences on these variables. Turning again to Figure 1, we see that there are 32 variables with arrows pointing toward them; all represent observed variables (SDQ1-SDQ66). Accordingly, these regression paths can be summarized in terms of 32 separate equations as follows: SDQ1 = F1 + E1 SDQ8 = F1 + E8 SDQ15.= F1 + E15 .. SDQ62 = F1 + E62 SDQ3 = F2 + E3 SDQ10.= F2 + E10 .. SDQ64 = F2 + E64 SDQ7 = F3 + E7 SDQ14.= F3 + E14 .. SDQ69 = F3 + E69

4

Factor Analysis: Confirmatory SDQ5 = F4 + E5 SDQ19.= F4 + E19 .. SDQ66 = F4 + E66 (1)

Although, in principle, there is a one-to-one correspondence between the schematic presentation of a model and its translation into a set of structural equations, it is important to note that neither one of these representations tells the whole story. Some parameters, critical to the estimation of the model, are not explicitly shown and thus may not be obvious to the novice CFA analyst. For example, in both the schematic model (see Figure 1) and the linear structural equations cited above, there is no indication that either the factor variances or the error variances are parameters in the model. However, such parameters are essential to all structural equation models and therefore must be included in the model specification. Typically, this specification is made via a separate program command statement, although some programs may incorporate default values. Likewise, it is equally important to draw your attention to the specified nonexistence of certain parameters in a model. For example, in Figure 1, we detect no curved arrow between E1 and E8, which would suggest the lack of covariance between the error terms associated with the observed variables SDQ1 and SDQ8. (Error covariances can reflect overlapping item content and, as such, represent the same question being asked, but with a slightly different wording.)

Testing a Hypothesized CFA Model Testing for the validity of a hypothesized CFA model requires the satisfaction of certain statistical assumptions and entails a series of analytic steps. Although a detailed review of this testing process is beyond the scope of the present chapter, a brief outline is now presented in an attempt to provide readers with at least a flavor of the steps involved. (For a nonmathematical and paradigmatic introduction to SEM based on three different programmatic approaches to the specification and testing of a variety of basic CFA models, readers are referred to [6–9]; for a more detailed and analytic approach to SEM, readers are referred to [3], [14], [16] and [17].)

Statistical Assumptions As with other multivariate methodologies, SEM assumes that certain statistical conditions have been met. Of primary importance is the assumption that the data are multivariate normal (see Catalogue of Probability Density Functions). In essence, the concept of multivariate normality embraces three requirements: (a) that the univariate distributions are normal; (b) that the joint distributions of all variable combinations are normal; and (c) that all bivariate scatterplots are linear and homoscedastic [14]. Violations of multivariate normality can lead to the distortion of goodness-of-fit indices related to the model as a whole (see e.g., [12]; [10]; and (see Goodness of Fit) to positively biased tests of significance related to the individual parameter estimates [14]).

Estimating the Model Once the researcher determines that the statistical assumptions have been met, the hypothesized model can then be tested statistically in a simultaneous analysis of the entire system of variables. As such, some parameters are freely estimated while others remain fixed to zero or some other nonzero value. (Nonzero values such as the 1’s specified in Figure 1 are typically assigned to certain parameters for purposes of model identification and latent factor scaling.) For example, as shown in Figure 1, and in the structural equation above, the factor loading of SDQ8 on Factor 1 is freely estimated, as indicated by the single-headed arrow leading from Factor 1 to SDQ8. By contrast, the factor loading of SDQ10 on Factor 1 is not estimated (i.e., there is no single-headed arrow leading from Factor 1 to SDQ10); this factor loading is automatically fixed to zero by the program. Although there are four main methods for estimating parameters in CFA models, maximum likelihood estimation remains the one most commonly used and is the default method for all SEM programs.

Evaluating Model Fit Once the CFA model has been estimated, the next task is to determine the extent to which its specifications are consistent with the data. This evaluative process focuses on two aspects: (a) goodness-of-fit of the model as a whole, and (b) goodness-of-fit of individual parameter estimates. Global assessment of fit

Factor Analysis: Confirmatory is determined through the examination of various fit indices and other important criteria. In the event that goodness-of-fit is adequate, the model argues for the plausibility of postulated relations among variables; if it is inadequate, the tenability of such relations is rejected. Although there is now a wide array of fit indices from which to choose, typically only one or two need be reported, along with other fit-related indicators. A typical combination of these evaluative criteria might include the Comparative Fit Index (CFI; Bentler, [1]), the standardized root mean square residual (SRMR), and the Root Mean Square Error of Approximation (RMSEA; [18]), along with its 90% confidence interval. Indicators of a well-fitting model would be evidenced from a CFI value equal to or greater than .93 [11], an SRMR value of less than .08 [11], and an RMSEA value of less than .05 [4]. Goodness-of-fit related to individual parameters of the model focuses on both the appropriateness (i.e., no negative variances, no correlations >1.00) and statistical significance (i.e., estimate divided by standard error >1.96) of their estimates. For parameters to remain specified in a model, their estimates must be statistically significant.

Post Hoc Model-fitting Presented with evidence of a poorly fitting model, the hypothesized CFA model would be rejected. Analyses then proceed in an exploratory fashion as the researcher seeks to determine which parameters in the model are misspecified. Such information is gleaned from program output that focuses on modification indices (MIs), estimates that derive from testing for the meaningfulness of all constrained (or fixed) parameters in the model. For example, the constraint that the loading of SDQ10 on Factor 1 is zero, as per Figure 1 would be tested. If the MI related to this fixed parameter is large, compared to all other MIs, then this finding would argue for its specification as a freely estimated parameter. In this case, the new parameter would represent a loading of SDQ10 on both Factor 1 and Factor 2. Of critical importance in post hoc model-fitting, however, is the requirement that only substantively meaningful parameters be added to the original model specification.

5

Interpreting Estimates Shown in Figure 2 are standardized parameter estimates resulting from the testing of the hypothesized CFA model portrayed in Figure 1. Standardization transforms the solution so that all variables have a variance of 1; factor loadings will still be related in the same proportions as in the original solution, but parameters that were originally fixed will no longer have the same values. In a standardized solution, factor loadings should generally be less than 1.0 [14]. Turning first to the factor loadings and their associated errors of measurement, we see that, for example, the regression of Item SDQ15 on Factor 1 (Physical SC; Appearance) is .82. Because SDQ15 loads only on Factor 1, we can interpret this estimate as indicating that Factor 1 accounts for approximately 67% (100 × .822 ) of the variance in this item. The measurement error coefficient associated with SDQ15 is .58, thereby indicating that some 34% (as a result of decimal rounding) of the variance associated with this item remains unexplained by Factor 1. (It is important to note that, unlike the LISREL program [13], which does not standardize errors in variables, the EQS program [2] used here does provide these standardized estimated values; see Structural Equation Modeling: Software.) Finally, values associated with the double-headed arrows represent latent factor correlations. Thus, for example, the value of .41 represents the correlation between Factor 1 (Physical SC; Appearance) and Factor 2 (Physical SC; Ability). These factor correlations should be consistent with the theory within which the CFA model is grounded. In conclusion, it is important to emphasize that only issues related to the specification of first-order CFA models, and only a cursory overview of the steps involved in testing these models has been included here. Indeed, sound application of SEM procedures in testing CFA models requires that researchers have a comprehensive understanding of the analytic process. Of particular importance are issues related to the assessment of multivariate normality, appropriateness of sample size, use of incomplete data, correction for nonnormality, model specification, identification, and estimation, evaluation of model fit, and post hoc model-fitting. Some of these topics are covered in other entries, as well as the books and journal articles cited herein.

6

Factor Analysis: Confirmatory

0.82 0.70* Physical SC (Appearance) F1

0.82* 0.85* 0.58* 0.72* 0.69* 0.71*

SDQ1

0.58

E1*

SDQ8

0.71

E8*

SDQ15

0.58

E15*

SDQ22

0.53

E22*

SDQ38

0.82

E38*

SDQ46

0.69

E46*

SDQ54

0.73

E54*

SDQ62

0.71

E62*

SDQ3

0.66

E3*

SDQ10

0.85

E10*

SDQ24

0.84

E24*

SDQ32

0.85

E32*

SDQ40

0.61

E40*

SDQ48

0.69

E48*

SDQ56

0.48

E56*

SDQ64

0.82

E64*

SDQ7

0.79

E7*

SDQ14

0.74

E14*

SDQ28

0.68

E28*

SDQ36

0.79

E36*

SDQ44

0.77

E44*

SDQ52

0.78

E52*

SDQ60

0.71

E60*

SDQ69

0.61

E69*

SDQ5

0.78

E5*

0.63 0.56*

SDQ19

0.83

E19*

0.60* 0.55* 0.66* 0.73* 0.83* 0.69*

SDQ26

0.80

E26*

SDQ34

0.83

E34*

SDQ42

0.76

E42*

SDQ50

0.68

E50*

SDQ58

0.55

E58*

SDQ66

0.73

E66*

0.41*

0.55*

Physical SC (Ability) F2

0.75 0.53* 0.54* 0.52* 0.80* 0.73* 0.88* 0.57*

0.52* 0.29* 0.61 0.67* Social SC (Peers) F3 0.21*

0.74* 0.61* 0.64* 0.62* 0.71* 0.79*

0.42*

Social SC (Parents) F4

Figure 2

Standardized estimates for hypothesized CFA model

Factor Analysis: Confirmatory

References [1] [2] [3] [4]

[5]

[6]

[7]

[8]

[9]

[10]

Bentler, P.M. (1990). Comparative fit indexes in structural models, Psychological Bulletin 107, 238–246. Bentler, P.M. (2004). EQS 6.1: Structural Equations Program Manual, Multivariate Software Inc, Encino. Bollen, K. (1989). Structural Equations with Latent Variables, Wiley, New York. Browne, M.W. & Cudeck, R. (1993). Alternative ways of assessing model fit, in Testing Structural Equation Models, K.A. Bollen & J.S. Long eds, Sage, Newbury Park, pp. 136–162. Bryant, F.B. & Yarnold, P.R. (1995). Principal-components analysis and exploratory and confirmatory factor analysis, in Reading and Understanding Multivariate Statistics, L.G. Grimm & P.R. Yarnold eds, American Psychological Association, Washington. Byrne, B.M. (1994). Structural Equation Modeling with EQS and EQS/Windows: Basic Concepts, Applications, and Programming, Sage, Thousand Oaks. Byrne, B.M. (1998). Structural Equation Modeling with LISREL, PRELIS, and SIMPLIS: Basic Concepts, Applications, and Programming, Erlbaum, Mahwah. Byrne, B.M. (2001a). Structural Equation Modeling with AMOS: Basic Concepts, Applications, and Programming, Erlbaum, Mahwah. Byrne, B.M. (2001b). Structural equation modeling with AMOS, EQS, and LISREL: comparative approaches to testing for the factorial validity of a measuring instrument, International Journal of Testing 1, 55–86. Curran, P.J., West, S.G. & Finch, J.F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis, Psychological Methods 1, 16–29.

7

[11]

Hu, L.-T. & Bentler, P.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives, Structural Equation Modeling 6, 1–55. [12] Hu, L.-T., Bentler, P.M. & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted? Psychological Bulletin 112, 351–362. [13] J¨oreskog, K.G. & S¨orbom, D. (1996). LISREL 8: User’s Reference Guide, Scientific Software International, Chicago. [14] Kline, R.B. (1998). Principles and Practice of Structural Equation Modeling, Guildwood Press, New York. [15] Marsh, H.W. (1992). Self Description Questionnaire (SDQ) I: A Theoretical and Empirical Basis for the Measurement of Multiple Dimensions of Preadolescent Self-concept: A Test Manual and Research Monograph, Faculty of Education, University of Western Sydney, Macarthur, New South Wales. [16] Maruyama, G.M. (1998). Basics of Structural Equation Modeling, Sage, Thousand Oaks. [17] Raykov, T. & Marcoulides, G.A. (2000). A First Course Oin Structural Equation Modeling, Erlbaum, Mahwah. [18] Steiger, J.H. (1990). Structural model evaluation and modification: an interval estimation approach, Multivariate Behavioral Research 25, 173–180.

(See also History of Path Analysis; Linear Statistical Models for Causation: A Critical Review; Residuals in Structural Equation, Factor Analysis, and Path Analysis Models; Structural Equation Modeling: Checking Substantive Plausibility) BARBARA M. BYRNE

FACTORIAL DESIGNS IN CLINICAL TRIALS

a recent chapter discussing factorial designs in medical studies given by Piantadosi (24).

Steven Piantadosi

1 BASIC FEATURES OF FACTORIAL DESIGNS

Johns Hopkins University Baltimore, MD, USA

The simplest factorial design has two treatments (A and B) and four treatment groups (Table 1). There might be n patients entered into each of the four treatment groups for a total sample size of 4n and a balanced design. One group receives neither A nor B, a second receives both A and B, and the other two groups receive one of A or B. This is called a 2 × 2 (two by two) factorial design. The design generates enough information to test the effects of A alone, B alone, and A plus B. The 2 × 2 design generalizes to higher order factorials. For example, a design studying three treatments, A, B, and C, is the 2 × 2 × 2. Possible treatment groups for this design are shown in Table 2. The total sample size is 8n if all treatment groups have n subjects. These examples highlight some of the prerequisites necessary for, and restrictions on, using a factorial trial. First, the treatments must be amenable to being administered in combination without changing dosage in the presence of each other. For example, in Table 1, we would not want to reduce the dose of A in the lower right cell where B is present. This requirement implies that the side effects of the treatments cannot be cumulative to the point where the combination is impossible to administer. Secondly, it must be ethically acceptable to withhold the individual treatments, or administer them at lower doses as the case may be. In some situations, this means having a no-treatment or placebo group in the trial. In other cases A and B may be administered in addition to a ‘‘standard’’ so that all groups receive some treatment. Thirdly, we must be genuinely interested in learning about treatment combinations; otherwise, some of the treatment groups might be unnecessary. Alternately, to use the design to achieve greater efficiency in

Factorial experiments test the effect of more than one treatment (factor) using a design that permits an assessment of interactions between the treatments. A treatment could be either a single therapy or a combination of interventions. The essential feature of factorial designs is that treatments are varied systematically (i.e. some groups receive more than one treatment), and the experimental groups are arranged in a way that permits testing whether or not the treatments interact with one another. The technique of varying more than one treatment in a single study has been used widely in agriculture and industry based on work by Fisher 10,11 and Yates (33). Influential discussions of factorial experiments were given by Cox (8) and Snedecor & Cochran (28). Factorial designs have been used relatively infrequently in medical trials, except recently in disease prevention studies. The discussion here will be restricted to randomized factorial clinical trials. Factorial designs offer certain advantages over conventional comparative designs, even those employing more than two treatment groups. The factorial structure permits certain comparisons to be made that cannot be achieved by any other design. In some circumstances, two treatments can be tested in a factorial trial using the same number of subjects ordinarily used to test one treatment. However, the limitations of factorial designs must be understood before deciding whether or not they are appropriate for a particular therapeutic question. Additional discussions of factorial designs in clinical trials can be found in Byar & Piantadosi (6) and Byar et al. (7). For a discussion of such designs related to cardiology trials, particularly in the context of the ISIS-4 trial (17), see Lubsen & Pocock (20). This article is based on

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

FACTORIAL DESIGNS IN CLINICAL TRIALS

Table 1. Treatment Groups and Sample Sizes in a 2 × 2 Balanced Factorial Design No Yes Total

n n 2n

n n 2n

2n 2n 4n

Table 2. Treatment Groups in a Balanced 2 × 2 × 2 Factorial Design 1 2 3 4 5 6 7 8

No Yes No No Yes No Yes Yes

No No Yes No Yes Yes No Yes

No No No Yes No Yes Yes Yes

n n n n n n n n

are group averages of some normally distributed response denoted by Y. The subscripts on Y indicate which treatment group it represents. Half the patients receive one of the treatments (this is also true in higher order designs). For a moment, further assume that the effect of A is not influenced by the presence of B. There are two estimates of the effect of treatment A compared to placebo in the design, Y A − Y 0 and Y AB − Y B . If B does not modify the effect of A, the two estimates can be combined (averaged) to estimate the overall effect of A(β A ), βA =

(Y A − Y 0 ) + (Y AB − Y B ) . 2

(1)

(Y B − Y 0 ) + (Y AB − Y A ) . 2

(2)

Similarly, studying two or more treatments, we must know that some interactions do not exist. Fourthly, the therapeutic questions must be chosen appropriately. We would not use a factorial design to test treatments that have exactly the same mechanisms of action (e.g. two ACE inhibitors for high blood pressure) because either would answer the question. Treatments acting through different mechanisms would be more appropriate for a factorial design (e.g. radiotherapy and chemotherapy for tumors). In some prevention factorial trials, the treatments tested also target different diseases. 1.1 Efficiency Factorial designs offer certain very important efficiencies or advantages when they are applicable. Consider the 2 × 2 design and the estimates of treatment effects that would result using an additive model for analysis (Table 3). Assume that the responses

Table 3. Treatment Effects from a 2 × 2 Factorial Design A No Yes No Yes

Y0 YA

YB Y AB

βB =

Thus, in the absence of interactions (i.e. the effect of A is the same with or without B, and vice versa), the design permits the full sample size to be used to estimate two treatment effects. Now suppose that each patient’s response has a variance σ 2 that is the same in all treatment groups. We can calculate the variance of β A to be var (βA ) =

1 4σ 2 σ2 × = . 4 n n

This is the same variance that would result if A were tested against placebo in a single twoarmed comparative trial with 2n patients in each treatment group. Similarly, var (βB ) =

σ2 . n

However, if we tested A and B in separate trials, we would require 4n subjects in each trial or a total of 8n patients to have the same precision. Thus, in the absence of interactions, factorial designs estimate main effects efficiently. In fact, tests of both A and B can be conducted in a single factorial trial with the same precision as two single-factor trials using twice the sample size.

FACTORIAL DESIGNS IN CLINICAL TRIALS

2

INTERACTIONS

for treatment A,

The effect of A might be influenced by the presence of B (or vice versa). In other words, there might be a treatment interaction. Some of the efficiencies just discussed will be lost. However, factorial designs are even more relevant when interactions are possible. Factorial designs are the only type of trial design that permits study of treatment interactions. This is because the design has treatment groups with all possible combinations of treatments, allowing the responses to be compared directly. Consider again the two estimates of A in the 2 × 2 design, one in the presence of B and the other in the absence of B. The definition of an interaction is that the effect of A in the absence of B is different from the effect of A in the presence of B. This can be estimated by comparing βAB = (Y A − Y 0 ) − (Y AB − Y B )

(3)

to zero. If β AB is near zero, we would conclude that no interaction is present. It is straightforward to verify that β AB = β BA . When there is an AB interaction present, we must modify our interpretation of the main effects. For example, the estimates of the main effects of A and B [(1) and (2)] assumed no interaction was present. We may choose to think of an overall effect of A, but recognize that the magnitude (and possibly the direction) of the effect depends on B. In the absence of the other treatment, we could estimate the main effects using βA
= (Y A − Y 0 )

(4)

βB
= (Y B − Y 0 ).

(5)

and

In the 2 × 2 × 2 design, there are three main effects and four interactions possible, all of which can be tested by the design. Following the notation above, the effects are βA = 14 [(Y A − Y 0 ) + (Y AB − Y B ) + (Y AC − Y C ) + (Y ABC − Y BC )],

3

(6)

βAB = 12 {[(Y A − Y 0 ) − (Y AB − Y B )] +[(Y AC − Y C ) − (Y ABC − Y BC )]}, (7) for the AB interaction, and βABC = [(Y A − Y 0 ) − (Y AB − Y B ) − (Y AC −Y C ) − (Y ABC − Y BC )]

(8)

for the ABC interaction. When certain interactions are present, we may require an alternative estimator for β A or β BA (or for other effects). Suppose that there is evidence of an ABC interaction. Then, instead of β A , one possible estimator of the main effect of A is βA
= 12 [(Y A − Y 0 ) + (Y AB − Y B )], which does not use β ABC . Other estimators of the main effect of A are possible. Similarly, the AB interaction could be tested by
= (Y A − Y 0 ) − (Y AB − Y B ), βAB

for the same reason. Thus, when treatment interactions are present, we must modify our estimates of main effects and lower order interactions, losing some efficiency. 2.1 Scale of Measurement In the examples just given, the treatment effects and interactions have been assumed to exist on an additive scale. This is reflected in the use of sums and differences in the formulas for estimation. Other scales of measurement may be useful. As an example, consider the response data in Table 4, where the effect of Treatment A is to increase the baseline response by 10 units. The same is

Table 4. Response Data from a Factorial Trial Showing no Interaction on an Additive Scale A No

Yes

No Yes

5 15

15 25

4

FACTORIAL DESIGNS IN CLINICAL TRIALS

true of B and there is no interaction between the treatments on this scale because the joint effect of A and B is to increase the response by 20 units. In contrast, in Table 5 are shown data in which the effects of both treatments are to multiply the baseline response by 3.0. Hence, the combined effect of A and B is a nine fold increase which is greater than the joint treatment effect for the additive case. If the analysis model were multiplicative, then Table 4 would show an interaction, whereas if the analysis model were additive, then Table 5 would show an interaction. Thus, to discuss interactions, we must establish the scale of measurement. 2.2 Main Effects and Interactions In the presence of an interaction in the 2 × 2 design, one cannot speak simply about an overall, or main, effect of either treatment. This is because the effect of A is different depending on the presence or absence of B. In the presence of a small interaction, where all patients benefit from A regardless of the use of B, we might observe that the magnitude of the ‘‘overall’’ effect of A is of some size and that therapeutic decisions are unaffected by the presence of an interaction. This is called ‘‘quantitative’’ interaction, so-named because it does not affect the direction of the treatment effect. For large quantitative interactions, it may not be sensible to talk about overall effects. In contrast, if the presence of B reverses the effect of A, then the interaction is ‘‘qualitative’’, and treatment decisions may need to be modified. Here, we cannot talk about an overall effect of A, because it could be positive in the presence of B, negative in the absence of B, and could yield an average effect near zero.

Yes

No Yes

5 15

Motivation for the estimators given above can be obtained using general linear models. There has been little theoretic work on analyses using other models. One exception is the work by Slud (27) describing approaches to factorial trials with survival outcomes. Suppose we have conducted a 2 × 2 factorial experiment with group sizes given by Table 1. We can estimate the AB interaction effect using the linear model E{Y} = β0 + βA XA + βB XB + βAB XA XB , (9) where the Xs are indicator variables for the treatment groups and β AB is the interaction effect. The design matrix has dimension 4n × 4 and is 

1 0
X = 0 0

... ... ... ...

1 1 0 0

... ... ... ...

15 45

1 0 1 0

... ... ... ...

1 1 1 1

 ... . . . , . . . ...

where there are four blocks of n identical rows representing each treatment group and the columns represent effects for the intercept, treatment A, treatment B, and both treatments, respectively. The vector of responses has dimension 4n × 1 and is Y
= {Y01 , . . . , YA1 , . . . , YB1 , . . . , YAB1 , . . .}. The ordinary least squares solution for the model (9) is βˆ = (X
X)−1 X
Y. The covariance matrix is (X’X)−1 σ 2 , where the variance of each observation is σ 2 . We have  4 2
 XX = n× 2 1 

Table 5. Response Data From a Factorial Trial Showing no Interaction on a Multiplicative Scale A No

2.3 Analysis

(X
X)−1

2 2 1 1

2 1 2 1

 1 1 , 1 1

 1 −1 −1 1 1 −1 2 1 −2 , = × n −1 1 2 −2 1 −2 −2 4

FACTORIAL DESIGNS IN CLINICAL TRIALS

5

and

and 



 3 1 1 2 4 −2 −2 σ
∗ } = cov{β × − 12 1 0 . n 1 −2 0 1

Y 0 + Y A + Y B + Y AB   Y A + Y AB
, XY =n×   Y B + Y AB Y AB where Y i denotes the average response in the ith group. Then,  Y0   −Y 0 + Y A , βˆ =    −Y 0 + Y B Y 0 − Y A − Y B + Y AB 

(10)

which corresponds to the estimators given above in (3)–(5). However, if we assume no interaction, then the β AB effect is removed from the model, and we obtain the estimator  3  1 1 1 4 Y 0 + 4 Y A + 4 Y B − 4 Y AB  1  1 1 1  βˆ ∗ =  − 2 Y 0 + 2 Y A − 2 Y B + 2 Y AB  . 1 1 1 1 − 2 Y 0 − 2 Y A + 2 Y B + 2 Y AB The main effects for A and B are as given above in (1) and (2). The covariance matrices for these estimators are   1 −1 −1 1 2 −1 2 1 −2 σ 
= × cov{β} −1 1 2 −2 n 1 −2 −2 4

In the absence of an interaction, the main effects of A and B are estimated independently and with higher precision than when an interaction is present. The interaction effect is relatively imprecisely estimated, indicating that larger sample sizes are required to have a high power to detect such effects. 3

EXAMPLES

Several clinical trials conducted in recent years have used factorial designs. A sample of such studies is shown in Table 6. One important study using a 2 × 2 factorial design is the Physicians’ Health Study 16,30. This trial has been conducted, in 22 000 physicians in the US and was designed to test the effects of (i) aspirin on reducing cardiovascular mortality and (ii) β-carotene on reducing cancer incidence. The trial is noteworthy in several ways, including its test of two interventions in unrelated diseases, use of physicians as subjects to report outcomes reliably, relatively low cost, and an all-male (high risk)

Table 6. Some Recent Randomized Clinical Trials using Factorial Designs Trial Design Physicians’ Health Study ATBC Prevention Trial Desipramine ACAPS Linxian Nutrition Trial Retinitis pigmentosa Linxian Cataract Trial Tocopherol/deprenyl Womens’ Health Initiative Polyp Prevention Trial Cancer/eye disease Cilazapril/hydrochlorothiazide Nebivolol Endophthalmitis vitrectomy study Bicalutamide/flutamide ISIS-4 Source: adapted from Piantadosi

Reference 2×2 2×2 2×2 2×2 24 2×2 2×2 23 2×2 2×2 4×3 4×3 2×2 2×2 23

Hennekens & Eberlein Heinonen et al. Max et al. ACAPS Group Li et al. Berson et al. Sperduto et al. Parkinson Study Group Assaf & Carleton Greenberg et al. Green et al. Pordy Lacourciere et al. Endophthalmitis Vitrectomy Study Group Schellhammer et al. ISIS-4 Collaborative Group

6

FACTORIAL DESIGNS IN CLINICAL TRIALS

study population. This last characteristic has led to some unwarranted criticism. In January 1988 the aspirin component of the Physicians’ Health Study was discontinued, because evidence demonstrated convincingly that it was associated with lower rates of myocardial infarction (20). The question concerning the effect of β-carotene on cancer remains open and will be addressed by continuation of the trial. In the likely absence of an interaction between aspirin and β-carotene, the second major question of the trial will be unaffected by the closure of the aspirin component. Another noteworthy example of a 2 × 2 factorial design is the α-tocopherol β-carotene Lung Cancer Prevention Trial conducted in 29 133 male smokers in Finland between 1987 and 1994 3,15. In this study, lung cancer incidence is the sole outcome. It was thought possible that lung cancer incidence could be reduced by either or both interventions. When the intervention was completed in 1994, there were 876 new cases of lung cancer in the study population during the trial. Alpha-tocopherol was not associated with a reduction in the risk of cancer. Surprisingly, β-carotene was associated with a statistically significantly increased incidence of lung cancer (4). There was no evidence of a treatment interaction. The unexpected findings of this study have been supported by the recent results of another large trial of carotene and retinol (32). The Fourth International Study of Infarct Survival (ISIS-4) was a 2 × 2 × 2 factorial trial assessing the efficacy of oral captopril, oral mononitrate, and intravenous magnesium sulfate in 58 050 patients with suspected myocardial infarction 12,17. No significant interactions among the treatments were observed and each main effect comparison was based on approximately 29 000 treated vs. 29 000 control patients. Captopril was associated with a small but statistically significant reduction in five-week mortality. The difference in mortality was 7.19% vs. 7.69% (143 events out of 4319), illustrating the ability of large studies to detect potentially important treatment effects even when they are small in relative magnitude. Mononitrate and magnesium therapy did not significantly reduce five-week mortality.

4 SIMILAR DESIGNS 4.1 Fractional and Partial Factorial Designs Fractional factorial designs are those which omit certain treatment groups by design. A careful analysis of the objectives of an experiment, its efficiency, and the effects that it can estimate may justify not using some groups. Because many cells contribute to the estimate of any effect, a design may achieve its intended purpose without some of the cells. In the 2 × 2 design, all treatment groups must be present to permit estimating the interaction between A and B. However, for higher order designs, if some interactions are thought biologically not to exist, omitting certain treatment combinations from the design will still permit estimates of other effects of interest. For example, in the 2 × 2 × 2 design, if the interaction between A, B, and C is thought not to exist, omitting that treatment cell from the design will still permit estimation of all the main effects. The efficiency will be somewhat reduced, however. Similarly, the two-way interactions can still be estimated without Y ABC . This can be verified from the formulas above. More generally, fractional high-order designs will produce a situation termed ‘‘aliasing’’, in which the estimates of certain effects are algebraically identical to completely different effects. If both effects are biologically possible, the design will not be able to reveal which effect is being estimated. Naturally, this is undesirable unless additional information is available to the investigator to indicate that some aliased effects are zero. This can be used to advantage in improving efficiency and one must be careful in deciding which cells to exclude. See Cox (8) or Mason & Gunst (21) for a discussion of this topic. The Women’s Health Initiative clinical trial is a 2 × 2 × 2 partial factorial design studying the effects of hormone replacement, dietary fat reduction, and calcium and vitamin D on coronary disease, breast cancer, and osteoporosis (2). All eight combinations of treatments are given, but participants may opt to join one, two, or all three of the randomized components. The study is expected to accrue over 64 000 patients and is projected to finish in the year 2007. The dietary

FACTORIAL DESIGNS IN CLINICAL TRIALS

component of the study will randomize 48 000 women using a 3:2 allocation ratio in favor of the control arm and nine years of followup. Such a large and complex trial presents logistical difficulties, questions about adherence, and sensitivity of the intended power to assumptions that can only roughly be validated. 4.2 Incomplete Factorial Designs When treatment groups are dropped out of factorial designs without yielding a fractional replication, the resulting trials have been termed ‘‘incomplete factorial designs’’ (7). In incomplete designs, cells are not missing by design intent, but because some treatment combinations may be infeasible. For example, in a 2 × 2 design, it may not be ethically possible to use a placebo group. In this case, one would not be able to estimate the AB interaction. In other circumstances, unwanted aliasing may occur, or the efficiency of the design to estimate main effects may be greatly reduced. In some cases, estimators of treatment and interaction effects are biased, but there may be reasons to use a design that retains as much of the factorial structure as possible. For example, they may be the only way in which to estimate certain interactions.

REFERENCES 1. ACAPS Group (1992). Rationale and design for the Asymptomatic Carotid Artery Plaque Study (ACAPS), Controlled Clinical Trials 13, 293–314. 2. Assaf, A. R. & Carleton, R. A. (1994). The Women’s Health Initiative clinical trial and observational study: history and overview, Rhode Island Medicine 77, 424–427. 3. ATBC Cancer Prevention Study Group (1994). The alpha-tocopherol beta-carotene lung cancer prevention study: design, methods, participant characteristics, and compliance, Annals of Epidemiology 4, 1–9. 4. ATBC Cancer Prevention Study Group (1994). The effect of vitamin E and beta carotene on the incidence of lung cancer and other cancers in male smokers, New England Journal of Medicine 330, 1029–1034.

7

5. Berson, E. L., Rosner, B., Sandberg, M. A., Hayes, K. C., Nicholson, B. W., WeigelDiFranco, C. & Willett, W. (1993). A randomized trial of vitamin A and vitamin E supplementation for retinitis pigmentosa, Archives of Ophthalmology 111, 761–772. 6. Byar, D. P. & Piantadosi, S. (1985). Factorial designs for randomized clinical trials, Cancer Treatment Reports 69, 1055–1063. 7. Byar, D. P., Herzberg, A. M. & Tan, W.-Y. (1993). Incomplete factorial designs for randomized clinical trials, Statistics in Medicine 12, 1629–1641. 8. Cox, D. R. (1958). Planning of Experiments. Wiley, New York. 9. Endophthalmitis Vitrectomy Study Group. Results of the Endophthalmitis Vitrectomy Study (1995). A randomized trial of immediate vitrectomy and of intravenous antibiotics for the treatment of postoperative bacterial endophthalmitis, Archives of Ophthalmology 113, 1479–1496. 10. Fisher, R. A. (1935). The Design of Experiments. Collier Macmillan, London. 11. Fisher, R. A. (1960). The Design of Experiments, 8th Ed. Hafner, New York. 12. Flather, M., Pipilis, A., Collins, R. et al. (1994). Randomized controlled trial of oral captopril, of oral isosorbide mononitrate and of intravenous magnesium sulphate started early in acute myocardial infarction: safety and haemodynamic effects, European Heart Journal 15, 608–619. 13. Green, A., Battistutta, D., Hart, V., Leslie, D., Marks, G., Williams, G., Gaffney, P., Parsons, P., Hirst, L., Frost, C. et al. (1994). The Nambour Skin Cancer and Actinic Eye Disease Prevention Trial: design and baseline characteristics of participants, Controlled Clinical Trials 15, 512–522. 14. Greenberg, E. R., Baron, J. A., Tosteson, T. D., Freeman, D. H., Jr, Beck, G. J., Bond, J. H., Colacchio, T. A., Coller, J. A., Frankl, H. D., Haile, R. W., Mandel, R. W., Nierenberg, J. S., Rothstein, D. W., Richard, S., Dale, C., Stevens, M. M., Summers, R. W. & vanStolk, R. U. (1994). A clinical trial of antioxidant vitamins to prevent colorectal adenoma. Polyp Prevention Study Group, New England Journal of Medicine 331, 141–147. 15. Heinonen, O. P., Virtamo, J., Albanes, D. et al. (1987). Beta carotene, alpha-tocopherol lung cancer intervention trial in Finland, in Proceedings of the XI Scientific Meeting of the International Epidemiologic Association, Helsinki, August, 1987. Pharmy, Helsinki.

8

FACTORIAL DESIGNS IN CLINICAL TRIALS

16. Hennekens, C. H. & Eberlein, K. (1985). A randomized trial of aspirin and beta-carotene among U.S. physicians, Preventive Medicine 14, 165–168.

cancer. Casodex Combination Study Group, Urology 45, 745–752. 27. Slud, E. V. (1994). Analysis of factorial survival experiments, Biometrics 50, 25–38.

17. ISIS-4 Collaborative Group (1995). ISIS-4: a randomized factorial trial assessing early captopril, oral mononitrate, and intravenous magnesium-sulphate in 58 050 patients with suspected acute myocardial infarction, Lancet 345, 669–685.

28. Snedecor, G. W. & Cochran, W. G. (1980). Statistical Methods, 7th Ed. The Iowa State University Press, Ames. 29. Sperduto, R. D., Hu, T. S., Milton, R. C., Zhao, J. L., Everett, D. F., Cheng, Q. F., Blot, W. J., Bing, L., Taylor, P. R., Li, J. Y. et al. (1993). The Linxian cataract studies. Two nutrition intervention trials, Archives of Ophthalmology 111, 1246–1253. 30. Stampfer, M. J., Buring, J. E., Willett, W. et al. (1985). The 2 × 2 factorial design: its application to a randomized trial of aspirin and carotene in U.S. physicians, Statistics in Medicine 4, 111–116. 31. Steering Committee of the Physicians’ Health Study Research Group (1989). Final report on the aspirin component of the ongoing physicians’ health study. New England Journal of Medicine 321, 129–135. 32. Thornquist, M. D., Owenn, G. S., Goodman, G. E. et al. (1993). Statistical design and monitoring of the carotene and retinol efficacy trial (CARET), Controlled Clinical Trials 14, 308–324.

18. Lacourciere, Y., Lefebvre, J., Poirier, L., Archambault, F. & Arnott, W. (1994). Treatment of ambulatory hypertensives with nebivolol or hydrochlorothiazide alone and in combination. A randomized double-blind, placebo-controlled, factorial-design trial, American Journal of Hypertension 7, 137–145. 19. Li, B., Taylor, P. R., Li, J. Y., Dawsey, S. M., Wang, W., Tangrea, J. A., Liu, B. Q., Ershow, A. G., Zheng, S. F., Fraumeni, J. F., Jr et al. (1993). Linxian nutrition intervention trials. Design, methods, participant characteristics, and compliance, Annals of Epidemiology 3, 577–585. 20. Lubsen, J. & Pocock, S. J. (1994). Factorial trials in cardiology (editorial), European Heart Journal 15, 585–588. 21. Mason, R. L. & Gunst, R. L. (1989). Statistical Design and Analysis of Experiments. Wiley, New York. 22. Max, M. B., Zeigler, D., Shoaf, S. E., Craig, E., Benjamin, J., Li, S. H., Buzzanell, C., Perez, M. & Ghosh, B. C. (1992). Effects of a single oral dose of desipramine on postoperative morphine analgesia, Journal of Pain & Symptom Management 7, 454–462. 23. Parkinson Study Group (1993). Effects of tocopherol and deprenyl on the progression of disability in early Parkinson’s disease, New England Journal of Medicine 328, 176–183. 24. Piantadosi, S. (1997). Factorial designs, in Clinical Trials: a Methodologic Perspective. Wiley, New York. See Chapter 15. 25. Pordy, R. C. (1994). Cilazapril plus hydrochlorothiazide: improved efficacy without reduced safety in mild to moderate hypertension. A double-blind placebo-controlled multicenter study of factorial design, Cardiology 85, 311–322. 26. Schellhammer, P., Shariff, R., Block, N., Soloway, M., Venner, P., Patterson, A. L., Sarosdy, M., Vogelzang, N., Jones, J. & Kiovenbag, G. (1995). A controlled trial of bicalutamide versus flutamide, each in combination with lutenizing hormone-releasing hormone analogue therapy, in patients with advanced prostate

33. Yates, F. (1935). Complex experiments (with discussion), Journal of the Royal Statistical Society, Series B 2, 181–247.

FAST TRACK ‘‘Fast Track’’ is a formal mechanism to interact with the U.S. Food and Drug Administration (FDA) using approaches that are available to all applicants for marketing claims. The Fast Track mechanism is described in the Food and Drug Administration Modernization Act of 1997 (FDAMA). The benefits of Fast Track include scheduled meetings to seek FDA input into development plans, the option of submitting a New Drug Application (NDA) in sections rather than all components simultaneously, and the option of requesting evaluation of studies using surrogate endpoints. The Fast Track designation is intended for the combination of a product and a claim that addresses an unmet medical need but is independent of the Priority Review and Accelerated Approval programs. An applicant may use any or all of the components of Fast Track without the formal designation. Fast Track designation does not necessarily lead to Priority Review or Accelerated Approval.

This article was modified from the website of the United States Food and Drug Administration (http://www.accessdata.fda.gov/scripts/cder/onctools /Accel.cfm#FastTrack) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

FDA DIVISION OF PHARMACOVIGILANCE AND EPIDEMIOLOGY (DPE)

Center with access to several large recordlinked databases. The reports produced by the Data Encryption Standard (DES) are integral to the ongoing risk assessment and the risk management performed by CDER review divisions of a product’s risk versus benefit profile. In addition, DPE epidemiologists are called on to meet with industry over important safety issues or to present their work before FDA advisory committees.

The Center for Drug Evaluation and Research’s (CDER) Division of Pharmacovigilance and Epidemiology (DPE) also carries out an epidemiologic function in the monitoring of drug safety. This function is performed by a multidisciplinary professional staff of physicians and Ph.D. epidemiologists, pharmacists, and program/project managers. The primary work is directed toward the evaluation and the risk assessment of drugs in the postmarketing environment using the tools of epidemiology. Epidemiologists integrate the medical/ clinical details of the underlying disease being treated with the influence of patient factors, concomitant diseases, and medications, as well as the clinical pharmacology of the specific product under study. DPE’s Epidemiology staff work closely with the Post-Marketing Safety Reviewers to provide clinical and epidemiologic case-series reviews of spontaneous adverse event reports submitted to the Food and Drug Administration (FDA). These data are used in a variety of ways to develop, refine, and investigate signals of clinical importance related to drug safety. As a complement, drug-use data are used frequently to estimate the size and to characterize the demographic composition of the population exposed to a given prescription product. Additionally, epidemiologists are involved in the design and the critique of Phase IV protocols for safety studies performed by industry and in the review of study findings. They also design, execute, and help to analyze data from epidemiologic studies performed through the mechanism of the DPE’s cooperative agreement program that provides the This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/epidemio.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

FDA MODERNIZATION ACT (FDAMA) OF 1997

drug and biological manufacturing changes, and to reduce the need for environmental assessment as part of a product application. The act also codifies FDA’s regulations and practice to increase patient access to experimental drugs and medical devices and to accelerate review of important new medications. In addition, the law provides for an expanded database on clinical trials that will be accessible by patients. With the sponsor’s consent, the results of such clinical trials will be included in the database. Under a separate provision, patients will receive advance notice when a manufacturer plans to discontinue a drug on which they depend for life support or sustenance, or for a treatment of a serious or debilitating disease or condition.

The Food and Drug Administration (FDA) Modernization Act (FDAMA), which was enacted November 21, 1997, amended the Federal Food, Drug, and Cosmetic Act relating to the regulation of food, drugs, devices, and biological products. With the passage of FDAMA, Congress enhanced FDA’s mission to recognize that the Agency would be operating in a twenty-first century characterized by increasing technological, trade, and public health complexities. 1

PRESCRIPTION DRUG USER FEES

The act reauthorizes, for five more years, the Prescription Drug User Fee Act of 1992 (PDUFA). In the past five years, the program has enabled the agency to reduce to 15 months the 30-month average time that used to be required for a drug review before PDUFA. This accomplishment was made possible by FDA managerial reforms and the addition of 696 employees to the agency’s drugs and biologics program, which was financed by $329 million in user fees from the pharmaceutical industry. 2

3 INFORMATION ON OFF-LABEL USE AND DRUG ECONOMICS The law abolishes the long-standing prohibition on dissemination by manufacturers of information about unapproved uses of drugs and medical devices. The act allows a firm to disseminate peer-reviewed journal articles about an off-label indication of its product, provided the company commits itself to file, within a specified time frame, a supplemental application based on appropriate research to establish the safety and effectiveness of the unapproved use. The act also allows drug companies to provide economic information about their products to formulary committees, managed care organizations, and similar large-scale buyers of health-care products. The provision is intended to provide such entities with dependable facts about the economic consequences of their procurement decisions. The law, however, does not permit the dissemination of economic information that could affect prescribing choices to individual medical practitioners.

FDA INITIATIVES AND PROGRAMS

The law enacts many FDA initiatives undertaken in recent years under Vice President Al Gore’s Reinventing Government program. The codified initiatives include measures to modernize the regulation of biological products by bringing them in harmony with the regulations for drugs and by eliminating the need for establishment license application, to eliminate the batch certification and monograph requirements for insulin and antibiotics, to streamline the approval processes for This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/opacom/backgrounders/mo dact.htm), (http://www.fda.gov/oc/fdama/default. htm) by Ralph D’Agostino and Sarah Karl.

4

PHARMACY COMPOUNDING

The act creates a special exemption to ensure continued availability of compounded drug

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

FDA MODERNIZATION ACT (FDAMA) OF 1997

products prepared by pharmacists to provide patients with individualized therapies not available commercially. The law, however, seeks to prevent manufacturing under the guise of compounding by establishing parameters within which the practice is appropriate and lawful. 5 RISK-BASED REGULATION OF MEDICAL DEVICES The act complements and builds on the FDA’s recent measures to focus its resources on medical devices that present the greatest risk to patients. For example, the law exempts from premarket notification class I devices that are not intended for a use that is of substantial importance to prevent impairment of human health, or that do not present a potential unreasonable risk of illness or injury. The law also directs FDA to focus its postmarket surveillance on high-risk devices, and it allows the agency to implement a reporting system that concentrates on a representative sample of user facilities—such as hospitals and nursing homes—that experience deaths and serious illnesses or injuries linked with the use of devices. Finally, the law expands an ongoing pilot program under which FDA accredits outside— so-called ‘‘third party’’— experts to conduct the initial review of all class I and low-tointermediate risk class II devices. The act, however, specifies that an accredited person may not review devices that are permanently implantable, life-supporting, life-sustaining, or for which clinical data are required. 6

FOOD SAFETY AND LABELING

The act eliminates the requirement of the FDA’s premarket approval for most packaging and other substances that come in contact with food and may migrate into it. Instead, the law establishes a process whereby the manufacturer can notify the agency about its intent to use certain food contact substances and, unless FDA objects within 120 days, may proceed with the marketing of the new product. Implementation of the notification process is contingent on additional appropriations to cover its cost to the agency. The

act also expands procedures under which the FDA can authorize health claims and nutrient content claims without reducing the statutory standard. 7 STANDARDS FOR MEDICAL PRODUCTS Although the act reduces or simplifies many regulatory obligations of manufacturers, it does not lower the standards by which medical products are introduced into the market place. In the area of drugs, the law codifies the agency’s current practice of allowing in certain circumstances one clinical investigation as the basis for product approval. The act, however, does preserve the presumption that, as a general rule, two adequate and well-controlled studies are needed to prove the product’s safety and effectiveness. In the area of medical devices, the act specifies that the FDA may keep out of the market products whose manufacturing processes are so deficient that they could present a serious health hazard. The law also gives the agency authority to take appropriate action if the technology of a device suggests that it is likely to be used for a potentially harmful unlabeled use.

FEDERAL FOOD, DRUG, AND COSMETIC ACT

shortcomings of the 1906 act and brought a new consciousness of consumer needs. After several unpopular attempts to revise the Pure Food and Drug Act during the administration of Franklin D. Roosevelt, public outcry over the ‘‘Elixir Sulfanilamide’’ disaster in 1937—a mass poisoning incident in which a popularly marketed drug killed over 100 people—led to the Food, Drug, and Cosmetic Act of June 25, 1938.

The 1906 Pure Food and Drug Act (the ‘‘Wiley Act’’) prohibited the manufacture, sale, and interstate shipment of ‘‘adulterated’’ and ‘‘misbranded’’ foods and drugs. Product labels were required to make a truthful disclosure of contents but were not required to state weights or measures. By 1913, food manufacturers had grown alarmed by growing variety of state-level weight and measure laws and sought uniformity at the federal level through the Gould Amendment, which required net contents to be declared, with tolerances for reasonable variations. Under the Wiley Act, the U.S. federal government’s Bureau of Chemistry—which in 1927 became the Food, Drug, and Insecticide Administration, then in 1931, the Food and Drug Administration—could challenge illegal products in court but lacked the affirmative requirements to guide compliance. Food adulteration continued to flourish because judges could find no specific authority in the law for the standards of purity and content that the Food and Drug Administration (FDA) had set up. Such products as ‘‘fruit’’ jams made with water, glucose, grass seed, and artificial color were undercutting the market for honest products. False therapeutic claims for patent medicines also escaped control in 1912 when Congress enacted an amendment that outlawed such claims but required the government to prove them fraudulent; to escape prosecution, defendants had only to show that they personally believed in the fake remedy, a major weakness in the law for 26 years. The 1906 law became obsolete because technological changes were revolutionizing the production and marketing of foods, drugs, and related products. In addition, economic hardships of the 1930s magnified the many

1

THE PREVENTIVE AMENDMENTS

The 1938 Food, Drug, and Cosmetic Act in conjunction with World War II greatly expanded the FDA’s workload. Wartime demands had stimulated the development of new ‘‘wonder drugs,’’ especially the antibiotics, which were made subject to FDA testing, beginning with penicillin in 1945. Although there was now a law requiring premarket clearance of new drugs, consumers continued to be guinea pigs for a host of new chemical compounds of unknown safety. The law prohibited poisonous substances but did not require proof that food ingredients were safe. It also provided exemptions and safe tolerances for unavoidable or necessary poisons such as pesticides. When the FDA attempted to set a pesticide tolerance, an adverse court decision showed that the lengthy procedure required by law was unworkable. The FDA could stop the use of known poisons and did so in numerous cases, but the vast research efforts needed to ensure that all food chemicals were safe were clearly beyond government resources. Thus, three amendments fundamentally changed the character of the U.S. food and drug law: the Pesticide Amendment (1954), the Food Additives Amendment (1958), and the Color Additive Amendments (1960). These laws provide that no substance can legally be introduced into the U.S. food supply unless there has been a prior determination that it is safe, and the manufacturers themselves are required to prove a product’s safety. Also very significant was the proviso in the food and color additive laws that no additive could be deemed safe (or

This article was modified from the website of the United States Food and Drug Administration (http://www.cfsan.fda.gov/∼lrd/histor1a.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

FEDERAL FOOD, DRUG, AND COSMETIC ACT

given FDA approval) if it was found to cause cancer in humans or experimental animals. Known as the ‘‘Delaney Clause,’’ this section was initially opposed by the FDA and by scientists, who agreed that an additive used at very low levels need not necessarily be banned only because it might cause cancer at high levels. However, its proponents justified the clause on the basis that cancer experts have not been able to determine a safe level for any carcinogen. This was the underlying basis for the 1959 nationwide FDA recall of cranberries contaminated by the weed killer aminotriazole, which was beneficial in convincing farmers that pesticides must be used with care. Preventing violations through premarket clearance has given consumers immeasurably better protection than merely prosecuting the few violations that could be proved by investigating after injuries were reported.

FEDERAL REGISTER

of the nature of the comments received. Each time a proposal is substantively revised or amended, a notice is published in the Federal Register.

The Federal Register is one of the most important sources for information on the activities of the U.S. Food and Drug Administration (FDA) and other government agencies. Published daily, Monday through Friday, the Federal Register carries all proposed and finalized regulations and many significant legal notices issued by the various agencies as well as presidential proclamations and executive orders. Subscriptions to the Federal Register can be purchased from the federal government’s Superintendent of Documents. As an alternative, copies can usually be found in local libraries, county courthouses, federal buildings, or on the Internet. 1

3

Ultimately, a ‘‘Final Rule’’ is published, which specifies the date when the new regulatory requirements or regulations become effective. 4

REGULATORY AGENDA

Twice a year (April and October), the entire Department of Health and Human Services, including the FDA, publishes an agenda in the Federal Register that summarizes policysignificant regulations, regulations that are likely to have a significant economic impact on small entities, and other actions under development. Each item listed includes the name, address, and telephone number of the official to contact for more information.

ADVANCE NOTICE

Often, the FDA publishes ‘‘Notices of Intent’’ in the Federal Register to give interested parties the earliest possible opportunity to participate in its decisions. These are notices that the FDA is considering an issue and that outside views are welcome before a formal proposal is made. 2

FINAL REGULATIONS

5

MEETINGS AND HEARINGS

Notices are published in the Federal Register announcing all meetings of the FDA’s advisory committees and all public meetings that provide an information exchange between FDA and industry, health professionals, consumers, and the scientific and medical communities. The notice contains the date, time, and place of the meeting as well as its agenda. The Federal Register also announces administrative hearings before the FDA and public hearings to gain citizen input into Agency activities (see ‘‘Citizen Petition’’).

PROPOSED REGULATIONS

When a formal proposal is developed, the FDA publishes a ‘‘Notice of Proposed Rulemaking’’ in the Federal Register. The notice provides the timeframe in which written comments about the proposed action can be submitted. A written request also can be submitted that FDA officials extend the comment period. If FDA extends the period, a notice of the extension is published in the Federal Register. Occasionally, a second or third proposal is published in the Federal Register because This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/ora/fed state/Small business /sb guide/fedreg.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

FILEABLE NEW DRUG APPLICATION (NDA) After a New Drug Application (NDA) is received by the U.S. Food and Drug Administration’s Center for Drug Evaluation and Research (CDER), it undergoes a technical screening, generally referred to as a completeness review. This evaluation ensures that sufficient data and information have been submitted in each area to justify ‘‘filing’’ the application—that is, to justify initiating the formal review of the NDA.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/fileable.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

FINANCIAL DISCLOSURE

The financial arrangements that must be disclosed include the following:

The U.S. Food and Drug Administration (FDA) reviews data generated in clinical studies to determine whether medical device applications are approvable. Financial interest of a clinical investigator is one potential source of bias in the outcome of a clinical study. To ensure the reliability of the data, the financial interests and arrangements of clinical investigators must be disclosed to the FDA. This requirement applies to any clinical study submitted in a marketing application that the applicant or the FDA relies on to establish that the product is effective or is used to show equivalence to an effective product, and to any study in which a single investigator makes a significant contribution to the demonstration of safety. The requirement does not apply to studies conducted under the emergency use, compassionate use, or treatment use provisions. Financial compensation or interests information is used in conjunction with information about the design and purpose of the study as well as information obtained through on-site inspections in the agency’s assessment of the reliability of the data. As of February 1999, anyone who submits a Premarket Approval (PMA) or Premarket Notification 510(k) that contains a covered clinical study must submit certain information concerning the compensation to and financial interests of any clinical investigator conducting clinical studies covered in the application. Applicants must certify the absence of certain financial interests of clinical investigators on Financial Interest Form (Certification: Financial Interests and Arrangements of Clinical Investigations, FDA Form 3454) or disclose those financial interests on Financial Interest Form (Disclosure: Financial Interests and Arrangements of Clinical Investigators, FDA Form 3455).

• Compensation made to the investigator

in which the value of the compensation could be affected by the study outcome. • Significant payments to the investigator or institution with a monetary value of $25,000 or more (e.g., grants, equipment, retainers for ongoing consultation, or honoraria) over the cost of conducting the trial. Any such payments to the investigator or institution during the time the investigator is conducting the study and for 1 year after study completion must be reported. • Proprietary interest in the device, such as a patent, trademark, copyright, or licensing agreement. • Significant equity interest in the sponsor such as ownership, interest, or stock options. All such interests whose value cannot be readily determined through reference to public prices must be reported. If the sponsor is a publicly traded company, any equity interest whose value is greater than $50,000 must be reported. Any such interests held by the investigator while the investigator was conducting the study and for 1 year after study completion must be reported. This requirement applies to investigators and subinvestigators, including their spouses and dependent children, but does not apply to full-time or part-time employees of the sponsor or to hospital or office staff. (For studies completed before February 1999, the requirements are reduced. That is, the sponsor does not need to report equity interest in a publicly held company or significant payments of other sorts. Other reporting still applies.) Sponsors are responsible for collecting financial information from investigators, and clinical investigators are responsible for providing financial disclosure information to the sponsor. The investigator’s agreement with the sponsor should require the investigator to provide the sponsor with accurate financial disclosure information. Certification or

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cdrh/devadvice/ide/financial. shtml) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

FINANCIAL DISCLOSURE

disclosure information should not be included in the Investigational Device Exemption (IDE) application. If the FDA determines that the financial interests of any clinical investigator raise a serious question about the integrity of the data, the FDA will take any action it deems necessary to ensure the reliability of the data, including: • Initiating agency audits of the data de-

rived from the clinical investigator in question. • Requesting that the applicant submit additional analyses of data (e.g., to evaluate the effect of the clinical investigator’s data on the overall study outcome). • Requesting that the applicant conduct additional independent studies to confirm the results of the questioned study. • Refusing to use the data from the covered clinical study as the primary basis for an agency action, such as PMA approval or 510(k) clearance.

FISHER’S EXACT TEST

obtained using the standard χ 2 test. But we cannot trust the accuracy of this approximation when it is based on observations on so few patients. Fisher’s exact test provides a way around this difficulty. The reasoning behind the test is as follows. Suppose that the treatment was totally ineffectual, and that each patient’s recovery over the subsequent five days was unaffected by whether the treatment were applied or not. Precisely three patients recovered. If the treatment was ineffectual, then these three, and only these three, individuals would have recovered regardless of whether they were assigned to the treatment or control group. The fact that all three did indeed appear in the treatment group would then have been just a coincidence whose probability could be calculated as follows. When four out of the eight subjects were randomly chosen for the treatment group, the chance that all three of those destined to recover should end up in the treatment group is given by the hypergeometric distribution as (3 C3 )(5 C1 ) = 0.071. 8 C4

RICK ROUTLEDGE Simon Fraser University Vancouver, British Columbia, Canada

Fisher’s exact test can be used to assess the significance of a difference between the proportions in two groups. The test was first described in independently written articles by Irwin (14) and Yates (25). Yates used the test primarily to assess the accuracy of his correction factor to the χ 2 test, and attributed the key distributional result underlying the exact test to R.A. Fisher. Fisher successfully promoted the test in 1935, presenting two applications, one to an observational study on criminal behavior patterns (8), and another to an artificial example of a controlled experiment on taste discrimination (9). Typical recent applications are to the results of simple experiments comparing a treatment with a control. The design must be completely randomized, and each experimental unit must yield one of two possible outcomes (like success or failure). Consider, for example, the study reported by Hall et al. (13). This was a randomized, double-blind, placebo-controlled study on the effect of ribavirin aerosol therapy on a viral infection (RSV) of the lower respiratory tract of infants. After five days of treatment, each infant was examined for the continued presence of viral shedding in nasal secretions. There were 26 patients in the randomized trial. For illustrative purposes, the following discussion focuses on hypothetical results from a smaller set of only eight patients. Also, a patient showing no signs of viral shedding in nasal secretions will be said to have recovered. Consider, then, the ‘‘results’’ displayed in Table 1. All three recoveries were in the treatment group. For a frequency table based on only four treatments and four control subjects, the evidence could hardly be more convincing, but is it statistically significant? Had the experiment included more patients, an approximate P value could have been

This is the standard P value for Fisher’s exact test of the null hypothesis of no treatment effect against the one-sided alternative that the treatment has a positive benefit. Consider the more general setting, as portrayed in Table 2. The P value for testing the null hypothesis that the treatment has no impact vs. the onesided alternative that it has a positive value is min(n,S)
(S Cy )(F Cn−y ) . (1) P= N Cn y=a For a two-sided alternative there is no universally accepted definition. The two most common approaches are (i) to double the onesided P value, or (ii) to extend the above sum over the other tail of the distribution, including all those terms which are less than or equal to the probability for the observed

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

FISHER’S EXACT TEST

Table 1. Results From a Small, Comparative Experiment

Treatment Control Totals

Recovered

Not recovered

Totals

3 0 3

1 4 5

4 4 8

Table 2. Notation for a 2 × 2 Frequency Table of Outcomes From a Comparative Experiment

Treatment Control Totals

Recovered

Not recovered

Totals

a c S

b d F

n m N

table. The latter strategy is deployed by the major statistical packages, BMDP (Two-Way Tables in (5)), JMP (Contingency Table Analysis in Fit Y by X in (21)), SAS (FREQ procedure in (20)), S-PLUS (function, fisher.test in (17)), SPSS (Crosstabs in (23)), StatXact (6), and Systat (Tables in (24)). Gibbons & Pratt (12) discuss possible alternatives. The test can be extended to an r × c contingency table, as proposed by Freeman & Halton (11). It is also used on r × 2 tables for multiple comparisons, with the usual controversy over adjustments for simultaneous inferences on a single data set (see (22) and references therein). 1

APPLICABILITY AND POWER

A major advantage of Fisher’s exact test is that it can be justified solely through the randomization in the experiment. The user need not assume that all patients in each group have the same recovery probability, nor that patient recoveries occur independently. The patients could, for example, go to one of four clinics, with two patients per clinic. Two patients attending the same clinic might well have experienced delayed recovery from having contacted the same subsidiary infection in their clinic, but the above argument would still be valid as long as individuals were randomly selected without restriction

from the group of eight for assignment to the treatment vs. control groups. If, however, the randomization was applied at the clinic level, with two of the four clinics selected for assignment to the treatment group, then the test would be invalid. Compared with the hypergeometric distribution, the data would most likely be overdispersed. Similarly, if the randomization was restricted by blocking with respect to clinic, the pair of individuals from each clinic being randomly split between the treatment and control groups, then the test would again be invalid. These alternative designs certainly have their place, particularly in larger experiments with more subjects, but the results would have to be analyzed with another test. The example also illustrates a major weakness of Fisher’s exact test. The evidence for a table based on only four subjects in each of two groups could hardly have been more favorable to the alternative. Yet the P value still exceeds 5%, and most observers would rate the evidence as not statistically significant. It is in general difficult to obtain a statistically significant P value with Fisher’s exact test, and the test therefore has low power. The most effective way to increase the power may well be to take quantitative measurements. Suppose, for instance, that all four patients who received the treatment showed reduced nasal shedding of the virus. By quantifying this evidence, and subjecting the quantitative measurements to a test of significance, the experimenter could, in many instances, generate a more powerful test. One could also, of course, consider running the study on a larger group of patients. 2 COMPETING BINOMIAL-MODEL TEST It is also possible to obtain greater power by analyzing the above table with another statistical model. The most commonly used competitor involves assuming that the numbers of recovered patients in each group are independently binomially distributed. The test was mentioned by Irwin (14), and promoted by Barnard (2). Although he soon withdrew his support (3), it has since become a popular alternative. Its increased power has been

FISHER’S EXACT TEST

amply demonstrated by D’Agostino et al. (7) and others. For the above table, the P value is 0.035 vs. the 0.071 for Fisher’s exact test. The P value based on this binomial model is typically smaller than the one generated by Fisher’s exact test. The main reason for the difference is that the standard definition of the P value contains the probability of the observed table, and this probability is higher for Fisher’s exact test than for the binomial model 1,10,18. Thus the null hypothesis is more frequently rejected, and the binomialmodel test is more powerful. This test is available in StatXact (6). However, the increased power comes at a cost. To justify the binomial model, one must either assume that all patients within each group have the same recovery probability, or envisage that the patients were randomly sampled from some larger group. The trial must also have been conducted so as to ensure that patient deaths occur independently. They cannot, for example, attend four clinics, with two patients per clinic. There is another, more subtle problem with the binomial model. Simple calculations show that had fewer than three or more than five patients recovered, then neither P value could possibly have been significant. This puts the researcher in an awkward quandary. For example, had only two patients recovered after 5 days, the researcher would have had an incentive either to present the results after more than five days of treatment when at least one more patient had recovered, or to incorporate more patients into the experiment. One does not win accolades for announcing the results of experiments that are not only statistically insignificant, but also apparently barely capable of ever producing significant results. These are important complications when it comes to interpreting the results of these sorts of small experiments. Suppose, for example, that in the above experiment the researcher was to have adjusted the five-day reporting time, if necessary, so as to guarantee between three and five recoveries. Then the binomial P value would be invalid. The probability of obtaining a table at least as favorable to the treatment as the above one can be shown to be 0.056, not 0.035, as generated by the standard binomial model.

3

3

THE MID-P VALUE

The P value of 0.071 generated by Fisher’s exact test is still large compared with the 0.056 figure produced by this modified binomial model. There is yet another alternative with important theoretical and practical advantages (see, for example, 16,4,1, and (19)). This is the mid-P value, first introduced in 1949 by Lancaster (15). In place of the standard definition, P value = Pr (evidence at least as favorable to Ha as observed |H0 ), they propose the alternative, mid-P value = Pr(evidence more favorable to Ha as observed |H0 ) + 12 Pr(evidence equally favorable to Ha as observed |H0 ). Table 3 summarizes the possible P values for the above example. This table illustrates that the mid-P has the potential to provide a smaller, more significant-looking P value, and to reduce the discrepancy between P values generated by competing models. However, by using a smaller P value, one may reject a valid null hypothesis too frequently. Fortunately, amongst other desirable attributes of the mid-P, its routine use does indeed control a quantity closely related to the type I error rate (see (19), and references therein). The computer package, StatXact (6) facilitates the calculation of the mid-P by providing the probability of the observed table along with the standard P value.

Table 3. Comparison of P Values for the Data in Table (1) Modified Fisher’s Binomial binomial exact test model model Standard P value Mid-P value

7.1% 3.6%

3.5% 2.0%

5.6% 3.0%

4

4

FISHER’S EXACT TEST

CONCLUSION

Fisher’s exact test provides a widely applicable way to assess the results of simple randomized experiments leading to 2 × 2 contingency tables. But it has low power, especially when the standard P value is used. The power can be increased considerably through (i) using the mid-P value, or (ii) carefully constructing a test based at least in part on a binomial model. Further power increases can be generated through (iii) taking quantitative measurements on each subject, or (iv) running the trial with a larger number of patients. REFERENCES 1. Agresti, A. (1990). Categorical Data Analysis. Wiley–Interscience, New York. 2. Barnard, G. A. (1945). A new test for 2 × 2 tables, Nature 156, 177. 3. Barnard, G. A. (1949). Statistical inference, Journal of the Royal Statistical Society, Series B 11, 115–139. 4. Barnard, G. A. (1989). On alleged gains in power from lower p-values, Statistics in Medicine 8, 1469–1477. 5. BMDP Statistical Software, Inc. (1990). BMDP Statistical Software Manual: To Accompany the 1990 Software Release. University of California Press, Berkeley. 6. Cytel Software Corporation (1995). StatXact-3 for Windows. Cytel Software Corporation, Cambridge, Mass. 7. D’Agostino, R. B., Chase, W. & Belanger, A. (1988). The appropriateness of some common procedures for testing the equality of two independent binomial populations, American Statistician 42, 198–202. 8. Fisher, R. A. (1935). The logic of inductive inference, Journal of the Royal Statistical Society, Series A 98, 39–84. 9. Fisher, R. A. (1935). The Design of Experiments. Oliver & Boyd, Edinburgh. 10. Franck, W. E. (1986). P-values for discrete test statistics, Biometrical Journal 4, 403–406. 11. Freeman, G. H. & Halton, J. H. (1951). Note on an exact treatment of contingency, goodness of fit and other problems of significance, Biometrika 38, 141–149. 12. Gibbons, J. D. & Pratt, J. W. (1975). Pvalues: interpretation and methodology, American Statistician 29, 20–25.

13. Hall, C. B., McBride, J. T., Gala, C. L., Hildreth, S. W. & Schnabel, K. C. (1985). Ribavirin treatment of respiratory syncytial viral infection in infants with underlying cardiopulmonary disease, Journal of the American Medical Association 254, 3047–3051. 14. Irwin, J. O. (1935). Tests of significance for differences between percentages based on small numbers, Metron 12, 83–94. 15. Lancaster, H. O. (1949). The combination of probabilities arising from data in discrete distributions, Biometrika 36, 370–382. 16. Lancaster, H. O. (1961). Significance tests in discrete distributions, Journal of the American Statistical Association 56, 223–234. 17. MathSoft, Inc. (1993). S-PLUS Reference Manual, Version 3.2. MathSoft, Inc., Seattle. 18. Routledge, R. D. (1992). Resolving the conflict over Fisher’s exact test, Canadian Journal of Statistics 20, 201–209. 19. Routledge, R. D. (1994). Practicing safe statistics with the mid-p, Canadian Journal of Statistics 22, 103–110. 20. SAS Institute, Inc. (1989). SAS/STAT User’s Guide, Version 6, 4th Ed., Vol. 1. SAS Institute Inc., Cary. 21. SAS Institute, Inc. (1995). JMP Statistics and Graphics Guide, Version 3.1. SAS Institute Inc., Cary. 22. Savitz, D. A. & Olshan, A. F. (1995). Multiple comparisons and related issues in the interpretation of epidemiological data, American Journal of Epidemiology 142, 904–908. 23. SPSS, Inc. (1991). SPSS Statistical Algorithms, 2nd Ed. SPSS Inc., Chicago. 24. SYSTAT, Inc. (1992). SYSTAT for Windows: Statistics, Version 5. SYSTAT, Inc., Evanston. 25. Yates, F. (1934). Contingency tables involving small numbers and the χ 2 test, Journal of the Royal Statistical Society, Supplement 1, 217–235.

FLEXIBLE DESIGNS

the adaptation rules need not be specified in advance. Different ways have been used to define flexible multi-stage designs with adaptive interim analyses (2–5). In order to control the overall type I error probability, they all adhere to a common invariance principle: Separate standardized test statistics are calculated from the samples at the different stages and aggregated in a predefined way to test statistics that are used for the test decisions. Under the null hypothesis, the distributions of these separate test statistics are known, for example, stage-wise Pvalues follow stochastically independent uniform distributions on [0, 1], or stage-wise Z-scores follow independent standard normal distributions. Assume that, given no design modifications are permitted, the test procedure applied to the aggregated test statistics controls the level α. Then, every design modification that preserves the distributional properties of the separate stage-wise test statistics does not inflate the level α of the test procedure (6). The method of Proschan and Hunsberger (7) based on the definition of the conditional error function can also be defined in terms of a test combining standardized stage-wise test statistics according to a prefixed rule (8–10). The self-designing method of Fisher (11, 12) for two stages also fits into this concept (8). However, it does not allow for an early rejection of the null hypothesis in the interim analysis. The method allows for multiple stages with design modifications (but no test decisions in the interim analyses), and it is up to the experimenter to decide if the trial is completed with a test decision after the next stage. Some additional flexibility exists because the experimenter can choose how future stages will be aggregated to the final test statistics. The weight of the last stage, however, is determined by the weights of the previous stages. A similar approach, that also allows for rejection of the null hypothesis in the interim analyses, has been proposed by Hartung and Knapp (13) and is based on the sum of χ 2 distributed test statistics. ¨ ¨ Muller and Schafer (14) use the notion of the conditional error function to extend the flexibility to the adaptive choice of the number

MARTIN POSCH PETER BAUER WERNER BRANNATH Medical University of Vienna Vienna, Austria

1

INTRODUCTION

Classical frequentist statistical inference is based on the assumption that the inferential procedure is completely specified in advance. Consequently, the consensus guideline International Conference on Harmonization (ICH) E9 on Statistical Principles for Clinical Trials (1) requires for confirmatory trials that hypotheses and the statistical analysis plan is laid down in advance. For the planning of optimal trial designs knowledge on quantities as the expected efficacy of a new treatment, the safety properties, the appropriate doses or application forms of a treatment, the success rate in the control group, and the variability of outcome parameters are essential. Typically, in the planning phase of a clinical trial, many of these quantities are unknown. However, relevant information may aggregate in the course of the trial. Based on this information, changes in the trial design can become desirable or even inevitable. The necessity to allow for cautious adaptations is also recognized in the ICH E9 document: If it becomes necessary to make changes to the trial, any consequent changes to the statistical procedures should be specified in an amendment to the protocol at the earliest opportunity, especially discussing the impact on any analysis and inferences that such changes may cause. The procedure selected should always ensure that the overall probability of type I error is controlled (1).

Statistical inference based on adaptive designs allows implementation of design adaptations without inflating the type I error. The crucial point is that the adaptations may be based on the unblinded data collected so far as well as external information and

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

FLEXIBLE DESIGNS

of interim analyses. The invariance principle behind their approach can be described in simple terms: After every stage, the remainder of the design can be replaced by a design that, given what has been observed up to now, would not result in a higher conditional type I error probability than the preplanned design. Or, in other words, design modifications at any time that preserve the conditional error probability of the original design do not compromise the overall type I error probability. This principle can be defined concisely also in terms of the recursive application of simple two-stage combination tests. This allows the construction of an overall P-value and confidence intervals (15). By this generalization in the interim analysis, an experimenter may decide to insert a further interim analysis for saving time (if he sees good chances to get an early rejection or if information from outside asks for quick decisions). Adaptive designs can also be applied to time-to-event data (16) when information from the first-stage sample is used also in the second-stage test statistics (e.g., exploiting the independent increment structure of the log-rank statistics). However, restrictions exist on the type of information from the first stage, which may be utilized for adaptations (17). Up to now, sample size reassessment (4, 5, 7, 10, 18–23) has been an issue of major interest. However, various other design modifications like changing doses (24), dropping treatment arms (25–27), redesigning multiple endpoints (28, 29), changing the test statistics (30–32), and selecting goals between non-inferiority and superiority (33, 34) have been considered.

is based on a suitably defined combination function C(p1 , p2 ), which is assumed to be left continuous, monotonically increasing in both arguments, and strictly increasing in at least one: If C(p1 , p2 ) ≤ c, one rejects the hypothesis, otherwise one accepts. Note that for α 0 = 1, no stopping for futility is applied. If, in addition, α 1 = 0, no early test decision will be taken and the interim analysis is only performed for adaptation purposes. The adaptive testing procedure is summarized in Fig. 1. If, under H 0 , the P-values are independently and uniformly distributed on [0, 1], then the level condition to determine c, α 1 , and α 0 can be written as
α1 +

α0 α1




1

1[C(x,y)≤c] dy dx = α

Here, the indicator function 1[·] equals 1 if C(x, y) ≤ c and 0 otherwise. One of the combination functions considered in the literature plays a special role because of its relationship to group sequential tests (see Group Sequential Designs). Assume that the one-sided null hypothesis H0 :µA = µB versus H 1 :µA > µB is tested for comparing the means of two normal distributions with known variance (w.l.o.g. σ 2 = 1). The

First Stage p1 a1

0

a0

Reject H01

2

(1)

0

1 Accept H01

THE GENERAL FRAMEWORK Adaptation

To start fairly general, let us assume that a one-sided null hypothesis H0 is planned to be tested in a two-stage design. The test decisions are performed by using the P-values p1 and p2 calculated from the samples at the separate stages. Early decision boundaries are defined for p1 : If p1 ≤ α 1 (where α 1 < α), one stops after the interim analysis with an early rejection; if p1 > α 0 (where α 0 > α), one stops with an acceptance of H0 (stopping for futility). In case of proceeding to the second stage, the decision in the final analysis

Second Stage C (p1,p2) 0

1

c

Reject H01

Accept H01

Figure 1. The adaptive testing procedure.

FLEXIBLE DESIGNS

weighted inverse normal combination function (4, 5) can be defined as C(p1 , p2 ) = 1 − [w1 zp1 + w2 zp2 ], 0 < wi < 1, w21 + w22 = 1

(2)

where zγ is the (1 − γ ) quantile of the standard normal distribution. For the group sequential test of the normal mean with samples balanced over treatments n1 = n1A = n1B and n 2 = n2A = n2B at the  two stages, and w1 = n1 /(n1 + n2 ), w2 = n2 /(n1 + n2 ). The term in the squared brackets of Equation (2) is simply the standardized difference of the treatment means calculated  n /(n from the total sample, Z = Z 1 1 1 + n2 ) +  Z2 n2 /(n1 + n2 ). Here, Z1 and Z2 are the standardized mean treatment differences calculated from the separate stages. Note that C(p1 , p2 ) as defined in Equation (2) is just the P-value for the test based on the total sample. Hence, if no adaptation is performed, the test decision is the same as in the classical group sequential test with an early rejection boundary zα1 . Given zα1 and an early stopping for futility boundary zα0 , the critical boundary zα2 for the test statistic Z in the final analysis can be derived from Equation (1), which then is equivalent to the common level condition Prob(Z1 ≥ zα1 ) + Prob(zα0 ≤ Z1 < zα1 , Z ≥ zα2 ) = α (3) for the corresponding group sequential test with obligatory stopping for futility. It is obvious how this analogy works for more than two stages. The conditional error function A(p1 ) = Prob(reject H 0 |p1 ) in the group sequential context leads to the so-called linear conditional error function (7) ⎧ 0 ⎪ ⎪ ⎪ ⎨1 − [(z √n + n α2 1 2 A(z1 ) = √ √ ⎪ −z n )/ n ] 1 1 2 ⎪ ⎪ ⎩ 1

if z1 < zα0 if zα0 ≤ z1 ≤ zα1 if z1 ≤ zα1 (4)

In the following, n˜ 2 denotes the adapted second-stage sample size, which may be

3

different to n2 planned a priori. Let further Z˜ 2 denote the standardized mean of the actual second-stage sample and  Z˜ = Z1 n1 /(n1 + n2 ) + Z˜ 2 n2 /(n1 + n2 ) the adaptive test statistics based on the preassigned weights tocombine the two stages. Now, setting wi = ni /(n1 + n2 ), i = 1, 2 then Z˜ ≥ zα2 is equivalent to C(p1 , p2 ) < α 2 , where C(p1 , p2 ) is defined in Equation (2) and p1 , p2 are the P-values of the first and the possibly adapted second stage. Note that Z˜ ≥ zα2 is also equivalent to Z˜ 2 ≥ zA(z1 ) , so that formally the test in the second-stage sample is performed at the level A(z1 ). Hence, when proceeding to the second stage, the choice of n˜ 2 can simply be based on sample size formulas for the fixed sample size case using the adjusted level A(z1 ). Some comments have to be made here. (1) The crucial property of these flexible designs is that the adaptation rule needs not to be specified a priori. (2) An alternative approach is to start with an a priori specified sample size reassessment rule n˜ 2 (p1 ) and to use the classical test statistics for the final test (19, 20). To control the type I error, either an adjusted critical boundary or constraints for the sample sizes have to be applied. As this procedure weights the stage-wise test statistics always according to the actual sample sizes, it can be expected to be more efficient than when fixed weights are used. Unless extreme sample sizes are used, this difference, however, is likely to be small (compare Reference 4). Note that with a predefined sample size reassessment rule, one can also define a combination test that uses the classical test statistics as long as one does not deviate from this prespecified rule. The corresponding combination function is identical to Equation (2), but in the weights wi the preplanned second-stage sample size n2 is replaced by n˜ 2 (p1 ). Using this combination function, one can also here deviate from the prespecified sample size reassessment rule. However, a deviation from this rule implies that the classical test statistics is no longer used. Hence, designs with prespecified mandatory adaptation rules can be looked at as a special case of a flexible design. (3) Clearly, combination functions can also be used for tests in a distributional environment completely different to the normal.

4

FLEXIBLE DESIGNS

As long as the stage-wise P-values are independent and uniformly distributed under the global null hypothesis, the methodology will also apply. By transforming the resulting Pvalues via the inverse normal combination method, one formally arrives at independent standard normal increments so that all the results known for group sequential trials under the normal distribution can also be utilized for completely different testing problems (4). (4) The assumptions can even be relaxed, only requiring that the stage-wise P-values follow a distribution that, given the results of the previous stages, is stochastically larger or equal to the uniform distribution (15). A very general formulation of adaptive designs is given in Reference 35. (5) Classical group sequential tests (see Group Sequential Designs) are a special case of the more general concept of adaptive combination tests, because they result from a special combination function for the aggregation of the stage-wise test statistics. (6) Hence, classical group sequential trials can be planned in the context of adaptive multistage designs. Moreover, if the trial has in fact been performed according to the preplanned schedule and no adaptations have been performed, no price at all has to be paid for the option to deviate from the preplanned design: then the classical test statistic and rejection boundaries can be used. However, if adaptations are performed because of the fixed weights, the classical test statistics is no longer used in the final analysis. (7) Estimation faces the problems of sequential designs and the lack of a complete specification of the future sample space because of the flexibility. Several proposals for the construction of point estimates and confidence intervals have been made (4, 15, 34, 36–40). The crucial question is how to use the wide field of flexibility opened by this general concept in practice. Note that, in principle, at every interim analysis, a ‘‘new’’ trial at a significance level equal to the conditional error probability can be planned. This conditional error probability accounts for the past and assures that the overall type I error probability for the future is always controlled. Furthermore, by adopting the concept of (formally) performing interim looks without early test decisions after every sample unit,

this concept of the conditional error function can also be applied for mid-trial design modifications in trials without any preplanned interim analyses (41). Clearly, because of the large number and diversity of possible adaptations, their merits, for example, for the practice of clinical trials, are difficult to establish. In the following, some types of adaptations are discussed. 3 CONDITIONAL POWER AND SAMPLE SIZE REASSESSMENT The conventional measure to assess the performance of tests in a particular design is the overall power: In the long run, the experimenter performing equally powered studies gets rejections with probability 1 − β under the alternative. In case of a sequential design with early decision, however, unblinded information on the observed effect is available. So when proceeding to the next stage, one is tempted to have a look at the chances to get rejection in the end, given the results up to now. The overall power takes expectation over all possible outcomes in the interim analysis. Now, when being halfway through the trial and having preliminary estimates, should the experimenter in a concrete situation average his perspectives over outcomes for which he definitely knows that they have not been observed in the current trial? Or should he argue based on what he already knows? The conditional power (given Z1 ) for the comparison of two normal means by the weighted inverse normal combination function in Equation (2) is given by (σ 2 = 1)  √ √ CP(z1 ) = 1 −  (zα2 n1 + n2 − z1 n1 )/ √  √  n˜ 2 n2 − √ 2

(5)

where  = µA − µB is the relevant difference on which the power is targeted (7, 10, 18, 21–23, 42). In the interim analysis, the second-stage sample size n˜ 2 to achieve a rejection probability of 1 − β in the final analysis is determined by solving the equation CP(z1 ) = 1 − β for n˜ 2 . The question of which value should be plugged in for the targeted effect  develops.

FLEXIBLE DESIGNS

3.1 Using the Estimated Effect It has been proposed to calculate the conditional power by replacing  by its firstˆ (12). Figure 2 shows this stage estimate  ‘‘predictive’’ conditional power for two group sequential tests (here, for comparability, α = 0.025, one-sided, 1 − β = 0.8 is chosen) with equal sample sizes at the two stages balanced over treatments as a function of z1 . Clearly, for small observed effects, the predictive conditional power does not promise a good chance to reject at the end; for large effects only slightly smaller than the rejection boundary zα1 , however, the predictive conditional power exceeds 0.8, particularly in the O’Brien-Fleming (43) design. The predictive conditional power is a random variable and Fig. 3 shows its distribution function given that the trial proceeds to the second stage. Under the alternative, the conditional probability to end up with a predictive conditional power below 0.5 (given the event that the trial continues) is 0.54 for the Pocock design (44) and 0.35 for the O’Brien-Fleming design. Under the null hypothesis, the predictive conditional power will remain below 0.2 in more than 80% of the cases for both designs, which explains the findings that sample size reassessment based on the ‘‘predictive’’ conditional power by using the estimate of the effect size will, in general, lead to large expected sample sizes. Jennison and Turnbull (10) have shown this when applying a sample size reassessment rule used by Cui za1

1

et al. (5) allowing the reassessed sample size to become as large as 25 times the preplanned one. They apply a very large zα1 (>4) and the trial may go on even when a negative effect has been observed. They suggest to instead use group sequential trials that are overpowered for the (optimistic) targeted treatment differences but still have reasonable power for a (pessimistic) smaller margin. The expected sample size for the larger treatment difference (although not being optimal for usual power values) then is still sufficiently small. Other arguments also exist that have been brought up against the use of the mid-trial estimate of the effect size for sample size recalculation. In such procedures, the power does not sufficiently increase with the effect size resulting in flat power curves (10). Moreover, when used in comparisons with placebo, they may aim at small and irrelevant effect sizes. The relevant effect size should follow from medical considerations accounting also for risks and costs, which is true in theory, but not enough knowledge of all issues concerned often exists in advance. Some of them may become clearer from the cumulating observations (e.g., when aggregating sufficient data on safety). In comparisons between active treatments, it is even more difficult to specify the relevant difference because any improvement of the standard therapy may be of interest (e.g., if safer therapies evolve). The current discussion around the choice of equivalence margins in large equivalence trials is a

0.8

0.6

0.6

0.4

0.4

0.2

0.2 0

1 z1

2

Pocock Boundaries

za1

1

0.8

−1

5

−1

0

1 z1

2

O’Brien & Fleming Boundaries

Figure 2. The conditional power (bold line) and the predictive conditional power (thin line) as a function of z1 for a group sequential design with balanced sample sizes over treatments and stages, overall power 80%, and α = 0.025. The dotted line denotes the conditional error function.

6

FLEXIBLE DESIGNS

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2 0.2

0.4 0.6 0.8 Conditional Power

1

Pocock Boundaries

0.2

0.4 0.6 0.8 Conditional Power

1

O’Brien & Fleming Boundaries

Figure 3. The conditional distribution functions (conditional on proceeding to the second stage) of the conditional power (bold lines) and the predictive conditional power (thin lines) for the null (dashed lines) and the alternative (continuous lines) hypothesis.

good example for this changed environment. More emphasis is put on precise confidence intervals of the treatment differences in order to be able to position the treatment within the set of comparators, which, quite naturally, leads to the consideration of families of null hypotheses with diverging equivalence margins (45), which complicates the choice of an a priori sample size (33, 34). 3.2 Staying with the a Priori Defined Effect Size One alternative is to base conditional power on the a priori fixed relevant treatment differences. Figure 2 gives the conditional power for this strategy depending on z1 for two different group sequential tests. When proceeding to the second stage under the alternative in Pocock’s design, only for very large z1 will the conditional power be above 0.8. In the Fleming-O’Brien design, the conditional power is higher because the final critical boundary zα2 is smaller. Figure 3 gives the distribution function of the conditional power given that one proceeds to the second stage. Under the alternative, this conditional power will stay below 0.8 in 74.6% of the cases for Pocock’s design and in only 46.5% for the O’Brien-Fleming design. On the other hand, under the null hypothesis conditional power values above 0.5 would be found in 14.8% and 20.8% of the cases respectively. Denne (21) looked at properties of the resulting test in terms of power and expected sample size when sample size reassessment is

based on conditional power. He shows that, compared with the group sequential design, the power is slightly higher when the effect is smaller than expected, however, for the price of serious inflations of the expected sample size. Posch et al. (46) considered designs with sample size reassessment according to a restricted conditional power rule, where an upper limit on the second-stage sample size is applied. Such a design leads to considerable savings in expected sample size, compared with a two-stage group sequential design, in an important region of the alternative hypothesis. These savings come for a small price to be paid in terms of expected sample size close to the null hypothesis and in terms of maximal sample size. Brannath and Bauer (42) derived the optimal combination test in terms of average sample size when the conditional power rule is applied. Admittedly, for multistage designs, arguments will become much more complicated so that it will be difficult to quantify the impact of sample size reassessment rules. 3.3 Overpowered Group Sequential Versus Adaptive Trials As mentioned above, cautious sample size reassessment rules based on conditional power have good properties in terms of expected sample size. Additionally, sample size reassessment in adaptive designs does not need to be performed according to strict predefined data-based rules. Information may develop from other sources (e.g.,

FLEXIBLE DESIGNS

safety considerations not having been incorporated in a fixed rule, or information from outside the trial) that could strongly favor a change of the preplanned sample size. For example, the demand for a larger sample size may come up to achieve an appropriate judgement of the risk-benefit relationship, or the treatment allocation rule may be changed to get more information on patients under a particular treatment. To adhere to the design and start another trial may then not be considered to be a reasonable strategy by ethical and economical reasons. Clearly, if the cautious approach to overpowered group sequential designs will be more extensively used in practice, then the stopping for futility option will have to be considered thoroughly and carefully in order to avoid radically large sample sizes under the null hypothesis. The very large maximal sample sizes to be laid down in the planning phase may be a further obstacle in the implementation of such designs. In practice, experimenters may rather tend to give up with a negative outcome if large increments in sample size are required to reach the next decision (and the chance for getting a positive decision is small). It is questionable if overall power rather than conditional power arguments will prevent such unscheduled stopping for futility decisions based on ongoing results. But then the perspectives of the overpowered group sequential designs may not apply in real-life scenarios. 3.4 Inconsistency of Rejection Regions At this point, the question about the price to be paid for the option of adaptations has to be answered. As mentioned in Section (2), no price has to be paid if the design is performed according to the preplanned group sequential design (if the appropriate inverse normal combination function is applied), which seems to be surprising and seems like a free lunch. However, the price to be paid may be the potential of being misled by the observed data and to modify a design, which, in fact, may be optimal for the true state of nature. Additionally, in case of an adaptation in terms of sample size reassessment, the decisions are based on statistics deviating from the minimal sufficient statistics.

7

Note that the way how to combine the data from the different stages has to be laid down before the stages are in fact performed (e.g., the weights when using the normal inverse normal method), either from the very beginning or recursively during the trial (14, 15). Denne (21) discussed the case where the adaptive test rejects but the group sequential two-stage test with weights corresponding to the actual sample sizes would not reject. He suggests to avoid such a type of inconsistency by rejecting in the adaptive test only if this group sequential test is rejected too. He shows that, for sample size reassessment based on conditional power, this additional condition has practically no impact on the performance of the adaptive test procedure. A more fundamental inconsistency occurs if the adaptive test rejects but the fixed sample size test would fail to reject (i.e., Z˜ ≥ zα2 ,   ˜ 2 n˜ 2 /(n1 + n˜ 2 ) < zα , but Z1 n1 /(n1 + n˜ 2 ) + Z where zα2 ≥ zα ). To get a complete picture, consider all possible constellations for which such inconsistencies occur with positive probability (i.e., there exist z˜ 2 values that lead to an inconsistency). It turns out that these constellations can be characterized in terms of the sample ratio r = n˜ 2 /(n1 + n˜ 2 ). Thus, if balanced stages were preplanned n1 = n2 ) r = 1/2 corresponds to the case of no sample size reassessment (n˜ 2 = n2 = n1 ), for n˜ 2 → ∞, one has r → 1, and if n˜ 2 = 1, then r = 1/(n1 + 1) (which approaches 0 for increasing n1 ). Figure 4 gives the regions where such inconsistencies occur with positive probability depending on Z1 when applying Pocock or O’Brien-Fleming boundaries with equal sample sizes per stage. The bad news is that such inconsistencies are possible for all values of Z1 . One sees that the area for such inconsistencies is smaller with constant rejection boundaries (Pocock), because the adaptive test has to exceed the larger final decision boundary. Furthermore, if the sample size is increased in case of small and decreased in case of large observed Z1 -values, which is a reasonable behavior in practice, no such inconsistencies may ever occur, which is good news. The lines in the figures denote the conditional power rule when performing sample size reassessment after halfway through the group sequential designs (α = 0.025, one-sided) with overall power 0.8. It can be

FLEXIBLE DESIGNS

1

1

0.75

0.75

0.5

0.5

r

r

8

0.25

0.25

0

0 0

za1 z1 Pocock Boundaries

0

za1 z1 O’Brien & Fleming Boundaries

Figure 4. For sample size reassessment [expressed in terms of the ratio r = n˜ 2 /(n1 + n˜ 2 )] in the shaded regions with a positive (but possibly very small) probability, the adaptive test may lead to a rejection, whereas the Z-test statistics of the pooled sample falls short of zα . For sample size reassessments in the white region, such inconsistencies never occur. The line corresponds to sample size reassessment according to the conditional power rule.

seen that, following this rule, inconsistencies never occur. A general way to deal with inconsistencies without imposing restrictions on the sample size reassessment rule is to reject in the adaptive design only if the fixed sample size test in the end also rejects at the level α. 4 EXTENDING THE FLEXIBILITY TO THE CHOICE OF THE NUMBER OF STAGES Interim analyses are not only costly and time consuming but unblinding may also have an unfavorable impact on the course of the remainder of a trial. Hence, interim analysis should only be performed if either relevant information for the adaptation of the trial design can be expected or if a good chance exists to arrive at an early decision. A priori, it is often difficult to assess the right number of interim analyses. Given the results of, for example, the first interim analysis, one might want to cancel the second if the conditional probability to get an early decision is small. If, on the other hand, a high chance exists to stop the trial early, one might want to add further interim analyses. Also, for external reasons, a further interim analysis might be favorable, for example, if a competitor enters the market such that an early decision would be a competitive advantage.

Assume that in the first interim analysis of a group sequential test with at least three stages, no early stopping condition applies. Then, by a generalization of Equation (4), the conditional error function A(z1 ) gives the probability (under the null hypothesis) that, given Z1 = z1 , the original design rejects at a later stage. Hence, it is the type I error probability that can be spent for later decisions. Thus, one can either spend this type I error probability by performing a single second stage with a final test at level A(z1 ) or, alternatively, proceed with the group sequential test. The decision of which way to go can be based on all information collected so far (14, 15). Posch et al. (46) investigated the expected sample size of a three-stage design with early rejection of the null hypothesis, where the second interim analysis is dropped when the chance for an early rejection is low. It is shown that such a design has nearly the expected average sample size of a three-stage group sequential test. At the same time, it has a lower maximal sample size and saves a considerable number on interim analysis, especially under the null hypothesis. Instead of dropping an interim analysis, as in the above example, one can also add further interim analyses. For example, one can start out with a two-stage design, then

FLEXIBLE DESIGNS

compute the conditional error A(z1 ) in the interim analysis and plan a further two-stage design with level A(z1 ). This procedure can be applied recursively (14, 15). 5

SELECTION OF THE TEST STATISTICS

Additionally, adaptive interim analysis give the opportunity to adaptively choose the test statistics used in the second stage, which allows one to select scores or contrasts based on the interim data (30, 31, 47, 48). If it turns out that a covariable can explain a substantial part of the variability of the primary variable, for the second-stage test an analysis accounting for this covariable can be specified. 6 MORE GENERAL ADAPTATIONS AND MULTIPLE HYPOTHESES TESTING 6.1 The Closed Testing Principle and Adaptive Designs A more general type of adaptation occurs if, in the interim analysis, the null hypothesis to be tested is changed, which is the case, for example, if doses are selected, endpoints are re-weighted, or the study population is adapted. If in the first stage a hypothesis, H 0,1 and, in the second stage a modified hypothesis, H0,2 are tested, then the combination test tests only the intersection hypothesis H0,1 ∩ H0,2 . The rejection of this intersection null hypothesis implies that H0,1 or H 0,2 is false. If, in a dose response setting, the proof of principle for efficacy of at least one dose is intended, it maybe sufficient. Also, when multiple endpoints are considered, the proof of principle for efficacy of at least one of the endpoints may suffice. To make inference on the individual hypotheses, a multiple testing procedure to control the multiple level (i.e., the probability to erroneously reject one or more null hypotheses) has to be applied. A general principle that guarantees control of the multiple level is the closed testing principle (49)! 6.1.1 Closed Testing Procedure. Assume a set I of null hypotheses is to be tested at multiple level α. To reject an individual

9

hypothesis j ∈ I, for all subsets J ⊂ I that contain j, the intersection hypothesis H0, J = ∩i∈J H0,i (stating that all hypotheses in J are true) has to be rejected at local level α. For example, the Bonferroni and Bonferroni Holm procedure can be formulated as special cases of this principle. The closed testing procedure can easily be integrated in adaptive designs by defining adaptive tests for all intersection hypotheses (26, 28, 29, 50), which opens a new dimension of flexibility: The hypotheses to be tested can also be adapted; some hypotheses can be dropped, and new hypotheses can be included in the interim analysis. 6.2 Selection and Addition of Hypotheses The general formulation of the adaptive multiple testing procedure is quite technical (29) such that one demonstrates the methodology with a simple example. Assume that in the first stage two hypotheses H0,A , H 0,B are tested (e.g., corresponding to two treatment groups that are compared with placebo). Then, according to the closed testing principle, level α tests for all intersection hypotheses H0,J , J ∈ J = {{A}, {B}, {A, B}} have to be defined. Thus, in the planning phase of the adaptive test for all hypotheses H0,J , J ∈ J, a level α combination test C(·,·) with decision boundaries α 0 , α 1 , cα has to be specified. In the following, assume for simplicity that all hypotheses are tested with the same combination test and set α 0 = 1, α 1 = 0 such that no early stopping is possible. Now, firststage tests and P-values p1, J , J ∈ J for all (intersection) hypotheses have to be defined. 6.2.1 Selecting a Single Hypothesis. Assume that, in the interim analysis, it is decided that only hypothesis H0,A is selected for the second stage. Then, only a second-stage test for H0,A with P-value p2,A is specified. H 0,A is rejected in the final analyses at multiple level α if both, C(p1,{A,B} , p2A ) ≤ cα (which is a test for the intersection hypothesis H 0,{A,B} ) and C(p1,A , p2,A ) ≤ cα . Note that, in this case, the second-stage test for H 0,A is also used as a test for the intersection hypothesis H0,{A,B} .

10

FLEXIBLE DESIGNS

6.2.2 Selecting Both Hypotheses. If, in the interim analysis, it has been decided to continue with both hypotheses, A and B, the second-stage test for the intersection hypothesis can be based on data for both hypotheses leading to a P-value p2,{A,B} . Then, the individual hypothesis i∈{A, B} can be rejected at multiple level α if C(p1,{A,B} , p2,{A,B} ) ≤ cα and C(p1,i , p2,i ) ≤ cα . 6.2.3 Adding a Hypothesis. Finally, assume that it is decided in the interim analysis to add a new hypothesis H0,C . For simplicity, it is additionally assumed that the hypotheses H0,A , H 0,B have been dropped in the interim analysis. Then H0,C can be rejected at multiple level α if all tests for the intersection hypotheses can be rejected:

Hypothesis

Test

H0,C ∩ H0,A ∩ H0,B H0,C ∩ H0,A H0,C ∩ H0,B H 0,C

C(p2,{A,B} , p2,C ) ≤ cα C(p2,A , p2,C ) ≤ cα C(p2,B , p2,C ) ≤ cα p2,C ≤ α

A simple way to construct P-values for the intersection hypotheses are Bonferroni adjusted P-values. In the example, one can set p1,{A,B} = min[2 min(p1,A , p1,B ),1]. More general tests for intersection hypotheses allow to give different weights to the individual hypothesis or to specify a hierarchical ordering among the individual hypotheses. If several doses are tested, the intersection hypothesis could be tested by testing for a positive trend. Note that the second-stage tests for all (intersection) hypothesis can be chosen adaptively based on the data from the first stage. Clearly, sample size reassessment can be performed additionally to the adaptive choice of hypotheses carried on to the second stage. The allocation ratio to different treatments could be changed, for example, investigating a larger sample for a test treatment. Also, more efficient tests could be planned for the second stage relying on the interim information.

6.3 Adaptation of Hypotheses in Clinical Trials 6.3.1 Treatment Selection. The selection of treatments or doses allows the integration of several phases in the drug development process into a single trial. Assume that in a first stage several dose groups are tested against placebo. In the interim analysis, one or more doses can be selected for the second stage. The selection process will typically be based on safety as well as efficacy information collected in the first stage as well as possible external information coming, for example, from other trials or experiments. 6.3.2 Adapting Multiple Endpoints. If multiple endpoints are considered, the adaptive interim analysis allows one to select or even add new endpoints in the second stage. Consequently, endpoints that appear to be highly variable or for which the interim data shows no efficacy at all can be dropped in the interim analysis. If a composite endpoint is used that summarizes multiple individual variables, the weights of these individual variables in the composite endpoint can be adapted. 6.3.3 Adapting the Population. Another option is to adapt the study population in the interim analysis, which may be desirable if, for example, the interim data show a strong treatment effect in a subpopulation that was not specified beforehand or safety problems in a subpopulation occur. 7 AN EXAMPLE The methodology has been exploited in a two-stage design for an international, multicenter, five-armed clinical phase II dose finding study (51, 52). The objectives for the first stage (433 patients recruited) was to obtain some initial evidence of the primary efficacy variable (infarct size measured by the cumulative release of alpha-HBDH from time 0 to 72 h), select a subset of doses to carry through to stage two and determine the sample size to be applied at stage two. No strict adaptation rules were laid down in the protocol because, in this early phase, the decisions were planned to be taken by utilizing all the

FLEXIBLE DESIGNS

information collected up to the interim analysis from in and outside the trial. A global proof of principle for the existence of a dose response relationship was intended by using the product p1 p2 of the stage-wise P-values pi as the predefined combination function. The predefined first-stage test to give p1 was a linear trend test among the increasing doses including placebo. Multiply controlled inference on the comparisons of the doses with a control was intended by applying the closed testing principle. The second and third highest doses were used in stage two in a balanced comparison with placebo. Based on a hierarchical testing strategy, the comparison of the highest dose applied in the second stage to placebo was laid down to create the secondstage P-value p2 . The second-stage sample size was fixed to 316 per group. The decisions were taken in a two-day meeting by a group of persons: an independent interim analysis group, the principal investigator, safety experts, and others. Finally, the overall product combination test (C(p1 , p2 ) = p1 p2 ) after the recruitment of 959 patients at the second stage failed to proof a dose response relationship because the promising first-stage result on the second highest dose could not be reproduced at the second stage. Still, it was argued a posteriori that the adaptive design saved time and resources to arrive at the decision as compared with other designs. 8

CONCLUSION

The crucial point in adaptive designs considered here is that the adaptation rule does not need to be fully specified in advance. Hence, information from all sources can be incorporated into the adaptation and a full modeling of the decision process is not required. Tsiatis (53) showed that, given a fixed adaptation rule, any adaptive design can be outperformed in terms of average sample size by a likelihood ratio-based sequential design with the same type I error spending function, which implies that in such a design, an interim look has to be performed at any sample size where the adaptive design has a positive probability to reject the null hypothesis. Hence, in case of sample size reassessment, one essentially ends up with continuous monitoring. But group sequential designs have

11

been introduced just to avoid this type of monitoring, which is usually not practical and too costly in clinical trials. Additionally, every group sequential design specifies a combination function and is, thus, a special case of an adaptive design. However, the adaptive design gives the opportunity of extra flexibility. If extensive adaptations are performed in the interim analysis as, for example, a reweighting of endpoints or a change of the study population, the transparency of the testing procedure may get lost and the trial can lose persuasiveness. Although many adaptations are possible in the sense that the type I error is controlled, not all of them are feasible as the interpretability of results may suffer. Another point to consider is how to keep the integrity of the trial by avoiding any negative impact by a leakage of interim results to investigators or other persons involved in the trial. Even the decisions taken in the interim analysis may allow to draw conclusions on the interim results: If, for example, the second-stage sample size is increased, it may indicate that a poor interim treatment effect has been observed. Although the availability of such information, in principle, does not harm the validity of the flexible design (the type I error is still controlled), it may give cause for problems concerning the motivation of investigators or recruitment. When performing adaptations, one has to keep in mind that findings from small first-stage samples (‘‘internal pilot studies’’) in general will be highly variable. So, by performing adaptations, the experimenter may quite frequently be detracted from a good design carefully laid down in the planning phase by looking at interim results. Adaptive designs open a wide field of flexibility with regard to mid-trial design modifications. The authors believe that adaptive treatment selection may be considered as the main advantage of adaptive designs that hardly can be achieved by other methods. However, sample size reassessment has attracted most of the attention up to now. Clearly, the acceptance of the methodology will be higher if the scope of adaptations to be performed is anticipated in the protocol, which, as in the example above, does not

12

FLEXIBLE DESIGNS

mean that the adaptation rule needs to be prespecified in any detail.

¨ ¨ 16. H. Schafer and H-H. Muller, Modification of the sample size and the schedule of interim analyses in survival trials based on data inspections. Stat. Med. 2001; 20: 3741–3751.

REFERENCES

17. P. Bauer and M. Posch, A letter to the editor. Stat. Med., 2004; 23: 1333–1334.

1. European Agency for the Evaluation of Medical Products, ICH Topic E9: Notes for Guidance on Statistical Principles for Clinical Trials, 1998. 2. P. Bauer, Sequential tests of hypotheses in consecutive trials. Biometr. J. 1989; 31: 663–676. 3. P. Bauer and K. K¨ohne, Evaluation of experiments with adaptive interim analyses. Biometrics 1994; 50: 1029–1041. 4. W. Lehmacher and G. Wassmer, Adaptive sample size calculations in group sequential trials. Biometrics 1999; 55: 1286–1290. 5. L. Cui, H. M. J. Hung, and S. Wang, Modification of sample size in group sequential clinical trials. Biometrics 1999; 55: 321–324. 6. P. Bauer, W. Brannath, and M. Posch, Flexible two stage designs: an overwiew. Meth. Inform. Med. 2001; 40: 117–121. 7. M. A. Proschan and S. A. Hunsberger, Designed extension of studies based on conditional power. Biometrics 1995; 51: 1315–1324. 8. M. Posch and P. Bauer, Adaptive two stage designs and the conditional error function. Biometr. J. 1999; 41: 689–696. 9. G. Wassmer, Statistische Testverfahren fur ¨ gruppensequentielle und adaptive Plane ¨ in klinischen Studien. M¨onch, Germany: Verlag Alexander, 1999. 10. C. Jennison and B. Turnbull, Mid-course sample size modification in clinical trials based on the observed treatment effect. Stat. Med. 2003; 22: 971–993. 11. L. D. Fisher, Self-designing clinical trials. Stat. Med. 1998; 17: 1551–1562. 12. Y. Shen and L. Fisher, Statistical inference for self-designing clinical trials with a one-sided hypothesis. Biometrics 1999; 55: 190–197. 13. J. Hartung and G. Knapp, A new class of completely self-designing clinical trials. Biometr. J. 2003; 45: 3–19. ¨ ¨ 14. H-H. Muller and H. Schafer, Adaptive group sequential designs for clinical trials: combining the advantages of adaptive and of classical group sequential approaches. Biometrics 2001; 57: 886–891. 15. W. Brannath, M. Posch, and P. Bauer, Recursive combination tests. J. Amer. Stat. Assoc. 2002; 97: 236–244.

18. M. Posch and P. Bauer, Interim analysis and sample size reassessment. Biometrics 2000; 56: 1170–1176. 19. Z. Shun, Sample size reestimation in clinical trials. Drug Inform. J. 2001; 35: 1409–1422. 20. Z. Shun, W. Yuan, W. E. Brady, and H. Hsu, Type I error in sample size re-estimations based on observed treatment difference. Stat. Med. 2001; 20: 497–513. 21. J. S. Denne, Sample size recalculation using conditional power. Stat. Med. 2001; 20: 2645–2660. 22. A. L. Gould, Sample size re-estimation: recent developments and practical considerations. Stat. Med. 2001; 20: 2625–2643. 23. T. Friede and M. Kieser, A comparison of methods for adaptive sample size adjustment. Stat. Med. 2001; 20: 3861–3874. 24. P. Bauer and J. R¨ohmel, An adaptive method for establishing a dose response relationship. Stat. Med. 1995; 14: 1595–1607. 25. M. Bauer, P. Bauer, and M. Budde, A simulation program for adaptive two stage designs. Comput. Stat. Data Anal. 1998; 26: 351–371. 26. P. Bauer and M. Kieser, Combining different phases in the development of medical treatments within a single trial. Stat. Med. 1999; 18: 1833–1848. 27. T. Friede, F. Miller, W. Bischoff, and M. Kieser, A note on change point estimation in dose-response trials. Comput. Stat. Data Anal. 2001; 37: 219–232. 28. M. Kieser, P. Bauer, and W. Lehmacher, Inference on multiple endpoints in clinical trials with adaptive interim analyses. Biometr. J. 1999; 41: 261–277. 29. G. Hommel and S. Kropf, Clinical trials with an adaptive choice of hypotheses. Drug Inform. J. 2001; 35: 1423–1429. 30. T. Lang, A. Auterith, and P. Bauer, Trendtests with adaptive scoring. Biometr. J. 2000; 42: 1007–1020. ¨ 31. M. Neuhausen, An adaptive location-scale test. Biometr. J. 2001; 43: 809–819. 32. M. Kieser, B. Schneider, and T. Friede, A bootstrap procedure for adaptive selection of the test statistic in flexible two-stage designs. Biometr. J. 2002; 44: 641–652.

FLEXIBLE DESIGNS 33. S-J. Wang, H. M. J. Hung, Y. Tsong, and L. Cui, Group sequential test strategies for superiority and non-inferiority hypotheses in active controlled clinical trials. Stat. Med. 2001; 20: 1903–1912. 34. W. Brannath, P. Bauer, W. Maurer, and M. Posch, Sequential tests for non-inferiority and superiority. Biometrics 2003; 59: 106–114. 35. Q. Liu, M. A. Proschan, and G. W. Pledger, A unified theory of two-stage adaptive designs. J. Amer. Statist. Assoc. 2002; 97: 1034–1041. 36. J. Lawrence and H. M. Hung, Estimation and confidence intervals after adjusting the maximum information. Biom. J. 2003; 45: 143–152. 37. W. Brannath, F. K¨onig, and P. Bauer, Improved repeated confidence bounds in trials with a maximal goal. Biom. J. 2003; 45: 311–324. 38. H. Frick, On confidence bounds for the BauerK¨ohne two-stage test. Biom. J. 2002; 44: 241–249. 39. S. Coburger and G. Wassmer, Conditional point estimation in adaptive group sequential test designs. Biometr. J. 2001; 43: 821–833. 40. S. Coburger and G. Wassmer, Sample size reassessment in adaptive clinical trials using a bias corrected estimate. Biometr. J. 2003; 45: 812–825. ¨ ¨ 41. H. H. Muller and H. Schafer, A general statistical principle for changing a design anytime during the course of a trial. Stat. Med. 2004; 23: 2497–2508. 42. W. Brannath and P. Bauer, Optimal conditional error functions for the control of conditional power. Biometrics 2004; 60: 715–723. 43. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics 1979; 5: 549–556. 44. S. J. Pocock, Group sequential methods in the design and analysis of clinical trials. Biometrika 1977; 64: 191–199. 45. P. Bauer and M. Kieser, A unifying approach for confidence intervals and testing of equivalence and difference. Biometrika 1996; 83: 934–937. 46. M. Posch, P. Bauer, and W. Brannath, Issues in designing flexible trials. Stat. Med. 2003; 30: 953–969. 47. J. Lawrence, Strategies for changing the test statistic during a clinical trial. J. Biopharm. Stat. 2002; 12: 193–205. 48. M. Kieser and T. Friede, Simple procedures for blind sample size adjustment that do not

13

affect the type i error rate. Heidelberg, Germany: Medical Biometry Unit, University of Heidelberg, Germany, 2002. 49. R. Marcus, E. Peritz, and K. R. Gabriel, On closed testing procedures with special reference to ordered analysis of variance. Biometrika 1976; 63: 655–660. 50. G. Hommel, Adaptive modifications of hypotheses after an interim analysis. Biometr. J. 2001; 43(5): 581–589. 51. U. Zeymer, H. Suryapranata, J. P. Monassier, G. Opolski, J. Davies, G. Rasmanis, G. Linssen, U. Tebbe, R. Schroder, R. Tiemann, T. Machnig, and K. L. Neuhaus, The Na+ /H + exchange inhibitor Eniporide as an adjunct to early reperfusion therapy for acute myocardial infarction. J. Amer. College Cardiol. 2001; 38: 1644–1651. 52. U. Zeymer, H. Suryapranata, and J. P. Monassier et al., Evaluation of the safety and cardioprotective effects of eniporide, a specific Sodium/Hydrogen exchange inhibitor, given as adjunctive therapy to reperfusion in patients with acute myocardial infarction. Heart Drug 2001; 1: 71–76. 53. A. A. Tsiatis and C. Mehta, On the inefficiency of the adaptive design for monitoring clinical trials. Biometrika 2003; 90: 367–378.

FOOD AND DRUG ADMINISTRATION (FDA)

responsible for advancing the public health by helping to speed innovations that make medicines and foods more effective, safer, and more affordable and by helping the public get the accurate, science-based information they need to use medicines and foods to improve their health. FDA is the federal agency responsible for ensuring that foods are safe, wholesome, and sanitary; human and veterinary drugs, biological products, and medical devices are safe and effective; cosmetics are safe; and electronic products that emit radiation are safe. FDA also ensures that these products are represented honestly, accurately and informatively to the public. Some of the specific responsibilities of the agency include the following:

The U.S. Food and Drug Administration is a scientific, regulatory, and public health agency that oversees items accounting for 25 cents of every dollar spent by consumers. Its jurisdiction encompasses most food products (other than meat and poultry); human and animal drugs; therapeutic agents of biological origin; medical devices; radiation-emitting products for consumer, medical, and occupational use; cosmetics; and animal feed. The agency grew from a single chemist in the U.S. Department of Agriculture in 1862 to a staff of approximately 9,100 employees and a budget of $1.294 billion in 2001, comprising chemists, pharmacologists, physicians, microbiologists, veterinarians, pharmacists, lawyers, and many others. About one third of the agency’s employees are stationed outside of the Washington, D. C. area, staffing over 150 field offices and laboratories, including 5 regional offices and 20 district offices. Agency scientists evaluate applications for new human drugs and biologics, complex medical devices, food and color additives, infant formulas, and animal drugs. Also, the FDA monitors the manufacture, import, transport, storage, and sale of about $1 trillion worth of products annually at a cost to taxpayers of about $3 per person. Investigators and inspectors visit more than 16,000 facilities a year and arrange with state governments to help increase the number of facilities checked. The FDA is responsible for protecting the public health by assuring the safety, efficacy, and security of human and veterinary drugs, biological products, medical devices, our national food supply, cosmetics, and products that emit radiation. The FDA is also

Biologics • Product and manufacturing establish-

ment licensing • Safety of the national blood supply • Research to establish product standards

and develop improved testing methods Cosmetics • Safety • Labeling

Drugs • Product approvals • OTC and prescription drug labeling • Drug manufacturing standards

Foods • Labeling • Safety of all food products (except meat

and poultry) • Bottled water

Medical Devices This article was modified from the website of the United States Food and Drug Administration (http: //www.fda.gov/oc/history/historyoffda/default.htm), (http://www.fda.gov/opacom/morechoices/mission. html), (http://www.fda.gov/comments/regs.html) by Ralph D’Agostino and Sarah Karl.

• Premarket approval of new devices • Manufacturing and performance stan-

dards • Tracking reports of device malfunction-

ing and serious adverse reactions

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

FOOD AND DRUG ADMINISTRATION (FDA)

Radiation-Emitting Electronic Products • Radiation safety performance standards

for microwave ovens, television receivers, and diagnostic equipment • X-ray equipment, cabinet x-ray systems (such as baggage x-rays at airports), and laser products • Ultrasonic therapy equipment, mercury vapor lamps, and sunlamps • Accrediting and inspecting mammography facilities Veterinary Products • Livestock feeds • Pet foods • Veterinary drugs and devices

FRAILTY MODELS

One can distinguish two broad classes of frailty models:

ANDREAS WIENKE

1. models with an univariate survival time as endpoint and 2. models that describe multivariate survival endpoints (e.g., competing risks, recurrence of events in the same individual, occurrence of a disease in relatives).

University Halle-Wittenberg Institute of Medical Epidemiology Biostatistics and Informatics Germany

The notion of frailty provides a convenient way to introduce random effects, association, and unobserved heterogeneity into models for survival data. In its simplest form, a frailty is an unobserved random proportionality factor that modifies the hazard function of an individual or related individuals. In essence, the frailty concept goes back to the work of Greenwood and Yule (1) on ‘‘accident proneness.’’ The term frailty itself was introduced by Vaupel et al. (2) in univariate survival models, and the model was substantially promoted by its application to multivariate survival data in a seminal paper by Clayton (3) (without using the notion ‘‘frailty’’) on chronic disease incidence in families. Frailty models are extensions of the proportional hazards model, which is best known as the Cox model (4), the most popular model in survival analysis. Normally, in most clinical applications, survival analysis implicitly assumes a homogenous population to be studied, which means that all individuals sampled into that study are subject in principle under the same risk (e.g., risk of death, risk of disease recurrence). In many applications, the study population cannot be assumed to be homogeneous, but must be considered as a heterogeneous sample (i.e., a mixture of individuals with different hazards). For example, in many cases, it is impossible to measure all relevant covariates related to the disease of interest, sometimes because of economical reasons, although the importance of some covariates is still sometimes unknown. The frailty approach is a statistical modeling concept that aims to account for heterogeneity, caused by unmeasured covariates. In statistical terms, a frailty model is a random effect model for time-to-event data, where the random effect (the frailty) has a multiplicative effect on the baseline hazard function.

In the first case, an univariate (independent) lifetime is used to describe the influence of unobserved covariates in a proportional hazards model (heterogeneity). The variability of survival data is split into a part that depends on risk factors, and is therefore theoretically predictable, and a part that is initially unpredictable, even when all relevant information is known. A separation of these two sources of variability has the advantage that heterogeneity can explain some unexpected results or give an alternative interpretation of some results, for example, crossing-over effects or convergence of hazard functions of two different treatment arms [see Manton and Stallard (5)] or levelingoff effects, which means the decline in the increase of mortality rates, which could result in a hazard function at old ages parallel to the x-axis [see Aalen and Tretli (6)]. More interesting, however, is the second case when multivariate survival times are considered in which one aims to account for the dependence in clustered event times, for example, in the lifetimes of patients in study centers in a multi-center clinical trial, caused by center-specific conditions [see Andersen et al. (7)]. A natural way to model dependence of clustered event times is through the introduction of a cluster-specific random effect—the frailty. This random effect explains the dependence in the sense that, had the frailty been known, the events would be independent. In other words, the lifetimes are conditional independent, given the frailty. This approach can be used for survival times of related individuals like family members or recurrent observations on the same person. Different extensions of univariate frailty models to multivariate models are possible and will be considered below.

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

FRAILTY MODELS

The author wants to explain the key ideas of univariate frailty models by an illustrative example from Aalen and Tretli (6). The authors analyzed the incidence of testis cancer by means of a frailty model based on data from the Norwegian Cancer Registry collected from 1953 to 1993. The incidence of testicular cancer is greatest among younger men, and then declines from a certain age. The frailty is considered to be established by birth, and caused by a mixture of genetic and environmental effects. The idea of the frailty model is that a subgroup of men is particularly susceptible to testicular cancer, which would explain why testis cancer is primarily a disease of young men. As time goes by, the members of the frail group acquire the disease, and at some age this group is more or less exhausted. Then the incidence, computed on the basis of all men at a certain age, will necessarily decline. 1

UNIVARIATE FRAILTY MODELS

The standard situation of the application of survival methods in clinical research projects assumes that a homogeneous population is investigated when subject under different conditions (e.g., experimental treatment and standard treatment). The appropriate survival model then assumes that the survival data of the different patients are independent form each other and that each patient’s individual survival time distribution is the same (independent and identically distributed failure times). This basic presumption implies a homogeneous population. However, in the field of clinical trials, one observes in many most practical situations that patients differ substantially. The effect of a drug, a treatment, or the influence of various explanatory variables may differ greatly between subgroups of patients. To account for such unobserved heterogeneity in the study population, Vaupel et al. (2) introduced univariate frailty models into survival analysis. The key idea is that individuals possess different frailties, and that those patients who are most frail will die earlier than the others. Consequently, systematic selection of robust individuals (that means patients with low frailty) takes place.

When mortality rates are estimated, one may be interested in how these rates change over time or age. Quite often it is observed that the hazard function (or mortality rate) rises at the beginning, reaches a maximum, and then declines (unimodal intensity) or levels off at a constant value. The longer the patient lives after manifestation of the disease, the more improved are his or her chances of survival. It is likely that unimodal intensities are often a result of a selection process acting in a heterogeneous population and do not reflect individual mortality. The population intensity may start to decline simply because the high-risk individuals have already died out. The hazard rate of a given individual might well continue to increase. If protective factors or risk factors are known, those could be included in the analysis by using the proportional hazards model, which is of the form µ(t, X) = µ0 (t) exp(β T X) where µ0 (t) denotes the baseline hazard function, assumed to be unique for all individuals in the study population. X is the vector of observed covariates and β the respective vector of regression parameters to be estimated. The mathematical convenience of this model is based on the separation of the effects of aging in the baseline hazard µ0 (t) from the effects of covariates in the parametric term exp(β T X). Two main reasons exist why it is often impossible to include all important factors on the individual level into the analysis. Sometimes too many covariates exist to be considered in the model, in other cases, the researcher does not know or is not able to measure all the relevant covariates. In both cases, two sources of variability exist in survival data: variability accounted for by measurable risk factors, which is thus theoretically predictable, and heterogeneity caused by unknown covariates, which is thus theoretically unpredictable, even if knowing all the relevant information. Advantages to separating these two sources of variability exist because heterogeneity in contrast to variability can explain some ‘‘unexpected’’ results or can provide an alternative explanation

FRAILTY MODELS

of some results. Consider, for example, nonproportional hazards or decreasing hazards when unexpected extra variability prevails. In a proportional hazards model, neglect of a subset of the important covariates leads to biased estimates of both regression coefficients and the hazard rate. The reason for such bias lies in the fact that the timedependent hazard rate results in changes in the composition of the study population over time with respect to the covariates. If two groups of patients exist in a clinical trial where some individuals experience a higher risk of failure, then the remaining individuals at risk tend to form a more or less selected group with a lower risk. An estimate of the individual hazard rate, without taking into account the unobserved frailty, would therefore underestimate the true hazard function and the extent of underestimation would increase as time progresses. The univariate frailty model extents the Cox model such that the hazard of an individual depends in addition on an unobservable random variable Z, which acts multiplicatively on the hazard function: µ(t, Z, X) = Zµ0 (t) exp(β T X)

(1)

Again, µ0 (t) is the baseline hazard function, β the vector of regression coefficients, X is the vector of observed covariates, and Z now is the frailty variable. The frailty Z is a random variable varying over the population, which lowers (Z < 1) or increases (Z > 1) the individual risk. Frailty corresponds to the notions liability or susceptibility in different settings (8). The most important point here is that frailty is unobservable. The respective survival function S, describing the fraction of surviving individuals in the study population, is given by
t S(t|Z, X) = exp(−Z

µ0 (s)ds exp(β T X)) (2) 0

S(t|Z, X) may be interpreted as the fraction of individuals surviving the time t after beginning of the follow-up given the vector of observable covariates X and frailty Z. Note, that Equation (1) and Equation (2) describe the same model using different notions. Up

3

to now, the model has been described at the level of individuals. However, this individual model is not observable. Consequently, it is necessary to consider the model at the population level. The survival function of the total population is the mean of the individual survival functions (Equation 2). It can be viewed as the survival function of a randomly drawn individual, and corresponds to that which is actually observed. It is important to note that the observed hazard function will not be similar to the individual hazard rate. What may be observed in the population is the net result for a number of individuals with different Z. The population hazard rate may have a completely different shape than the individual hazard rate as shown in Fig. 1. One important problem in the area of frailty models is the choice of the frailty distribution. The frailty distributions most often applied are the gamma distribution (2, 3), the positive stable distribution (9), a three-parameter distribution (PVF) (10), the compound Poisson distribution (11, 12) , and the log-normal distribution (13). Univariate frailty models are widely applied. A few examples that can be consulted for more details are listed here. Aalen and Tretli (6) applied the compound Poisson distribution to testicular cancer data already introduced above. The idea of the model is that a subgroup of men is particularly susceptible to testicular cancer, which results in selection over time. Another example is the malignant melanoma dataset including records of patients who had radical surgery for malignant melanoma (skin cancer) at the University Hospital of Odense in Denmark. Hougaard (14) compared the traditional Cox regression model with a gamma frailty and PVF frailty model, respectively, to these data. The third example deals with the time from insertion of a catheter into dialysis patients until it has to be removed because of infection. A subset of the complete data, including the first two infection times of 38 patients, was published by McGilchrist and Aisbett (13). To account for heterogeneity within the data, Hougaard (14) used a univariate gamma frailty model. Henderson and Oman (15) tried to quantify the bias that may occur in estimated

4

FRAILTY MODELS

Conditional and unconditional hazard rates Z=1

Z=2 hazard rates

Z = 0.5

unconditional

0

10

20

30

40

50 60 age

70

80

90 100 110

covariate effects, and fitted marginal distributions when frailty effects are present in survival data, but the latter are ignored in a misspecified proportional hazards analysis. Congdon (16) investigated the influence of different frailty distributions (gamma, inverse Gaussian, stable, binary) on total and cause-specific mortality from the London area (1988–1990).

Figure 1. Conditional and unconditional hazard rates in a simulated data set of human mortality. The red lines denote the conditional (individual) hazard rates for individuals with frailty 0.5, 1, and 2, respectively. The blue line denotes the unconditional (population) hazard rate

Averaging over an assumed distribution for the latent variables (e.g., using a gamma, log-normal, stable distribution) then induces a multivariate model for the observed data. In the case of paired observations, the twodimensional survival function is of the form
∞ S(t1 , t2 ) =

S(t1 |z, X 1 )S(t2 |z, X 2 )g(z)dz 0

2

MULTIVARIATE FRAILTY MODELS

A second important application of frailty models is in the field of multivariate survival data. Such kind of data occurs, for example, if lifetimes (or times of onset of a disease) of relatives (twins, parent-child) or recurrent events like infections in the same individual are considered. In such cases, independence between the clustered survival times cannot be assumed. Multivariate models are able to account for the presence of dependence between these event times. A commonly used and very general approach is to specify independence among observed data items conditional on a set of unobserved or latent variables (14). The dependence structure in the multivariate model develops from a latent variable in the conditional models for multiple observed survival times, for example, let S(t1 |Z, X 1 ) and S(t2 |Z, X 2 ) be the conditional survival functions of two related individuals with different vectors of observed covariates X 1 and X 2 , respectively [see Equation (2)].

where g denotes the density of the frailty Z. In the case of twins, S(t1 ,t2 ) denotes the fraction of twins pairs with twin 1 surviving t1 and twin 2 surviving t2 . Frailty models for multivariate survival data are derived under conditional independence assumption by specifying latent variables that act multiplicatively on the baseline hazard. 2.1 The Shared Frailty Model The shared frailty model is relevant to event times of related individuals, similar organs, and repeated measurements. Individuals in a cluster are assumed to share the same frailty Z, which is why this model is called shared frailty model. It was introduced by Clayton (3) and extensively studied in Hougaard (14). The survival times are assumed to be conditional independent with respect to the shared (common) frailty. For ease of presentation, the case of groups with pairs of individuals will be considered (bivariate failure times,

FRAILTY MODELS

for example, event times of twins or parentchild). Extensions to multivariate data are straightforward. Conditional on the frailty Z, the hazard function of an individual in a pair is of the form Zµ0 (t)exp(β T X), where the value of Z is common to both individuals in the pair, and thus is the cause for dependence between survival times within pairs. Independence of the survival times within a pair corresponds to a degenerate frailty distribution (Z = 1, V(Z) = σ 2 = 0). In all other cases with σ 2 > 0, the dependence is positive by construction of the model. Conditional on Z, the bivariate survival function is given as S(t1 , t2 |Z) = S1 (t1 )Z S2 (t2 )Z In most applications, it is assumed that the frailty distribution (i.e., the distribution of the random variable Z) is a gamma distribution with mean 1 and variance σ 2 . Averaging the conditional survival function produces under this assumption survival functions of the form 2

2

S(t1 , t2 ) = (S1 (t1 )−σ + S2 (t2 )−σ − 1)1/σ

2

Shared frailty explains correlations between subjects within clusters. However, it does have some limitations. First, it forces the unobserved factors to be the same within the cluster, which may not always reflect reality. For example, at times it may be inappropriate to assume that all partners in a cluster share all their unobserved risk factors. Second, the dependence between survival times within the cluster is based on marginal distributions of survival times. However, when covariates are present in a proportional hazards model with gamma distributed frailty, the dependence parameter and the population heterogeneity are confounded, which implies that the joint distribution can be identified from the marginal distributions (10). Third, in most cases, a one-dimensional frailty can only induce positive association within the cluster. However, some situations exist in which the survival times for subjects within the same cluster are negatively associated. For example, in the Stanford Heart Transplantation Study, generally the longer an individual must wait for an available heart, the shorter he or she is likely to survive after

5

the transplantation. Therefore, the waiting time and the survival time afterwards may be negatively associated. To avoid the above-mentioned limitations of shared frailty models, correlated frailty models were developed. 2.2 The Correlated Frailty Model Originally, correlated frailty models were developed for the analysis of bivariate failure time data, in which two associated random variables are used to characterize the frailty effect for each pair. For example, one random variable is assigned for partner 1 and one for partner 2 so that they would no longer be constrained to have a common frailty. These two variables are associated and have a joint distribution. Knowing one of them does not necessarily imply knowing the other. A restriction no longer exists on the type of correlation. These two variables can also be negatively associated, which would induce a negative association between survival times. Assuming gamma distributed frailties, Yashin and Iachine (17) used the correlated gamma frailty model resulting in a bivariate survival distribution of the form S(t1 , t2 ) =

S1 (t1 )1−ρ S2 (t2 )1−ρ (S1 (t1 )−σ 2 + S2 (t2 )−σ 2 − 1)ρ/σ 2

Examples of the use of multivariate frailty models are various and emphasize the importance of this family of statistical models for survival data. – a shared log-normal frailty model for the catheter infection data mentioned above used by McGilchrist and Aisbett (13); – a shared frailty model with gamma and log-normal distributed frailty, applied to the recurrence of breast cancer by dos Santos et al. (18); – a shared positive stable frailty model, applied by Manatunga and Oakes (19) to the data from the Diabetic Retinopathy Study, which examined the effectiveness of laser photo-coagulation in delaying the onset of blindness in patients with diabetic retinopathy. The positive stable frailty allows for proportional hazards both in the marginal and the conditional model;

6

FRAILTY MODELS

– a study of Andersen et al. (7), who tested for center effects in multi-center survival studies by means of a frailty model with unspecified frailty distribution; – a correlated gamma-frailty model, applied by Pickles et al. (20) to age of onset of puberty and antisocial behavior in British twins; – a correlated gamma-frailty model by Yashin and Iachine (17) and Yashin et al. (21) to analyze mortality in Danish twins; – a correlated gamma-frailty model by Wienke et al. (22) and Zdravkovic et al. (23) to analyze genetic factors involved in mortality caused by coronary heart disease in Danish and Swedish twins, respectively; – an extension of the correlated gammafrailty model by Wienke et al. (25) used to model death due to coronary heart disease in Danish twins; – different versions of the correlated gamma-frailty model applied by Zahl (26) to causespecific cancer mortality in Norway to model the excess hazard.

3

SOFTWARE

Stata 7 (procedure st streg) allows one to explore univariate models with gamma and inverse Gaussian distributed frailty. aML 2 supports log-normal frailty models in univariate analysis. WinBugs is designed for analysis of shared frailty models with different frailty distribution, using Markov Chain Monte Carlo methods. On the Internet, several SAS, GAUSS, S-Plus, and R routines are available dealing with different frailty models. REFERENCES 1. M. Greenwood and G. U. Yule, An inquiry into the nature of frequency distributions representative of multiple happenings with particular reference to the occurrence of multiple attacks of disease or of repeated accidents. J. Royal Stat. Soc. 1920; 83: 255–279.

2. J. W. Vaupel, K. G. Manton, and E. Stallard, The impact of heterogeneity in individual frailty on the dynamics of mortality. Demography 1979; 16: 439–454. 3. D. G. Clayton, A model for association in bivariate life tables and its application in epidemiological studies of familial tendency in chronic disease incidence. Biometrika 1978; 65: 141–151. 4. D. R. Cox, Regression models and life-tables. J. Royal Stat. Soc. B 1972; 34: 187–220. 5. S. Manton, Methods for evaluating the heterogeneity of aging processes in human populations using vital statistics data: explaining the black/white mortality crossover by a model of mortality selection. Human Biol. 1981; 53: 47–67. 6. O. O. Aalen and S. Tretli, Analysing incidence of testis cancer by means of a frailty model. Cancer Causes Control 1999; 10: 285–292. 7. P. K. Andersen, J. P. Klein, and M-J. Zhang, Testing for centre effects in multi-centre survival studies: a Monte Carlo comparison of fixed and random effects tests. Stat. Med. 1999; 18: 1489–1500. 8. D. S. Falconer, The inheritance of liability to diseases with variable age of onset, with particular reference to diabetes mellitus. Ann. Human Genet. 1967; 31: 1–20. 9. P. Hougaard, A class of multivariate failure time distributions. Biometrika 1986; 73: 671–678. 10. P. Hougaard, Survival models for heterogeneous populations derived from stable distributions. Biometrika 1986; 73: 671–678. 11. O. O. Aalen, Heterogeneity in survival analysis. Stat. Med. 1988; 7: 1121–1137. 12. O. O. Aalen, Modeling heterogeneity in survival analysis by the compound Poisson distribution. Ann. Appl. Probabil. 1992; 4(2): 951–972. 13. C. A. McGilchrist and C. W. Aisbett, Regression with frailty in survival analysis. Biometrics 1991; 47: 461–466. 14. P. Hougaard, Analysis of Multivariate Survival Data. New York: Springer, 2000. 15. R. Henderson and P. Oman, Effect of frailty on marginal regression estimates in survival analysis. J. Royal Stat. Soc. B 1999; 61: 367–379. 16. P. Congdon, Modeling frailty in area mortality. Stat. Med. 1995; 14: 1859–1874. 17. A. I. Yashin and A. I. Iachine, Genetic analysis of durations: correlated frailty model applied

FRAILTY MODELS

18.

19.

20.

21.

22.

23.

to survival of danish twins. Genet. Epidemiol. 1995; 12: 529–538. D. M. dos Santos, R. B. Davies, and B. Francis, Nonparametric hazard versus nonparametric frailty distribution in modeling recurrence of breast cancer. J. Stat. Plan. Infer. 1995; 47: 111–127. A. K. Manatunga and D. Oakes, Parametric analysis of matched pair survival data. Lifetime Data Anal. 1999; 5: 371–387. A. Pickles et al., Survival models for developmental genetic data: age of onset of puberty and antisocial behavior in twins. Genet. Epidemiol. 1994; 11: 155–170. A. I. Yashin, J. W. Vaupel, and I. A. Iachine, Correlated individual frailty: an advantageous approach to survival analysis of bivariate data. Math. Pop. Studies 1995; 5: 145–159. A. Wienke, N. Holm, A. Skytthe, and A. I. Yashin, The heritability of mortality due to heart diseases: a correlated frailty model applied to Danish twins. Twin Res. 2001; 4: 266–274. S. Zdravkovic et al., Heritability of death from coronary heart disease: a 36-year follow-up of 20,966 Swedish twins. J. Intern. Med. 2002; 252: 247–254.

7

24. S. Zdravkovic et al., Genetic influences on CHD-death and the impact of known risk factors: Comparison of two frailty models. Behavior Genetics 2004; 34: 585–591. 25. A. Wienke, K. Christensen, A. Skytthe, and A. I. Yashin, Genetic analysis of cause of death in a mixture model with bivariate lifetime data. Stat. Model. 2002; 2: 89–102. 26. P. H. Zahl, Frailty modelling for the excess hazard. Stat. Med. 1997; 16: 1573–1585.

FURTHER READING P. Hougaard, Modeling heterogeneity in survival data. J. Appl. Probabil. 1991; 28: 695–701. A. Wienke, P. Lichtenstein, K. Czene, and AA. I. Yashin, The role of correlated frailty models in studies of human health, ageing and longevity. In: Applications to Cancer and AIDS Studies, Genome Sequence Analysis, and Survival Analysis. N. Balakrishnan, J.-L. Auget, M. Mes¨ bah, G. Molenberghs (eds.), Birkhauser, 2006, pp. 151–166.

FUTILITY ANALYSIS

with a one-sided hypothesis. Key contributions to development of the futility monitoring concept include introduction of stochastic curtailment by Lan et al. (6) and adaptation of this methodology to the futility index by Ware et al. (1). First, we present the common statistical approaches to futility monitoring. Two examples of studies stopped for futility are then provided in detail. Finally, practical issues of futility monitoring are discussed.

BORIS FREIDLIN Biometric Research Branch National Cancer Institute Bethesda, Maryland

Interim monitoring of outcome data has become a well-accepted component of randomized clinical trials (RCT); a trial can be stopped early for efficacy if a treatment arm appears definitively better than another arm. Most clinical trials are designed to demonstrate benefit of experimental vs. standard treatment and are thus implicitly addressing a one-sided hypothesis. The one-sided nature of the question provides rationale for both efficacy and futility monitoring. In technical terms, futility monitoring refers to a statistical procedure for stopping the trial early if it appears that the experimental arm is unlikely to be shown definitively better than the control arm if the trial is continued to the final analysis. In the context of evidence-based clinical research, the primary goal of a phase III trial is to provide data on the benefit-to-risk profile of the intervention that is sufficiently compelling to change medical practice. From this perspective, a futility boundary should be interpreted as the point at which convincing evidence exists to resolve the original question—that is, to convince the clinical community that the new treatment is not beneficial. The advantages of early futility stopping are obvious in terms of minimizing patient exposure to ineffective potentially toxic experimental treatments as well as in terms of optimizing the use of resources (1). However, these potential advantages should be weighed against the risk of stopping short of obtaining sufficiently compelling evidence that the new treatment is not beneficial (2–4). Such premature stopping wastes time and resources that went into the design and conduct of the trial and may adversely affect similar ongoing and future trials. There is a rich literature on futility monitoring. DeMets and Ware (5) were among the first to point out the need for a more aggressive monitoring for futility in studies

1 COMMON STATISTICAL APPROACHES TO FUTILITY MONITORING 1.1 Statistical Background For simplicity, the following presentation assumes that the study outcome has a normal distribution (or has been transformed to a normal outcome). The results are readily adapted to other common clinical outcome types (e.g., time-to-event or binary endpoints) where the normal approximation is routinely used in the monitoring of accumulating data (7, 8). Consider a RCT that compares an experimental arm (arm A) to the control arm (arm B). Let θ denote the primary measure of treatment effect (e.g., difference in cholesterol levels or log hazard ratio for survival) with θ = 0 corresponding to no difference between the arms and positive values of θ corresponding to an advantage of the experimental treatment. The trial is designed to test H 0 :θ = 0 versus HA :θ = θ A , where θ A > 0 is the minimally clinically meaningful treatment effect. Relevant design calculations are θˆ often based on the test statistic Z = SE( θˆ ) ˆ is an estimate of the standard where SE(θ) ˆ The quantity SE(θ) ˆ −2 is called error of (θ). the information (I) about the parameter θ . The design rejects H 0 if at the final analysis Z > cα (cα denotes the 1 − α quantile of the standard normal distribution). Type I error and type II error of the design are defined as PH0 {Z > cα } and PH1 {Z ≤ cα }, respectively. The amount of information needed to detect θ A with type I error α and type II error β is:
I=

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

cα + cβ θA

2 .

2

FUTILITY ANALYSIS

CPk (θ ) = P(Zk > cα |Zk = zk , θ )

curtailment (6). The most common implementation of stochastic curtailment calculates conditional power at the minimally clinically meaningful treatment effect θ A and stops at the kth analysis if CPk (θ A ) < γ for some prespecified γ . This formulation of futility rule is easily understood by nonstatistical consumers of clinical trials such as clinicians and patients (1). Use of formal rules based on conditional power at the design alternative CPk (θ A ) is sometime criticized because it conditions on current data and a value of θ that may be inconsistent with the current data. This criticism is somewhat misplaced because CPk (θ A ) is intended to quantify the degree of consistency between the observed data and the trial goal as represented by parameters (θ A ,α,β). In any case, the real life applications of stochastic curtailment method to futility monitoring typically involve evaluation of conditional power over a range of potential values of parameter θ that includes both biologically plausible values consistent with the accumulated data and the treatment effect used in the design of the study (11). In another variation of stochastic curtailment, the stopping rule is based on the conditional power calculated under the current estimate of treatment effect θˆk , that is, stop at kth analysis if CPk (θˆk ) < γ . This so-called ‘‘current trend’’ approach is problematic as it may lead to premature stopping, especially early into the trial when the estimate θˆk may be highly unreliable (see the antenatal corticosteroids example). As an alternative, Pepe and Anderson (12) proposed a more conservative approach using CPk (θˆk + SE(θˆk )). Futility boundaries based on conditional power can be derived as follows. At the kth interim analysis, the conditional distribution is √ ZK , given √ Zk = zk is normal with mean zk tk + θ IK (1 − tk ) and variance 1 − tk . The conditional power is then   √ √ cα − zk tk − θ IK (1 − tk ) CPk (θ ) = 1 −  √ 1 − tk

The conditional power is an appealing way to summarize the accumulated data to quantify the feasibility of continuing the study to the full accrual. If CPk (θ ) is low for θ in the desirable range, the study may be stopped for futility. This approach is known as stochastic

where ( ) is the cumulative standard normal distribution function. Hence the condition CPk (θ ) < γ corresponds to stopping for futility at kth analysis if √  (1 − tk ) 1 − tk cα − θ IK √ Zk < √ − cγ √ tk tk tk

For a specific outcome type, information I is translated into the required sample size. For ethical and economic reasons, it is important to stop the trial as soon as sufficiently compelling evidence has accumulated for either rejection of H0 in favor of H A (efficacy monitoring) or if there is no longer a reasonable chance that H0 can be rejected (futility monitoring). Most RCTs have prospectively specified interim monitoring guidelines that integrate the interim and the final analysis into a single framework. A typical interim analysis plan specifies K analyses, with the first K − 1 analyses designated as interim and the Kth as the final analysis. The test statistic and information at the kth analysis (k = 1, . . . , K) are denoted by Zk and Ik , respectively. The quantity tk = Ik /IK (0 < tk ≤ tK = 1) represents the proportion of the total information at the kth analysis and is often referred to as the ‘‘information time’’ (9). Information time provides a convenient universal notation and will be used in the following presentation. Formal derivation of futility boundaries is based on the following property that holds for normal outcomes with independent observations: statistics (Z1 , . . . , ZK ) are multivariate with √ normal √ √ K-dimensional mean vector √ θ IK ( t1 , . . . , tK ) and cov(Zl , Zk ) = tl /tk for l < k. It can also be shown that under some mild conditions the result holds asymptotically for most non-normally distributed outcomes that have been transferred to normal outcomes (10). 1.2 Conditional Power and Stochastic Curtailment Conditional power at the kth analysis is defined as the probability of rejecting the null hypothesis at the final analysis given the currently observed data and a hypothesized true treatment effect θ :

FUTILITY ANALYSIS

Because futility monitoring does not involve rejecting H 0 , the type I error of the study is not inflated. Futility monitoring does inflate the type II error. Lan et al. (6) have shown that continuous application of the stochastic curtailment futility rule CPk (θ A ) < γ on a fixed sample size study with type II error β has type II error bounded by β/(1 − γ ). Because in most implementations interim monitoring is not continuous, this bound is very conservative. A more accurate bound can be found in Davis and Hardy (13). See Lachin (14) and Moser and George (15) for a discussion of integrating the stochastic curtailment futility boundary with an efficacy boundary for simultaneous control of type I and type II error rates. An alternative to calculating conditional power at a fixed value of parameter θ is the use of predictive power (16, 17). It is derived by averaging the conditional power with respect to the posterior distribution for parameter θ :  PPk = CPk (θ )π (θ |Dk )dθ where π (θ |Dk ) denotes the posterior distribution and Dk is the observed data at the kth analysis. 1.3 Group Sequential Formulation of Futility Boundary Another common approach to futility monitoring is the one-sided group sequential rule of Pampallona and Tsiatis (18) (herein referred to as PT) that is based on the power family group sequential tests (19). The rule combines both efficacy and futility boundaries to control the overall type I and type II error rates. Boundaries depend on a parameter  and are defined by pairs of critical values (ak ,bk ) for k = 1, . . . , K, subject to ak ≥ bk and aK = bK . At the kth analysis time, the decision rule is: if Zk > ak , stop for efficacy; if Zk < bk , stop for futility; otherwise, continue. The critical values ak and bk are of the form ak = C1 t−0.5 k and , bk = θA Ik0.5 − C2 t−0.5 k

3

where constants C1 and C2 are selected to satisfy the specified type I and type II error rates for fixed . Here, we focus on the futility boundary {bk }. This futility procedure corresponds to a one-sided group sequential boundary for testing the hypothesis HA against H 0 . The boundary uses a spending function to control the overall error rate at level β. The shape of the spending function is determined by parameter . For example,  = 0 corresponds to an O’Brien–Fleming type boundary (20) whereas  = 0.5 corresponds to a Pocock type boundary (21). In practice, the most commonly used value of  is 0. It can be shown that a PT futility boundary with  = 0 is similar to the stochastic curtailment with γ = 0.5 (22). More generally, Moser and George (15) show that a PT boundary can be expressed in terms of the generalized stochastic curtailment procedure and vice versa. A number of published group sequential approaches to futility are based on repeated tests of the alternative hypothesis. For example, the following rule is suggested (23, 22): at an interim analysis, terminate the trial for futility if a one-sided test of consistency of the data with HA is statistically significant at some fixed nominal significance level (e.g., 0.005 or 0.001). This procedure has negligible effect on power (22). The repeated test rules are statistically equivalent to the approaches based on repeated confidence intervals (4, 24, 25). 1.4 Other Statistical Approaches to Constructing Futility Boundaries Another simple futility rule was proposed by Wieand et al. (26) (see also Ellenberg and Eisenberg [27]). A single futility analysis is performed when half of the total trial information is available. If the estimate of the treatment effect θˆ < 0 (e.g., the log hazard ratio of the control arm to the experimental arm is less than 0), early termination of the trial is recommended. This simple rule results in negligible deflation of the type I error and less than 2% loss of the fixed-sample-size design power. This approach was used for futility stopping in randomized studies of the somatostatin analogue octreotide for advanced colon cancer

4

FUTILITY ANALYSIS

(28) and for advanced pancreatic cancer (29). Although the simplicity of this rule is appealing, having long intervals between scheduled analyses may not be appropriate in some settings (30). A Bayesian framework can offer a useful tool for futility monitoring deliberations. Bayesian analysis provides a formal way for combining the totality of the prior and/or external information with the data accumulated in the trial. A futility decision is typically based on considering an ‘‘enthusiastic’’ prior—that is, a prior reflecting the opinion of the proponents of the new treatment. For example, in a normal setting, such a prior may be defined by centering on the alternative hypothesis θ A and fixing the probability of no benefit (θ < 0) at some low value (e.g., 5%). The posterior distribution of the treatment effect θ given the current data can then be used to model how the accumulated data affect the beliefs of the proponents of the intervention and whether the observed negative trend is sufficiently compelling to stop the trial for futility. For further discussion, see Spiegelhalter et al. (31). 2

EXAMPLES

Examples of futility stopping from two randomized clinical trials are now presented in detail. 2.1 Optimal Duration of Tamoxifen in Breast Cancer National Surgical Breast and Bowel Project (NSABP) B-14 trial investigated the use of

tamoxifen in early-stage breast cancer. The relevant portion of the study for the discussion here was designed to address the optimal duration of tamoxifen administration. Breast cancer patients who had completed 5 years of tamoxifen therapy free of disease recurrence were randomized to either tamoxifen or placebo for another 5 years (second randomization). The study was designed to have 85% power (α = 0.05 one-sided) to detect a 40% reduction in disease-free survival (DFS) hazard rate after observing 115 events (an event was defined as recurrence of breast cancer, occurrence of new primary cancer, or death from any cause). The study was closed after the third interim analysis with the conclusion that a meaningful benefit of continuing tamoxifen would not be demonstrated. This result went against the prevailing clinical opinion at that time. The study team provided a detailed account of the various futility monitoring approaches that had lead to the decision (32). The summary of DFS and the secondary endpoint overall survival (OS) from the three interim analyses is given in Table 1. At the time of the first interim analysis, a developing negative trend was observed but the data were considered immature. At the time of the second interim analysis, conditional power (at 115 events) under the design alternative of 40% reduction in hazard was less than 5%. In fact, even under a threefold reduction (67%) the conditional power was less than 5%. However, an argument was made that [1] the design alternative (40% reduction in hazard) that was roughly equivalent to the size of benefit for the patients

Table 1. Interim Analysis of Results from the National Surgical Breast and Bowel Project (NSABP) B-14 trial Analysis number Information time (observed/planned number of events) Placebo arm: DFS events (deaths) Tamoxifen arm: DFS events (deaths) Hazard ratio (tamoxifen/placebo) DFS P-value (two-sided) Conditional power under the design alternative

1

2

3

40% (46/115)

58% (67/115)

76% (88/115)

18 (6) 28 (9) NA NA NA

24 (10) 43 (19) 1.8 0.028 <5%

32 (13) 56 (23) 1.7 0.015 0

Note: DFS, disease-free survival; NA, not applicable. Source: Dignam et al. Control Clin Trials. 1998; 19: 575–588.

FUTILITY ANALYSIS

receiving the first 5 years of tamoxifen was not realistic, and [2] the sizing of the study was driven in part by the limitations of the eligible population pool. A trial with same parameters targeting a more realistic 30% reduction in hazard would require 229 events. Based on this projection, conditional power was approximately 10% under a 30% reduction in hazard and 40% under a 40% reduction in hazard (see Figure 3 in Dignam et al. [32]). According to the study team, these conditional power calculations suggested that a ‘‘sufficient statistical power to detect the benefit’’ was still present and that these considerations contributed to the decision to continue the trial at that time. At the third interim analysis, there was no possibility for a statistically significant result under the existing 115-event design (even if all 27 remaining events would occur in the control arm, the final test would not achieve the predetermined level of statistical significance). For a 229-event design, the conditional power was approximately 3% under a 30% reduction in hazard and 15% under a 40% reduction in hazard. A Bayesian approach to futility, using a normal approximation for the distribution of the log hazard ratio, was also examined by the investigators. An enthusiastic prior was centered at the alternative treatment effect θ A (40% reduction in hazard), with the variance corresponding to a 5% probability of no benefit (θ ≤ 0). At the third interim analysis, the posterior probability of any benefit for tamoxifen arm (θ > 0) was 13%, and the posterior probability of the treatment effect exceeding θ A was 0.00003.

2.2 A Randomized Study of Antenatal Corticosteroids Pregnant women at risk for preterm delivery are often given weekly courses of antenatal corticosteroids. To assess the efficacy of weekly administration, a randomized doubleblind, placebo-controlled, phase III trial of weekly administration versus a single course was conducted Guinn et al. (33). The trial was designed to enroll 1,000 women to give 90% power to detect a 33% reduction in composite neonatal morbidity from 25% to 16.5% (0.66 relative risk) at the 0.05 two-sided significance level (note that while a two-sided significance level was used for sample size calculation the study is addressing a one-sided question). The study results are summarized in Table 2 (34). At a planned interim analysis after 308 women had been randomized and evaluated, observed composite morbidity was 24% on the weekly arm and 27% on single-course groups, corresponding to only a 11% relative reduction. The investigators reported that under the ‘‘observed trend’’ the conditional power at 1,000 patients was less than 2%. (Guinn et al. do not detail the derivation of the 2% result, but based on the reported data the conditional power under ‘‘trend observed’’ in the first 308 patients was approximately 14%.) The reported conditional power (along with some external safety reports) was used as the rationale to stop the trial after accrual of 502 patients (194 additional patients). Had the investigators calculated conditional power under the design alternative, they would have obtained conditional power of 74%. After the study was finished, observed morbidity rates in the last

Table 2. Interim Analysis Results from Antenatal Corticosteroids Trial (as reported in [34]) First 308 patients Weekly # of patients # of morbidities (%) RR 95% CI Conditional power ‘‘current trend’’ Conditional power design alternative

Single

Last 194 patients Weekly

Single

161 147 39 (24.2) 40 (27.2) .89 (.61, 1.30) 14%

95 99 17 (17.8) 26 (26.2) .68 (.40, 1.74) 75%

74%

77%

Source: Jenkins et al. JAMA. 2002; 287: 187–188.

5

Total 502 patients Weekly

Single

256 246 56 (21.9) 66 (26.8) .82 (.6, 1.11) 40% 77%

6

FUTILITY ANALYSIS

194 patients were 17.9% in the weekly arm and 26.2% in the single-course arm, corresponding to a 32% reduction (very close to the 34% target effect) (34). At the time of the trial closure with 502 patients, conditional power (under the design alternative) was approximately 77%, and conditional power under the ‘‘current trend’’ was 40%. These data also illustrate the unstable nature of the ‘‘current trend’’ approach early in the study: conditional power ranged from 14% in the first 308 patients to 75% in the next 194. 3

DISCUSSION

The decision to stop a trial for futility is inherently complex. In most cases, it involves considerations beyond whether the primary outcome crossed a formal boundary. Such a fundamental decision requires a thorough evaluation of all relevant data (external and internal). Issues requiring careful consideration include consistency of results across primary and secondary outcomes, treatment group similarity, and complete and unbiased evaluation of outcome (35). The degree of evidence required to address the study objectives as well as justify early futility stopping depends on the nature of the intervention, the context of the disease, and the current standard of care. Randomized clinical trials are often conducted in the presence of efficacy evidence from randomized trials in similar clinical settings, epidemiologic studies, and surrogate endpoints. In some situations, experimental intervention is already in limited use in the community, and the trial is designed to provide definitive evidence to support widespread use. In such cases, stopping the trial early requires a clear refutation of meaningful benefit (4). Hence, futility decisions are typically made only after a definitive negative trend is established. This point of view is often taken in large trials in cardiovascular, cancer-prevention, and adjuvant cancer settings (32, 36). In contrast, in settings such as advanced cancer, the requirements for preliminary evidence of efficacy are relatively low and a large proportion of agents tested in phase III studies turn out to be inactive. In such settings, there is a need for an efficient approach for testing new

agents and weaker evidence (or rather a lack of evidence of activity) is sufficient to affect medical practice (28, 29). These more aggressive futility standards allow one to redirect patients to other trials. A discrepancy between the futility monitoring rule and the degree of evidence required in the particular setting may result in the failure to answer an important clinical question as well as waste time and resources. A somewhat extreme example is provided by the use of the ‘‘current trend’’ conditional power in the antenatal corticosteroids trial (33). The investigators used conditional power under the ‘‘observed trend’’ with less than one-third of the total information available to justify stopping the trial. At that time, conditional power under the design alternative was 74%. Moreover, the observed data at that time did not rule out the target relative risk of 0.66. Consequently, the use of ‘‘current trend’’ conditional power early in this study was inappropriate. One can argue that the study failed to address its goal (37). Some of the rules described previously suggest stopping for futility with the experimental arm doing nontrivially better than the control arm. For example, consider the commonly used PT futility approach with  = 0. For a trial designed to detect a hazard ratio of 1.5 with four interim analyses (one-sided α = 0.025 and β = 0.1), the rule calls for stopping for futility at the fourth analysis (with 80% of total information) if the observed hazard ratio is (just less than) 1.22 with 90% confidence interval (0.98, 1.52; P = 0.068). For a trial that is designed to show a 50% improvement in median survival, a 22% increase in median survival time is a meaningful clinical effect in many cases. Furthermore, the upper confidence limit includes the design target of 1.5 while the lower confidence limit excludes all but a negligible inferiority of the experimental arm. Individuals reading the results of the study may infer that the experimental treatment is worth pursuing even though it was not as effective as the study investigators may have hoped. This leads to questions as to whether stopping the trial early was a correct decision. This example reiterates the point that investigators designing a trial should carefully examine the monitoring boundaries they are considering to ensure that they

FUTILITY ANALYSIS

would be comfortable stopping the trial for futility for certain observed positive differences (22). In addition to the potential harm to the integrity of the ongoing study, a premature stopping may jeopardize completion of other ongoing trials addressing this or similar questions (38). The decision to stop (or not to stop) a large study that is addressing a major public health question is often challenged by a wider audience. Even after thorough deliberation, some may question the wisdom and implications of the ultimate decision. For example, the early stopping of NSABP B14 was criticized (39). Some interim monitoring approaches integrate efficacy and futility boundaries under a single framework (15, 18). This allows simultaneous control of the type I and type II error rates. Others take a more informal approach to futility and implicitly argue that because futility monitoring does not inflate type I error no formal adjustment is needed. In general, due to the complexity of the monitoring process, interim monitoring boundaries are considered to be guidelines rather than strict rules. This is especially relevant to the futility stopping that in practice is often based on a synthesis of efficacy, safety, and feasibility considerations both internal and external to the ongoing trial (by contrast, common efficacy monitoring scenarios are driven by strong internal evidence of benefit with respect to the primary clinical outcome). On a purely mathematical level, the integration of the efficacy and futility boundaries typically involves adjusting the upper boundary downward to compensate for the deflation of the type I error due to the possibility of futility stopping. If the futility boundary is not strictly adhered to, the overall type I error of the design is not maintained. Therefore, due to the different nature of the efficacy and futility stopping and the priority to control type I error in many settings, the downward adjustment of the efficacy boundary may not be justified. This can potentially become an issue when the study is intended to support approval by a regulatory agency (40). Thus far, an implicit assumption in the discussion has been that the design treatment effect θ A represents a minimally clinically meaningful benefit and that it is within

7

the range of biological plausibility. However, some RCTs are sized using a hypothesized treatment effect that is a compromise between a realistic target supported by clinical/biological evidence and the feasibility considerations (achievable sample size and timely completion of the study). As a result, some studies, especially those in rare diseases, are often sized using an unrealistic target treatment effect. When an interim futility analysis conditioned on the observed data suggests that the study is unlikely to be positive if continued to the planned sample size, this may to a large degree reflect the unrealistic target effect. In such situations, more conservative futility boundaries should be used. The exact cut-off value γ used in stochastic curtailment may vary depending on the context of the trial. However, the use of γ = 0.5 or higher (i.e., to stop a trial for futility with a conditional power of 0.5 or higher) seems to be inconsistent with the original design. Recall that this conditional power calculation is done under the alternative hypothesis such that at the beginning of the trial the conditional power is the unconditional power (0.9, or possibly 0.8). This high unconditional power reflects the supporting preliminary data and rationale that are used to justify the expending of effort and resources to set up and conduct a trial. Therefore, once the trial is started, the drop in power from 0.9 (or 0.8) to 0.5 might not provide sufficient evidence to stop the trial early and risk wasting the resources that went into setting up and conducting the trial. It is instructive to draw an analogy with the widely accepted O’Brien–Fleming efficacy boundary. As mentioned previously, the boundary corresponds to stopping with conditional power 0.5 under the null (CPk (0) = 0.5); that is, the ‘‘conditional type I error’’ is 0.5. Thus, for a trial designed with ‘‘unconditional’’ 0.05 (0.025) type I error, this rule requires a 10- (20)-fold increase in conditional power to justify stopping the trial for efficacy. Although the exact choice of γ depends on circumstances, values in the 0.1 to 0.3 range are generally more consistent with common trial designs.

8

FUTILITY ANALYSIS

In many cases, it may be difficult to identify a simple expression for a stopping boundary that has the desired properties throughout the trial. Many commonly used futility rules may appear to be too conservative or too liberal at different information/logistic stages of a study. A futility boundary based on one statistical approach can usually be defined through other formulations (e.g., under a general definition, any futility stopping rule can be expressed in terms of the general stochastic curtailment approach or a repeated testing of the alternative hypothesis approach [15]). This can be useful in developing futility rules with desirable operational characteristics, in elucidating these rules to the data monitoring committee members, and in presetting the study results to the clinical community (41). In addition, increased flexibility can be achieved by selecting different values for the conditional power cut-off γ (or, equivalently, selecting different significance levels for the alternative hypothesis testing formulation), depending on the proportion of available information or depending on where the trial is in terms of accrual and follow-up. For example, different values of conditional power can be used during [1] the accrual period and [2] the follow-up period when all of the patients are off treatment. This strategy adjusts for the different impact of stopping for the study patients and the results. The first scenario implies terminating accrual and possible treatment change for the patients who are still on the study medication; some data are lost, and the total design information is not reached. The second scenario means only releasing the study data early; complete follow-up data are generally available after an additional follow-up period. In general, we recommend [1] scheduling futility analyses at the same time as efficacy analyses (starting at 25% to 30% of the total information) (30), and [2] requiring a stronger level of evidence for stopping early in the trial. The responsibility of monitoring of RCTs is best handled by a data monitoring committee that is independent from the principal investigator and the sponsor and thus is free from potential conflicts of interest. A failure to provide transparent and independent futility monitoring may lead to controversy. For example, Henke et al. (42) reported results

from a randomized, placebo-controlled trial of erythropoietin for head and neck cancer. The paper was unclear as to who was doing the interim monitoring and stated that the drug company sponsoring the study had decided to omit the second of the two planned interim analyses. Two years after the scheduled date of the omitted interim analysis, the final analysis was carried out. It revealed a significant impairment in cancer control and in survival for the erythropoietin arm relative to the placebo arm: the locoregional progression-free survival (the primary endpoint) hazard ratio was 1.62 (P = 0.0008), and the overall survival hazard ratio was 1.39 (P = 0.02). Although the study outcome at the time of the omitted analysis is not available, it is hypothetically possible that if the omitted interim analysis were carried out the study would have closed for futility and the medical community would have had the benefit of such information 2 years earlier than they did. The fact that the decision to cancel the second interim analysis was made by the sponsor rather than by an independent data monitoring committee created an apparent potential conflict of interest (43). In addition to the futility considerations based on low conditional power (disappointing observed treatment effect), futility may also be affected by such factors as lower than expected event rates or slower than expected accrual (44). If the observed control arm event rate is lower than that used at the design stage (to calculate the sample-size/duration needed for the desired power), the study may not be able to accumulate enough information (within a reasonable time period) to achieve adequate power and reliably address its goals. For example, one of the considerations in the early termination of the aspirin component of the Physicians Health Study (45) was that, due to a fivefold lower than expected cardiovascular death rate, the trial would have to be extended another 10 years to achieve the prespecified power (46). Note that such post hoc unconditional power analyses are usually used in conjunction with a conditional power analysis (11). In summary, a clinical trial to which substantial time and resources have been expended should provide an unequivocal resolution of the primary question. Premature

FUTILITY ANALYSIS

stopping may have a major impact on the future of the intervention, clinical care, and research directions. Therefore, futility stopping rules should be consistent with the study objectives in requiring a degree of evidence that is sufficiently convincing to the relevant medical community. Futility procedures should be carefully outlined in the protocol. If a trial is stopped early, then a clear explanation of the procedure and justification should be reported. REFERENCES 1. J. H. Ware, J. E. Muller, and E. Braunwald, The futility index. Am J Med. 1985; 78: 635–643. 2. S. J. Pocock, When to stop a clinical trial. BMJ. 1992; 305: 235–240. 3. P. W. Armstrong and C. D. Furberg, Clinical trial data and safety monitoring boards. Circulation. 1995; 91: 901–904. 4. D. L. DeMets, S. J. Pocock, and D. G. Julian, The agonizing negative trend in monitoring of clinical trials. Lancet. 1999; 354: 1983–1988. 5. D. L. DeMets and J. H. Ware, Group sequential methods for clinical trials with one-sided hypothesis. Biometrika. 1980; 67: 651–660. 6. K. K. G. Lan, R. Simon, and M. Halperin, Stochastically curtailed tests in long-term clinical trials. Commun Stat Theory Methods. 1982; 1: 207–219. 7. J. Whitehead, The Design and Analysis of Sequential Clinical Trials, 2nd ed. Chichester, UK: Wiley, 1997. 8. C. Jennison and B. W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. Boca Raton, FL: Chapman & Hall/CRC, 2000. 9. K. K. G. Lan and J. Wittes, The B-value: a tool for monitoring data. Biometrics. 1988; 44: 579–585. 10. D. O. Scharfstein, A. A. Tsiatis, and J. M. Robins, Semiparametric efficiency and its implications on the design and analysis of the group-sequential trials. J Am Stat Assoc. 1997; 92: 1342–1350. 11. Lachin JM and Lan SH, for the Lupus Nephritis Collaborative Study Group. Termination of a clinical trial with no treatment group difference: the lupus nephritis collaborative study. Control Clin Trials. 1992; 13: 62–79. 12. M. S. Pepe and G. L. Anderson, Two-stage experimental designs: early stopping with a negative result. Appl Stat. 1992; 41: 181–190.

9

13. B. R. Davis and H. J. Hardy, Upper bounds on type I and type II error rates in conditional power calculations. Commun Stat Theory Methods. 1990: 19: 3572–3584. 14. J. M. Lachin, A review of methods for futility stopping based on conditional power. Stat Med. 2005; 24: 2747–2764. 15. B. K. Moser and S. L. George, A general formulation for a one-sided group sequential design. Clin Trials. 2005; 2; 519–528. 16. S. C. Choi, P. J. Smith, and D. P. Becker, Early decision in clinical trials when treatment differences are small. Control Clin Trials. 1985; 6: 280–288. 17. D. J. Spiegelhalter, L. S. Freedman, and P. R. Blackburn, Monitoring clinical trials: conditional or predictive power. Control Clin Trials. 1986; 7: 8–17. 18. S. Pampallona and A. A. Tsiatis, Group sequential designs for one-sided and two-sided hypothesis testing with provision for early stopping in favor of the null hypothesis. J Stat Plan Inference. 1994; 42: 19–35. 19. S. K. Wang and A. A. Tsiatis, Approximately optimal one-parameter boundaries for group sequential trials. Biometrics. 1987; 43: 193–200. 20. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics. 1979; 35: 549–556. 21. S. J. Pocock, Interim analyses for randomized clinical trials: the group sequential approach. Biometrics. 1982; 38: 153–162. 22. B. Freidlin and E. L. Korn, A comment on futility monitoring. Control Clin Trials. 2002; 23: 355–366. 23. T. R. Fleming, D. P. Harrington, and P. C. O’Brien, Designs for group sequential tests. Control Clin Trials. 1984; 5: 348–361. 24. T. R. Fleming and L. F. Watelet, Approaches to monitoring clinical trials. J Natl Cancer Inst. 1989; 81: 188–193. 25. C. Jennison and B. W. Turnbull, Interim analyses: the repeated confidence interval approach. J R Stat Soc Ser B Methodol. 1989: 51: 305–361. 26. S. Wieand, G. Schroeder, and J. R. O’Fallon, Stopping when the experimental regimen does not appear to help. Stat Med. 1994; 13: 1453–1458. 27. S. S. Ellenberg and M. A. Eisenberger, An efficient design for phase III studies of combination chemotherapies. Cancer Treat Rep. 1985; 69: 1147–1152.

10

FUTILITY ANALYSIS

28. R. M. Goldberg, C. G. Moertel, H.S. Wieand, J. E. Krook, A. J. Schutt, et al., A phase III evaluation of a somatostatin analogue (octreotide) in the treatment of patients with asymptomatic advanced colon carcinoma. Cancer. 1995; 76: 961–966. 29. P. A. Burch, M. Block, G. Schroeder, J. W. Kugler, D. J. Sargent, et al., Phase III evaluation of octreotide versus chemotherapy with 5-fluorouracil or 5-fluorouracil plus leucovorin in advanced exocrine pancreatic cancer. Clin Cancer Res. 2000; 6: 3486–3492. 30. B. Freidlin, E. L. Korn, and S. L. George, Data monitoring committees and interim monitoring guidelines. Control Clin Trials. 1999; 20: 395–407. 31. D. J. Spiegelhalter, L. S. Freedman, and M. K. B. Parmar, Bayesian approaches to randomized trials. J R Stat Soc Ser A Stat Soc. 1994; 157: 357–416. 32. J. J. Dignam, J. Bryant, H. S. Wieand, B. Fisher, and N. Wolmark, Early stopping of a clinical trial when there is evidence of no treatment benefit: protocol B-14 of the National Surgical Adjuvant Breast and Bowel Project. Control Clin Trials. 1998; 19: 575–588. 33. D. A. Guinn, M. W. Atkinson, L. Sullivan, M. Lee, S. MacGregor, et al., Single vs weekly courses of antenatal corticosteroids for women at risk of preterm delivery: A randomized controlled trial. JAMA. 2001; 286: 1581–1587. 34. T. M. Jenkins, R. J. Wapner, E. A. Thom, A. F. Das, and C. Y. Spong, Are weekly courses of antenatal steroids beneficial or dangerous? JAMA. 2002; 287: 187–188. 35. P. J. Schwartz, How reliable are clinical trials? The importance of the criteria for early termination. Eur Heart J. 1995; 16(Suppl G): 37–45. 36. G. S. Omenn, G. E. Goodman, M. D. Thornquist, J. Balmes, M. R. Cullen, et al., Effects of a combination of beta carotene and vitamin A on lung cancer and cardiovascular disease. N Engl J Med. 1996; 334: 1150–1155. 37. K. E. Murphy, M. Hannah, and P. Brocklehurst, Are weekly courses of antenatal steroids beneficial or dangerous? [letter]. JAMA. 2002; 287: 188. 38. S. Green, J. Benedetti, and J. Crowley, Clinical Trials in Oncology. London: Chapman & Hall, 2002.

41.

42.

43.

44.

45.

46.

47.

48.

49.

50.

51.

(CHMP). Reflection Paper on Methodological Issues in Confirmatory Clinical Trials with Flexible Design and Analysis Plan. Draft. March 23, 2006. Available at: http://www.emea.eu.int/pdfs/human/ewp/ 245902en.pdf S. S. Emerson, J. M. Kittelson, and D. L. Gillen, On the Use of Stochastic Curtailment in Group Sequential Clinical Trials. University of Washington Biostatistics Working Paper Series, no. 243. Berkeley, CA: Berkeley Electronic Press, March 9, 2005. Available at: http://www.bepress.com/uwbiostat/paper243/ ¨ M. Henke, R. Laszig, and C. Rube, Erythropoietin to treat head and neck cancer patients with anemia undergoing radiotherapy: randomized, double-blind, placebo controlled trial, Lancet. 2003; 362: 1255–1260. B. Freidlin and E. L. Korn, Erythropoietin to treat anemia in patients with head and neck cancer [letter]. Lancet. 2004; 363: 81. E. L. Korn and R. Simon, Data monitoring committees and problems of lower-thanexpected accrual or events rates. Control Clin Trials. 1996; 17: 526–535. Steering Committee of the Physicians’ Health Study Research Group. Final report on the aspirin component of the ongoing Physicians’ Health Study. N Engl J Med. 1989; 321: 129–135. L. Friedman, C. D. Furberg, and D. L. DeMets, Fundamentals of Clinical Trials, 3rd ed. New York: Springer, 1999. S. S. Ellenberg, T. R. Fleming, and D. L. DeMets. Data Monitoring Committees in Clinical Trials. Chichester, UK: Wiley, 2002. A. Dmitrienko and M. D. Wang, Bayesian predictive approach to interim monitoring in clinical trials. Stat Med. 2006; 25: 2178–2195. K. K. Lan and D. M. Zucker, Sequential monitoring of clinical trials: the role of information and Brownian motion. Stat Med. 1993; 12: 753–765. P. K. Andersen, Conditional power calculations as an aid in the decision whether to continue a clinical trial. Control Clin Trials. 1987; 8: 67–74. O. M. Bautista, R. P. Bain, and J. M. Lachin, A flexible stochastic curtailing procedure for the log-rank test. Control Clin Trials. 2000; 21: 428–439.

39. R. Peto, Five years of tamoxifen-or more? J Natl Cancer Inst.. 1996; 88: 1791–1793.

FURTHER READING

40. European Medicines Agency, Committee for Medicinal Products for Human Use

Excellent overviews of the futility monitoring are provided in Friedman, Furberg, and DeMets

FUTILITY ANALYSIS 1996 (46), and Ellenberg, Fleming, and DeMets 2002 (47). A review of recent developments in Bayesian approach to futility monitoring is provided in Dmitrienko and Wang (48). A comprehensive review of the statistical methodology can be found in Whitehead (7), Jennison and Turnbull (8), and Emerson, Kittelson and Gillen (41). A detailed discussion of information quantification in various outcome settings is provided in Lan and Zucker (49). Practical applications of stochastic curtailment for the time to event endpoint are presented in Andersen (50) and Bautista et al. (51).

CROSS-REFERENCES Conditional power Interim analysis Group sequential designs Stopping boundaries Trial monitoring

11

GENERALIZED ESTIMATING EQUATIONS

the entire class of models addressed in their proposed framework. The theoretical justification of and the practical application of GLMs have since been described in many articles and books; McCullagh and Nelder (2) is the classic reference. GLMs address a wide range of commonly used models, which include linear regression for continuous outcomes, logistic regression for binary outcomes, and Poisson regression for count data outcomes. A particular GLM requires specification of a link function to characterize the relationship of the mean response to a vector of covariates, and specification of a function to characterize the variance of the outcomes in terms of the mean. The derivation of the iteratively reweighted least squares algorithm appropriate for fitting GLMs begins with the likelihood specification for the single parameter exponential family of distributions. Within the usual iterative Newton-Raphson algorithm, an updated estimate of the coefficient vector can be computed via weighted ordinary least squares. This estimation is then iterated to convergence, for example, until the change in the estimated coefficient vector is smaller than some specified tolerance. The GLM response, with or without the conditioning of predictors, is a member of the single parameter exponential family of distributions described by

JAMES W. HARDIN University of South Carolina, Columbia South Carolina

JOSEPH M. HILBE Arizona State University, Tempe Arizona

Parametric model construction specifies the systematic and random components of a model. Inference from maximum likelihood (ML)-based models relies on the validity of these specified components, and model construction proceeds from the specification of a likelihood based on some distribution to the implied estimating equation. In the case of ML models, an estimating equation is defined as the derivative of the log-likelihood function (with respect to one of the parameters of interest) set to zero where there is one estimating equation for each unknown parameter. Solution of the (vector-valued) estimating equation then provides point estimates for each unknown parameter. In fact, the estimating equation is so called because its solution leads to such point estimates. Two obvious approaches to the analysis of correlated data include fixed-effects and random-effects models. Fixed-effects models incorporate a fixed increment to the model for each group (allowing group-specific intercepts), whereas random-effects models assume that the incremental effects from the groups are perturbations from a common random distribution; in such a model, the parameter (variance components) of the assumed random-effects distribution is estimated rather than the much larger collection of individual effects. 1

f(y) = exp{[y θ − b(θ )]/φ + c(y, φ)} where θ is the canonical parameter, φ is a proportionality constant or scale, y is the response, and b(θ ) is the cumulant, the moments of which describe the mean and the variance of the random response variable. After introducing covariates and associated regression parameters, this density function leads to an estimated p × 1 regression coefficient vector β estimated by solving the estimating equation

GENERALIZED LINEAR MODELS

The theory and an associated computational method for obtaining estimates in which the response variable follows a distribution from the single parameter exponential family was introduced in Nelder and Wedderburn (1). The authors introduced the term generalized linear models (GLMs), which refers to

(β) =

n


i =

i=1

n


Xi T (yi − µi )/[φV(µi )]

i=1

[∂µi /∂ηi ] = 0(p×1)

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

(1)

2

GENERALIZED ESTIMATING EQUATIONS

In the estimating equation, Xi is the ith row of an n × p matrix of covariates X, µi = g(xi β), which represents the mean or expected outcome E(y) = b
(θ ). µi is a transformation of the linear predictor ηi = xi β via a monotonic (invertible) link function g(), and the variance V(µi ) is a function of the expected value proportional to the variance of the outcome Var(yi ) = φ V(µi ). If the link-variance pair of functions coincide with those functions implied by a specific member of the exponential family of distributions, then the resulting coefficient estimates are equivalent to maximum likelihood estimates. However, data analysts are not limited to only those pairs of link and variance functions. When selection of variance and link functions do not coincide to the canonical form of a particular exponential family member distribution, the estimating equation is said to imply the existence of a quasilikelihood, and the resulting estimates are referred to as maximum-quasilikelihood estimates. 2 THE INDEPENDENCE MODEL FOR CORRELATED DATA When observations are clustered because of repeated measurements on the sampling unit or because the observations are grouped by identification of a cluster identifier variable, the model is written in terms of the observations yit for the clusters I = 1, . . . , n and the within-cluster (repeated) observations t = 1, . . . , ni . The total number of observations is then N =  i ni . In this presentation, the clusters i are independent, but the withinclusters observations are assumed to be correlated. The individual-level model, which is otherwise known as the independence model, assumes no within-cluster correlation and is written by considering the n vector-valued observations yi as if they defined N independent observations. The independence model is a special case of correlated data models (such as those specified through GEEs). Although the independence model assumes that the repeated measures are independent, the model still provides consistent estimators in the presence of correlated data. This consistency is

paid for through inefficiency, but Glonek and McCullagh (3) show that loss of efficiency is not always large. As such, this simple model remains an attractive alternative because of its computational simplicity as well as its straightforward interpretation. In addition, the independence model serves as a reference model in the derivation of diagnostics for more sophisticated models for clustered and longitudinal data (such as GEE models). 3 ESTIMATING VARIANCE The validity of the (naive) model-based variance estimators depends on the correct specification of the variance; in turn, this value depends on the correct specification of the working correlation model. A formal justification for an alternative estimator known as the sandwich variance estimator is given in Huber (4) and discussed at length in Hardin (5) and Hardin and Hilbe (6). Analysts can use the independence model to obtain point estimates along with standard errors based on the modified sandwich variance estimator to ensure that inference is robust to any type of within-cluster correlation. Although the inference regarding marginal effects is valid (assuming that the model for the mean is correctly specified), the estimator from the independence model is not efficient when the data are correlated. It should be noted that, assuming independence is not always conservative, the modelbased (naive) variance estimates based on the observed or expected Hessian matrix are not always smaller than those of the modified sandwich variance estimator. Because the sandwich variance estimator is sometimes called the robust variance estimator, this result may seem counterintuitive. However, this result is easily understood by assuming negative within-cluster correlation that leads to clusters with both positive and negative residuals. The cluster-wise sums of those residuals will be small, and the resulting modified sandwich variance estimator will yield smaller standard errors based on squaring the group-wise sum of residuals than the model-based Hessian variance estimators based on squaring and summing each residual.

GENERALIZED ESTIMATING EQUATIONS

4 SUBJECT SPECIFIC VERSUS POPULATION-AVERAGED MODELS Two main approaches are used to deal with correlation in repeated or longitudinal data. The population-averaged approach focuses on the marginal effects averaged across the individuals. The subject-specific approach focuses on the effects for given values of the random effects by fitting parameters of the assumed random-effects distribution. The population-averaged approach models the average response for observations that share the same covariates (across all of the clusters or subjects), whereas the subjectspecific approach explicitly models the source of heterogeneity so that the fitted regression coefficients have an interpretation in terms of the individuals. The most commonly described GEE model was introduced in Liang and Zeger (7). This method is a population-averaged approach. Although it is possible to derive subjectspecific GEE models, such models are not commonly supported in commercial software packages and so do not appear nearly as often in the literature. The basic idea behind the populationaveraged approach is illustrated as follows. We initially consider the estimating equation for GLMs. The estimating equation, in matrix form, for the exponential family of distributions can be expressed as (β) =

n


i =

i=1

=

Xi T D[∂µi /∂ηi ]

i=1

V n


n


−1

(µi ) (yi − µi )/φ

Xi T D[∂µi /∂ηi ] V −1/2 (µi )I(n×n)

i=1

V −1/2 (µi ) (yi − µi )/φ = 0(p×1) (2) This equation corresponds to the independence model we have previously discussed. However, the specification of the identity matrix between the factors of the variance matrix signals the point at which secondorder variance (within-cluster correlation) can be introduced. Formally, Liang and Zeger (7) introduce a second estimating equation for the structural parameters of the working correlation

3

matrix. The authors then establish the properties of the estimators that result from the solution of these estimating equations. The GEE moniker was applied because the model is derived through a generalization of the GLM estimating equation; the second-order variance components are introduced directly into the estimating equation rather than appearing in consideration of a multivariate likelihood. Many major statistical software packages support estimation of these models, including R, SAS, S-PLUS, STATA, LIMDEP, SPSS, GENSTAT, and SUDAAN. R and S-PLUS users can easily find userwritten software tools for fitting GEE models, whereas such support is included in the other packages. 5 ESTIMATING THE WORKING CORRELATION MATRIX One should carefully consider the parameterization of the working correlation matrix because including the correct parameterization leads to more efficient estimates. We carefully consider this choice even if we employ the modified sandwich variance estimator for calculation of the standard errors of the regression parameters estimates. Although the use of the modified sandwich variance estimator assures robustness in the case of misspecification of the working correlation matrix, the advantage of more efficient point estimates is still worth the effort of trying to identify the correct structural constraints to place on the correlation matrix. No controversy surrounds the fact that the GEE estimates are consistent, but there is some controversy with regard to their efficiency. Typically, a careful analyst chooses some small number of candidate parameterizations. Pan (8) discusses the quasilikelihood information criterion measures for choosing between candidate parameterizations of the correlation matrix. This criterion measure is similar to the well-known Akaike information criterion. The most common choices for the working correlation R matrix are given by structural constraints that parameterize the elements of the matrix as provided in Table 1.

4

GENERALIZED ESTIMATING EQUATIONS Table 1. Common correlation structures Ruv = 0 Ruv = α Ruv = α|u−v | Ruv = α|u−v | if |u-v| ≤ k 0 otherwise Ruv = α (u,v) if |u-v| ≤ k 0 otherwise Ruv = α (u,v)

Independent Exchangeable Autocorrelated − AR1 Stationary (k) Nonstationary (k) Unstructured Values are given for u = v; Ruu

=

1.

The independence model admits no extra parameters, and the resulting model is equivalent to a generalized linear model specification. The exchangeable correlation parameterization admits one extra parameter. The most general approach is to consider the unstructured (only imposing symmetry) working correlation parameterization that admits M(M − 1)/2 − M extra parameters where M = maxi {ni }. The exchangeable correlation specification, which is the most commonly used correlation structure for GEEs, is also known as equal correlation, common correlation, and compound symmetry. The elements of the working correlation matrix are estimated using Pearson residuals, which are calculated following each iteration of model fit. Estimation alternates between estimating the regression parameters β, assuming the that the initial estimates of α are true, and then obtaining residuals to update the estimate of α, and then using new estimates of α to calculate updated parameter estimates, and so forth until convergence. GEE algorithms are typically estimated using the same routines used for estimation of GLMs with an additional subroutine called to update values of α. Estimation of GEE models using other correlation structures use a similar methodology; only the properties of each correlation structure differs. A schematic for representing how the foremost correlation structures appear is found below. Discussion on how the elements in each matrix are to be interpreted in terms of model fit can be found in Twisk (9) and Hardin and Hilbe (10).

Independent Exchangeable 1 0 0 0

1 0 0

1 0

1 p p p

1

1 p p

1 p

1

Stationary or M-Dependent (2-DEP) Autoregressive (AR-1) 1 p1 p2 0

1 p1 p2

1 p1

1

1 p p2 p3

1 p p2

1 p

1

1 p6

1

Nonstationary (2-DEP) Unstructured 1 p1 p2 0

1 p3 p4

1 p5

1

1 p1 p2 p3

1 p4 p5

6 EXAMPLE The progabide data are commonly used as an example to demonstrate the various GEE correlation structures. The data are available in Thall and Vail (11). The data are from a panel study in which four 2-week counts of seizures were recorded for each epileptic patient. The response variable, or dependent variable of interest, is seizure, which is a count that ranges from 0 to 151. Explanatory predictors include time (1 = followup; 0 = baseline), progabide (1 = treatment; 0 = placebo), and time × prog (an interaction term). The natural log of the time period, which ranges from 2–8, is entered into the model as an offset.

GENERALIZED ESTIMATING EQUATIONS

5

GENERALIZED LINEAR MODEL seizures

time progabide timeXprog cons lnPeriod

Coef.

.111836 .0275345 −.1047258 1.347609 (offset)

Robust Std. Err.

z

.1169256 .2236916 .2152769 .1587079

0.96 0.12 −0.49 8.49

P > |z|

0.339 0.902 0.627 0.000

[95% Conf. Interval]

−.1173339 −.410893 −.5266608 1.036547

.3410059 .465962 .3172092 1.658671

GEE MODEL ASSUMING INDEPENDENCE seizures

time progabide timeXprog cons lnPeriod

Coef.

.111836 .0275345 −.1047258 1.347609 (offset)

Robust Std. Err.

z

.1169256 .2236916 .2152769 .1587079

0.96 0.12 −0.49 8.49

P > |z|

0.339 0.902 0.627 0.000

[95% Conf. Interval]

−.1173339 −.410893 −.5266608 1.036547

.3410059 .465962 .3172092 1.658671

Estimated within-id correlation matrix R: r1 r2 r3 r4 r5

c1 1.0000 0.8102 0.6565 0.5319 0.4309

c2

c3

c4

c5

1.0000 0.8102 0.6565 0.5319

1.0000 0.8102 0.6565

1.0000 0.8102

1.0000

In all, 59 patients participated in the study. Patients are identified with the variable identification (id). A robust cluster or modified sandwich variance estimator, clustered on id, is applied to the standard errors of the parameter estimates. Because the response is a count, we use a Poisson regression to model the data. We first model the data using a generalized linear model, which assumes that the observations are independent. Because five observations were obtained per patient, the data are likely correlated (on id); therefore, we use a robust variance estimator clustered on id to adjust the standard errors for the extra correlation. We subsequently model the data using GEE with various correlation structures. Each type of structure attempts to capture the correlation in a specific manner. Only the table of parameter estimates and associated statistics is shown for each model. GEE models have an accompanying within-id correlation matrix displayed to show how the model is adjusted.

The independence model is identical to the generalized linear model; that is, the correlation in the data caused by the clustering effect of id is not adjusted by an external correlation structure. It is only adjusted by means of a robust variance estimator. The exchangeable correlation matrix is nearly always used with clustered, nonlongitudinal data. A single correlation parameter is associated with this structure, which means that each cluster is assumed to be internally correlated in a similar manner. Subjects are assumed to be independent. The autoregressive correlation structure assumes that within time intervals, the correlation coefficients decrease in value based on the respective power increase in measurement. The values are the same for each respective off diagonal. Theoretically, the second-level diagonal values are the squares of the first-level diagonals. The third-level diagonals are the cube of the first, with a likewise increase in power for each larger diagonal. A large working correlation matrix,

6

GENERALIZED ESTIMATING EQUATIONS

GEE MODEL ASSUMING EXCHANGEABLE CORRELATION seizures time progabide timeXprog cons lnPeriod

Coef.

Semi-robust Std. Err.

.111836 .0275345 −.1047258 1.347609 (offset)

P > |z|

z

.1169256 .2236916 .2152769 .1587079

0.96 0.12 −0.49 8.49

0.339 0.902 0.627 0.000

[95% Conf. Interval]

−.1173339 −.410893 −.5266608 1.036547

.3410059 .465962 .3172092 1.658671

Estimated within-id correlation matrix R: r1 r2 r3 r4 r5

c1 1.0000 0.7767 0.7767 0.7767 0.7767

c2

c3

c4

c5

1.0000 0.7767 0.7767 0.7767

1.0000 0.7767 0.7767

1.0000 0.7767

1.0000

GEE MODEL ASSUMING AUTOREGRESSIVE (AR-1) CORRELATION seizures time progabide timeXprog cons lnPeriod

Coef.

Semi-robust Std. Err.

.1522808 .019865 −.1292328 1.3128 (offset)

P > |z|

z

.1124191 .2135299 .2620191 .1631003

1.35 0.09 −0.49 8.05

0.176 0.926 0.622 0.000

[95% Conf. Interval]

−.0680567 −.3986458 −.6427809 .9931291

.3726183 .4383758 .3843153 1.632471

Estimated within-id correlation matrix R: r1 r2 r3 r4 r5

c1 1.0000 0.8102 0.6565 0.5319 0.4309

c2

c3

c4

c5

1.0000 0.8102 0.6565 0.5319

1.0000 0.8102 0.6565

1.0000 0.8102

1.0000

which represents more within-group observations, will have increasingly small values in the extreme off diagonals. If the actual correlation structure varies considerably from the theoretical, then one should investigate using another structure. The nonstationary correlation structure is like the stationary except that the offdiagonal values are not the same. We would want to use these structures if we want to account for measurement error at each time period or measurement level. We observe above the same stopping point as in the stationary structure.

The unstructured correlation structure assumes that all correlations are different. This structure usually fits the model the best, but it does so by losing its interpretability—especially for models that have more than three predictors. Note that the number of unstructured correlation coefficients is based on the number of predictors p. Therefore the number of coefficients equals p(p − 1)/2.

GENERALIZED ESTIMATING EQUATIONS

7

GEE MODEL ASSUMING STATIONARY(2) or M-DEPENDENT(2) CORRELATION seizures time progabide timeXprog cons lnPeriod

Coef.

Semi-robust Std. Err.

.0866246 .0275345 −.1486518 1.347609 (offset)

P > |z|

z

.1739279 .2236916 .2506858 .1587079

0.50 0.12 −0.59 8.49

0.618 0.902 0.553 0.000

[95% Conf. Interval]

−.2542677 −.410893 −.639987 1.036547

.4275169 .465962 .3426833 1.658671

Estimated within-id correlation matrix R: r1 r2 r3 r4 r5

c1 1.0000 0.8152 0.7494 0.0000 0.0000

c2

c3

c4

c5

1.0000 0.8152 0.7494 0.0000

1.0000 0.8152 0.7494

1.0000 0.8152

1.0000

GEE MODEL ASSUMING NONSTATIONARY(2) CORRELATION seizures time progabide timeXprog cons lnPeriod

Coef.

Semi-robust Std. Err.

.0866246 .0275345 −.1486518 1.347609 (offset)

P > |z|

z

.1739279 .2236916 .2506858 .1587079

0.50 0.12 −0.59 8.49

0.618 0.902 0.553 0.000

[95% Conf. Interval]

−.2542677 −.410893 −.639987 1.036547

.4275169 .465962 .3426833 1.658671

Estimated within-id correlation matrix R: r1 r2 r3 r4 r5

c1 1.0000 0.9892 0.7077 0.0000 0.0000

c2

c3

c4

c5

1.0000 0.8394 0.9865 0.0000

1.0000 0.7291 0.5538

1.0000 0.7031

1.0000

It is possible to use other correlation structures with GEE models, but the ones shown above are the most common. More details on these structures can be found in Hardin and Hilbe (10). 7

CONCLUSION

GEE models are an extension of generalized linear models. GLMs are based on likelihoods (or quasilikelihoods), which assume the independence of observations in the model. When this assumption is violated because of clustering or longitudinal effects, then an appropriate adjustment needs to be made to the

model to accommodate the violation. GEE is one such method; fixed and random effects models are alternatives to GEE. Likewise, hierarchical and mixed models have been used to deal with clustering and longitudinal effects, which bring extra correlation into the model. The GEE approach is generally known as a population-averaging approach in which the average response is modeled for observations

8

GENERALIZED ESTIMATING EQUATIONS

GEE MODEL ASSUMING UNSTRUCTURED CORRELATION seizures time progabide timeXprog cons lnPeriod

Coef.

Semi-robust Std. Err.

.0826525 .0266499 −.1002765 1.335305 (offset)

P > |z|

z

.1386302 .224251 .2137986 .1623308

0.60 0.12 −0.47 8.23

0.551 0.905 0.639 0.000

[95% Conf. Interval]

−.1890576 −.4128741 −.5193139 1.017142

.3543626 .4661738 .318761 1.653467

Estimated within-id correlation matrix R: r1 r2 r3 r4 r5

c1 1.0000 0.9980 0.7149 0.8034 0.6836

c2

c3

c4

c5

1.0000 0.8290 0.9748 0.7987

1.0000 0.7230 0.5483

1.0000 0.6983

1.0000

across clusters and longitudinal subjects. This method varies from the subject-specific approach, which provides information concerning individual observations rather than averages of observations. The method has been used extensively in clinical research and trials, and it is widely available in the leading software packages. REFERENCES 1. J. A. Nelder and R. W. M. Wedderburn, Generalized linear models. J. Royal Stat. Soc. Series A 1972; 135: 370–384. 2. P. McCullagh and J. A. Nelder, Generalized Linear Models, 2nd ed. London: Chapman & Hall, 1989. 3. G. F. V. Glonek and R. McCullagh, Multivariate logistic models. J. Royal Stat. Soc. Series B 1995; 57: 533–546. 4. P. J. Huber, The behavior of maximum likelihood estimates under nonstandard conditions. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1. Berkeley, CA: University of California Press, 1967, pp. 221–223. 5. J. W. Hardin, The sandwich estimate of variance. In: T. B. Fomby and R. C. Hill (eds.), Advances in Econometrics, Vol. 17. 2006, pp. 45–73. 6. J. W. Hardin and J. M. Hilbe, Generalized Linear Models and Extensions, 2nd ed. College Station, TX: Stata Press, 2007.

7. K.-Y. Liang and S. L. Zeger, Longitudinal data analysis using generalized linear models. Biometrika 1986; 73: 13–22. 8. W. Pan, Akaike’s information criterion in generalized estimating equations. Biometrics 2001; 57: 120–125. 9. J. W. R. Twisk, Applied Longitudinal Data Analysis for Epidemiology: A Practical Guide. Cambridge, UK: Cambridge University Press, 2003. 10. J. W. Hardin and J. M. Hilbe, Generalized Estimating Equations. Boca Raton, FL: Chapman & Hall/CRC Press, 2002. 11. P. F. Thall and S. C. Vail, Some covariance models for longitudinal count data with overdispersion. Biometrics 1990; 46: 657–671.

CROSS-REFERENCES Generalized linear models mixed-effects models sandwich variance estimator

GENERALIZED LINEAR MODELS

functional relationship between y and x. As η is linear in the parameters, it is also called the linear predictor. Beside the structural part, the model contains the residual ε capturing all variation that is not included in η, which is the stochastic variability of y. Conventionally, one assumes normality for ε with zero mean and variance σ 2 . With this in mind, the structural part of model (1) can be written as

¨ GORAN KAUERMANN

University Bielefeld Postfach 300131, Bielefeld, Germany

JOHN NORRIE University of Aberdeen Health Services Research Unit Polwarth Building, Foresterhill, Aberdeen, Scotland, United Kingdom

1

µ = E(y|x) = η

That is, the mean of the response y given the explanatory covariate x is related in a linear fashion to x through the linear predictor η. Even though Equation (1) is a cornerstone in statistics, it is not an appropriate model if normality of the residuals does not hold. This applies, for instance, if y is a binary response variable taking values 0 and 1. To accommodate such a response, data model (1) needs to be generalized, and the decomposition of the structural and the stochastic part plays the key role in the GLM. For the stochastic part, one assumes that y for a given linear predictor η follows an exponential family distribution. This step provides an elegant mathematical framework and embraces several commonly used distributions, including the normal (or Gaussian), binomial, Poisson, exponential, gamma, and inverse normal distributions. Generalization of the structural part results as follows. Note that for normal response data, both sides of Equation (2) take values in the real numbers. This is not the case for other distributions. If, for instance, y is Binomially distributed, then E(y|x) takes values in [0, 1], so that the set of possible values for the left- and right-hand sides of Equation (2) differ. To overcome this problem, one introduces a transformation function g(·) called the link function and generalizes (2) to

INTRODUCTION

1.1 Motivation Generalized linear models (GLMs) provide a flexible and commonly used tool for modeling data in a regression context. The unifying approach traces back to Nelder and Wedderburn (1), although its general breakthrough was about a decade later, initiated by the book of McCullagh and Nelder (2). The availability of software to fit the models increased the acceptance of GLMs (see Reference 3). The approach has had a major impact on statistical modeling technology with research on a wide range of extensions continuing today. Recent references include Myers, et al. (4) and Dobson (5), which are of an introductory level, and Fahrmeir and Tutz (6), which concentrates on extensions toward multivariate models. For a concise overview of GLMs, see also Firth (7). For clarity of presentation, the generalization of regression models starts with the classic linear model, which can be written as y = β0 + xβx + ε

(2)

(1)

with y as response, x as some explanatory variable(s), and ε as the residual having mean zero. Model (1) plays a central role in statistics and mathematics, developed in the early nineteenth century by Gauß and Legendre (see Reference 8). A more recent reference is Stuart and Ord (9). This statistical model comprises two components: a structural and a random part. The structural part is given by η = β0 + xβx and models the

g(µ) = g{E(y|x)} = η

(3)

The link function g(·) guarantees that the possible values of both sides of Equation (3) remain the real numbers. For a Binomially distributed response, a convenient choice for g(·) is the logit link g(µ) = logit(µ) = log(µ) − log(1 − µ), which uniquely transforms the set

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

GENERALIZED LINEAR MODELS

[0, 1] into the real numbers. Alternatively, the probit link g(µ) = −1 (µ) can be used, with (·) as standard normal distribution function. If the response follows a Poisson distribution, as is often the case for count data, E(y|x) is a positive number. In this case, a suitable link is taking g(·) as the log function. Both examples will be revisited later. Note that classic linear regression is included by taking g(·) as the identity function.

where x = 1 indicates the CBT + TAU group and x = 0 is the TAU group as reference category. The parameter estimates (with standard errors in brackets) are βˆ0 = −0.631(0.247) and βˆx = −0.881(0.393). For binary response, the odds ratio is a nicely interpretable quantity that results via exp(βx ) = exp{logit(E(y = 1|x = 1)) − logit(E(y = 1|x = 0))}

1.2 Example: Logistic Regression In a randomized controlled trial, the relapse in 144 schizophrenics was observed. Let y = 1 indicate the occurrence of a relapse measured by the positive and negative symptom score (PANSS) psychopathology scale, and let y = 0 otherwise. In this trial, there were two randomized treatments, cognitive behavioral therapy (CBT) on top of treatment as usual (TAU) against treatment as usual. The 144 schizophrenics (who fulfilled the American Psychiatric Association DSM-IV [1994] criteria for schizophrenia) were all receiving antipsychotic medication and considered relapse prone (relapse within last 2 years or living in stressful environment or alone or problems with taking antipsychotic medication). The PANSS is a 30-item observer-rated scale, scored as 1 (no psychopathology) to 7 (severe). The first seven items comprise the positive score, and a relapse was defined as admission to hospital or a 50% increase in positive score sustained for 1 week (for those with a positive item at baseline ≥3) or reaching a positive item ≥3 (if no positive item ≥3 at baseline). Further details of the methods and results are reported by Gumley et al. (10). Note that in that report, the relapse outcome was modeled using Cox regression, incorporating the time to relapse. The exposition here uses logistic regression, ignoring the time to relapse and concentrating on the occurrence of the event within the first 12 months only. During this 12-month followup, 13 out of 72 (18.1%) of those randomized to CBT + TAU relapsed, compared with 25 out of 72 (34.7%) in the TAU group. A logistic regression model was fitted with logit{E(y|x)} = β0 + xβx

Here, the odds ratio of relapse on CBT + TAU against TAU is estimated by 0.414 with confidence interval (0.191, 0.896). This example is revisited later. 2 GENERALIZED LINEAR MODELS 2.1 Modeling 2.1.1 Stochastic Component. For the stochastic part, one assumes that response y given predictor η = g(µ), with g(·) the link function and µ the mean of y given the explanatory covariates, follows the exponential family distribution. The likelihood can be written in the form 
yθ − b(θ ) + h(y, φ) (4) L(µ, φ) = exp wφ where b(·) and h(·) are known functions, φ and w are dispersion parameters, and θ = θ (µ) is called the natural or canonical parameter. The dispersion is thereby decomposed into two parts: a known weight w, which might vary between observations, and a dispersion parameter φ (sometimes referred to as a nuisance parameter), which is the same for all observations. The latter depends on the underlying distribution and is either known or unknown. The natural parameter θ and the mean µ are related via the first-order partial derivative of b(·) with respect to θ ; that is, ∂b(θ ) = µ(θ ) ∂θ

(5)

Thus, the natural parameter θ determines the mean value µ and vice versa. As the function is invertible, θ can also be written in dependence of µ as θ (µ).

GENERALIZED LINEAR MODELS

Important and commonly used statistical distributions belong to the exponential family [Equation (4)]. Taking, for instance, y ∈ {0, 1} as binary response, one obtains the likelihood function L(µ, ϕ) = µy (1 − µ)(1−y) = exp{yθ − b(θ )} (6) with θ = log{µ/(1 − µ)} and b(θ ) = log[1/{1 + exp(θ )}]. The joint likelihood for m independent identically distributed binary responses yi (a Binomial distribution) is written as
  yθ − b(θ ) m L(µ, ϕ) = exp + log , (7) y (1/m) with y = i yi /m and w = 1/m as the weight. For a Poisson distributed response variable y with mean µ, the likelihood can be written as L(µ, ϕ) =

µy exp{−µ} y!

= exp{y log(µ) − µ − log(y!)}

(8)

so that θ = log(µ) and b(θ ) = exp(θ ). Finally, for Normally distributed response y, one gets   L(µ, ϕ) = exp (2yµ − µ2 )/(2σ 2 ) + h(y, σ 2 ) with h(·) collecting terms that are not dependent on the mean, and θ = 2µ and b(θ ) = µ2 . Table 1 gives an overview of commonly used distributions belonging to Equation (4). 2.1.2 Variance Function. The exponential family implies a dispersion structure for response y. By standard results from exponential families, one finds by differentiation ∂ 2 b(θ ) 1 =: v(µ) Var(y|x) = wφ ∂θ 2

(9)

with v(µ) also called the variance function, which captures the dependence of the variance of y on the mean value. For binomial response, the variance function is v(µ) = µ (1 − µ), whereas for Poisson data, it is v(µ) = µ. For normal responses v(µ) = 1, reflecting that the variance does not depend on the mean. The role of v(µ) is then to allow variance heterogeneity to be automatically incorporated into the estimation.

3

2.1.3 Structural Component. For the structural part of the model, the mean value of y is linked to the linear predictor η via the link function g(·). This function must be appropriately chosen so that both sides of Equation (2) have the same range of possible values. The choice of g(·) can therefore in principle be made in isolation from the stochastic part with the only requirement being that g(·) maps the space of mean values for y to the real numbers and vice versa. However, natural candidates for the link function are suggested by the stochastic part of the model. Such link functions are called natural or canonical links. They are obtained by setting θ = η in Equation (4); that is, the linear predictor is set to equal the natural parameter of the exponential family. This result gives µ(θ ) = µ(η) from Equation (5), which is mathematically as well as numerically a convenient choice. Natural links enjoy widespread use in practice, with the logit link µ(θ ) = µ(η) = exp(η)/(1 + exp(η)) the natural choice for the Binomial response, whereas for Poisson response, the natural link is µ(θ ) = exp(θ ). For a normally distributed response y, the natural link is the identity function, so that the classic regression model (1) results as a special case. 2.2 Estimation 2.2.1 Maximum Likelihood Estimation. Taking the logarithm of Equation (4) yields the log likelihood l(µ, φ) = {yθ − b(θ )}/(wφ) + h(y, φ).

(10)

It is notationally convenient to parameterize the likelihood by β instead of µ. Assuming independent observations yi , i = 1, . . . , n, with corresponding covariates xi , one obtains from the individual log likelihood contributions [Equation (10)] the log likelihood function l(β, φ) =

n 

li (β, φ)

i=1

=

n  [{yi θi − b(θi )}/(wi φ) i=1

+ {h(yi , φ)}]

(11)

4

GENERALIZED LINEAR MODELS

Table 1. Examples of Exponential Family Distributions Distribution N(µ, σ 2 )

Normal

b(θ)

µ(θ) = b (θ)

v(µ) = b (θ)

φ

θ 2 /2

1

σ2

µ(1 − µ)

1

exp(θ) − log(−θ)

θ exp(θ) 1 + exp(θ) exp(θ) −1/θ

exp(θ) µ2

1 ν −1

−(−2θ)1/2

(−2θ)−1/2

µ3

σ2

θ(µ)

Binomial

B(1, µ)

Poisson Gamma Inverse Normal

P(µ) G(µ, ν)

 µ

µ log (1 − µ) log(µ) −1/µ

IN(µ, σ 2 )



log(1 + exp(θ))

1/µ2

with θi = θ (µi ) = θ {g−1 (ηi )} and linear predictor ηi = xi β. The maximum likelihood estimate for β is now obtained by maximizing Equation (11), which is achieved by setting the first derivative of l(β, φ) to zero. Simple differentiation using the chain rule leads to the score equation 0=

n  ∂ ηˆ i ∂ µˆ i ∂ θˆi (yi − µˆ i ) ∂β ∂η ∂µ

(12)

i=1

with ηˆ i = βˆ0 + xi βˆx , µˆ i = g−1 (ηˆ i ), and θˆi = θ (µˆ i ). If g(·) is chosen as natural link, one has θ = η so that (12) becomes 0=

n  ∂ ηˆ i i=1

∂β

2.3 Example: Multiple Logistic Regression (yi − µˆ i ).

(13)

Note that η = (1, x)β so that ∂η/∂β = (1, x)T . Hence, for natural links, Equation (13) has a simple structure. Moreover, with normal response y and natural link, Equation (13) reads as X T (Y − Xβ), with X T = ((1, x1 )T , . . . , (1, xn )T ) and Y = (y1 , . . . , yn ), which is easily solved analytically by βˆ = (X T X)−1 X T Y. In general, however, µi = µ(θi ) is a nonlinear function so that even for natural links, no analytic solution of Equation (13) will be available. Iterative fitting, for instance, based on Fisher scoring, is therefore required. 2.2.2 Fisher Scoring. A solution of Equation (12) can be found following a Newton Raphson strategy. Taking β (0) as starting value, then obtain an update β (t+1) from β (t) , t = 0, 1, 2, . . ., from the iterative procedure  β

(t+1)

=β

(t)

−

In practice, for ease of computation, it is convenient to replace the second-order derivative in Equation (14) by its expectation, otherwise known as the Fisher matrix. The resulting estimation routine is therefore called Fisher scoring. The procedure is sometimes also called iterative reweighted least squares, because the second-order derivative as well as the first-order derivative contain dispersion weights v(µi ), which depend on the parameter β and have to be updated iteratively using weighted least squares in each step.

∂ 2 l(β (t) , φ) ∂β∂β T

−1

∂l(β (t) , φ) . ∂β (14)

The subsequent example is taken from the textbook Brown (11). For 53 patients, nodal involvement of tumor was recorded as response variable y, indicating whether the tumor had spread to neighboring lymph nodes (y = 1) or not (y = 0). Explanatory variables are all binary, xage indicating whether the patient is 60 years or older (=1), xstage equal to 1 if the tumor has been classified as serious (=0 otherwise), xgrade describing the pathology of the tumor (=1 if assessed as serious, =0 otherwise), xxray if X-ray of the tumor led to a serious classification, and finally xacid indicating whether the serum acid phosphatase level exceeds the value 0.6. Parameter estimates of the multivariable model are shown in Table 2, with positive estimates indicating a tendency for an increased likelihood of tumor spread. The t-value in Table 2 is calculated by dividing the estimate by its standard deviation. The significance of the covariate can be assessed by comparing the t-value to standard normal distribution quantiles. By so doing, one should keep in mind the

GENERALIZED LINEAR MODELS Table 2. Parameter Estimates and Standard Deviation for Logistic Regression Example Covariate

Estimate

Standard Deviation

t-Value

Intercept age stage grade xray acid

−3.08 −0.29 1.37 0.87 1.80 1.68

0.98 0.75 0.78 0.81 0.81 0.79

−3.13 −0.39 1.75 1.07 2.23 2.13

5

Table 3. Parameter Estimates and Standard Deviation for Poisson Example

β0 βt β tt

model with linear time

model with quadratic time

5.178 (0.046) −0.0628 (.0057) –

5.011 (0.076) 0.0081 (.0238) −0.0043 (.0015)

quadratic fit is plotted in Fig. 1. The example is picked up again later. effects shown are conditional effects, meaning given the values of the other covariates. For instance, age exhibits a negative effect given the explanatory information contained in stage, grade, xray, and acid. It seems that some covariates show a significant effect, with their t-values taking values larger than 2 in absolute term, whereas, for instance, age does not show a significant effect, again conditional on the other variates in the model (stage, grade, xray, acid). A variable selection to classify covariate effects as significant and nonsignificant will be demonstrated in a subsequent section.

3

INFERENCE

As observed in the above three examples, it is necessary to have inferential tools at hand to select an appropriate model. This model contains only those covariates that need to be in the model (1) either because they are prognostically valuable or they are necessary for external reasons, or (2) they show significant effect. The following chapter discusses alternatives for the assessment of the latter point (2).

2.4 Poisson Regression Poisson regression can be useful for responses that are counts or frequencies. Consider, for example, the number of deaths from injury in childhood reported to the Registrar General in Scotland between 1981 and 1995. Morrison et al. (12) give full details of this data and explore socioeconomic differentials. Here the data are considered as overall number of deaths by year. Figure 1 shows the number of deaths per year, indicating a clear decline over the 15 years of observation, from a high of 173 in the late 1980s to a low of just over 50 in 1995. The number of deaths in any year are modeled by a Poisson distribution with mean µ as a function of calender year T, starting with 1981 (T = 1) to 1995 (T = 15). The simplest model would relate time T to the mean in a linear fashion: µ = exp(β0 + tβt ). In addition, a more complicated function, such as a quadratic function, can be used to accommodate nonlinearity: µ = exp(α + βt t + t2 βtt ). Table 3 provides the corresponding parameter estimates. The resulting log linear and log

3.1 Variance of Estimates 3.1.1 Fisher Matrix. Standard likelihood theory can be applied to obtain variances for the maximum likelihood estimates βˆ = (βˆ0 , βˆx ), which requires the calculation of the Fisher matrix by second-order differentiation of Equation (11). As ∂η/∂β = (1, x)T does not depend on β, one finds for the second-order derivative of l(β) an expression consisting of two components:  ∂ηi ∂θi v(µi ) ∂θi ∂ηi ∂ 2 l(β) =− T ∂β ∂η wi φ ∂η ∂β T ∂β∂β n

(15)

i=1

+

n  ∂ηi ∂ 2 θi ∂ηi yi − µi ∂β ∂η2 ∂β T wi φ

(16)

i=1

The first component in Equation (15) is a deterministic term, whereas Equation (16) is a stochastic one. Keeping in mind that

6

GENERALIZED LINEAR MODELS

170 150

(log)quadratic

Deaths

130 110

(log)linear

90 70 Figure 1. Accidental deaths in children in Scotland 1981 (T = 1) to 1995 (T = 15).

50 T Deaths

1

2

3

4

5

6

7

8

9

10 11 12 13 14 15

135 148 143 173 148 123 124 105 88 82 120 70 88 67 56

E(yi |xi ) = µi , the stochastic term has mean zero so that the Fisher matrix equals   ∂ 2 l(β) F(β) = E − ∂β∂β T =

n  ∂ηi ∂θi v(µi ) ∂θi ∂ηi ∂β ∂η wi φ ∂η ∂β T

(17)

i=1

For natural link functions, the structure nicely simplifies because with θ (η) = η, one has ∂θi /∂η = 1 and ∂ 2 θi /∂η2 = 0. Hence, Equation (16) vanishes and Equation (15) has the simple structure

F(β) =

n  ∂ηi v(µi ) ∂ηi ∂β wi φ ∂β T i=1

In particular, in this case, the expected and the observed Fisher matrices coincide. From standard likelihood theory, we know that the variance of the maximum likelihood estimate βˆ is asymptotically equal to the Fisher a ˆ = F(β)−1 , and the cenmatrix, that is, Var(β) tral limit theorem provides asymptotic normality: a βˆ ∼ N(β, F(β)−1 )

Therefore, standard statistical methods can be used for the parameter estimation. 3.1.2 Dispersion Parameter. The variance formulas above contain the dispersion parameter φ. Dependent on the stochastic model

being used, φ is either known or unknown. For instance, for binomial as well as for Poisson responses, one has φ = 1, which follows directly from Equation (6) and Equation (8), respectively. Hence, estimation of φ is not necessary. In contrast, for normally distributed response, component φ is the residual variance that is typically unknown and has to be estimated from the data. In principle, this estimation can be done by maximum likelihood theory based on the likelihood function (11). For the general case, however, maximum likelihood estimation of φ is not recommended (see Reference 2, p. 295) and instead a moment-based approach should be preferred. Based on variance formula (9), an estimate is found by φˆ =

n  (yi − µˆ i )2 /n. wi v(µˆ i )

(18)

i=1

For normal response models, the momentbased estimate is identical to the maximum likelihood estimate. This case is, however, special, and it does not hold in general. Moreover, replacing the denominator in Equation (18) by n − p, with p as number of parameters, reduces the bias occurring because of the use of fitted residuals (see Reference 2 for details). 3.2 Variable Selection It can be desirable within a statistical model with many explanatory covariates to only include covariates that make a substantive contribution to the understanding of the

GENERALIZED LINEAR MODELS

relationship between the covariates and the response. This question is one of parsimony, that is, to check whether all covariates are needed or whether a model with less covariates fits the data similarly well. A large amount of literature on variable selection in regression is available, with a particularly readable overview given by Harrel (13). Also, the issue of model inconsistency should be considered, particularly in the nonlinear model (see Reference 14) as one adds and subtracts covariates from the model. The problem can be tackled by testing the significance of subsets of covariate effects, and there are three different possibilities in common use. All three methods coincide for normal response y, and they are asymptotically equivalent for the general case. 3.2.1 Wald Statistics. Let the set of explanatory variables contained in x be split into xa and xb , that is, x = (xa , xb ). Model (3) is rewritten as g(µ) = xa βa + xb βb

(19)

For notational convenience, the intercept is included in xa . The focus of interest is to test the hypothesis βb = βb0 , with βb0 some given vector. The primary focus is usually on testing βb0 = 0. Taking advantage of standard likelihood theory, one can test this hypothesis using a Wald Statistic. Let the Fisher matrix F(β) be decomposed to  F(β) =

Faa (β) Fab (β) Fba (β) Fbb (β)



with submatrices matching to the dimensions of βa and βb . The Wald Statistic is then defined as wb = (βˆb − βb0 )T (F bb (β))−1 (βˆb − βb0 )

(20)

with F bb (β) as the bottom right matrix of F(β)−1 . For βb = βb0 , the test statistic follows asymptotically a chi-squared distribua tion with p degrees of freedom; i.e., wb ∼ χp , where p is the dimension of βb . 3.2.2 Score Statistics. A second test statistic is available from the score contributions, that is, the first derivatives of the

7

likelihood function. Let β˜ = (β˜a , βb0 ) be the maximum likelihood estimate in the hypothetical model with βb = βb0 . The Score Statistic is then defined by

sb =

˜ φ) ∂l(β, ∂βbT

˜ F bb (β)

˜ φ) ∂l(β, ∂βb

(21)

The idea behind the Score Statistic is that if βb = βb0 , then the score contribution at β˜ will be close to zero, which indicates that the maximum likelihood estimate βˆb in the complete model is close to βb0 . Asymptotic a likelihood theory shows sb ∼ χp . 3.2.3 Likelihood Ratio Statistic. Two models are called nested if the smaller one results in setting some parameters in the larger model to zero (or to some specific value). In this respect, the model with βb = βb0 is nested in the larger model with no constraints on βb . Testing βb = βb0 can then be pursued in a general framework using the Likelihood Ratio Statistics: ˜ φ) − l(β, ˆ φ)} λb = −2{l(β,

(22)

Again, standard likelihood theory indicates a the asymptotic distribution λb ∼ χp , assuming the smaller model to holds. 3.2.4 Remarks for Practical Application. It should be noted that the Wald Statistic is ˆ the estimate in the altercalculated from β, native model, whereas the Score Statistics ˜ the estimate in the is calculated from β, hypothetical model. For the Likelihood Ratio Statistics, both estimates are required for calculation. Standard software usually calculates the Wald Statistic or the likelihood ratio statistic only, whereas the score statistic is typically not provided. In terms of asymptotic behavior, all three test statistics follow standard likelihood theory (see Reference 15) based on the central limit theorem. The fundamental assumption for the asymptotics to hold even in small samples is that Fisher matrices are well conditioned and grow asymptotically at the same rate as the sample size. ‘‘Well conditioned’’ means here that both the design matrix of covariates x as well as the coefficients β

8

GENERALIZED LINEAR MODELS

are well behaved. The first is a common feature in any linear regression model, whereas the second is a special feature occurring in GLMs. For example, consider a binary covariate x taking values 0 and 1. If the binary response y = 1 for x = 1 and y = 0 for x = 0, then regardless of the design of x, the Fisher matrix will not have full rank because estimates tend to infinity in absolute terms. A thorough investigation of the effect of design on the asymptotic behavior is found in Fahrmeir and Kaufmann (16, 17). General guidelines on when the asymptotic approximations above are good enough to be reliable are difficult to derive, because the behavior depends strongly on the underlying stochastic model. Some discussion is found in Agresti (18, p. 246) or Fahrmeir and Tutz (6, p. 50). In the case of binomial and Poisson responses, Cressie and Read (19) introduced a power divergence statistics with improved small sample behavior. Further discussion of this point is also found in Santner and Duffy (20). As rule of thumb, one can take that asymptotic results can be satisfactorily applied in finite samples if (1) the covariate design is well conditioned and (2) the mean value µ is not close to to the boundary of its parameter space (for example, for a binary response y, we want µ to be not close to 0 or 1). If one of the two conditions is violated, the small sample behavior can be doubtful. 3.3 Example: Logistic Regression (Revisited) The different test statistics are exemplified with the logistic regression example on schizophrenic relapse from the first section by testing the treatment effect and are given in Table 4. All three quantities are of similar size and show a significant effect of treatment based on an asymptotic χ 2 distribution with one degree of freedom. The corresponding P-values are also shown in Table 4. 3.4 Deviance and Goodness of Fit The deviance is strongly linked to the likelihood ratio statistics. It results by comparing the model fit with the saturated model, that is, the model that fully describes the data via E(yi ) = ηi for i = 1, . . . , n. Let η˜ = (η˜ 1 , . . . , η˜ n ) be the maximizer in the saturated model,

Table 4. Test Statistics for the Logistic Regression Example Statistics

Estimate

P-Value

5.006 5.148 5.217

0.0253 0.0233 0.0224

Wald Score Likelihood ratio

and denote with li (ηi ) the likelihood contributions in Equation (11) evaluated at the linear predictor. The deviance is then defined as D(Y, µ) = 2φ



wi {li (η˜ i ) − li (ηˆ i )}

i

where Y = (y1 , . . . , yn ) and µ = (µ1 , . . . , µn ), The deviance is a helpful tool for model selection. Let, therefore, Ma denote the model with g(µ) = xa βa . Accordingly, Mab denotes the model built from g(µ) = xβ, with x = (xa , xb ) and β = (βa , βb ). The models are nested, and one can test Ma against Mab by making use of the asymptotic result that under the hypothetical model Ma a

D(Y, µMa ) − D(Y, µMab ) ∼ χ 2 (dfa −dfab )

(23)

where D(M) is the deviance calculated under the corresponding model and df a and df ab are the degrees of freedom for the two considered models, that is, n minus the number of estimated parameters. The deviance can also be used to validate the overall goodness of fit of a model. Assume that x is a metrically scaled covariate, which is included in the model in a linear form; that is, η = β0 + xβx . A general and important question is whether the model is in fact linear or whether the functional relationship is different. Note that theoretically the model could be checked by comparing its deviance with the deviance of the saturated model, that is, using D(Y, µ) − D(Y, Y). Based on Equation (23), this would follow a χ 2 distribution with dfa − 0 degrees of freedom. As n → ∞, one has dfa → ∞ and it results that the convergence rate is useless for practical proposes. This in fact forbids the deviance to be used in this way for checking the overall goodness of fit (see

GENERALIZED LINEAR MODELS

Reference 7 for more details). Instead, one can extend the model and test it against a more complex functional form. To check, for instance, the linear model η = β0 + xβx , one can test it against an extended polynomial model; e.g., η = β0 + xβx + x2 βxx + . . . . In this case, the difference of the deviance for both models follows a well-behaved asymptotic pattern and with Equation (23) the linear model can be checked. Over the last decade, such parametric model checks have increasingly been replaced by nonparametric methods where the alternative model is specified through g(µ) = m(x), with m(·) as unknown but smooth function to be estimated from the data. The idea can be found in Firth et al. (21); for further references and examples, one should consult Bowman and Azzalini (22). 3.5 Example: Multiple Logistic Regression (Revisited) For the nodal involvement example from above, a variable selection is desirable to include only those covariates in the model that are significant. Two methods are illustrated here. The first is a Wald test using Equation (20); the second is a likelihood ratio test stated in Equation (22). The latter is equivalent to a deviance comparison making use of Equation (23). Table 5 shows the deviance for several models. The models are nested, in that in each new row, a parameter of the precedent row is set to zero. Using Equation (23), the difference of the deviance in two consecutive rows is chisquared distributed with 1 degree of freedom. The corresponding P-value is also shown in Table 5. In the same form, the P-value for the Wald Statistic is provided, testing the significance of the effect, which is omitted from the preceding row. It would seem that the model including the stage of the tumor, the X-ray assessment, and the serum acid phosphatase level seems appropriate for describing the data.

9

4 MODEL DIAGNOSTICS AND OVERDISPERSION 4.1 Residuals It is a wise principle to check the model assumptions carefully before relying on inferential conclusions drawn from it. A first and helpful step in this direction is the numerical and graphical investigation of the fitted residuals yi − µˆ i . However, even though this is a manifest approach in standard regression with normally distributed residuals, in the generalized case, one is faced with two additional features. First, residuals are not homoscedastic, that is, they have different variances, and second, residuals can take clustered values so that visual investigation is cumbersome. Both of these problems have been adequately addressed. Pierce and Schafer (23) provide an early discussion of this point; a very readable account of regression diagnostics is provided by Atkinson (24). For a detailed discussion of residual based model checking, one may consult Cook and Weisberg (25). Heterogeneity of the residuals can be taken into account by standardization that leads to the Pearson residuals defined by yi − µˆ i εˆ Pi =

v(µˆ i )/wi

(24)

The sum of the squared Pearson residuals is also known as the Pearson statistics: X 2 = (yi − µˆ i )2 /{wi v(µˆ i )}. Beside the Pearson residuals, the deviance residual is in frequent use. This residual is defined as εˆ Di =

√ wi sign(η˜ i − ηˆ i )[2{li (η˜ i ) − li (ηˆ i )}]1/2 (25)

with sign(·) as the sign function and ηˆ i = βˆ0 + xi βˆx . Finally, a third type of residual has been suggested, given by transforming yi by the dµ function T(·) defined through T(·) = , v1/3 (µ) which leads to the so-called Anscombe residuals εˆ Ai =

ˆ T(yi ) − E{T(y i )}

Var{T(yi )}

10

GENERALIZED LINEAR MODELS

Standard software packages give both Pearson and deviance residuals. In contrast, Anscombe residuals are not well established in software packages, and therefore, they are less frequently used, principally because of the additional numerical burden resulting from estimating both the expectation and the variance of T(yi ). All three residuals correct for heterogeneity. For normal responses, they all reduce to the classic fitted residuals εˆ i = yi − µˆ i . For discrete data, however, the residuals exhibit an additional problem because they take clustered values. For instance, if yi is binary in a logistic regression on x, the residuals will mirror the binary outcomes of yi , which makes simple residual exploration by eye complicated or even impossible. To overcome this point, one can plot the residuals against x and apply some smoothing methods to explore whether there is any structure left that would indicate a lack of fit. Such approaches have been suggested by, among others, le Cessie and van Houwelingen (26), and they have become increasingly popular over the last couple of years. Bowman and Azzalini (22) can serve as a general reference.

4.2 Example: Multiple Logistic Regression (Revisited) For the nodal involvement data, the Pearson is Plotted against deviance residuals for the model, including xstage , xxray , and xacid as covariates. For better visibility, some small random noise has been added to avoid overlaying points in the plot. There is one observation with a large Pearson residual and a somewhat inflated deviance residual. This observation would need some extra consideration (Fig. 2).

4.3 Overdispersion For several distributions, the variance of y is completely determined by the mean. For such distributions, the dispersion parameter φ is a known constant and the variance of yi depends exclusively on v(µi ) and some known weights wi . Examples are the binomial distribution and the Poisson distribution. When using such distributions to model data, the analyst sometimes encounters the phenomena of overdispersion, in which the empirical residual variance is larger than the variance determined by the model. Overdispersion can arise from several sources. A possible explanation is that relevant covariates have been omitted in the model, which could, for instance, happen if covariates have not been recorded or are unmeasurable (or latent). Two commonly used modeling approaches correct for overdispersion. First, instead of assuming φ to be fixed and determined from the stochastic model, one allows φ to be unknown. One assumes Var(yi |ηi ) = v(µi )wi φ with wi as known weights, v(·) as a known variance function, and φ as an unknown dispersion parameter. Estimation can then be carried out with Equation (18). The resulting stochastic model with φ unknown is not generally in a tractable form. There are, however, some exceptions. For instance, if y is Poisson distributed with mean m, say, and m is assumed to be random following a gamma distribution with mean µ and variance τ , one obtains a negative binomial distribution for y. Integrating out the random mean m yields the overdispersed model with E(y) = µ and Var(y) = µφ, where φ = (1 + τ )/τ . More details are found in McCullagh and Nelder (2). Alternatively it has been suggested to accommodate overdispersion by fitting a mixed model with latent random effect. This random effect mimics the latent covariates that are assumed to be responsible for

Table 5. Deviance and Wald Statistics for Logistic Regression Example Model xray + acid + stage + grade + age xray + acid + stage + grade xray + acid + stage xray + acid

Deviance

P-Value for Difference of Deviance

P-Value for Wald Statistic

47.61 47.76 49.18 54.79

0.699 0.233 0.024

0.698 0.235 0.018

GENERALIZED LINEAR MODELS

11

Deviance Residuals

4

• 2

••• ••••

0

••••••••

•••• ••••• •••••• •• •• • •••

−2

•

−2

Figure 2. Pearson versus deviance residuals for nodal involvement data.

the observed overdispersion. Estimation is then carried out by numerical or approximative integration over the random effect (see Reference 27 for more details). The approach is attractive because it remains in the distributional framework. It requires, however, additional numerical effort. Tests on overdispersion can be constructed in two ways. First, one can use a Likelihood Ratio Statistic comparing the fitted models with and without overdispersion. For instance, in the Poisson/gamma model sketched above, one can test whether a random mean m has variance 0; i.e., τ = 0 (without overdispersion). The resulting test based on a log likelihood ratio is, however, nonstandard, because the hypothesis τ = 0 lies on the boundary of the parameter space τ > 0. Second, one can simply fit a model with a variance specified by Var(yi |xi ) = µi + αh(µi ), where h(·) is a prespecified, e.g., quadratic function. Overdispersion is then tested by α = 0. Details on tests of overdispersion are found in Cameron and Trivedi (28). Joint modeling of mean and dispersion is treated in more depth in Jørgensen (29). 4.4 Example: Poisson Regression (Revisited) In Table 6, the different goodness of fit measures are given for the Poisson data example from above. As can be observed, the difference between deviance and Pearson statistics

0 2 Pearson Residuals

4

Table 6. Goodness of Fit for Poisson Data Example

X2 D df

Model with Linear Time

Model with Quadratic Time

39.59 38.56 13

31.52 30.67 12

is negligible. Based on the difference of the goodness of fit statistics in the linear and quadratic model, the quadratic time effect seems significant with P-values 0.0045 and 0.0049 for the Pearson and deviance statistics, respectively. There is no free dispersion parameter for the Poisson distribution, and so for a wellfitting model, the ratio of the Pearson χ 2 to its degrees of freedom should tend toward unity. A simple correction for overdispersion, proposed by McCullagh and Nelder (2), is to inflate the standard errors of the parameter estimates by an estimated dispersion parameter, and so give some protection against over-interpreting the data because of inappropriately small confidence intervals around the parameter estimates. Based on Equation (18), this means

that confidence intervals are inflated by φˆ = X 2 /df . In the linear time model, this is 9.59/13 or 1.745, and

12

GENERALIZED LINEAR MODELS

in the quadratic model, it is 31.52/12 or 1.621. After this correction, the estimated standard error for the quadratic time parameter increases to 0.0015 × 1.621 = 0.0024. In terms of estimating the change in accidents over, say, a 10-year period, the first model, linear in time but with scale parameter fixed at 1, would estimate this as a 47% reduction (95% confidence interval 40% to 52%), whereas the linear model with scale parameter estimated would return the same point estimate, i.e., 47%, but now the 95% confidence interval would stretch from 35% to 56%. For the quadratic model with scale parameter estimated, the change for the first 10 years 1981 to 1991 would be estimated as a 39% reduction with a 95% confidence interval of 23% to 52%. 5

5.2 Quasi-Likelihood

EXTENSIONS

5.1 Multicategorial and Ordinal Models In the standard form of a GLM, one assumes that the response y is univariate. It can be readily extended to accommodate multivariate response variables, for example, if y is a categorical response with possible outcomes y ∈ {1, 2, . . . , k}. Recoding y by the dummy variables
1 if y = r y˜ r = 0 otherwise for r = 1, . . . , k − 1, the distribution of the resulting vector y˜ = (˜y1 , . . . , y˜ k−1 )T can be written in exponential family form exp{˜yT θ˜ − b(θ˜ )} with multivariate parameter vector θ˜ . Accordingly, the structural part of the model (3) is written in multivariate form

g(µ) = g{E(˜y|x)} = η

different influence on the separate cells of the discrete distribution. Multicategorical responses are often measured on an ordinal scale. In this case, it is plausible to model cumulated probabilities P(y ≤ r) = rl=1 E(˜yl ) for r = 1, . . . , k − 1. Taking a logit-based link leads to the Cumulative Logistic Model or Proportional Odds Model, respectively, defined by the link funk g(·) with components gr (µ) = log{P(y ≤ r)/P(y > r)}. The linear predictor should now be specified as ηr = β0r + xβx , mirroring a joint influence of covariate x on all categories of x but with separate category specific intercepts, with β0r ≤ β0r+1 for r = 1, . . . , k − 2. The model can also be interpreted via a latent univariate response variable as motivated in depth in McCullagh and Nelder (2); see also in Fahrmeir and Tutz (6).

(26)

with g(·) = {g1 (·), . . . , gk−1 (·)}T and η˜ = (η˜ 1 , . . . , η˜ k−1 )T as a multivariate linear predictor. The natural link is then gr (η) = log{P(y = r)/P(y = k)} with P(y = r) = E(˜yr ) and P(y = k) = 1 − k−1 yl ). For notational simplicl=1 E(˜ ity, the dependence on x is thereby omitted. For the linear predictor, one may set ηr = β0r + xβxr to allow covariate x to have a

The cornerstone of so-called quasi-likelihood estimation (see Reference 30) is that the variance function v(µ) is inherited from the assumed distribution, but otherwise the distributional assumption is given up. As a consequence, no proper likelihood function is available for estimation, and instead, estimating equations are used. The structural part is extended to accommodate both mean and variance. The latter also allows multivariate response models. The mean specification can be written in matrix form as E(Y|X) = µ(β), where Y = (y1 , . . . yn ) is the response vector and X is the design matrix comprising the rows (1, xi ). Note that the explicit dependence on parameters β is not now of primary interest, but it is retained for notational convenience. The distributional assumption is now expressed as Var(Y|X) = V(µ)Wφ, with W = diag(wi ) the matrix of known weights and V(µ) as the prespecified variance matrix. In particular, correlated observations can be incorporated by appropriate specification of nondiagonal entries in V(µ). In the case of independent errors, V(µ) simplifies to diag{v(µi )}. Based on estimating equation theory (see Reference 31), the best estimating equation for β becomes ˆ Y − µˆ −1 ∂ µ ˆ := {V(µ)W} s(β) ˆ ∂β T φ

(27)

GENERALIZED LINEAR MODELS

which can be solved by Newton Raphson or Fisher scoring in the same way as in a likelihood framework. An attractive feature of the approach is that score and information identities are inherited; that is, if the prespecified variance is the true variance, one gets E{s(β)} = 0,

E(−

∂s(β) ) = Var(s(β)) ∂β

This approach became fashionable in the early 1990s with the introduction of generalized estimating equations (GEEs) (see Reference 32). 6

SOFTWARE

The software package GLIM (see Francis et al. (1993)) was to some extent a precursor on the computational side of GLMs. Nowadays, however, most standard software packages are well equipped with fitting routines for GLMs. SPSS (www.spss.com SPSS Inc, 233 S. Wacker Dr, Chicago, IL 60606, USA) allows one to fit most parametric GLMs including multinomial and ordinal models (see GLM procedure). The same holds for SAS (www.sas.com SAS Inst Inc, 100 SAS Campus Dr Cary, NC 27513, USA; see, for instance, PROC, GENMOD, and MIXED). The widest range of fitting possibilities, in particular using nonparametric approaches, is found in Splus (www.insightful.com Insightful, ¨ Kagensti 17 4153 Reinach, Switzerland) or its open source clone R (www.r-project.org; see function glm). Beside these mainstream programs, several smaller and specialized tools provide estimation routines for GLMs. A comprehensive overview is found in Fahrmeir and Tutz (6). REFERENCES 1. J. A. Nelder and R. W. M. Wedderburn, Generalized linear models. Journal of the Royal Statistical Society, Series B 1972; B 34: 370– 384. 2. P. McCullagh and J. A. Nelder, Generalized Linear Models. 2nd ed. New York: Chapman and Hall, 1989. 3. M. Aitkin, D. Anderson, B. Francis, and J. Hinde, Statistical Modelling in GLIM. Oxford, U.K.: Oxford University Press, 1989.

13

4. R. H. Myers, D. C. Montgomery, and G. G. Vining, Generalized Linear Models: With Applications in Engineering and the Sciences. New York: Wiley, 2001. 5. A. J. Dobson, Introduction to Generalized Linear Models. Boca Raton, FL: Chapman & Hall/CRC, 2001. 6. L. Fahrmeir and G. Tutz, Multivariate Statistical Modelling Based on Generalized Linear Models. 2nd ed. New York: Springer Verlag, 2001. 7. D. Firth, Generalized linear models. In: D. V. Hinkley, N. Reid, and E. J. Snell (eds.), Statistical Theory and Modelling. London: Chapman and Hall, 1991. 8. S. M. Stigler, The History of Statistics: The Measurement of Uncertainty Before 1900. Cambridge, MA: Harvard University Press, 1986. 9. A. Stuart and J. K. Ord, Kendall’s Advanced Theory of Statistics, Vol. 2A: Classical Inference & the Linear Model. New York: Oxford University Press, 1999. 10. A. Gumley, M. O’Grady, L. McNay, J. Reilly, K. Power, and J. Norrie, Early intervention for relapse in schizophrenia: Results of a 12 month randomised controlled trial of cognitive behavioural therapy. Psychological Medicine 2003; 419–431. 11. B. Brown, Prediction analysis for binary data. In: R. Miller, B. Efron, B. Brown, and L. Moses (eds.), Biostatistics Casebook. New York: Wiley, 1980, pp. 3–18. 12. A. Morrison, D. H. Stone, A. Redpath, H. Campbell, and J. Norrie, Trend analysis of socio-economic differentials in deaths from injury in childhood in scotland, 1981–95. British Medical Journal 1999; 318: 567– 568. 13. F. E. J. Harrel, Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. New York: Springer Series in Statistics, 2001. 14. J. Norrie and I. Ford, The role of covariates in estimating treatment effects and risk in long-term clinical trials. Statistics in Medicine 2002; 21(19): 2899–2908. 15. T. A. Severini, Likelihood Methods in Statistics. Oxford, U.K.: Oxford University Press, 2000. 16. L. Fahrmeir and H. Kaufmann, Consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models. The Annals of Statistics 1985; 13: 342–368.

14

GENERALIZED LINEAR MODELS

17. L. Fahrmeir and H. Kaufmann, Asymptotic inference in discrete response models. Statistical Papers 1986; 27: 179–205. 18. A. Agresti, Categorical Data Analysis. New York: Wiley, 1990. 19. N. Cressie and T. R. C. Read, Multinomial goodness-of-fit tests. Journal of the Royal Statistical Society, Series B 1984; 46: 440– 464. 20. T. J. Santner and D. E. Duffy, The Statistical Analysis of Discrete Data. New York: Springer Verlag, 1990. 21. D. Firth, J. Glosup, and D. V. Hinkley, Model checking with nonparametric curves. Biometrika 1991; 78: 245–252. 22. A. W. Bowman and A. Azzalini, Applied Smoothing Techniques for Data Analysis: The Kernel Approach with S-Plus Illustrations. Oxford, U.K.: Oxford University Press, 1997. 23. D. A. Pierce and D. W. Schafer, Residulas in Generalized Linear Models. Journal of the American Statistical Association 1986; 81: 977–986. 24. A. Atkinson, Plots, Transformations and Regression. An Introduction to Graphical Methods of Diagnostic Regression Analysis. Oxford Statistical Science Series. Oxford, U.K.: Clarendon Press, 1985. 25. R. D. Cook and S. Weisberg, Applied Regeression including Computing and Graphics. New York: Wiley, 1999.

26. S. le Cessie and J. van Houwelingen, A goodness-of-fit test for binary regression models, based on smoothing methods. Biometrics 1991; 47: 1267–1282. 27. M. Aitkin, A general maximum likelihood analysis of variance components in generalized linear models. Biometrics 1999; 55: 218–234. 28. A. Cameron and P. Trivedi, Regression Analysis of Count Data. Cambridge, U.K.: Cambridge University Press, 1998. 29. B. Jørgensen, The Theory of Dispersion Models. Boca Raton, FL: Chapman & Hall, 1997. 30. R. W. M. Wedderburn, Quasi-likelihood functions, generalized linear models, and the gauss-newton method. Biomtrika 1974; 61: 439–447. 31. V. Godambe and C. Heyde, Quasi-likelihood and optimal estimation. International Statistical Review 1987; 55: 231–244. 32. Diggle, P. J., K.-Y. Liang, and S. L. Zeger, Analysis of Longitudinal Data. Oxford, U.K.: Oxford University Press, 1994. 33. B. Francis, M. Green, C. Payne. The GLIM System: Release 4 Claredon Press,

GENERIC DRUG REVIEW PROCESS

the application contains all the necessary components, then an ‘‘acknowledgment letter’’ is sent to the applicant to indicate its acceptability for review and to confirm its filing date. Once the application has been determined to be acceptable for filing, the Bioequivalence, Chemistry/Microbiology, and Labeling reviews may begin. If the application is missing one or more essential components, a ‘‘Refuse to File’’ letter is sent to the applicant. The letter documents the missing component(s) and informs the applicant that the application will not be filed until it is complete. No additional review of the application occurs until the applicant provides the requested data and the application is acceptable and complete. The Bioequivalence Review process establishes that the proposed generic drug is bioequivalent to the reference listed drug, which is based on a demonstration that both the rate and the extent of absorption of the active ingredient of the generic drug fall within established parameters when compared with that of the reference listed drug. The FDA requires an applicant to provide detailed information to establish bioequivalency. Applicants may request a waiver from performing in vivo (testing done in humans) bioequivalence studies for certain drug products where bioavailability (the rate and extent to which the active ingredient or active moiety is absorbed from the drug product and becomes available at the site of action) may be demonstrated by submitting data such as (1) a formulation comparison for products whose bioavailability is self evident, for example, oral solutions, injectables, or ophthalmic solutions where the formulations are identical or (2) comparative dissolution. Alternatively, in vivo bioequivalence testing that compares the rate and the extent of absorption of the generic versus the reference product is required for most tablet and capsule dosage forms. For certain products, a head-to-head evaluation of comparative efficacy based on clinical endpoints may be required. On filing an ANDA, an establishment evaluation request is forwarded to the Office of

An Applicant is any person (usually a pharmaceutical firm) who submits an abbreviated new drug application (ANDA), or an amendment or supplement to them, to obtain Food and Drug Administration (FDA) approval to market a generic drug product, and/or is any person (or firm) who owns an approved application or abbreviated application. A generic drug product is one that is comparable to an innovator drug product [also known as the reference listed drug (RLD) product as identified in the FDA’s list of Approved Drug Products with Therapeutic Equivalence Evaluations] in dosage form, strength, route of administration, quality, performance characteristics, and intended use. The ANDA contains data that, when submitted to the FDA’s Center for Drug Evaluation and Research, Office of Generic Drugs, provides for the review and ultimate approval of a generic drug product. Once approved, an applicant may manufacture and market the generic drug product provided all issues related to patent protection and exclusivity associated with the RLD have been resolved. Generic drug applications are termed ‘‘abbreviated’’ in that generally they are not required to include preclinical (animal) and clinical (human) data to establish safety and effectiveness. These parameters were established on the approval of the innovator drug product, which is the first version of the drug product approved by the FDA. An application must contain sufficient information to allow a review to be conducted in an efficient and timely manner. On receipt of the application, a pre-filing assessment of its completeness and its acceptability is performed by a project manager within the Regulatory Support Branch, Office of Generic Drugs. If this initial review documents that This article was modified from the website of the United States Food and Drug Administration http://www.fda.gov/cder/handbook/ by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

GENERIC DRUG REVIEW PROCESS

Compliance to determine whether the product manufacturer, the bulk drug substance manufacturer, and any outside testing or packaging facilities are operating in compliance with current Good Manufacturing Practice (cGMP) regulations as outlined in 21 CFR 211. Each facility listed on the evaluation request is evaluated individually and an overall evaluation for the entire application is made by the Office of Compliance. Furthermore, a preapproval product specific inspection may be performed on to assure data integrity of the application. The Chemistry/Microbiology review process provides assurance that the generic drug will be manufactured in a reproducible manner under controlled conditions. Areas such as the applicant’s manufacturing procedures, raw material specifications and controls, sterilization process, container and closure systems, and accelerated and room temperature stability data are reviewed to assure that the drug will perform in an acceptable manner. The Labeling review process ensures that the proposed generic drug labeling (package insert, container, package label, and patient information) is identical to that of the reference listed drug except for differences caused by changes in the manufacturer, distributor, pending exclusivity issues, or other characteristics inherent to the generic drug product (tablet size, shape or color, etc.). Furthermore, the labeling review serves to identify and to resolve issues that may contribute to medication errors such as similar sounding or appearing drug names, and the legibility or prominence of the drug name or strength. If at the conclusion of the Bioequivalence Review, it is determined that deficiencies exist in the bioequivalence portion of the application, a Bioequivalence Deficiency Letter is issued by the Division of Bioequivalence to the applicant. This deficiency letter details the deficiencies and requests information and data to resolve them. Alternatively, if the review determines that the applicant has satisfactorily addressed the bioequivalence requirements, the Division of Bioequivalence will issue a preliminary informational letter to indicate that no additional questions exist at this time. If deficiencies are involved in the Chemistry/Manufacturing/Controls, Microbiology,

or Labeling portions of the application, these deficiencies are communicated to the applicant in a facsimile. The facsimile instructs the applicant to provide information and data to address the deficiencies and provides regulatory direction on how to amend the application. Once the above sections are found to be acceptable, as well as the preapproval inspection and bioequivalence portion of the application, then the application moves toward approval. If after undergoing a final Office-level administrative approval review by all review disciplines no additional deficiencies are noted, then the application can be approved. A satisfactory recommendation from the Office of Compliance based on an acceptable preapproval inspection is required prior to approval. The preapproval inspection determines compliance with cGMPs and a product specific evaluation that concerns the manufacturing process of the application involved. If an unsatisfactory recommendation is received, a ‘‘not approvable’’ letter may be issued. In such a case, approval of the generic drug product will be deferred pending a satisfactory reinspection and an acceptable recommendation. After all components of the application are found to be acceptable, an ‘‘Approval’’ or a ‘‘Tentative Approval’’ letter is issued to the applicant. The letter details the conditions of the approval and allows the applicant to market the generic drug product with the concurrence of the local FDA district office. If the approval occurs prior to the expiration of any patents or exclusivities accorded to the reference listed drug product, a tentative approval letter is issued to the applicant that details the circumstances associated with the tentative approval of the generic drug product and delays final approval until all patent/exclusivity issues have expired. A tentative approval does not allow the applicant to market the generic drug product.

GENE THERAPY

Table 1. Gene therapy clinical trial indications (from www.wiley.co.uk/genemed/clinical)

SAMANTHA L GINN and IAN E ALEXANDER

Indication

Gene Therapy Research Unit Children’s Medical Research Institute and The Children’s Hospital at Westmead NSW, Australia

Cancer diseases Cardiovascular diseases Monogenic diseases Infectious diseases Neurological diseases Ocular diseases Other diseases Gene marking Healthy volunteers Total

Gene therapy was conceived originally as an approach to the treatment of inherited monogenic diseases; it is defined by the United Kingdom’s Gene Therapy Advisory Committee as ‘‘the deliberate introduction of genetic material into human somatic cells for therapeutic, prophylactic or diagnostic purposes (1).’’ Ongoing improvements in gene-transfer technologies and biosafety have been accompanied by a growing appreciation of the broader scope of gene therapy. Pathophysiological processes such as wound healing (2), chronic pain (3) and inflammation (4), cancer (5), and acquired infections such as HIV-1 (6) are now becoming realistic targets for this promising therapeutic modality. The first authorized gene transfer study took place at the National Institutes of Health (NIH) in 1989. In this marker study, tumor-infiltrating lymphocytes were harvested, genetically tagged using a retroviral vector, and reinfused with the intention of examining the tumor-homing capacity of these cells. This landmark study provided the first direct evidence that human cells could be genetically modified and returned to a patient without harm (7). Since then, over 1300 trials have been approved, initiated, or completed worldwide, which are performed predominantly in the United States (8, 9). Most studies have focused on cancer, with cardiovascular and monogenic diseases the next most frequent indications (Table 1). These predominantly early-phase trials have provided invaluable proof-of-concept for gene therapy by confirming that desired changes to the target cell phenotype can be achieved successfully. In most trials, however, an insufficient number of cells have been genetically modified to achieve therapeutic benefit.

Number of protocols 871 (66.5%) 119 (9.1%) 109 (8.3%) 85 (6.5%) 20 (1.5%) 12 (0.9%) 21 (1.6%) 50 (3.8%) 22 (1.7%) 1309

Notable exceptions to the lack of therapeutic efficacy have been the successful treatment of several primary immunodeficiencies that affect the hematopoietic compartment (10–16). 1 REQUIREMENTS FOR SUCCESSFUL THERAPEUTIC INTERVENTION The prerequisites for successful gene therapy are complex and disease-specific, but invariably they include the availability of an efficient gene delivery technology and an understanding of its properties, capacity and limitations, a detailed knowledge of the biology of the target cell population, and a precise understanding of the molecular basis of the target disease (Fig. 1). Although significant progress is being made in each of these areas, the ability to achieve efficient gene delivery has been described as ‘‘the Achilles heel of gene therapy (17).’’ 1.1 Gene Delivery Technology Gene delivery systems can be classified into two broad categories: nonviral physicochemical approaches and recombinant viral systems. The comparative strengths of nonviral approaches include ease of chemical characterization, simplicity and reproducibility of production, larger packaging capacity, and reduced biosafety risks (18, 19). Gene delivery, however, is relatively inefficient and

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

GENE THERAPY

Gene Delievery Technology

Gene Therapy Target Cell Biology

Understanding of the Target Disease

Figure 1. Venn diagram of requirements for successful therapeutic intervention.

the effects are often transient. Examples of nonviral systems include microinjection of individual cells, DNA-calcium phosphate coprecipitation, and the formulation of DNA into artificial lipid vesicles (20–24). In contrast, viral systems, which are commonly modified to render them replicationincompetent, are markedly more efficient and exploit favorable aspects of virus biology (17, 25–27). Viral vectors can be divided into two main categories: nonintegrating and integrating, based on the intracellular fate of their genetic material. These properties are important when considering the required duration of the treatment. Nonintegrating vectors are maintained extrachromosomally in the nucleus, whereas the genome of integrating vectors becomes incorporated into the host cell genome that provides the potential for stable long-term gene expression. Limitations include increased biosafety risks caused by contamination by replication-competent virus, the presence of potentially toxic viral gene products, and insertional mutagenesis when integrating vectors are used (28). Irrespective of vector type, another important limitation is the induction of unwanted immune responses directed against components of the delivery system and/or the encoded transgene product. These responses may be either cell-mediated or humoral and depend on several variables that include

the nature of the transgene, the vector and promoter used, the route and site of administration, the vector dose, and the host factors. Ultimately, host-vector immune responses have the potential to influence clinical outcomes negatively (29–32). Accordingly, the optimization of gene delivery systems and strategies to evade deleterious immune responses remains a fundamentally important challenge (33–35). Difficulty in producing high-titre vector preparations and constraints on packaging capacity are also drawbacks. Hybrid vectors that combine the advantageous features of more than one virus have also been developed, although their application has been largely in in vitro models (36–40). Despite these limitations, the relative efficiency of gene transfer has resulted in the predominant use of viral vectors in pre-clinical and clinical gene therapy studies up to the present time. The uses of gene-correction rather than gene-addition strategies, which include targeted recombination (41–43), antisense oligonucleotide-induced exon skipping (44), and RNA interference (45), are also being investigated. Such strategies will be particularly important in the context of dominant disease processes in which the simple addition of a functionally normal gene is insufficient. Currently, these approaches lack the efficiency required for human gene therapy applications. Finally, although recombinant viral vectors have the most immediate potential for clinical use, it is envisaged that these systems will be supplanted by hybrid and derivative systems that combine the simplicity and safety of nonviral gene delivery with favorable aspects of viral biology. Virosomes are a prototypic example of such a system and among these, the Hemagglutinating Virus of Japan liposome has been the most extensively investigated (46, 47). 1.2 Target Cell Biology Each viral vector system possesses a unique set of biological properties (Table 2), and their use is governed largely by the biology of the target cell. For example, integration provides the molecular basis for stable long-term gene expression as would be required for the

GENE THERAPY

3

Table 2. Properties of widely used viral vector systems Vector

Adenovirus

Retrovirus∗

Lentivirus†

AAV

Genome Insert capacity Location in cell nucleus Cell-cycle dependent gene transfer Duration of transgene expression Functional titre (per mL) Immunogenicity

dsDNA 7 to 30 kb extrachromosomal

ssRNA 7 kb integrated

ssRNA 10 kb integrated

no

yes

no

ssDNA 4.8 kb integrated/ extrachromosomal no

transient

long-term

long-term

long-term

≥ 1012

≥ 107

≥ 109

≥ 109

high

low

low

low

AAV, adeno-associated virus. ∗ Most commonly derived from Moloney-Murine Leukemia virus (MoMLV). † Most commonly derived from HIV-1.

treatment of genetic disease in replicating cell populations. Integration also provides the potential for gene-modified cells to be expanded from a modest number of progenitors. This feature is of paramount importance when the genetic modification of cells capable of enormous proliferative expansion is required. This feature has been powerfully illustrated in the first successful treatment of a genetic disease by gene therapy (10, 11). In this study, a selective growth and survival advantage was conferred on hematopoietic stem cells following transduction with a retroviral vector based on the Moloney murine leukemia virus. Despite the advantages of viral integration, nonintegrating vectors are potentially effective if the target cell is nondividing. Efforts are also being made to develop integration-deficient forms of integrating vectors in an attempt to increase biosafety (48). Additional constraints, such as the replication state of the target cell or whether the target cell is amenable to ex vivo manipulation define vector choice and gene transfer protocols even more. For example, vectors based on lentiviruses and adeno-associated viruses can modify postmitotic and nondividing cells (49–52), which makes them of particular interest for targets such as muscle and the nervous system.

1.3 Disease Pathophysiology Before a gene-therapy approach can be considered feasible for a disease or physiological process, a precise understanding of the underlying molecular basis is required. The requirement for transient or persistent transgene expression must also be considered. For example, the extrachromosomal nature of adenoviral vectors has the potential to limit the duration of gene expression by dilutional loss during cell division (53). In some contexts, in which only transient gene expression is required, such as anticancer gene therapy, this may be a positive attribute. The pathophysiology of the target disease also defines the number of cells that must be successfully gene-modified to achieve the desired therapeutic effect. For example, it is anticipated that for the treatment of hemophilia B, levels of Factor IX as low as 1% of normal will be therapeutic (54). For some more demanding disease phenotypes, expansion of gene-modified cells through in vivo selection is one strategy by which the fundamental challenge of gene-modifying sufficient cells to achieve therapeutic benefit may be overcome. Ultimately, however, the development of more efficient gene delivery technologies is required to allow the effective treatment of the many human diseases and pathophysiological processes that are potentially amenable to gene therapy.

4

2

GENE THERAPY

PRECLINICAL RESEARCH

Before a clinical gene therapy protocol can be considered for human application, extensive preclinical testing is required. The data generated is vital in establishing whether the target cell can be safely gene-modified to produce the phenotypic changes required for disease correction. This involves years of preclinical experimental testing progressing from tissue culture models to small animal models (most commonly mice), and finally to large animal models when feasible. 2.1 In Vitro Studies An important first step in establishing proofof-concept data for a clinical gene therapy protocol is provided by in vitro studies. The manipulation of mammalian cells in culture can help define several important experimental parameters. Taking into account the biology of the target cell, several gene transfer approaches could potentially be available. For example, if neuron-targeted gene transfer is required, several recombinant viral vector systems are available, such as those based on adeno-associated virus, herpes simplex virus, and lentiviruses (55–60). Using cells in culture, it is relatively easy to select the vector system that is most effective in genetically modifying the cell type of interest. Important parameters that can be determined in vitro include the tropism of the virus for the relevant target cell population, the minimum vector dose required to achieve the desired phenotypic changes, the level and the duration of transgene expression, and vector toxicity. In vitro systems also allow aspects of expression cassette design to be examined such as the use of tissue-specific promoters or regulated gene expression. Immortalized cell lines are commonly used for such studies, but frequently they are transduced more readily than primary cells and do not consistently model the challenge of transducing specific target cell populations in vivo. Culture of primary cells, and tissue explants where possible. Therefore, is preferable to offer a more realistic representation of the target cell population in its native state before proceeding to animal models.

2.2 Animal Models Another prerequisite for successful gene therapy is the availability of an animal model that approximates the human disease for preclinical testing. Indeed, successful phenotype correction in mouse models of human disease is now frequently reported (61–66). Up to the present time, these successes have rarely been replicated in large animal models or human clinical trials. The explanation for this is primarily quantitative. Success in larger subjects demands that proportionally greater numbers of target cells be gene-modified to reach a therapeutically meaningful threshold. The average child, for example, is approximately 1000-fold bigger than a mouse, and, therefore, presents a correspondingly greater gene transfer challenge. In addition to the challenge of size, other factors, such as species-specific differences, exert potent effects in some contexts. Animal models also provide valuable safety data required by regulatory bodies before approving early-phase human trials, but these models do not always accurately predict adverse effects subsequently observed in human subjects. For example, in a therapeutically successful gene therapy trial for the X-linked form of severe combined immunodeficiency (SCID-X1) 4 of 11 patients developed a leukemia-like illness as a direct consequence of retroviral insertion into chromosomal DNA (28). The risk of insertional mutagenesis when using integrating vectors had long been recognized, but formerly considered low because retroviral vectors had previously been employed extensively without incident in animal models and in almost 300 documented human clinical protocols (9). Interestingly, concurrent with the above report of vector-mediated insertional mutagenesis in humans, the first report of essentially the same phenomenon in mice was published (67). Collectively, these events illustrate the inherent challenge in predictive safety testing, whether in animal models or in early phase human clinical trials. Preferably such tests must be configured with specific adverse events in mind, and where possible, in a manner that accommodates the possible contribution of the particular disease for which gene therapy is being contemplated.

GENE THERAPY

3 CONDUCTING A HUMAN CLINICAL GENE THERAPY TRIAL In comparison with drug-based clinical trials, several additional factors must be considered when undertaking a human gene therapy application. These include additional layers of regulatory oversight, ethical considerations related to the genetic modification of the subject’s cells and availability of appropriate skills, infrastructure, and reagents. 3.1 Regulatory Oversight Clinical research with genetic material poses safety and methodological challenges not shared by other forms of human investigation. As a result, several regulatory requirements must be satisfied before human studies that involve gene transfer can be initiated. In most countries, compared with requirements for pharmaceutical products, these requirements are achieved through an additional layer, or layers, of regulation. Depending on the host country, regulatory oversight can be complicated even more by the fact that existing regulatory frameworks have evolved for more conventional therapeutic products. As for all human clinical research gene therapy, trials must also be conducted according to a set of standards referred to as Good Clinical Practice that are based on the Declaration of Helsinki (68). In the United States, it is a federal requirement that clinical protocols that involve the administration of recombinant DNA products be reviewed and approved by filing an investigational new drug application with the Food and Drug Administration (FDA). In addition, applications must be approved by local institutional human ethics and biosafety committees. The key regulatory issues for U.S.-based clinical gene therapy trials have been reviewed by Cornetta and Smith (69). In the United Kingdom, gene therapy applications are similarly regulated by the Medicines and Healthcare Products Regulatory Agency, the Gene Therapy Advisory Committee and local institutional committees. This regulatory complexity is particularly burdensome in the context of multinational trials and is a major driver behind efforts for the global harmonization.

5

Such efforts will not only facilitate international studies but also improve data quality and participant safety (70). 3.2 Special Ethical Considerations In contrast to drug-based clinical trials, several special ethical issues must be considered for human gene transfer studies. These issues include the possibility of inadvertent germ-line transmission and, depending on the type of delivery vehicle used, the ability to introduce lifelong modifications to the subject’s chromosomal DNA, the latter resulting in the need for long-term clinical follow-up. Currently, only somatic cell gene therapy protocols have been initiated. The use of germline manipulation, in which the genomes of germ cells are deliberately modified with the likely effect of passing on changes to subsequent generations, is opposed at this time (71). Although genetic manipulation of the human germ-line is illegal in many countries, this consensus is not unanimous and its use remains the subject of vigorous debate (72, 73). For any research team who attempt to develop a new medical treatment, patient safety is of paramount importance, and the decision to proceed with a gene therapy approach requires a careful balancing of the associated risks and benefits. For example, bone marrow transplantation from an HLAmatched sibling donor is the treatment of choice for diseases such as SCID-X1. Unfortunately, however, this option is available for only one in five affected infants. The alternative is transplantation from a haploidentical or matched unrelated donor, and it caries a substantial risk of severe immunologic complications and of early mortality (74). For these children, a gene therapy approach may carry a lower risk even when possible adverse events associated with gene therapy, such as leukemia induction through insertional mutagenesis, are taken into account (75). Another ethical concern for gene therapy trials is the enrollment of infants and children given their inability to provide informed consent. Although it may be preferable to undertake early phase clinical trials in adults, many severe disease phenotypes are restricted to the pediatric age group, or

6

GENE THERAPY

where meaningful intervention would only be possible early in the course of the target disease. Examples include SCID-X1 (10, 76) and cystic fibrosis (77). Accordingly, depending on the disease context, equally potent counterbalancing ethical arguments are in favor of early phase trials in the pediatric population. Another important consideration is whether a need exists for long-term monitoring of the subject after the gene transfer protocol. Parameters that include the ability of the delivery vehicle to integrate into the genome, the site of integration, vector persistence, biology of the target cell, and transgene-specific effects all influence the risk associated with the treatment. If no vector persistence exists, the risk is analogous to that of any new drug, and long-term follow-up may not necessarily be required (78). 3.3 Skills, Infrastructure, and Reagents To undertake a gene therapy clinical trial, a research team requires access to specialized facilities as well as appropriately trained staff to perform procedures in accordance with required standards. In most countries, the rigor of these requirements increases in late-phase trials to the level of good manufacturing practice. For each gene therapy protocol, the set of skills required are governed largely by the biology of the target cell. For example, in trials that involve gene transfer to hematopoietic stem cells, an ex vivo approach is the method of choice. This approach requires a medical team to harvest the subject’s bone marrow and personnel who can maintain the target cells in sterile culture for up to five days after harvest (10). This approach also requires the availability of an on-site clean-room to perform the cellular manipulations. Alternatively, it might be necessary to deliver the vector directly to the target cell in vivo. Examples of this approach include gene transfer to organs such as the eye or brain or in gene therapy protocols that deliver oncolytic agents to a solid tumor. Experimental products used for gene transfer studies are often complex and difficult to characterize completely in comparison with conventional pharmaceutical agents, which is true particularly for virus-based gene delivery systems that are also challenging to produce on a large-scale and cannot

be sterilized by autoclaving or radiation. Biological variability may also result from the packaging of empty virions, titre differences between different production runs, and loss of titre during storage. The propensity of vectors to undergo inadvertent sequence alteration during production through mechanisms such as recombination or via transcription by error prone polymerases (79) must also be monitored. 4 CLINICAL TRIALS Although originally conceived as a treatment for monogenic diseases, the major indication addressed by gene therapy trials to date has been cancer (66.5%, Table 1). This finding is predominantly caused by the poor prognosis of many cancer phenotypes that make the risk/benefit ratio more favorable for experimental intervention. Although initial trials have been largely unsuccessful, some positive outcomes have occurred. For example, in 2006, Morgan et al. (5) have observed cancer regression in patients who received autologous peripheral blood lymphocytes when modified by a retroviral vector to express a T cell receptor. Although regression was observed in only 2 of 15 patients (13%), which is considerably lower than the 50% response rate achieved when patients received tumorinfiltrating lymphocytes (TILs) in a similar trial (80), this method may prove useful in instances in which TILs are not available. Cardiovascular and monogenic diseases are the next most frequently addressed indications with 119 (9.1%) and 109 (8.3%) trials, respectively, approved worldwide (Table 1). Shortly after the first authorized gene transfer trial was undertaken in 1989 (7), the first therapeutic trial that involved two children who suffered from a severe combined immunodeficiency caused by adenosine deaminase deficiency (ADA-SCID) was approved. This trial was unsuccessful for several reasons that include maintaining the patients on PEG-ADA therapy and using patient T lymphocytes as the target cell population (81). Removal of PEG-ADA coupled with a myeloablative conditioning regime and the targeting of hematopoietic stem cells with an improved transduction protocol resulted

GENE THERAPY 2007 2006 2005 2004 2003 2002 2001 2000 1999 1998 1997 1996 1995 1994 1993 1992 1991 1990 1989 Unknown

7

33 97 98 95 81 89 108 95 116 68 82 51 67 38 37 14 8 2 1 129

Figure 2. Number of gene therapy clinical trials approved worldwide from 1989 to 2007 (reprinted with permission from http://www.wiley.co.uk/genmed/clinical/).

in the successful treatment of patients in subsequent trials for ADA-SCID (13, 14), which highlighted the need to impart a positive growth or survival advantage to the transduced cells. After initiation of the first therapeutic trial, a progressive increase occurred in the number of gene therapy trials approved in the following years (Fig. 2). This increase slowed briefly in the mid-1990s after an NIH review committee co-chaired by Stuart Orkin and Arno Motulsky concluded that ‘‘Significant problems remain in all basic aspects of gene therapy. Major difficulties at the basic level include shortcomings in all current gene transfer vectors and an inadequate understanding of the biological interaction of these vectors with the host (82).’’ This trend toward increasing numbers of gene therapy trials leveled off after 1999 coincident with report of the first severe adverse event as a direct consequence of gene therapy (83). In a Phase I dose escalation study that investigated the safety of an adenoviral vector for the treatment of ornithine transcarbamylase deficiency, a young adult trial participant died as a result of a severe systemic inflammatory reaction to the injected adenoviral vector. This incident resulted in the suspension of all trials at the host institution by the

FDA, and a senate subcommittee investigation that also looked more broadly at clinical trial safety and reporting across the United States (84). In light of the information that emerged, a renewed emphasis exists on critical evaluation of risk/benefit ratios for trial participants and comprehensive reporting of trial outcomes and adverse events. To date, gene therapy trials have been performed in 28 countries (Fig. 3) spanning five continents. These trials have been most comprehensively reviewed by Edelstein et al. (8). Most trials have been conducted in the United States with 864 trials (66%) followed by the United Kingdom (150 trials, 11.5%). Viral vectors are the most frequently used gene delivery system (Table 3) because of their superior gene transfer efficiency over nonviral methods. Of the viral gene delivery systems available, adenoviral and retroviral vectors (derived from murine retroviruses) have been most commonly used accounting for 24.9% and 23.1% of trials, respectively (Table 3). Lentiviral vectors derived from Human Immunodeficiency virus Type 1 (HIV1) are now being used clinically (85) after safety considerations relating to the inherent properties of the parent virus were addressed. Because of differences in their integration site patterns, lentiviral vectors may offer a

8

GENE THERAPY

Figure 3. Geographical distribution of gene therapy clinical trials (reprinted with permission from http://www.wiley.co.uk/genmed/clinical/). Table 3. Gene delivery systems in clinical trial use (from www.wiley.co.uk/genemed/clinical) Vector

Number of protocols

Adenovirus Retrovirus Naked/plasmid DNA Lipofection Vaccinia virus Poxvirus Adeno-associated virus Herpes simplex virus Poxvirus and vaccinia virus RNA transfer Lentivirus Flavivirus Gene Gun Others Unknown Total

326 (24.9%) 302 (23.1%) 239 (18.3%) 102 (7.8%) 65 (5.0%) 60 (4.6%) 48 (3.7%) 43 (3.3%) 26 (2.0%) 17 (1.3%) 9 (0.7%) 5 (0.4%) 5 (0.4%) 22 (1.7%) 40 (3.1%) 1309

safer alternative to retroviral vectors derived from murine retroviruses (86–88). 5

LESSONS LEARNED

It is now more than 15 years since the first authorized gene transfer trial was undertaken in 1989. Since then, over 1300 clinical trials have been initiated worldwide (Table 1) with several notable successes since 2000. Although results from these predominantly early-phase trials have been largely

unsuccessful in providing clinical benefit to human subjects, they have (1) provided clear proof-of-concept for gene therapy, (2) demonstrated that gene therapy is relatively safe, and (3) highlighted several important issues that must be considered to advance the field. The field has also recently experienced the first commercial release of a gene therapy treatment in China by Shenzhen SiBiono GenTech’s Gendicine for head-andneck squamous cell carcinoma (89, 90).

GENE THERAPY

5.1 The Power of In Vivo Selection The major reason gene therapy has been unsuccessful in providing clinical benefit in many disease contexts is low gene transfer efficiencies. Expansion of gene-modified cells, through in vivo selection, is one strategy by which the fundamental challenge of genemodifying sufficient cells to achieve therapeutic benefit can be overcome (91). The power of in vivo selection has been impressively illustrated in the SCID-X1 trial, which is the first successful treatment of a genetic disease by gene therapy (10). For most diseases, however, the gene corrected cells will not have a selective growth or survival advantage. Therefore, efforts to develop strategies for providing modified cells with an exogenous selective advantage are being made. One such strategy exploits mutant forms of the DNA repair enzyme methylguanine methyltransferase and targeting expression to hematopoietic progenitor cells using integrating vector systems. This strategy imparts genetic chemoprotection to the gene-modified cells and has been successfully employed in large animal models (92, 93). 5.2 Insertional Mutagenesis Insertional mutagenesis is now recognized as a clinically established risk associated with the use of integrating vector systems. Random integration events have the potential to drive a cell toward neoplastic transformation through inappropriate activation or inactivation of genes involved in the regulation of cell growth and differentiation. The risk of mutagenesis of cellular sequences promoting a malignant phenotype has been estimated to be about 10−7 per insertion (94). Although avoidance of integrating vector systems is not currently a viable option for gene therapy targeting the hematopoietic compartment, two broad strategies by which the risk of insertional mutagenesis can be significantly reduced include (1) reduction in the absolute number of integration sites to which patients are exposed and (2) reduction of the risk associated with individual integration events. Achievement of the former will depend on more sharply defining both the desired target cell population and the minimum effective dose of gene-corrected cells

9

to produce the desired phenotypic effect as well as optimization of transduction conditions. Reduction of the risk associated with individual integration events is theoretically achievable by careful choice of integrating vector system and optimized expression cassette design that lacks strong viral promoter/ enhancer elements. Whether leukemia represents a generic risk associated with the use of integrating vectors to target hematopoietic progenitor cells, or is linked more directly to specific features of the SCID-X1 gene therapy trial has yet to be established. Whatever the answer, future gene therapy studies that employ integrating vectors must be assessed against the, as yet, poorly quantified risk of the development of neoplasia as a consequence of insertional mutagenesis. 6

THE WAY FORWARD

The capacity of gene therapy to cure human disease is now an established reality, but for now, most disease phenotypes and pathophysiological processes potentially amenable to this exciting therapeutic approach lie beyond the reach of existing technology. The major challenge for the future, therefore, is to address the inherent shortcomings in the efficacy and safety of available gene delivery systems. Developments in this area will be complemented by an improved understanding of target cell biology, in particular the capacity to identify and manipulate stem cell populations. Finally, unwanted host vector interactions, such as immune responses directed against the vector and encoded transgene product, must be better understood and avoided. Such progress is fundamentally dependent on sound basic and pre-clinical research coupled with iterative human clinical trials. REFERENCES 1. Gene Therapy Advisory Committee (GTAC). United Kingdom Department of Health. Available: http://www.advisorybodies.doh.gov.uk/ genetics/gtac. 2. L. K. Branski, C. T. Pereira, D. N. Herndon, and M. G. Jeschke, Gene therapy in wound healing: present status and future directions. Gene Ther. 2007; 14: 1–10.

10

GENE THERAPY

3. E. D. Milligan, E. M. Sloane, S. J. Langer, T. S. Hughes, B. M. Jekich, M. G. Frank, J. H. Mahoney, L. H. Levkoff, S. F. Maier, P. E. Cruz, T. R. Flotte, K. W. Johnson, M. M. Mahoney, R. A. Chavez, L. A. Leinwand, and L. R. Watkins, Repeated intrathecal injections of plasmid DNA encoding interleukin-10 produce prolonged reversal of neuropathic pain. Pain 2006; 126: 294–308. 4. C. H. Evans, P. D. Robbins, S. C. Ghivizzani, M. C. Wasko, M. M. Tomaino, R. Kang, T. A. Muzzonigro, M. Vogt, E. M. Elder, T. L. Whiteside, S. C. Watkins, and J. H. Herndon, Gene transfer to human joints: progress toward a gene therapy of arthritis. Proc. Natl. Acad. Sci. U.S.A. 2005; 102: 8698–8703. 5. R. A. Morgan, M. E. Dudley, J. R. Wunderlich, M. S. Hughes, J. C. Yang, R. M. Sherry, R. E. Royal, S. L. Topalian, U. S. Kammula, N. P. Restifo, Z. Zheng, A. Nahvi, C. R. de Vries, L. J. Rogers-Freezer, S. A. Mavroukakis, and S. A. Rosenberg, Cancer regression in patients after transfer of genetically engineered lymphocytes. Science 2006; 314: 126–129. 6. O. ter Brake, P. Konstantinova, M. Ceylan, and B. Berkhout, Silencing of HIV-1 with RNA interference: a multiple shRNA approach. Mol. Ther. 2006; 14: 883–892. 7. S. A. Rosenberg, P. Aebersold, K. Cornetta, A. Kasid, R. A. Morgan, R. Moen, E. M. Karson, M. T. Lotze, J. C. Yang, and S. L. Topalian, Gene transfer into humans-immunotherapy of patients with advanced melanoma, using tumor-infiltrating lymphocytes modified by retroviral gene transduction. N. Engl. J. Med. 1990; 323: 570–578. 8. M. L. Edelstein, M. R. Abedi, J. Wixon, and R. M. Edelstein, Gene therapy clinical trials worldwide 1989–2004-an overview. J. Gene Med. 2004; 6: 597–602. 9. Gene Therapy Clinical Trials Worldwide. The Journal of Gene Medicine. Available: http:// www.wiley.co.uk/genmed/clinical/2007. 10. M. Cavazzana-Calvo, S. Hacein-Bey, G. de Saint Basile, F. Gross, E. Yvon, P. Nusbaum, F. Selz, C. Hue, S. Certain, J. L. Casanova, P. Bousso, F. L. Deist, and A. Fischer, Gene therapy of human severe combined immunodeficiency (SCID)-X1 disease. Science 2000; 288: 669–672. 11. S. Hacein-Bey-Abina, F. Le Deist, F. Carlier, C. Bouneaud, C. Hue, J. P. de Villartay, A. J. Thrasher, N. Wulffraat, R. Sorensen, S. Dupuis-Girod, A. Fischer, E. G. Davies, W. Kuis, L. Leiva, and M. Cavazzana-Calvo, Sustained correction of X-linked severe combined

immunodeficiency by ex vivo gene therapy. N. Engl. J. Med. 2002; 346: 1185–1193. 12. M. G. Ott, M. Schmidt, K. Schwarzwaelder, S. Stein, U. Siler, U. Koehl, H. Glimm, K. Kuhlcke, A. Schilz, H. Kunkel, S. Naundorf, A. Brinkmann, A. Deichmann, M. Fischer, C. Ball, I. Pilz, C. Dunbar, Y. Du, N. A. Jenkins, N. G. Copeland, U. Luthi, M. Hassan, A. J. Thrasher, D. Hoelzer, C. von Kalle, R. Seger, and M. Grez, Correction of X-linked chronic granulomatous disease by gene therapy, augmented by insertional activation of MDS1-EVI1, PRDM16 or SETBP1. Nat. Med. 2006; 12: 401–409. 13. A. Aiuti, S. Vai, A. Mortellaro, G. Casorati, F. Ficara, G. Andolfi, G. Ferrari, A. Tabucchi, F. Carlucci, H. D. Ochs, L. D. Notarangelo, M. G. Roncarolo, and C. Bordignon, Immune reconstitution in ADA-SCID after PBL gene therapy and discontinuation of enzyme replacement. Nat. Med. 2002; 8: 423–425. 14. A. Aiuti, S. Slavin, M. Aker, F. Ficara, S. Deola, A. Mortellaro, S. Morecki, G. Andolfi, A. Tabucchi, F. Carlucci, E. Marinello, F. Cattaneo, S. Vai, P. Servida, R. Miniero, M. G. Roncarolo, and C. Bordignon, Correction of ADA-SCID by stem cell gene therapy combined with nonmyeloablative conditioning. Science 2002; 296: 2410–2413. 15. H. B. Gaspar, K. L. Parsley, S. Howe, D. King, K. C. Gilmour, J. Sinclair, G. Brouns, M. Schmidt, C. von Kalle, T. Barington, M. A. Jakobsen, H. O. Christensen, A. Al Ghonaium, H. N. White, J. L. Smith, R. J. Levinsky, R. R. Ali, C. Kinnon, A. J. Thrasher, Gene therapy of X-linked severe combined immunodeficiency by use of a pseudotyped gammaretroviral vector. Lancet 2004; 364: 2181–2187. 16. H. B. Gaspar, E. Bjorkegren, K. Parsley, K. C. Gilmour, D. King, J. Sinclair, F. Zhang, A. Giannakopoulos, S. Adams, L. D. Fairbanks, J. Gaspar, L. Henderson, J. H. Xu-Bayford, E. G. Davies, P. A. Veys, C. Kinnon, and A. J. Thrasher, Successful reconstitution of immunity in ADA-SCID by stem cell gene therapy following cessation of PEG-ADA and use of mild preconditioning. Mol. Ther. 2006; 14: 505–513. 17. N. Somia and I. M. Verma, Gene therapy: trials and tribulations. Nat. Rev. Genet. 2000; 1: 91–99. 18. S. D. Li and L. Huang, Gene therapy progress and prospects: non-viral gene therapy by systemic delivery. Gene Ther. 2006; 13: 1313–1319.

GENE THERAPY

11

19. D. J. Glover, H. J. Lipps, D. A. Jans, Towards safe, non-viral therapeutic gene expression in humans. Nat. Rev Genet. 2005; 6: 299–310.

32. A. K. Zaiss and D. A. Muruve, Immune responses to adeno-associated virus vectors. Curr. Gene Ther. 2005; 5: 323–331.

20. C. C. Conwell and L. Huang,. Recent advances in non-viral gene delivery. Adv. Genet. 2005; 53: 3–18.

33. D. B. Schowalter, L. Meuse, C. B. Wilson, P. S. Linsley, and M. A. Kay, Constitutive expression of murine CTLA4Ig from a recombinant adenovirus vector results in prolonged transgene expression. Gene Ther. 1997; 4: 853–860.

21. S. Mehier-Humbert and R. H. Guy, Physical methods for gene transfer: improving the kinetics of gene delivery into cells. Adv. Drug Deliv. Rev. 2005; 57: 733–753. 22. S. Simoes, A. Filipe, H. Faneca, M. Mano, N. Penacho, N. Duzgunes, and M. P. de Lima, Cationic liposomes for gene delivery. Expert. Opin. Drug Deliv. 2005; 2: 237–254. 23. C. Louise, Nonviral vectors. Methods Mol. Biol. 2006; 333: 201–226. 24. M. D. Lavigne and D. C. Gorecki, Emerging vectors and targeting methods for nonviral gene therapy. Expert. Opin. Emerg. Drugs 2006; 11: 541–557. 25. W. Walther and U. Stein, Viral vectors for gene transfer: a review of their use in the treatment of human diseases. Drugs 2000; 60: 249–271. 26. M. A. Kay, J. C. Glorioso, and L. Naldini, Viral vectors for gene therapy: the art of turning infectious agents into vehicles of therapeutics. Nat. Med. 2001; 7: 33–40. 27. I. M. Verma and M. D. Weitzman, Gene therapy: twenty-first century medicine. Annu. Rev Biochem. 2005; 74: 711–738. 28. S. Hacein-Bey-Abina, C. von Kalle, M. Schmidt, F. Le Deist, N. Wulffraat, E. McIntyre, I. Radford, J. L. Villeval, C. C. Fraser, M. Cavazzana-Calvo, and A. Fischer, A serious adverse event after successful gene therapy for X-linked severe combined immunodeficiency. N. Engl. J. Med. 2003; 348: 255–256. 29. N. Chirmule, K. Propert, S. Magosin, Y. Qian, R. Qian, and J. Wilson, Immune responses to adenovirus and adeno-associated virus in humans. Gene Ther. 1999; 6: 1574–1583.

34. E. Dobrzynski, J. C. Fitzgerald, O. Cao, F. Mingozzi, L. Wang, and R. W. Herzog, Prevention of cytotoxic T lymphocyte responses to factor IX-expressing hepatocytes by gene transfer-induced regulatory T cells. Proc. Natl. Acad. Sci. U.S.A. 2006; 103: 4592–4597. 35. B. D. Brown, M. A. Venneri, A. Zingale, S. L. Sergi, and SL, Naldini, L. Endogenous microRNA regulation suppresses transgene expression in hematopoietic lineages and enables stable gene transfer. Nat. Med. 2006; 12: 585–591. 36. F. G. Falkner and G. W. Holzer, Vaccinia viral/retroviral chimeric vectors. Curr. Gene Ther. 2004; 4: 417–426. 37. A. L. Epstein and R. Manservigi, Herpesvirus/retrovirus chimeric vectors. Curr. Gene Ther. 2004; 4: 409–416. 38. A. Oehmig, C. Fraefel, X. O. Breakefield, and M. Ackermann, Herpes simplex virus type 1 amplicons and their hybrid virus partners, EBV, AAV, and retrovirus. Curr. Gene Ther. 2004; 4: 385–408. 39. A. Recchia, L. Perani, D. Sartori, C. Olgiati, and F. Mavilio, Site-specific integration of functional transgenes into the human genome by adeno/AAV hybrid vectors. Mol. Ther. 2004; 10: 660–670. 40. H. Wang and A. Lieber, A helper-dependent capsid-modified adenovirus vector expressing adeno-associated virus rep78 mediates sitespecific integration of a 27-kilobase transgene cassette. J. Virol. 2006; 80: 11699–11709. 41. D. W. Russell and R. K. Hirata, Human gene targeting by viral vectors. Nat. Genet. 1998; 18: 325–330.

30. C. S. Manno, V. R. Arruda, G. F. Pierce, B. Glader, M. Ragni, J. Rasko, M. C. Ozelo, K. Hoots, P. Blatt, B. Konkle, M. Dake, R. Kaye, M. Razavi, A. Zajko, J. Zehnder, H. Nakai, A. Chew, D. Leonard, et al., Successful transduction of liver in hemophilia by AAV-Factor IX and limitations imposed by the host immune response. Nat. Med. 2006; 12: 342–347.

42. R. M. Kapsa, A. F. Quigley, J. Vadolas, K. Steeper, P. A. Ioannou, E. Byrne, and A. J. Kornberg, Targeted gene correction in the mdx mouse using short DNA fragments: towards application with bone marrow-derived cells for autologous remodeling of dystrophic muscle. Gene Ther. 2002; 9: 695–699.

31. E. K. Broberg and V. Hukkanen, Immune response to herpes simplex virus and gamma134.5 deleted HSV vectors. Curr. Gene Ther. 2005; 5: 523–530.

43. D. de Semir and J. M. Aran, Targeted gene repair: the ups and downs of a promising gene therapy approach. Curr. Gene Ther. 2006; 6: 481–504.

12

GENE THERAPY

44. G. McClorey, H. M. Moulton, P. L. Iversen, S. Fletcher, and S. D. Wilton, Antisense oligonucleotide-induced exon skipping restores dystrophin expression in vitro in a canine model of DMD. Gene Ther. 2006; 13: 1373–1381. 45. M. Schlee, V. Hornung, and G. Hartmann, siRNA and isRNA: two edges of one sword. Mol. Ther. 2006; 14: 463–470. 46. Y. Kaneda, Y. Saeki, and R. Morishita, Gene therapy using HVJ-liposomes: the best of both worlds? Mol. Med. Today 1999; 5: 298–303. 47. Y. Kaneda, Virosomes: evolution of the liposome as a targeted drug delivery system. Adv. Drug Deliv. Rev. 2000; 43: 197–205. 48. R. J. Yanez-Munoz, K. S. Balaggan, A. Macneil, S. J. Howe, M. Schmidt, A. J. Smith, P. Buch, R. E. Maclaren, P. N. Anderson, S. E. Barker, Y. Duran, C. Bartholomae, C. von Kalle, J. R. Heckenlively, C. Kinnon, R. R. Ali, and A. J. Thrasher, Effective gene therapy with nonintegrating lentiviral vectors. Nat. Med. 2006; 12: 348–353. 49. L. Naldini, U. Blomer, P. Gallay, D. Ory, R. Mulligan, F. H. Gage, I. M. Verma, and D. Trono, In vivo gene delivery and stable transduction of nondividing cells by a lentiviral vector. Science 1996; 272: 263–267. 50. L. Naldini, U. Blomer, F. H. Gage, D. Trono, and I. M. Verma, Efficient transfer, integration, and sustained long-term expression of the transgene in adult rat brains injected with a lentiviral vector. Proc. Natl. Acad. Sci. U.S.A. 1996; 93: 11382–11388. 51. M. A. Adam, N. Ramesh, A. D. Miller, and W. R. Osborne, Internal initiation of translation in retroviral vectors carrying picornavirus 5’ nontranslated regions. J. Virol. 1991; 65: 4985–4990. 52. P. E. Monahan and R. J. Samulski, AAV vectors: is clinical success on the horizon? Gene Ther. 2000; 7: 24–30. 53. M. Ali, N. R. Lemoine, and C. J. Ring, The use of DNA viruses as vectors for gene therapy. Gene Ther. 1994; 1: 367–384. 54. A. C. Nathwani and E. G. Tuddenham, Epidemiology of coagulation disorders. Baillieres Clin. Haematol. 1992; 5: 383–439. 55. D. J. Fink, L. R. Sternberg, P. C. Weber, M. Mata, W. F. Goins, and J. C. Glorioso, In vivo expression of beta-galactosidase in hippocampal neurons by HSV-mediated gene transfer. Hum. Gene Ther. 1992; 3: 11–19. 56. X. Xiao, J. Li, T. J. McCown, and R. J. Samulski, Gene transfer by adeno-associated virus

vectors into the central nervous system. Exp. Neurol. 1997; 144: 113–124. 57. U. Blomer, L. Naldini, T. Kafri, D. Trono, I. M. Verma, and F. H. Gage, Highly efficient and sustained gene transfer in adult neurons with a lentivirus vector. J. Virol. 1997; 71: 6641–6649. 58. J. C. Glorioso, N. A. DeLuca, and D. J. Fink, Development and application of herpes simplex virus vectors for human gene therapy. Annu. Rev. Microbiol. 1995; 49: 675–710. 59. W. T. Hermens and J. Verhaagen, Viral vectors, tools for gene transfer in the nervous system. Prog. Neurobiol. 1998; 55: 399–432. 60. J. Fleming, S. L. Ginn, R. P. Weinberger, T. N. Trahair, J. A. Smythe, and I. E. Alexander, Adeno-associated virus and lentivirus vectors mediate efficient and sustained transduction of cultured mouse and human dorsal root ganglia sensory neurons. Hum. Gene Ther. 2001; 12: 77–86. 61. R. O. Snyder, C. Miao, L. Meuse, J. Tubb, B. A. Donahue, H. F. Lin, D. W. Stafford, S. Patel, A. R. Thompson, T. Nichols, M. S. Read, D. A. Bellinger, K. M. Brinkhous, and M. A. Kay, Correction of hemophilia B in canine and murine models using recombinant adenoassociated viral vectors. Nat. Med. 1999; 5: 64–70. 62. G. M. Acland, G. D. Aguirre, J. Ray, Q. Zhang, T. S. Aleman, A. V. Cideciyan, S. E. PearceKelling, V. Anand, Y. Zeng, A. M. Maguire, S. G. Jacobson, W. W. Hauswirth, and J. Bennett, Gene therapy restores vision in a canine model of childhood blindness. Nat. Genet. 2001; 28: 92–95. 63. T. M. Daly, K. K. Ohlemiller, M. S. Roberts, C. A. Vogler, and M. S. Sands, Prevention of systemic clinical disease in MPS VII mice following AAV-mediated neonatal gene transfer. Gene Ther. 2001; 8: 1291–1298. 64. A. Bosch, E. Perret, N. Desmaris, D. Trono, and J. M. Heard, Reversal of pathology in the entire brain of mucopolysaccharidosis type VII mice after lentivirus-mediated gene transfer. Hum. Gene Ther. 2000; 11: 1139–1150. 65. R. Pawliuk, K. A. Westerman, M. E. Fabry, E. Payen, R. Tighe, E. E. Bouhassira, S. A. Acharya, J. Ellis, I. M. London, C. J. Eaves, R. K. Humphries, Y. Beuzard, R. L. Nagel, and P. Leboulch, Correction of sickle cell disease in transgenic mouse models by gene therapy. Science 2001; 294: 2368–2371. 66. T. H. Nguyen, M. Bellodi-Privato, D. Aubert, V. Pichard, A. Myara, D. Trono, and N. Ferry, Therapeutic lentivirus-mediated neonatal in

GENE THERAPY vivo gene therapy in hyperbilirubinemic Gunn rats. Mol. Ther. 2005; 12: 852–859. 67. Z. Li, J. Dullmann, B. Schiedlmeier, M. Schmidt, C. von Kalle, J. Meyer, M. Forster, C. Stocking, A. Wahlers, O. Frank, W. Ostertag, K. Kuhlcke, H. G. Eckert, B. Fehse, and C. Baum, Murine leukemia induced by retroviral gene marking. Science 2002; 296: 497. 68. Declaration of Helsinki. The World Medical Association. Available: http://www.wma. net/e/policy/b3.htm. 69. K. Cornetta and F. O. Smith, Regulatory issues for clinical gene therapy trials. Hum. Gene Ther. 2002; 13: 1143–1149. 70. S. M. Dainesi, Seeking harmonization and quality in clinical trials. Clinics 2006; 61: 3–8. 71. J. Spink and D. Geddes, Gene therapy progress and prospects: bringing gene therapy into medical practice: the evolution of international ethics and the regulatory environment. Gene Ther. 2004; 11: 1611–1616. 72. D. B. Resnik and P. J. Langer, Human germline gene therapy reconsidered. Hum. Gene Ther. 2001; 12: 1449–1458. 73. M. Fuchs, Gene therapy. An ethical profile of a new medical territory. J. Gene Med. 2006; 8: 1358–1362. 74. C. Antoine, S. Muller, A. Cant, M. CavazzanaCalvo, P. Veys, J. Vossen, A. Fasth, C. Heilmann, N. Wulffraat, R. Seger, S. Blanche, W. Friedrich, M. Abinun, G. Davies, R. Bredius, A. Schulz, P. Landais, and A. Fischer, Long-term survival and transplantation of haemopoietic stem cells for immunodeficiencies: report of the European experience 1968–99. Lancet 2003; 361: 553–560. 75. M. Cavazzana-Calvo, A. Thrasher, and F. Mavilio, The future of gene therapy. Nature 2004; 427: 779–781. 76. A. J. Thrasher, S. Hacein-Bey-Abina, H. B. Gaspar, S. Blanche, E. G. Davies, K. Parsley, K. Gilmour, D. King, S. Howe, J. Sinclair, C. Hue, F. Carlier, C. von Kalle, B. G. de Saint, F. Le Deist, A. Fischer, and M. CavazzanaCalvo, Failure of SCID-X1 gene therapy in older patients. Blood 2005; 105: 4255–4257.

13

79. Structural characterization of gene transfer vectors. Federal Drug Authority. Available: http://www.fda.gov/OHRMS/DOCKETS/ ac/00/backgrd/3664b1a.doc. 80. M. E. Dudley, J. R. Wunderlich, P. F. Robbins, J. C. Yang, P. Hwu, D. J. Schwartzentruber, S. L. Topalian, R. Sherry, N.P. Restifo, A. M. Hubicki, M. R. Robinson, M. Raffeld, P. Duray, C. A. Seipp, L. Rogers-Freezer, K. E. Morton, S. A. Mavroukakis, D. E. White, and S. A. Rosenberg, Cancer regression and autoimmunity in patients after clonal repopulation with antitumor lymphocytes. Science 2002; 298: 850–854. 81. R. M. Blaese, K. W. Culver, A. D. Miller, C. S. Carter, T. Fleisher, M. Clerici, G. Shearer, L. Chang, Y. Chiang, P. Tolstoshev, et al. T lymphocyte-directed gene therapy for ADASCID: initial trial results after 4 years. Science 1995; 270: 475–480. 82. S. H. Orkin and A. G. Motulsky, Report and recommendations of the panel to assess the NIH investment in research on gene therapy. National Institutes of Health. Available: http://www.nih.gov/news/panelrep.html. 83. S. E. Raper, N. Chirmule, F. S. Lee, N. A. Wivel, A. Bagg, G. P. Gao, J. M. Wilson, and M. L. Batshaw, Fatal systemic inflammatory response syndrome in a ornithine transcarbamylase deficient patient following adenoviral gene transfer. Mol. Genet. Metab. 2003; 80: 148–158. 84. J. Savulescu, Harm, ethics committees and the gene therapy death. J. Med. Ethics 2001; 27: 148–150. 85. B. L. Levine, L. M. Humeau, J. Boyer, R. R. Macgregor, T. Rebello, X. Lu, G. K. Binder, V. Slepushkin, F. Lemiale, J. R. Mascola, F. D. Bushman, B. Dropulic, and C. H. June, Gene transfer in humans using a conditionally replicating lentiviral vector. Proc. Natl. Acad. Sci. U.S.A. 2006; 103: 17372–17377. 86. X. Wu, Y. Li, B. Crise, and S. M. Burgess, Transcription start regions in the human genome are favored targets for MLV integration. Science 2003; 300: 1749–1751.

77. A/ Jaffe, S. A. Prasad, V. Larcher, and S. Hart, Gene therapy for children with cystic fibrosis-who has the right to choose? J. Med. Ethics 2006; 32: 361–364.

87. R. S. Mitchell, B. F. Beitzel, A. R. Schroder, P. Shinn, H. Chen, C. C. Berry, J. R. Ecker, and F. D. Bushman, Retroviral DNA integration: ASLV, HIV, and MLV show distinct target site preferences. PLoS. Biol. 2004; 2: E234.

78. K. Nyberg, B. J. Carter, T. Chen, C. Dunbar, T. R. Flotte, S/ Rose, D. Rosenblum, S. L. Simek, and C. Wilson, Workshop on long-term followup of participants in human gene transfer research. Mol. Ther. 2004; 10: 976–980.

88. P. Hematti, B. K. Hong, C. Ferguson, R. Adler, H. Hanawa, S. Sellers, I. E. Holt, C. E. Eckfeldt, Y. Sharma, M. Schmidt, C. von Kalle, D. A. Persons, E. M. Billings, C. M. Verfaillie, A. W. Nienhuis, T. G. Wolfsberg, C. E. Dunbar,

14

89.

90.

91.

92.

93.

GENE THERAPY and B. Calmels, Distinct genomic integration of MLV and SIV vectors in primate hematopoietic stem and progenitor cells. PLoS. Biol. 2004; 2:E423. Z. Peng, Current status of gendicine in China: recombinant human Ad-p53 agent for treatment of cancers. Hum. Gene Ther. 2005; 16: 1016–1027. H. Jia, Controversial Chinese gene-therapy drug entering unfamiliar territory. Nat. Rev Drug Discov. 2006; 5: 269–270. T. Neff, B. C. Beard, and H. P. Kiem, Survival of the fittest: in vivo selection and stem cell gene therapy. Blood 2006; 107: 1751–1760. T. Neff, B. C. Beard, L. J. Peterson, P. Anandakumar, J. Thompson, and H. P. Kiem, Polyclonal chemoprotection against temozolomide in a large-animal model of drug resistance gene therapy. Blood 2005; 105: 997–1002. T. Neff, P. A. Horn, L. J. Peterson, B. M. Thomasson, J. Thompson, D. A. Williams, M. Schmidt, G. E. Georges, C. von Kalle, and H. P. Kiem, Methylguanine methyltransferasemediated in vivo selection and chemoprotection of allogeneic stem cells in a large-animal model. J. Clin. Invest. 2003; 112: 1581–1588.

94. Stocking, C, Bergholz, U, Friel, J, Klingler, K, Wagener, T, Starke, C, Kitamura, T, Miyajima, A, Ostertag, W. Distinct classes of factor-independent mutants can be isolated after retroviral mutagenesis of a human myeloid stem cell line. Growth Factors 1993; 8: 197–209.

FURTHER READING J. A. Wolff and J. Lederberg, An early history of gene transfer and therapy. Hum. Gene Ther. 1994; 5: 469–480.

CROSS-REFERENCES Risk-benefit analysis, safety, translation

GENETIC ASSOCIATION ANALYSIS

can also perform Fisher’s exact test, which is more accurate but more time-consuming. In many situations, the effects of SNP alleles are roughly additive; that is, the probability of response for the heterozygote (i.e., genotype aA) is intermediate between those of the two homozygotes (i.e., genotypes aa and AA). Then it is desirable to use the Armitage (1) trend test statistic:

D. Y. LIN Department of Biostatistics, University of North Carolina, Chapel Hill, North Carolina

In genetic association analysis, we assess the association between a genetic marker and a response variable. The association can originate in two ways: (1) The genetic marker is causally related to the response variable; and (2) the genetic marker is correlated with a causal variant. The latter might suffice prediction/classification purposes, whereas the former would be necessary for identifying potential drug targets. The most commonly used genetic markers are single nucleotide polymorphisms or SNPs. Virtually all SNPs are biallelic (i.e., having only two possible nucleotides or alleles). Thus, three possible genotypes exist at each SNP site: homozygous with allele A, homozygous with allele a, or heterozygous with one allele A and one allele a, where A and a denote the two possible alleles. In the case of a binary response variable with a total of n individuals, the data are represented in a 3 × 2 table:

nn.1 n.2 (n21 /2n.1 + n31 /n.1 − n22 /2n.2 − n32 /n.2 )2 n(n2. /2 + n3. ) − (n2. /2 + n3. )2 (2)

which has one degree of freedom. This test will be more powerful than Equation (1) if the genetic effects are indeed additive. We can also tailor our analysis to other genetic models. Under the dominant and recessive modes of inheritance, the test statistics become

Yes

No

Total

aa aA AA Total

n11 n21 n31 n.1

n12 n22 n32 n.2

n1. n2. n3. n

i=1 j=1

(nij − eij )2 , eij

n{n32 (n11 + n21 ) − n31 (n12 + n22 )}2 (n1. + n2. )n3. n.1 n.2

(4)

respectively. Again, Fisher’s exact tests can also be used. We measure the strength of association by the odds ratio or the difference of response rates. The choice of the test statistic should ideally be driven by the model of inheritance. Unfortunately, this knowledge is rarely available. The most common practice is to use the Armitage trend test, which should perform well unless the effects are far from additive. Test statistics (2)-(4) can be obtained as the score statistics under the logistic regression model:

To test the independence between the genotype and the response, we can use the well-known Pearson’s chi-squared statistic 3
2


(3)

and

Response Genotype

n{n11 (n22 + n32 ) − n12 (n21 + n31 )}2 n1. (n2. + n3. )n.1 n.2

logit{Pr(Y = 1|X)} = α + βX,

(5)

where Y is the response variable, X is the genotype score, α is the intercept, and β is the log odds ratio. Under the additive mode of inheritance, X denotes the number of A alleles; under the dominant model, X is the indicator for at least one A allele; and under the recessive model, X is the indicator for

(1)

where eij = ni. n.j /n. The null distribution of this test statistic is asymptotically chisquared with two degrees of freedom. We

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

GENETIC ASSOCIATION ANALYSIS

genotype AA. Inference on the odds ratio (i.e., eβ ) is based on the maximum likelihood theory. Test statistic (1) can also be generated under model (5). An important advantage of the logistic modeling is that it can readily accommodate multiple SNPs, environmental factors (e.g., treatment assignments), and gene-environment interactions. We adopt the general form of the logistic regression model: logit{Pr(Y = 1|X)} = α + β
X where X consists of genotype scores of (possibly) multiple SNPs, environmental factors, and products of genotype scores and environmental factors, and β is a vector of log odds ratios. In particular, we can assess the interactions between treatment assignments and genetic markers under this model. If the response variable is continuous, then we replace the logistic regression model with the familiar linear regression model. Indeed, we can use any generalized linear model (2). The only difference from the traditional regression analysis lies in the incorporation of appropriate genotype scores. If the response variable is survival time or event time, then we employ the proportional hazards model (3). All the analysis can be carried out in standard statistical software. It can be problematic to include many SNPs, some of which are highly correlated, in the regression analysis. An alternative approach is to consider haplotypes. A haplotype contains alleles from one parent. Association analysis based on haplotypes can reduce the degrees of freedom and increase the power to capture the combined effects of multiple causal variants, as compared with SNPsbased analysis. The current genotyping technologies do not separate the two homologous chromosomes of an individual, so haplotypes are not directly observable. Maximum likelihood methods have been developed to perform haplotype analysis based on genotype data (4). Haplotypes can also be used to infer the alleles of an untyped SNP (i.e., an SNP that is not on the genotyping chip) and thus allows analysis of association between an untyped SNP and a response variable (5). A potential drawback of genetic association analysis is that spurious associations

may result from unknown population structure or stratification. This problem originates when a genetic subgroup has a higher response rate than another, so that any SNP with allele proportions that differ among the subgroups will appear to be associated with the response variable. Several methods have been proposed to deal with population stratification. The most popular current approach is to infer axes of genetic variation from genomic markers through principal components analysis and then include those axes as covariates in the association analysis (6). Multiple testing is a serious issue in genetic association analysis, especially if many markers are examined. The common practice is to use the Bonferroni correction, which essentially divides the overall significance level by the number of tests performed. This strategy is conservative, especially when the markers are highly correlated. Accurate control of the type I error can be achieved by incorporating the correlations of the test statistics into the adjustments of multiple testing (7). The description thus far has been focused on the frequentist paradigm. There is an increasing interest in using Bayesian methods in genetic association analysis (8). REFERENCES 1. P. Armitage, Test for linear trend in proportions and frequencies. Biometrics 1955; 11:375–386. 2. P. McCullagh and J. A. Nelder, Generalized Linear Models, 2nd ed. New York: Chapman & Hall, 1989. 3. D. R. Cox. Regression models and life-tables (with discussion). J. R. Stat. Soc. Ser. B 1972; 34:187–220. 4. D. Y. Lin and D. Zeng. Likelihood-based inference on haplotype effects in genetic association studies (with discussion). J. Am. Stat. Assoc. 2006; 101:89–118. 5. D. Y. Lin, Y. Hu, and B. E. Huang. Simple and efficient analysis of SNP-disease association with missing genotype data. Am J. Hum. Genet. In press. 6. A. L. Price, N. J. Patterson, R. M. Plenge, M. E. Weinblatt, N. A. Shadick, and D. Reich. Principal components analysis corrects for stratification in genome-wide association studies. Nat. Genet. 2006; 38:904–909.

GENETIC ASSOCIATION ANALYSIS 7. D. Y. Lin. An efficient Monte Carlo approach to assessing statistical significance in genomic studies. Bioinformatics 2005; 21:781–787. 8. J. Marchini, B. Howie, S. Myers, G. McVean, and P. Donnelly. A new multipoint method for genome-wide association studies by imputation of genotypes. Nat. Genet. 2007; 39:906–913.

3

GLOBAL ASSESSMENT VARIABLES

treatment benefit (or harm)?’’ with all outcomes expected to have an effect in the same direction. To answer the first question, separate tests for each single outcome can be performed. Methods for these comparisons are discussed later in this section in comparison to global assessment. To address the directional hypothesis requires either a composite outcome or a global assessment of treatment benefit. Composite outcomes (see Composite Outcomes) refer to the situation where multiple outcomes are combined using a scoring system, which is defined a priori and considered to be clinically meaningful. For the situation where there is no such scoring system, O’Brien (2) as well as Pocock et al. (3) introduced the novel global statistical test (GST) methodology to combine information across outcomes and examine whether a treatment has a global benefit. GSTs have optimal statistical power when treatment effect is of similar magnitude across all outcomes (common dose effect). Because these methods can test whether a treatment has a global benefit without the need of multiple tests, the GST approach has been widely applied in medical studies (4–8), and the extensions of the GST method have appeared in the literature. Below we review different versions of the GST and describe where each GST can be appropriately used.

BARBARA TILLEY Medical University of South Carolina, Charleston, South Carolina

PENG HUANG John Hopkins University

PETER C. O’BRIEN Mayo Clinic

Although most clinical trials are designed using a single primary outcome, in some trials it is difficult to find the single most appropriate outcome for the main objective of the study. In the NINDS Stroke Trial, a consistent and persuasive poststroke improvement on multiple outcomes was required to define treatment efficacy (1). In Parkinson’s disease clinical trials, a treatment would not be considered to slow progression if the treatment improved only motor score and other measures of outcome deteriorated. In studying quality of life, we may be interested in treatment effects on all subscales being measured. In all cases, no single measure is sufficient to describe the treatment effect of interest, and no validated summary score exists. In a quality-of-life example, we could measure an overall quality-of-life score by summing across subscales, but we may under or overestimate the effect of treatment depending on how the treatment affects the individual subscales.

2 GENERAL COMMENTS ON THE GST 2.1 Definition of a GTE

1 SCIENTIFIC QUESTIONS FOR MULTIPLE OUTCOMES

Many GSTs can be described through two quantities. The first we call the global treatment effect (GTE), and it measures a treatment’s overall benefit across multiple outcomes. The concept of GTE was first introduced by Huang et al. (9); they defined it as a scale-free quantity. However, treatment’s overall benefit for many GSTs is defined through some scale-dependent quantity. We continue to call such a quantity a GTE with the understanding that it may not be exactly the same as that defined by Huang et al. (9) The other quantity is the statistic that tests the significance of this GTE. For those GSTs without GTE defined, the interpretation of

The choice of statistical method used for assessing treatment effect on multiple outcomes depends on the scientific question under investigation. Two types of questions lead to two different classes of tests. The first type is a directionless question: ‘‘Is there any difference, helpful or harmful between the two treatments in these outcomes?’’ A treatment difference could be detected if both strong positive and negative effects were observed. The other approach addressed a directional question: ‘‘Is there any global

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

GLOBAL ASSESSMENT VARIABLES

GST is more difficult as is the qualification of treatment benefit. The GTE from Huang et al. (9) is defined as the difference of two probabilities: the probability that a control group patient will do better if the patient is switched to the treatment, and the probability that a treatment patient will do better if the patient is switched to the control. Larger positive GTE value corresponds to stronger evidence of treatment benefit. Huang et al. (10) gave an unbiased estimate of the GTE for a ranksum GST. Detailed discussion of GTE was given in Huang et al. (9). A major advantage of using a GTE is that it can easily combine a treatment’s effects across different outcomes, no matter which scales are used and whether the outcomes have skewed distributions. Such advantage is not achieved by GTEs from parametric GSTs. A good discussion of advantages of using such a scale-free measure is given by Acion et al. (11). Based on the rank-sum GST, suppose the GTE = 0.4 or 40%. This value implies that a patient in the control group would have a [(70%−P/2) = (100% + 40%−P)/2] probability of a better overall outcome if he/she had been assigned to the treatment group, where P is the probability that the two groups being compared, have tied observations. If no ties are observed, then the probability would be equal to 70%. As the types of GSTs are described, the GTEs will be indicated or we will indicate that the GTE cannot be computed. 3

RECODING OUTCOME MEASURES

Because a global assessment of outcome variables is sensitive to the direction of the treatment effect on each of the variables, it is important to reverse code any variables in which the treatment effect would be measured in the opposite direction. For example, one can multiply variables by (− 1) if smaller observations indicate better outcomes for some but not all variables. In other words, all variables are coded in the way such that larger observations are preferred for all variables before applying a GST. Hereafter, we assume that all variables are coded in the way that larger value is preferred, although

the procedure could easily be applied when the all variables are coded such that the smaller value is preferred. 3.1 Assumptions Most GSTs require a common dose assumption, which implies that the treatment has a similar effect on all outcomes. The parametric GSTs must also meet the distributional assumptions of the test.

4 TYPES OF GLOBAL STATISTICAL TESTS (GSTS) 4.1 Rank-Sum-Type GST O’Brien (2) proposed a simple nonparametric GST for multiple outcomes using ranks. First, for each outcome, all patients are ranked according to their values on this outcome. Second, for each patient, the patient’s ranks are summed to obtain a rank score. Finally, a t-test (or a one-way ANOVA if more than 2 groups are involved) is applied to compare the rank scores between the treatment and control group. More generally, any univariate statistical procedure for comparing two samples may be applied to the rank sum scores. The GTE of this test is a rescaled probability that a control group patient will do better if the patient switches to the treatment, as defined by Huang et al. (9). Because a probability takes value between 0 and 1, they linearly rescaled the probability by minus 0.5 and then multiply by 2 to a value between −1 and +1 so that GTE = 0 implies no overall treatment benefit, a positive GTE value implies treatment benefit, and a negative GTE value implies treatment detrimental. The rank-sum-type GST is very simple to carry out. It is flexible in its ability to analyze multiple outcomes without the need of parametric assumptions regarding the distributions of the outcomes and the correlation among outcomes. The test is invariant to any monotone transformation of the data and is robust to outliers. Thus, rank-sum-type GST is applicable to broad medical research settings. Huang et al. (9) provided sample size computation formulas and corresponding Splus program code when rank-sum-type GST is used as the primary analysis.

GLOBAL ASSESSMENT VARIABLES

4.2 Adjusted Rank-Sum-Type GST Huang et al. (9) extended O’Brien’s ranksum-type GST to the case where variances in two treatment groups are different. The adjusted GST is computed similar to O’Brien’s unadjusted rank-sum GST, but it is divided by an adjusting factor. As for O’Brien’s rank sum test, the GTE is defined as the difference of two probabilities: the probability that a control group patient will do better if the patient is switched to the treatment, and the probability that a treatment patient will do better if the patient is switched to control. If no tied observations are recorded, then this GTE measures the probability that a patient will do better if he/she switched to the new treatment. Because this adjusted GST does not require the two treatment groups to have the same distribution under both the null and the alternative hypotheses, it can be applied to the Behrens-Fisher’s problem. The sample size formulas given by Huang et al. (9) can be applied to this adjusted rank-sum-type test. 4.3 Ordinary Least Square (OLS)-Based GST and Generalized Least Square (GLS)-Based GST These two tests were designed to have optimal power when the common-dose assumption is met (2, 3). The test statistics for OLS-based GST uses the standardized arithmetic mean (with equal weights) of all outcomes, and it is constructed based on the ordinary least square estimate of the GTE. The common standardized mean difference is the GTE. The test statistic for GLS-based GST uses a weighted average of all outcomes with weights depending on the correlation among the outcomes, and it is constructed based on the generalized least square estimate of GTE. These two GSTs are most appropriate for cases in which the treatment effect is measured by the change in outcome means. When the common-dose assumption holds, the GLSbased GST has higher power than other tests with the same type I error rate. Its limitation is that when some weights are negative, the test will be difficult to interpret. It will be difficult to determine whether a larger or a smaller test statistic is preferred or indicates treatment benefit. The negative weights can develop when the outcomes are diverse and

3

do not all have the same correlation with each other. Because of this problem with negative weights, several authors recommend the use of unweighted ordinary least squares GST (3, 11, 12). 4.4 Likelihood Ratio (LR)-Based GST for Normal Outcomes Recognizing that the common-dose assumption may not hold in practice and the GLS may lose power in this case, Tang et al. (12) studied the GLS for normally distributed outcomes assuming that treatment effects on all outcomes are in the same direction (i.e., all improved or all worsened), but the magnitudes of changes are a multiplier of (or equivalent, proportional to) a set of prespecified and possibly different values (13). The GTE of the LR-based GST is this common multiplier. The test statistic for LR-based GST is constructed in a similar manner as the GLS-based GST, but the outcome weights are determined by the prespecified magnitudes of change and the correlations among the outcomes. This GST is also designed for normally distributed outcomes. Tang et al. (14) illustrated how to apply the LR based GST in interim analysis and clinical trial design. Like GLS-based GST, the LR-based GST has a good power when the prespecified magnitudes of changes are true for the data, but in practice, these weights are not always known. 4.5 Approximate Likelihood Ratio (ALR)-Based GST for Normal Outcomes Although the LR-based GST has good power when true magnitudes of treatment effects on all outcomes are proportional to some prespecified values, its power decreases rapidly as the true treatment effects deviate from the prespecified values. Because it would be difficult for investigators to know the true treatment effects in advance, Tang et al. (14) proposed an ALR-based GST that maintains a good power in all possible magnitudes of treatment effects (all positive values). This GST is also designed for normally distributed outcomes. Its GTE is not easy to define. The test statistic is more complicated to compute than the previous GSTs. Simulations showed that ALR test is more powerful than

4

GLOBAL ASSESSMENT VARIABLES

O’Brien’s GLS-based GST when treatment effects are larger on some outcomes than on others. However, the authors noticed that the ALR test ignores differences going in the wrong direction, so caution was needed when using one-sided multivariate testing. 4.6 GST Using Scaled Linear Mixed Model for Continuous Outcomes When all outcomes are normally distributed and are measuring the same underlying event, Lin et al. (15) proposed a scaled linear mixed model to estimate treatment’s common dose effect. The method is flexible in studying multiple outcomes measured in different scales. Another advantage of this method is that it can adjust for confounding covariates and can be implemented by repeatedly fitting standard mixed models (See Mixed Models). 4.7 Follmann’s GST for Normal Outcomes Follmann proposed a simple test that has a good power when the sum of treatment effects on all outcomes is positive (17). Follmann’s GST uses Hotelling’s T2 test statistic to make a decision. It rejects the null hypothesis when T2 exceeds its 2α critical value and the sum of treatment differences on all outcomes is positive. The test controls the type I error α whether the covariance matrix of the outcomes is known. Under the common dose assumption, Follmann’s test is less powerful than O’Brien’s GLS-based GST. However, when the common dose assumption is violated, Follmann’s test could have a higher power than O’Brien’s GST and should avoid problems in interpretation if the correlation structure leads to negative weights for the GST. 4.8 GST for Binary Outcomes Lefkopoulou et al. (18) applied the quasilikelihood methods to model multiple binary outcomes. Later they extended these methods to incorporate analyses of clusters of individuals and they derived a scores test for multiple binary outcomes (19). They showed the scores test to be a special case of Pocock et al. (3). The common relative risk is the GTE measure of the GST, between different groups across all outcomes and represents

the assumed common-dose effect. The relative risk can be derived from the common odds ratio using the methods of Lu and Tilley (20). Binary GSTs without the common dose assumption were also compared using GEE by Lefkopoulou et al. (18) and Legler et al. (21). As the number of unaffected outcomes included in the GST increases, the global treatment effect is diluted, and the statistical power of the GST decreases quickly. 4.9 GST for Binary and Survival Outcomes Pocock et al. (3) gave a computational formula for GST when outcomes consist of binary outcomes and a survival outcome. First, a test statistic is constructed for each single outcome. Then, a multivariate test statistic is constructed based on these univariate test statistics and covariances among them. Rejection of the null hypothesis implies that treatment difference is observed on at least one outcome. Because this test does not assess whether a treatment has a global benefit, no GTE can be defined for the test. Also, because a global benefit is not computed, it could be considered a composite test (See Composite Endpoints). Simpler methods for accommodating binary time to event endpoints might also be considered. Specifically, one could use the actual binary values and the rank sum scores associated with each type of endpoint, respectively. 4.10 GST Using Bootstrap Bloch et al. (22) defined the treatment benefit as an improvement on at least one outcome and non-inferior on all outcomes. For example, if we have two outcomes, 1 is the mean treatment improvement on the first outcome, and 2 is the mean treatment improvement on the second outcome. They proposed to prespecify two positive numbers δ 1 and δ 2 such that treatment benefit is defined by the satisfaction of the following three conditions simultaneously: max {1 , 2 } > 0 and 1 > − δ1 and 2 > − δ2 This method is very similar to the approach proposed earlier by Jennison and Turnbull

GLOBAL ASSESSMENT VARIABLES

(23) who used an L-shaped subset in parameter space to define treatment benefit. The test statistic T is similar to the Hotelling’s T2 test statistic, but T is positive only when estimated 1 and 2 values are greater than some critical values u1 and u2 , respectively, and T = 0 otherwise. The null hypothesis of no treatment benefit is rejected when T > c for some critical value c. The critical values u1 , u2 , and c need to be determined to control the type I error rate. Because it is difficult to derive the null distribution of T analytically, the authors use bootstrap methods to estimate u1 , u2 , and c. Their simulations show that proposed method controls type I error rate and has good power for both normal and non-normal data, such as a mixture of normal distributions and a mixture of exponential distributions. Again, GTE as a global treatment effect is not computed and again, this method could be considered more like a composite test (See Composite Endpoints). 4.11 Extension of GST in Group Sequential Trial Design When all observations are accrued independently, all GSTs discussed above can be extended to group sequential design settings. Tang et al. (14) discussed how to apply likelihood ratio based GST for normal outcomes in group-sequential trials. A more general theory is given by Jennison and Turnbull (24) when the joint distribution of multiple outcomes is known except for a few parameters. The key assumption in their methods is that the additional data used in the next interim analysis must be independent to the data used in the previous interim analyses. For example, if the first interim analysis uses data from n1 subjects, the second interim analysis uses data from original n1 subjects plus n2 additional subjects. Observations from these n2 subjects are the additional data in the second interim analysis, and they must be independent to the data used in the first interim analysis. If data are not accrued independently, such as the case when patients are measured repeatedly and the data from the same patients are used in different interim analyses, then the computation of the critical values for the

5

stopping rule will be much more complicated. When data are not normally distributed but the normal theory is used to determine the critical values for the test statistics in different interim analyses, both the sample size at each interim analysis and the sample size difference between two consecutive interim analyses need to be large to make the Central Limit Theory applicable. 5 OTHER CONSIDERATIONS 5.1 Power of a GST A GST generally provides a univariate test statistic to describe the overall benefit (GTE). Because data from multiple outcomes are used to assess the GTE, it generally has a higher power than a univariate test (3,14). More importantly, a GST helps us to make a decision regarding whether a treatment is preferred using single test, rather using than multiple tests on different outcomes with the associated penalty for multiple testing. Where some outcomes strongly suggest benefit and others strongly suggest harm, power would be reduced, and we would be unlikely to reject the null hypothesis using a GST because we are testing the benefit of treatment. In the latter case, Hotellings T2 would be likely to reject the null hypothesis (See Multiple Comparisons). Because the power of the GST is generally greater than or equal to the power of the corresponding univariate test, using any single outcome one can use the smallest sample size calculated from the corresponding univariate tests as the sample size for the GST. However, this result may lead to a larger sample size than needed. Rochon (25) applied generalized estimating equation (GEE) approach to develop a sample size computation method for univariate outcome measured longitudinally. His method can be used for a cross-sectional study with multiple outcomes. Rochon’s approach is established under assumptions that variance–covariance matrix of outcomes is a function of outcome means and observations from all subjects in the same subgroup have exact the same covariates, mean values, and variance–covariance matrix. A sample size computational algorithm is provided along with a table of minimum required sample sizes for binary outcomes.

6

GLOBAL ASSESSMENT VARIABLES

5.2 Interpreting the GST The interpretation of GST depends on the variables included in the GST and the type of GST used. When it can be computed, the GTE measures a treatment’s overall benefit across multiple outcomes. For those GSTs without GTE defined, the interpretation of GST is more difficult. Estimation of GTE and its confidence interval may often provide more information about treatment effect than the binary decision regarding to whether the null hypothesis is rejected. Framing the hypothesis in terms of the GTE is helpful in sample size estimation and the design of clinical trials. For normal outcomes, if a null hypothesis is rejected by the OLS-based GST or by the GLS-based GST, then it is implied that, for all outcomes included in the GST, the common mean difference between the two groups, which is its GTE measure, is claimed to be significantly different from zero. For LR-based GST, a rejection of the null hypothesis implies that treatment improves variable means by the same factor, which is measured by the GTE. Because a GTE is not defined for ALR test, it is not easy to quantify treatment effect when the null hypothesis is rejected. What we can say about ALR test is that treatment improves all outcomes, but the amount of improvement for each outcome is not provided. For binary outcomes, because the GTE is defined as the common odds ratio of the success probabilities between the two groups, a rejection of the null hypothesis by the 1-degree-of-freedom GST implies that treatment improves success probabilities by the same factor whose value equals to the exponential of the GTE. For both rank-sum-type GST and adjusted rank-sum-type GST, the interpretation of treatment effect is similar. Rejection of a null hypothesis by these tests implies that treatment group patients have a greater chance of better overall outcome than the control group patients do. It is important to note that GST is designed to test treatment’s global effect on multiple outcomes. It does not provide separated assessment of the treatment effect on single outcomes. If an investigator is interested to

test the treatment effect for each of the single outcomes after the global null hypothesis is rejected, several sequential procedures are available to do so 26–29. These procedures listed do not allow repeated tests on the same outcome such as we do in the interim analysis. Kosorok et al. (30) proposed a group sequential design in which each single outcome is tested repeatedly in the interim analyses by partitioning the alternative region into mutually exclusive rectangular subregions, each corresponding to a combination of decisions based on all of the single outcomes. The decisions on each outcome can be ‘‘the treatment is better,’’ ‘‘the treatment is worse,’’ or ‘‘undetermined’’. Thus, a total of 3K − 1 subregions exist. The global type I error is defined as the probability that the test statistic incorrectly rejects the global null hypothesis of no treatment effect on any of the outcomes. The type II error is defined as the probability that the test statistic incorrectly leads to a conclusion that is inconsistent with the true distribution. For each outcome, stopping boundaries are constructed by spending its type I and type II errors separately, then combined to form multivariate stopping boundaries. To preserve the overall type I and type II errors, the boundaries are multiplied by a constant c that is determined through simulation. The advantage of using this method in clinical trials is that when a trial is stopped in an interim analysis, investigators can determine which outcomes are improved, which are worsened, and which are unchanged without inflation of the type I error. The limitation is its intensive computation. The authors provide software for this computation under the link http://www.bios.unc.edu/∼kosorok/clinicaltrial/main.html. 5.3 Choosing Outcome Variables to Include in the GST All variables included in the GST must be related to the scientific question to be addressed and should all be expected to respond to treatment. If the common dose assumption is required, then the treatment is expected to have a similar benefit on all outcomes. If several correlated variables are considered,

GLOBAL ASSESSMENT VARIABLES

whether one of them, some combinations of them, or all of them should be included into a GST depends on how we want to express treatment’s effect on these variables. Adding redundant variables into GST can bias the conclusion of GST. Simulations of Khedouri have shown that inclusion of highly correlated redundant variables artificially affects the power of different GSTs, which includes the OLS/GLS based GSTs, the GEE score equation derived GST, and rank-sum-type GST (31). 6

OTHER METHODS

The main rationale for using a global outcome in a clinical trial is that, by providing a more comprehensive and thus more informative assessment of the patient, it also provides a more clinically meaningful primary endpoint than use of any single variable by itself. Two other approaches for using multiple endpoints in the primary analysis of a clinical trial are the T2 statistic and use of multiple comparison procedures (See Multiple Comparisons). 6.1 T2 Statistic The T2 statistic computes the mean difference between groups for each of the endpoints, and then computes the distance between these two sets of means relative to a multivariate measure of the variability in the data that accounts for the correlation structure among the individual variables. Importantly, it does not take into account whether individual group differences were in a positive or negative direction. Thus, by itself the statistic is not meaningful. Furthermore, the failure to use this information about the derivation of the differences results in very low power for identifying efficacy. For these reasons, it is rarely used as a primary analysis in clinical trials. 6.2 Bonferroni Multiple comparison procedures are commonly used, especially Bonferroni adjustment of individual P-values. Specifically, the two treatment groups are compared in the usual way for each variable, and then the

7

several P-values are adjusted by multiplying them by the number of tests carried out. Statistical significance is claimed only if the smallest of these adjusted P-values is less than the prespecified threshold (typically .05). For example, if five primary endpoints are observed, then the smallest unadjusted Pvalue must be less than .01 to achieve overall significance at the 0.05 level. An argument sometimes made in favor of the Bonferroni approach is that interpretation is more straightforward and less ambiguous than using a global outcome, because when overall statistical significance is achieved, the endpoint that produced the smallest individual P-value can be reliably identified as an endpoint that is improved by the experimental treatment. Although this decreased ambiguity is an advantage, closer inspection of the Bonferroni approach reveals that interpretation may actually be more problematic than with a global outcome. To illustrate the problem with the Bonferroni method, imagine two primary endpoints, and suppose that the evidence for a treatment effect was identical for each variable with univariate P-values of .01, except that the direction of the treatment effect favored the experimental treatment for one endpoint and the placebo treatment for the other. Using the Bonferroni approach, the experimental treatment is considered efficacious, P = .02. However, using the rank sum method for defining a global outcome, the mean difference between groups in rank sum scores would be 0, which appropriately reflects that no overall benefit was associated with either treatment. The problem with the Bonferroni approach is that it fails to rely on a comprehensive assessment of the patient. This failure to use important information is reflected in lower power (See Power) for the Bonferroni approach if the experimental treatment has a beneficial effect across all primary endpoints. The ability of the various tests to control the overall type I error rate (falsely declaring efficacy when none exists) at the specified level of .05 and the power of the procedures to identify treatment effects is shown in Tables 1 and 2, respectively [adapted from a simulation study (2)]. In these tables, n indicates the sample size in

8

GLOBAL ASSESSMENT VARIABLES

Table 1. Observed Type I Error Rates for Various Tests Type I Error Rate∗ n

K

Rank-sum

GLS

Least Squares

Equal

5 20 20 50 5

2 2 5 5 50

.045 .050 .051 .048 .041

.041 .050 .059 .058 —

.062 .053 .045 .053 .061

Normal

Unequal

20 50 5

5 5 50

.047 .053 .050

.047 .051 —

.060 .049 .053

Skewed

Equal

5 20 20 50 5

2 2 5 5 50

.043 .062 .056 .035 .052

.041 .063 .064 .038 —

.045 .064 .063 .036 .054

Outliers

Equal

5 20 20 50 5

2 2 5 5 50

.047 .063 .051 .052 .053

.021 .032 .027 .036 —

.024 .032 .028 .037 .028

Distribution

Correlation

Normal

∗α =

.05. Data from O’Brien (2)

Table 2. Comparison of Power for Various Tests Power∗ Distribution

Treatment Effect

Correlation

n

K

Ranksum

GLS

Least squares

Bonferroni

T2

Equal

Normal

Equal

20 50 5

5 5 50

.64 .91 .28

.67 .92 —

.67 .92 .31

.52 .85 .17

.26 .66 —

Staggered

Normal

Equal

20 50 5

5 5 50

.32 .62 .14

.35 .64 —

.32 .64 .17

.34 .69 .10

.20 .58 —

Equal

Normal

Unequal

20 50 5

5 5 50

.62 .92 .18

.74 .97 —

.64 .93 .18

.51 .84 .06

.33 .78 —

Equal

Skewed

Equal

20 50 5

5 5 50

.77 .99 .38

.66 .93 —

.66 .93 .37

.57 .86 .16

.29 .67 —

Equal

Outliers

Equal

20 50 5

5 5 50

.23 .42 .11

.07 .08 —

.08 .08 .06

.02 .02 .01

.05 .05 —

∗α =

.05. Data from O’Brien (2)

GLOBAL ASSESSMENT VARIABLES

each treatment group, and K represents the number of univariate primary endpoints. As expected, all procedures control the overall type I error rate when sampling from a normal distribution, but only the rank sum test does so reliably for non-normal distributions (Table 1). Simulations to compare power (Table 2) assumed that the treatment improved all endpoints equally (the case for which the rank sum, least squares, and GLS tests should perform well) except for a ‘‘staggered effect size,’’ in which the effects ranged from 1/K, 2/K, . . . , K/K for endpoints 1, 2, . . . , K, respectively. As expected, the Bonferroni method had lower power for the equal effects situation, but can provide greater power when effects vary among the endpoints, depending on the variability, the number of endpoints, and the sample size. The T2 test has uniformly low power. 6.3 Composite Endpoints Lefkopoulou and Ryan (19) compared two different methods to assess treatment’s global effect on multiple binary outcomes. One is the test of composite outcomes (referred to as a combined test) that collapses multiple outcomes into a single binary outcome: A success is observed if success on any one of multiple binary outcomes is observed (See Composite Endpoints). The other is the 1-degree-offreedom GST that assumes a common odds ratio. The GST can be constructed using GEE (See Generalized Estimating Equations). When the response proportions are small (> 50%), both the composite test and the GST are approximately equally efficient. However, as the response proportions increase, the GST becomes much more efficient than the composite (combined) test (19). When a combined composite endpoint is used (any one of a set of outcomes, or any two, etc.), information from all outcomes is not taken into account. Someone with a stroke and myocardial infarction (MI) would be rated the same as someone having only an MI, and an increased risk of stroke, for example, could be missed. See Composite Outcomes (8) for a discussion of the limitation of the composite outcome and the ACR criteria used in the study of rheumatoid arthritis. However, computation of composite outcomes generally does not

9

require a common dose assumption. Also, for outcomes such as the ACR criteria, individual subjects are classified as successes or failures that are not generally the case for GSTs. 7

EXAMPLES OF THE APPLICATION OF GST

The NINDS t-PA Stroke Group performed two randomized, double-blind, placebo-controlled trials for patients with acute ischemic stroke (1). In both trials, the treatment success was defined as a ‘‘consistent and persuasive difference’’ in the proportion of patients who achieve favorable outcomes on the Barthel Index, Modified Rankin Scale, Glasgow Outcome Scale, and National Institutes of Health Stroke Scale. Four primary outcomes were used because a positive result from any single outcome was not believed to provide a sufficient evidence of treatment efficacy. Trial Coordinating Center proposed the use of a GST. All four outcomes are dichotomized to represent an outcome of normal or mild disability versus more severe disability or death. Trial investigators believed the assumption of a common dose effect to be valid for all four binary outcome measures. Table 3 gives the odds ratios for individual tests of each outcome and the 95% confidence intervals computed using a Mantal-Haenszel approach. The GST was computed using methods of Lefkopoulou and Ryan for binary outcomes (19). The GTE (odds ratio) for the GST is 1.73. No single outcome would be considered significant based on the Bonferroni adjustment. However, because the GST indicated an overall favorable outcome (P = 0.008), it was (a priori) considered to provide weak protection of alpha, and a test of the individual outcomes were conducted at the 0.05 level. Using these criteria, all individual outcomes indicated a benefit of treatment. See Table 3. Tilley et al. (8) used GSTs to analyze clinical trials in rheumatoid arthritis. O’Brien used GST in analysis of a randomized trial comparing two therapies, experimental and conventional, for the treatment of diabetes. The objective of the study was to determine whether the experimental therapy resulted in better nerve function, as measured by 34 electromyographic (EMG) variables, than the

10

GLOBAL ASSESSMENT VARIABLES

Table 3. NINDS t-PA Stroke Trial, Part II Data∗ Proportion With Favorable Outcome Outcome Barthel Modified Rankin Glasgow NIH Stroke Scale GST∗

t-PA (n = 168) 50 39 44 31

Placebo (n = 165) 38 26 32 20

OR 1.63 1.68 1.64 1.72 1.73

95% CI 1.06–2.49 1.09–2.59 1.06–2.53 1.05–2.84 1.16–2.60

P .026 .019 .025 .033 .008

∗ Global statistical test for binary outcomes developed by Lefkopoulou and Ryan (19). Data from the NINDS t-PA Stroke Trial (1).

standard therapy. Six subjects were randomized to standard therapy, and five subjects were randomized to experimental therapy. Despite the small sample size, the objective of the study was viewed primarily as confirmatory rather than exploratory. The medical question at issue was controversial, which required an overall quantitative and objective probability statement. There were six EMG variables for which the difference between groups was statistically significant (P > .05). The two smallest P-values were P = .002 and P = .015. The treated group did better in 28 of 34 variables, with P > .50 as a criterion; this result indicated the type of main effect for which the authors had hoped. Although these results seem to support the hypothesis of a beneficial effect associated with the experimental group, an overall test was needed. A T2 test was not appropriate. Application of a multiple-comparisontype per-experiment error rate is also meaningless here because of the small sample size relative to the large number of variables. When applied to all the data, the GST yielded a P-value of .033 (2).

8

CONCLUSIONS

Global statistical tests and associated GTEs summarize a treatment’s global benefit without the need of performing multiple univariate tests for individual outcomes. In contrast to composite tests, some GSTs take the correlation among outcomes into account. When a treatment shows benefit on each of the single outcomes, the GST often has a higher power than univariate tests because of increased information included in the GST. When a

treatment shows strong benefit on some outcomes and harm on other outcomes, the GST generally loses power compared with Bonferroni adjustment. This loss of power is also an advantage of the GST because investigators will be reluctant to accept a treatment if it shows strong harmful effect on equally important outcomes. The GST approach of analyzing multiple primary outcomes is a new emerging area in both statistical research and clinical applications. A variety of GSTs is available to address diverse situations. We believe that GSTs will play a significant role in medical research and future scientific findings. REFERENCES 1. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group, Tissue plasminogen activator for acute ischemic stroke. N. Engl. J. Med. 1996; 333: 1581–1587. 2. P. C. O’Brien, Procedures for comparing samples with multiple endpoints. Biometrics 1984; 40: 1079–1087. 3. S. J. Pocock, N. L. Geller, and Tsiatis AA. The analysis of multiple endpoints in clinical trials. Biometrics 1987; 43: 487–498. 4. L. Hothorn, Multiple comparisons in longterm toxicity studies. Environmental Health Perspectives. 1994; 102(suppl 1): 33–38. 5. K. D. Kaufman, E. A. Olsen, D. Whiting, R. Savin, R. DeVillez, W. Bergfeld, et al., Finasteride in the treatment of men with androgenetic alopecia. Finasteride Male Pattern Hair Loss Study Group. J. Am. Acad. Dermatol. 1998; 39: 578–589. 6. D. K. Li, G. J. Zhao, and D. W. Paty, Randomized controlled trial of interferon-beta-1a in secondary progressive MS: MRI results. Neurology 2001; 56: 1505–1513.

GLOBAL ASSESSMENT VARIABLES 7. R. S. Shames, D. C. Heilbron, S. L. Janson, J. L. Kishiyama, D. S. Au, and D. C. Adelman, Clinical differences among women with and without self-reported perimenstrual asthma. Ann. Allergy Asthma Immunol. 1998; 81: 65–72. 8. B. C. Tilley, S. R. Pillemer, S. P. Heyse, S. Li, D. O. Clegg, and G. S. Alarcon, Global statistical tests for comparing multiple outcomes in rheumatoid arthritis trials. MIRA Trial Group. Arthritis Rheum. 1999; 42: 1879–1888. 9. P. Huang, R. F. Woolson, and P.C. O’Brien, A rank-based sample size method for multiple outcomes in clinical trials. Stat. Med. In press. 10. P. Huang, B. C. Tilley, R. F. Woolson, and S. Lipsitz, Adjusting O’Brien’s test to control type I error for the generalized nonparametric Behrens-Fisher problem. Biometrics 2005; 61: 532–539. 11. L. Acion, J. J. Peterson, S. Temple, and S. Arndt, Probabilistic index: an intuitive nonparametric approach to measuring the size of treatment effects. Stats. Med. 2006; 25: 591–602. 12. D. I. Tang, N. L. Geller, and S. J. Pocock, On the design and analysis of randomized clinical trials with multiple endpoints. Biometrics 1993; 49: 23–30. 13. D. Follmann, Multivariate tests for multiple endpoints in clinical trials Stats. Med. 1995; 14: 1163–1175. 14. D. Tang, C. Gnecco, and N. Geller, Design of group sequential clinical trials with multiple endpoints. J. Am. Stat. Assoc. 1989; 84: 776–779. 15. D. Tang, C. Gnecco, and N. Geller An approximate likelihood ratio test for a normal mean vector with nonnegative components with application to clinical trials. Biometrika 1989; 76: 577–583. 16. X. Lin, L. Ryan, M. Sammel, D. Zhang, C. Padungtod, and X. Xu, A scaled linear mixed model for multiple outcomes. Biometrics 2000; 56: 593–601. 17. D. Follmann, A simple multivariate test for one-sided alternatives. J. Am. Stat. Assoc. 1996; 91: 854–861. 18. M. Lefkopoulou, D. Moore, and L. Ryan, The analysis of multiple correlated binary outcomes: application to rodent teratology experiments. J. Am. Stat. Assoc. 1989; 84: 810–815. 19. M. Lefkopoulou and L. Ryan Global tests for multiple binary outcomes. Biometrics 1993; 49: 975–988. 20. M. Lu and B. C. Tilley, Use of odds ratio or relative risk to measure a treatment effect in

11

clinical trials with multiple correlated binary outcomes: data from the NINDS t-PA Stroke Trial. Stats. Med. 2001; 20: 1891–1901. 21. J. Legler, M. Lefkopoulou, and L. Ryan, Efficiency and power of tests for multiple binary outcomes. J. Am. Stat. Assoc. 1995; 90: 680–693. 22. D. A. Bloch, T. L. Lai, and P. Tubert-Bitter, One-sided tests in clinical trials with multiple endpoints. Biometrics 2001; 57: 1039–1047. 23. C. Jennison and B. Turnbull, Group Sequential tests for bivariate response: interim analysis of clinical trials with both efficacy and safety endpoints. Biometrics 1993; 49: 741–752. 24. C. Jennison and B. Turnbull, Group Sequential Methods with Applications to Clinical Trials. 2000. CRC Press Inc., Boca Raton, FL. 25. J. Rochon, Application of GEE procedures for sample size calculations in repeated measures experiments. Stats. Med. 1998; 17: 1643–1658. 26. R. Falk, Hommel’s Bonferroni-type inequality for unequally spaced levels. Biometrika 1989; 76: 189–191. 27. S. Holm, A simple sequentially rejective multiple test procedure. Scand. J. Stats. 1979; 6: 65–70. 28. W. Lehmacher, G. Wassmer, and P. Reitmeir, Procedures for two-sample comparisons with multiple endpoints controlling the experimentwise error rate. Biometrics 1991; 47: 511–521. 29. R. Marcus, E. Peritz, and K. R. Gabriel On closed testing procedures with special references to ordered analysis of variance. Biometrika 1976; 63: 655–660. 30. M. R. Kosorok, S. Yuanjun, and D. L. DeMets, Design and analysis of group sequential clinical trials with multiple primary endpoints. Biometrics 2004; 60: 134–145. 31. C. Khedouri, Correcting Distortion in Global Statistical Tests With Application to Psychiatric Rehabilitation. Charleston, SC: Medical University of South Carolina, 2004. 32. T. Karrison and P. O’Brien, A rank-sum-type test for paired data with multiple endpoints. J. Appl. Stat. 2004; 31: 229–238. 33. N. Freemantle, M. Calvert, J. Wood, J. Eastaugh, and C. Griffin, Composite outcomes in randomized trials: greater precision but with greater uncertainty? JAMA 2003; 289: 2554–2559.

12

GLOBAL ASSESSMENT VARIABLES

CROSS-REFERENCES Type I Error Mixed Effect Models Multiple Comparisons Composite Variables (Endpoints or Indices) Generalized Estimating Equations Composite Outcomes

GOLD STANDARD

2

Many diagnostic tests have a binary response. An error-free, binary diagnostic test differentiates perfectly between patients who have a disease and those who do not, and it is referred to as a ‘‘perfect’’ gold standard test. Perfect gold standard tests are usually not available or not feasible because of technical, economic, or implementation constraints. Instead, investigators use tests or procedures with acceptably low error rates. An example is the use of clinical signs and symptoms, enzyme assays, and electrocardiograms instead of direct examination of the tissue of the heart to determine whether a patient has had a myocardial infarction. If a test is the best available, even though it may be imperfect and subject to error, it is called a gold standard test and is used as the reference with which other tests are compared. When a diagnostic test is not perfect, other characteristics of the test must be considered before declaring it a gold standard. Sensitivity is the conditional probability that the test will be positive given that the subject has the disease. Specificity is the conditional probability that the test will be negative given that the subject does not have the disease (3, p. 30). Very high values for both sensitivity and specificity are desirable if the test is to be considered useful. A test that has high sensitivity will be good at detecting a disease if it is present, and a test with high specificity will be good at ruling out a disease if it is not. For example, Jobst et al. (4) examined various tests used to diagnose Alzheimer’s disease. The study population consisted of normal elderly subjects as well as of subjects with non-Alzheimer’s disease dementias or Alzheimer’s disease that had been histopathologically confirmed (the gold standard test). A clinical diagnosis of probable Alzheimer’s disease according to the National Institute of Neurological and Communicative Disorders and Stroke–Alzheimer’s Disease and Related Disorders Association (NINCDSADRDA) had only 49% sensitivity in the Alzheimer’s subjects but 100% specificity in the subjects who did not have Alzheimer’s disease.

MARILYN A. AGIN Pfizer Global Research and Development, Ann Arbor, Michigan

In clinical trials, the concept of a gold standard is most often applied to diagnostic tests or methods of measurement that are considered to be the best in some general sense among all other tests or methods available. The gold standard test or measurement may be error-free or have an acceptably small error rate. New tests or measurements are compared with the gold standard. Closely related are the agreement of two or more types of tests or measurements when neither is a gold standard, and the reproducibility of a given test result or measurement. This article provides an overview of gold standard tests and measurements as used in clinical trials. 1

DIAGNOSTIC TESTS

THE GOLD STANDARD

In economics, the gold standard refers to a monetary standard under which the currency of a country is equal in value to and redeemable by a certain quantity of gold (1). Currencies of different countries can thus be compared with each other by assessing their relative value in gold. The United States was on the gold standard from time to time until 1971 (2). Today, informal usage of the term ‘‘gold standard’’ has expanded beyond economics to denote a practice or principle that is a model of excellence with which things of the same class can be compared. In clinical trials, the concept of a gold standard is most often applied to diagnostic tests or methods of measurement that are considered to be the best in some general sense among all other tests or methods available. New tests or measurements are compared with the gold standard. Closely related are the agreement of two or more types of tests or measurements when neither is a gold standard, and the reproducibility of a given test result or measurement. This article provides an overview of gold standard tests and measurements as used in clinical trials.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

GOLD STANDARD

What a clinician usually wants to know, however, is the probability that a patient has a disease given that a test is positive or the probability that a patient does not have a disease given that a test is negative. If the proportion of patients in the population who have the disease is known, then Bayes’ theorem can be used to estimate these conditional probabilities retrospectively (3, p. 12). Most often, this proportion is not known and must be approximated. Other important aspects of a diagnostic test are its error rates. A false-positive error means that the test result was positive but the subject does not have the disease. A falsenegative error means that the test result was negative but the subject does have the disease. The error rate of the reference test is sometimes overlooked in evaluating the new test, and this can lead to bias in estimating the errors in the new test. When declaring an imperfect test to be a gold standard, the purpose for which the test is administered should be considered along with the sensitivity, specificity, and error rates. Sometimes two tests are available, and one or neither is considered a gold standard. When comparing a new test with a gold standard, the objective is to be able to declare the new test almost as good as (or no worse than) the gold standard. The null hypothesis should state that the two tests are different. The alternative hypothesis should state that the new test is as good as (equivalence) or no worse than (noninferiority) the gold standard since this is what one wishes to conclude. If the two statistical hypotheses had been reversed, then no claim could have been made about the new test being as good as or no worse than the gold standard. Not being able to reject the null hypothesis is not evidence that it is true, i.e., an ‘‘absence of evidence is not evidence of absence’’ (5). If neither test is a gold standard, then the alternative hypothesis should state that the two tests are equivalent or similar. A practical difficulty encountered when testing for equivalence or noninferiority is quantifying how similar the tests must be in order to claim they are not different. Chen et al. (6) propose a method to test the hypothesis of noninferiority with respect to sensitivity and specificity when

the response is binary. The new test is compared with a reference test and with a perfect gold standard test under retrospective and prospective sampling plans. A latent continuous or categorical variable is one that cannot be observed directly (7). In clinical trials, when a perfect gold standard test does not exist, then the true disease state can be thought of as a latent variable. For example, postmortem examination of tissue may be the only way to detect a certain type of pneumonia or a recurrence of cancer so the true disease state is latent as long as the patient is alive. If two imperfect diagnostic tests are used, a common statistical assumption is that the results of the tests are conditionally independent given the latent variable. 3 MEASUREMENT METHODS As with diagnostic tests, clinical measurements often cannot be made directly so indirect methods are used. For example, the duration of ventricular depolarization and repolarization of the heart is evaluated indirectly by measuring the QT interval on an electrocardiogram (ECG). Two laboratories using different methods of measurement on the same ECG (8), a technician reading the same ECG twice (9), or two different computer algorithms applied to the same ECG (10) could produce two different values for the QT interval. Since the true value of the QT interval is unknown, no method provides an unequivocally correct measurement (perfect gold standard) and the degree to which the measurements agree must be assessed. Since the act of measuring almost always involves some inherent measurement error or subjective judgment, a gold standard method of measurement is usually not perfect but is widely accepted and the reference to which other methods are compared. For example, an established assay for determining the plasma concentration of a drug is usually the reference for a new assay. The manner in which two or more sets of measurements agree or disagree is important since tests of agreement are usually specific to certain types of disagreement. For example, Pearson’s correlation coefficient

GOLD STANDARD

indicates only the linear relationship between the two sets of measurements. The correlation could be very strong, but one measurement could be consistently longer than another, or the measurements could follow a nonlinear relationship very closely but with little correlation. A paired t-test will detect only if the means of the two measurement groups are different and not provide any information about the marginal distributions of each measurement. Regression analysis, Lin’s (8) concordance correlation coefficient, and the intraclass correlation coefficient (9) address other types of agreement and reproducibility. Altman and Bland (10) and Bland and Altman (11,12) provide a clear explanation of the problem of comparing two methods of measurement and statistical principles of a graphical method for assessing agreement. The differences between measurements on the same observational unit are plotted against their averages. The within- and between-unit variability and the dependence of the difference between measurements on the mean are examined. Bartko (13) expand on Altman and Bland’s analyses by suggesting a bivariate confidence ellipse to amplify dispersion. St. Laurent (14) and Harris et al. (15) propose models and estimators for assessing agreement of one or more approximate methods of measurement with an accepted gold standard. Agreement is assessed by a ratio of variances equal to the square of the correlation between the approximate and the gold standard measurements. This correlation depends on the variability in the gold standard measurements and is identical in form to the intraclass correlation coefficient used when comparing two approximate methods. An acceptable degree of agreement depends on the error in the approximate method and can be defined for the specific clinical application under investigation. 4

CONCLUSION

When assessing diagnostic tests or methods of measurement, the first step should be to plot the data from each test or method against the others. Understanding how the tests or

3

sets of measurements agree or differ and the potential for incorrect conclusions can help to determine how to improve a test or measurement, assess the results statistically, and provide evidence for replacing a former imperfect or accepted gold standard test or method of measurement with a new one. REFERENCES 1. The American Heritage College Dictionary, 3rd ed. New York: Houghton Mifflin Company, 1997, p. 585. 2. World Book Encyclopedia Online. 2004. World Book, Inc. 3. A. Agresti, Categorical Data Analysis. New York: Wiley, 1990. 4. K. A. Jobst, L. P. D. Barnetson, and B. J. Shepstone, Accurate Prediction of Histologically Confirmed Alzheimer’s Disease and the Differential Diagnosis of Dementia: The Use of NINCDS-ADRDA and DSM-III-R Criteria, SPECT, X-Ray CT, and Apo E4 in Medial Temporal Lobe Dementias. International Psychogeriatrics 1998; 10: 271-302. Published online by Cambridge University Press 10Jan2005. 5. D. G. Altman and J. M. Bland, Absence of evidence is not evidence of absence. BMJ 1995; 311: 485. 6. J. J. Chen, H. Hsueh, and J. Li, Simultaneous non-inferiority test of sensitivity and specificity for two diagnostic procedures in the presence of a gold standard. Biomet. J. 2003; 45: 47-60. 7. P. F. Lazarsfeld, The logical and mathematical foundation of latent structure analysis. In: S. Stouffer (ed.), Measurement and Prediction. Princeton, NJ: Princeton University Press, 1950. 8. S. Patterson, M. A. Agin, R. Anziano, T. Burgess, C. Chuang-Stein, A. Dmitrienko, G. Ferber, M. Geraldes, K. Ghosh, R. Menton, J. Natarajan, W. Offen, J. Saoud, B. Smith, R. Suresh, and N. Zariffa, Investigating druginduced QT and QTc prolongation in the clinic: A review of statistical design and analysis considerations: Report from the Pharmaceutical Research and Manufacturers of America QT Statistics Expert Team. Drug Inf J. 2005; 39: 243-264. 9. N. Sarapa, J. Morganroth, J. P. Couderc, S. F. Francom, B. Darpo, J. C. Fleishaker, J. D. McEnroe, W. T. Chen, W. Zareba, A. J. Moss, Electrocardiographic identification of

4

GOLD STANDARD

drug-induced QT prolongation: Assessment by different recording and measurement methods. Ann Noninvas. Electrocardiol. 2004; 9(1): 48-57. 10. J. L. Willems, P. Arnaud, J. H. van Bemmel, P. J. Bourdillon, C. Brohet, S. Dalla Volta, J. D. Andersen, R. Degani, B. Denis, and M. Demeester, Assessment of the performance of electrocardiographic computer programs with the use of a reference data base. Circulation 1985; 71(3): 523-534. 11. L. I. Lin, A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989; 45(1): 255-268. 12. P. E. Shrout and J. L. Fleiss, Intraclass correlations: Uses in assessing rater reliability. Psychol. Bull. 1979; 2: 420-428. 13. D. G. Altman and J. M. Bland, Measurement in medicine: The analysis of method comparison studies. Statistician 1983; 32: 307-317. 14. J. M. Bland and D. G. Altman, Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;(February): 307-310. 15. J. M. Bland and D. G. Altman, Measuring agreement in method comparison studies. Stat. Methods Med. Res. 1999; 8:135-160. 16. J. J. Bartko, Measures of agreement: a single procedure. Stat. Med. 1994; 13:737-745. 17. R. T. St. Laurent, Evaluating agreement with a gold standard in method comparison studies. Biometrics 1998; 54: 537-545. 18. I. R. Harris, B. D. Burch, and R. T. St. Laurent, A blended estimator for a measure of agreement with a gold standard. J. Agricult. Biol. Environ. Stat. 2001; 6: 326-339. 19. S. D. Walter and L. M. Irwig, Estimation of test error rates, disease prevalence and relative risk from misclassified data: A review. Clin. Epidemiol. 1988; 41: 923-937. 20. S. L. Hui and X. H. Zhou, Evaluation of diagnostic tests without gold standards. Stat. Methods Med. Res. 1998; 7: 354-370. 21. L. Joseph, Reply to comment on ‘‘Bayesian estimation of disease prevalence and the parameters of diagnostic tests in the absence of a gold standard’’. Am. J. Epidemiol. 1997; 145: 291-291.

22. N. Dendukuri and L. Joseph, Bayesian approaches to modeling the conditional dependence between multiple diagnostic tests. Biometrics 2001; 57: 208-217. 23. N. Dendukuri, E. Rahme, P. B´elisle, and L. Joseph, Bayesian sample size determination for prevalence and diagnostic test studies in the absence of a gold standard test. Biometrics 2004; 60: 388-397. 24. J. L. Fleiss, 1999. The Design and Analysis of Clinical Experiments. New York: Wiley, 1999.

CAMBRIDGE JOURNALS ONLINE Further reading Walter and Irwig (19) review publications on error rates of diagnostic tests. Hui and Zhou (20) review statistical methods for estimating sensitivity and specificity of diagnostic tests when one or more new diagnostic tests are evaluated and no perfect gold standard test exists, with or without an imperfect gold standard test as a reference. They also discuss models that relax the assumption of conditional independence between tests given the true disease status. Joseph (21), Dendukuri and Joseph (22), and Dendukuri et al. (23) use a Bayesian approach to account for the potential dependence between the tests instead of assuming conditional independence. Dallal’s website http://www.tufts.edu/ gdallal/compare.html provides an excellent introduction to the problem of comparing two measurement devices. Fleiss (24, Chapter 1) provides a comprehensive overview of the reliability of quantitative measurements.

CROSS-REFERENCES False Positive – False Negative Inter-rater reliability Intra-rater reliability Intraclass Correlation Coefficient Sensitivity Specificity Type I Error Type II

GOOD CLINICAL PRACTICE (GCP)

based on adequately performed laboratory and animal experimentation as well as on a thorough knowledge of the scientific literature. The right of the research subject to safeguard his/her integrity must always be respected. Physicians should abstain from engaging in research projects that involve human subjects unless they are satisfied that the hazards involved are predictable and should cease any investigations if the hazards are found to outweigh the potential benefits. Clinical tests in humans are performed under many regulatory controls. Currently, these controls include the regulations promulgated under the Food Drug and Cosmetic Act, European Union rules, and guidelines issued by the International Conference on Harmonization (ICH) (1). These regulations and guidances are collectively termed ‘‘Good Clinical Practices.’’ The testing of medical products, which include pharmaceuticals, biologicals, and medical devices on humans, is regulated by Good Clinical Practice standards (i.e., regulations and guidances). These standards are applied to the conduct of these tests or clinical investigations. International regulatory bodies require that these standards be met for the medical products to be approved for use. The experimental protocol must be reviewed by an independent body [Independent Ethics Committee (IEC) or Institutional Review Board (IRB)] constituted of medical professionals and nonmedical members, whose responsibility is to ensure the protection of the rights, safety and well-being of human subjects involved in a clinical investigation. This independent body provides continuing review of the trial protocol and amendments and of all materials to be used in obtaining and documenting the informed consent of the trial subjects.

DAVID B. BARR Kendle International, Cincinnati, Ohio

The ultimate goal of most clinical investigations is to obtain marketing approval of the product. In the case of a drug product, the Food and Drug Administration (FDA) requires the submission of a New Drug Application (NDA) that consists of a compilation of all clinical and other test data from the clinical studies plus extensive information on the development, manufacture, and testing of the drug product. This document is often massive, running in excess of 100 volumes. The FDA reviews this information. The review often includes meetings with the sponsor and may include a request for additional testing by the sponsor. Drug approval is for a specific drug, as well as the chemical entity, formulation, and the patient population on which it is approved for use, and the indications or disease state for which it is approved for use. For a sponsor to add other indications or patient populations to the drug labeling usually requires additional clinical studies to support these additional uses. 1

HUMAN RIGHTS AND PROTECTIONS

The need to protect human subjects during clinical studies is paramount, and the need for these protections is, in large part, based on abuses that had occurred in many countries including the medical experiments conducted by the Nazis and certain studies conducted in other countries without the full knowledge of the subjects. The rights of the subjects in a clinical trial are based on the principles stated in Declaration of Helsinki in 1964. Good clinical investigations must be conducted in accordance with these stated ethical principles. The rights, safety, and well-being of the trial subjects are paramount. Clinical research on human subjects must conform to generally accepted scientific principles and should be

2

INFORMED CONSENT

Subjects must volunteer to enter a clinical investigation and must be fully informed of their rights and a description of the study that includes risks and benefits. The investigator is required to provide each subject with

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

GOOD CLINICAL PRACTICE (GCP)

a written consent form approved by the IRB, which includes the following basic elements: a statement that the study involves research, the expected duration of the subject’s participation, a description of the procedures to be followed, identification of any procedures that are experimental, a description of any reasonably foreseeable risks or discomforts, a disclosure of any alternative procedures or courses of treatment that might be advantageous to the subject, an explanation of who to contact, and a statement that participation is voluntary. This consent form must be given to the subject, explained, and signed by the subject or his/her representative [21 CFR 50.25 -50.27 (2)]. Informed consent must be signed prior to initiation of the subject into the study. 3

INVESTIGATIONAL PROTOCOL

The clinical investigation must be described in a written protocol, which includes: a summary of information from both nonclinical studies that potentially have clinical significance and from clinical trials that have been conducted; a summary of the known and potential risks and benefits; information about the investigational product(s), route of administration, dosage, dosage regimen, and treatment periods; the population to be studied; references to literature and data that are relevant and provide background material for the trial; a description of the trial design, a statement of primary endpoints and any secondary endpoints, measures to be taken to minimize bias, discontinuation criteria, investigational product accountability procedures, identification of data to be recorded directly on the Case Report Form (CRF); criteria for the selection and withdrawal of subjects; description of subject treatment including the name(s) of all products, dosing schedule, route/mode of administration, treatment period(s) including follow-up period(s); permitted and prohibited medications, procedures for monitoring subject compliance; methods of assessments of efficacy and safety; statistical methods; quality control and quality assurance procedures; data handling and record-keeping requirements; and any supplements to the protocol.

4 INVESTIGATOR’S BROCHURE The Investigator’s Brochure (IB) is a compilation of the relevant clinical and nonclinical data on the investigational product(s). The IB must be provided to IRBs and IECs for their evaluation into whether the state of knowledge of the drug justifies the risk/benefit balance for the clinical trial. The IB is given to each investigator to provide to the clinical team with an understanding of the study details and facilitate their compliance with the protocol. 5 INVESTIGATIONAL NEW DRUG APPLICATION FDA requires the submission of an Investigational New Drug Application (IND) if a sponsor intends to conduct most clinical investigations (some exemptions apply see Reference 2). The IND is required to contain information on the sponsor and the investigational drug. The IND also contains brief descriptions of the drug, any relevant previous human experience with the drug, the overall investigational plan, nonclinical studies and safety information, protocol(s), and the IB. IND filed for Phase 1 studies are relatively brief at about one or two volumes, whereas those intended for Phase 2 and 3 typically contain detailed protocols that describe all aspects of the study. The IND must be approved before any human trial is initiated. Many other countries require similar applications. 6 PRODUCTION OF THE INVESTIGATIONAL DRUG Investigational products should be manufactured, handled, and stored in accordance with applicable current Good Manufacturing Practices (cGMPs ) and used in accord with the protocol. It is important that all product formulation, manufacturing process, specification, and test method development activities for pharmaceutical ingredients and products that support submissions of the IND and marketing applications be properly conducted and documented. Such information

GOOD CLINICAL PRACTICE (GCP)

may be requested for examination by FDA during preapproval inspections. All development activities should be performed applying the principles of Good Scientific Practices (GSPs) to ensure accuracy and reproducibility of data and other information. Examples of GSPs include accurate and complete documentation of all research and development activities; use of properly maintained and calibrated equipment; use of reagents, standards, and solutions that meet compendia standards or internal specifications; use of standard operating procedures, protocols, or guidelines; and performance of activities by appropriately qualified and trained personnel. At each stage of development, critical parameters should be defined for manufacturing processes (e.g., temperature, time) and product specifications (e.g., pH, viscosity). Test results and other data collected on product quality characteristics should be obtained using credible analytical methods. Critical parameters from production processes for product stability should be performed according to written procedures or prospective protocols typically with approval by the company’s quality unit. Changes in the product, process, specifications, and test methods should be controlled and documented. All data/information, whether favorable or unfavorable, should be recorded and included in development history documents. 7

CLINICAL TESTING

Clinical testing is performed according to the written protocols and procedures. A welldesigned and well-executed clinical investigation consists of many individual tests, stages, and subjects. The trial must be in compliance with the approved protocol and all pertinent regulations. Trials must be scientifically sound and described in a clear, detailed experimental protocol. Biomedical research that involves human subjects should be conducted only by scientifically qualified persons and under the supervision of a clinically competent medical person. In 1962, the Kefauver-Harris amendments to the Food Drug and Cosmetic Act (FD&C Act) increased FDA’s regulatory authority

3

over the clinical testing of new drugs. These amendments required that new drugs be approved by the FDA and that the application include ‘‘full reports of investigations which have been made to show whether or not such drug is safe for use and whether such drug is effective in use . . . ’’ [FD&C Act section 505 (3)]. Once compounds have been identified as potential drugs, preclinical testing begins. Clinical testing of new drugs and biologicals is conducted only after extensive preclinical animal testing to determine the safety of the potential drug. Currently, typical clinical testing consists of several phases, tests, and human subjects. Many tests may be required for these experimental products. For instance, in pharmaceuticals several tests are needed to permit the development of the estimates of the safe starting dose for use in clinical investigations. Tests are generally conducted for: single and repeated dose toxicity studies, reproductive toxicity, carcinogenic potential, genotoxicity, local tolerance, and pharmacokinetics. The ICH and several regulatory bodies have published guidelines on the conduct of these studies. Additionally, regulatory requirements control of many of these studies, for example, FDA’s Good Laboratory Practice regulations [GLPs at 21CFR 58 (4)]. Once animal safety studies are conducted and show relative safety, the experimental medical product may then begin testing in humans. An experimental drug requires a sponsor. The FDA defines a sponsor in the code of federal regulations [21CFR 312.3 (5)] as ‘‘a person who takes responsibility for and initiates a clinical investigation. The sponsor may be an individual or a pharmaceuticalcompany, governmental agency, academic institution, private organization, or other organization.’’ Before testing may be conducted on humans an Investigational New Drug (IND) or Investigational Device Exemption (IDE) application needs to be approved by regulatory bodies such as the FDA. Preapproval clinical testing is generally performed in three phases; each successive phase involves more subjects. These phases are conducted in accord with regulations [21 CFR 312.21 (5)], not by requirements of law.

4

GOOD CLINICAL PRACTICE (GCP)

Some clinical testing may also be performed post-approval. Phase One studies are closely controlled and monitored studies performed primarily to assess the safety of the drug in a few patients (typically 20–80) or healthy subjects. These studies are designed to determine the actions of the drug in the human body: absorption, metabolism, and pharmacological actions including degradation and excretion of the drug, side effects associated with increasing doses, and, if possible, to gain early evidence on effectiveness. These studies may also investigate the best method of administration and the safest dosage as well as side effects that occur as dosage levels are increased. Phase Two studies are usually the initial, well monitored, and well-controlled studies conducted to evaluate the effectiveness of the drug for a particular indication or condition. These studies generally involve a limited number of subjects, up to several hundred, who have the disease or condition under study. These studies are typically conducted to: evaluate the efficacy of the drug, determine the dose range(s), determine drug interactions that may occur, to confirm the safety of the drug, and to compare it with similar approved drugs. These studies may or may not be blinded. Phase Three studies are typically the last studies performed prior to filing the NDA with the regulatory bodies (e.g., FDA or European Agency for the Evaluation of Medicinal Products). These studies are expanded controlled and uncontrolled trials. They are performed after preliminary evidence that suggests effectiveness of the drug has been obtained. These studies are intended to gather the additional information about effectiveness and safety that is needed to evaluate the overall risk/benefit relationship of the drug and to provide and adequate basis for physician labeling. These trials are often multicenter studies with several thousands of subjects. These studies: provide the statistically significant data for the efficacy of the drug, assess the safety data, and determine the final dosage(s) and dosage forms. These studies are typically conducted as doubleblinded studies against similar approved drugs and/or placebos.

Phase Four studies are postmarketing studies that may be conducted by the sponsor and often required by FDA concurrent with marketing approval to delineate additional information about the risks, benefits, and optimal use of the drug. These studies include those studying different doses or schedules of administration, the use of the drug in other patient populations or stages of disease, or usage over a longer time period (see Fig. 1). Good Clinical Practice standards include most processes and organizations involved in the conduct of a pharmaceutical clinical study, such as the following: • Sponsors • Monitors • Contract Research Organizations

(CROs) • Institutional Research Boards (IRBs) or

Independent Ethics Committees (IECs) • Investigators

FDA regulations that govern the clinical investigations are found in Title 21 of the Code of Federal Regulations and in Guidelines promulgated by FDA and other regulatory bodies as well as those developed and issued by ICH. The ICH is an organization whose membership includes the regulators and Industry members from the EU, Japan, and the United States. As such, the ICH guidelines are in use in all of the EU countries, Japan and the United States and are widely used as guidelines worldwide. 8 SPONSORS The sponsor of an experimental drug initiates the studies and assumes the ultimate responsibility for compliance with all legal and regulatory requirements. The sponsor is responsible for the selection of all the parties utilized in the conduct of the clinical investigation including the monitors and investigators. According to FDA ‘‘The sponsor may be an individual or a pharmaceutical company, governmental agency, academic institution, private organization, or other organization.’’ The 21 Code of Federal Regulations, part

GOOD CLINICAL PRACTICE (GCP)

5

Compound Success Rates by Stage: Discovery: (2 to 10 years)

5,000 to 10,000 Screened

Preclinical Testing: Laboratory and animal testing

250 enter Preclinical testing

Phase I: 20 to 80 volunteers used to determine safety and dosage

5 enter Clinical testing:

Phase II: 100 to 300 volunteers; monitor for efficacy and side effects Phase III: 1,000 to 5,000 volunteers; monitor for adverse reactions to long-term use 1 FDA Approval

FDA Review/Approval Postmarket Testing 0

2

4

6

8 10 Years

12

14

16

Figure 1.

312 (5) (drugs) and part 812 (6) ( biologicals) contain the regulations that include the sponsors’ obligations. The obligations of sponsors include that they perform the following duties:

9. Ensure that the clinical investigation is conducted in accordance with the general investigational plan and protocols contained in the IND. 10. Maintain a current IND with respect to the investigations. 11. Evaluate and report adverse experiences. 12. Ensure that FDA and all participating investigators are promptly informed of significant new adverse effects (SADEs) or risks with respect to the drug.

1. Ensure that all clinical studies are conducted according to the regulations and an approved study protocol. (Both the FDA and an IRB must approve the study protocol.) 2. Obtain regulatory approval, where necessary, before studies begin. 3. Manufacture and label investigational products appropriately. 4. Initiate, withhold, or discontinue clinical investigations as required. 5. Refrain from commercialization investigational products.

of

6. Select qualified investigators to conduct the studies. 7. Provide the investigators with the information they need to conduct an investigation properly (training, written guidance including the investigators brochure). 8. Ensure proper monitoring of the clinical investigation.

Sponsors are responsible for reviewing data from the studies and from all other sources as soon as the materials are received to ensure that the safety of the subjects is not compromised. In any case, where the compliance of the investigator (or investigator site) has deviated from the approved protocol or regulations, it is the responsibility of the sponsor to ensure the site is either brought into compliance or, if compliance cannot be attained, to stop the investigation and assure the return of all clinical supplies. 9

CONTRACT RESEARCH ORGANIZATION

Contract Research Organization (CRO) is defined by FDA [(21 CFR 312.3 (b) (5)] as

6

GOOD CLINICAL PRACTICE (GCP)

‘‘a person that assumes, as an independent contractor with the sponsor, one or more of the obligations of a sponsor, e.g. design of a protocol, selection or monitoring of investigations, evaluation of reports, and preparation of materials to be submitted to the Food and Drug Administration.’’ The transfers of obligations to a CRO must be described in writing. If all obligations are not transferred, then the sponsor is required to describe each of the obligations being assumed by the CRO. If all obligations are transferred, then a general statement that all obligations have been transferred is acceptable [21 CFR 312.52 (5)] 10

MONITORS

Monitors are persons who oversee the progress of a clinical investigation and ensure that it is conducted, recorded, and reported accurately. The purposes of monitoring are to verify that the rights and well-being of the subjects are protected; the reported trial data are accurate, complete, and verifiable from source documents; and the conduct of the trial complies with the currently approved protocol amendments, GCP, and all applicable regulatory requirements. It is the responsibility of the sponsor to ensure that the clinical investigations are monitored adequately. Monitors are selected by the sponsor and must be adequately trained; have the scientific and/or clinical knowledge needed to monitor the trial adequately; and be thoroughly familiar with the investigational product, protocol, informed consent form, and any other materials presented to the subjects, the sponsors Standard Operating Procedures (SOPs), GCP, and all other applicable regulatory requirements. The extent and nature of monitoring should be set by the sponsor and must be based on multiple considerations such as the objective, purpose, design, complexity, blinding, size, and endpoints of the trial. Typically, on-site monitoring occurs before, during, and after the trial. 11

INVESTIGATORS

Investigators are the persons who actually conduct the clinical investigation (i.e., under

whose immediate direction the drug is administered or dispensed to a subject). In the event a team of individuals conducts an investigation, the investigator is the responsible leader of the team. ‘‘Subinvestigator’’ includes any other individual member of that team 21 CFR 312.3(b) (5)]. Investigators have several responsibilities, which include the following: 1. To ensure the investigation is conducted according to all applicable regulations and the investigational plan 2. To ensure the protection of the subjects’ rights, safety, and welfare 3. To ensure control of the drugs under investigation

12 DOCUMENTATION Acceptable documentation, from the sponsor, monitor, investigator, IRB, and so on, is critical for ensuring the success of clinical investigations. Sponsors and monitors must document their procedures and policies. SOPs are commonly used and must cover all aspects of their operations including Quality Assurance and monitoring. Investigators also require written procedures and CRFs. It is essential for investigators to assure the CRFs are up to date and complete and to have source documents (laboratory reports, hospital records, patient charts, etc.) that support the CRFs. All documentation in support of an NDA must be retained. FDA regulations require that for investigational drugs, all records and reports required during the studies must be retained for at least 2 years after the marketing application is approved. If the application is not approved, all applicable records should be retained for 2 years after the shipment of the study drug is discontinued and the FDA notified. The quality of the documentation is always critical. Records, including the CRFs must be legible; corrections should be cross outs with dates and initials and never obliterate the original data. The records should be controlled and retrievable for the monitors, FDA inspection, and other quality assurance

GOOD CLINICAL PRACTICE (GCP)

needs. Records must be credible and complete. They need to convince third parties (monitors, auditors, FDA Investigators, and reviewers) of their merit. The presence of accurate and reliable data and information in an application submitted to the FDA for scientific review and approval is essential. If a submission is misleading because of the inclusion of incorrect or misleading data, the FDA may impose the Application Integrity Policy (AIP). The AIP deals with applications that contain unreliable or inaccurate data. This policy enables the agency to suspend review of an applicant’s pending submitted applications that are in the review and approval process at FDA when a pattern of data integrity problems have been found in one or more of those applications and those data integrity problems are determined to be material to the review. Where data is collected electronically, such as in the use of electronic patient diaries, the FDA regulations on Electronic Records and Electronic Signatures [21 CFR Part 11 (7)] must be followed. These rules require many controls to ensure the authenticity, integrity, and confidentiality of electronic records. The required controls include (among many others) the validation of the system(s) assuring accuracy, reliability, and the ability to discern invalid or altered records. 13

CLINICAL HOLDS

A clinical hold is an order issued by the FDA to the sponsor to delay a proposed clinical investigation or to suspend an ongoing investigation. Clinical holds may apply to one or more of the investigations covered by an IND. When a proposed study is placed on clinical hold, subjects may not be given the investigational drug. When an ongoing study is placed on clinical hold, no new subjects may be recruited to the study and placed on the investigational drug; in the interest of patient safety, patients already in the study should be taken off therapy that involves the investigational drug unless specifically permitted by the FDA (2). Clinical holds are imposed when the FDA finds issues that may negatively affect patient safety, which include the following:

7

1. Human subjects are or would be exposed to an unreasonable and significant risk of illness or injury. 2. The investigator(s) are not qualified. 3. The investigators brochure is misleading, erroneous, or materially incomplete. 4. The IND does not contain sufficient information to assess the risks to the subjects. For Phase 2 and 3 studies, clinical holds may also be imposed if the plan or protocol is clearly deficient in design to meet its stated objectives. 14

INSPECTIONS/AUDITS

Audits and regulatory inspections are conducted to ensure that the clinical investigations are conducted correctly, data is accurately recorded and reported, and all procedures and applicable rules and regulations are followed. During the course of the study, the sponsor’s Clinical Quality Assurance staff generally conducts audits. These audits cover the Investigator(s) sites, data collection, and other sites that are pertinent to the clinical investigation. Any problems or issues revealed during the course of these audits typically result in corrections that enable the study to proceed without any major incidents or delays. Regulatory inspections, such as those conducted by FDA, are typically conducted after the sponsor files for marketing approval such as an NDA (the regulatory bodies may conduct inspections during the course of the study although it is generally only done when an issue develops). Regulatory bodies typically inspect a sampling of clinical sites from any given marketing application. Inspections of other facilities, such as sponsors, monitors, Institutional Review Boards, and so on, are typically performed from a sampling of all such facilities in their establishment inventory. The FDA has established a Bioresearch Monitoring Program that is used to provide oversight of the conduct of clinical studies. FDA has several Compliance Programs (CPs)

8

GOOD CLINICAL PRACTICE (GCP)

that provide guidance and specific instruction for inspections of: Investigators (CP7348.811) Sponsors/Monitors (CP7348.810) Institutional Review Boards (CP7348.809) Nonclinical Laboratories (CP7348.808) These CPs are available at www.fda.gov/ ora/cpgm. REFERENCES 1. International Conference on Harmonization, (ICH-E6), Good Clinical Practice: Consolidated Guidance, 1996. Available: www.ich.org. 2. 21 Code of Federal Regulations, Part 50, Protection of Human Subjects. 3. FD&C Act, Section 505. 4. 21 Code of Federal Regulations, Part 58. 5. 21 Code of Federal Regulations, Part 312, Investigational New Drug Application. 6. 21 Code of Federal Regulations, Part 812. 7. 21 Code of Federal Regulations, Part 11, Electronic Records and Signatures.

FURTHER READING Guideline for Industry Clinical Safety Data Management: Definition and Standards for Expedited Reporting (ICH-E2A), Federal Register, March 1, 1995 (60 FR 11284). W. K. Sietsema, Preparing the New Drug Application: Managing Submissions Amid Changing Global Requirements. FDAnews. Washington, DC, 2006. W. K. Sietsema, Strategic Clinical Development Planning: Designing Programs for Winning Products. FDAnews. Washington, DC, 2005. Food and Drug Administration. Available: http://www.fda.gov. Food and Drug Administration; Center for Drug Evaluation and Research. Available: http:// www.fda.gov/cder. Department of Health and Human Services. Available: http://www.os.dhhs.gov. European Union. Available: http://www.europa. eu.int. International Conference on Harmonization. Available: http://www.ich.org.

GOOD LABORATORY PRACTICE (GLP)

The regulations establish standards for the conduct and the reporting of nonclinical laboratory studies and are intended to assure the quality and the integrity of safety data submitted to the FDA. The FDA relies on documented adherence to GLP requirements by nonclinical laboratories in judging the acceptability of safety data submitted in support of research and/or marketing permits. The FDA has implemented this program of regular inspections and data audits to monitor laboratory compliance with the GLP requirements.

The Federal Food, Drug, and Cosmetic Act and Public Health Service Act require that sponsors of Food and Drug Administration(FDA)-regulated products submit evidence of their product’s safety in research and/or marketing applications. These products include food and color additives, animal drugs, human drugs and biological products, human medical devices, diagnostic products, and electronic products. The FDA uses the data to answer questions regarding: • The toxicity profile of the test article. • The

observed no-adverse-effect dose level in the test system. • The risks associated with clinical studies that involve humans or animals. • The potential teratogenic, carcinogenic, or other adverse effects of the test article. • The level of use that can be approved. The importance of nonclinical laboratory studies to the FDA’s public health decisions demands that they be conducted according to scientifically sound protocols and with meticulous attention to quality. In the 1970s, FDA inspections of nonclinical laboratories revealed that some studies submitted in support of the safety of regulated products had not been conducted in accord with acceptable practice; accordingly, data from such studies were not always of the quality and of the integrity to assure product safety. As a result of these findings, the FDA promulgated the Good Laboratory Practice (GLP) Regulations, 21 CFR (Code of Federal Regulations) Part 58, on December 22, 1978 (43 FR (Federal Register) 59986). The regulations became effective June 1979. This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/ora/compliance ref/bimo/7348 808/part I.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

GOODNESS OF FIT

continuous or a step function, later discussion will deal with continuous and discrete X i separately. There has been a recent resurgence of interest in the theory of goodness-of-fit tests. Technical advances have been made with some of the older tests, while new tests have been proposed and their power properties examined. This progress can be attributed in part to the availability of mathematical development in the theory of probability* and stochastic processes*. However, it is also in large measure due to the advent of the highspeed computer, the associated numerical technology, and the increased demand for statistical services. This article can only summarize some of the available results and refer the reader to special sources for further detail. Many statistical texts have introductory chapters on goodness-of-fit testing. For example, Kendall and Stuart [18, Vol. 2] and Lindgren (21) contain pertinent material lucidly presented. Pearson and Hartley (25) also contains accounts of specific tests illustrated by numerical examples. The following general notation will be adopted, additional special symbols being introduced as required:

G. M. TALLIS In general, the term ‘‘goodness of fit’’ is associated with the statistical testing of hypothetical models with data. Examples of such tests abound and are to be found in most discussions on inference*, least-squares* theory, and multivariate analysis.* This article concentrates on those tests that examine certain features of a random sample to determine if it was generated by a particular member of a class of cumulative distribution functions* (CDFs). Such exercises fall under the broad heading of hypothesis testing*. However, the feature that tends to characterize these ‘‘goodness-of-fit tests’’ is their preoccupation with the sample CDF, the population CDF, and estimates of it. More specifically, let X 1 , X 2 , . . . , X n be a random sample generated by CDF GX (x). It is required to test H0 : GX (x) = FX (x, θ ),

θ ∈
,

(1)

where θ is a q-dimensional vector of parameters belonging to the parameter space
. If θ is fixed at some value θ 0 , say, then F X (x, θ 0 ) = F X (x) is fully specified and H 0 is simple. Otherwise, the hypothesis states that GX (x) is some unspecified member of a family of CDFs and is composite*. As an example, consider the normal* family N X (x;θ ), θ  = (θ 1 , θ 2 ), where θ 1 is the mean and θ 2 the variance of N X . In this case
= (−∞, ∞) × (0, ∞) and it might be required to test whether or not a sample was generated by N X (x;θ ) for some unknown θ ∈
. Intuitively, and in fact, this is an intrinsically more difficult problem than testing whether the sample was generated by a particular normal CDF with known mean and variance. The latter case can always be reduced to the standard situation of testing GX (x) = N(x;θ 0 ), θ  0 = (0, 1). Most useful tests are parameter-free; i.e., the distribution of the test statistics does not depend on θ . Among such tests are found both parametric and nonparametric tests which are either distribution specific or distributionfree. Since tests may require F(x, θ ) to be

Probability density function* (PDF) corresponding to F X (x, θ ) and GX (x) (when they exist): f X (x, θ ), gX (x). Order statistics*: X i ≤ X  2 ≤ . . . ≤ X  n . Expected values of order statistics: E[X  i ] = ηi . Sample CDF: Gn (x) = [no. of X i ≤ x]/n. A chi-square random variable with d degrees of freedom: χ 2 (d). 100(1 − α) percentile of the chi-square distribution* with d degrees of freedom: 2 (d) . χ1−α The uniform density on [0, 1]: U [0, 1]. If X n is a sequence of random variables, L

then Xn → χ 2 (d) will indicate convergence in law to a chi-square distribution with d degrees of freedom; if X n is a sequence of ranL

dom vectors, then Xn → N(µ, ) will indicate

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

GOODNESS OF FIT

convergence* in law to a normal distribution with mean vector µ and covariance matrix*. 1

DISCRETE RANDOM VARIABLES

1.1 Simple H0 Suppose that X is a discrete random variable, Pr X = x = f X (x, θ ) for θ ∈
and x ∈ X, where X is a finite or countable set of real numbers. By suitable definition, categorical data* with no evident numerical structure can be brought within this framework. The simplest case is where X = 1, 2, . . . , k, f X (j, θ 0 ) = f j is fully specified, and it is required to test H 0 : gX (j) = f j . Let N j be the number of X i in the sample such that X i = j; then the probability under H 0 of obtaining the particular
outcome vector, n = n1 , n2 , . . . , nk , where j nj = n, is n!  nj fj . P(n) =  j nj !

(2)

j

An exact test of H 0 can, in principle, be constructed as follows: 1. Calculate P(m) for all possible outcome vectors, m. 2. Order the P(m). 3. Sum all P(m) which are less than or equal to P(n). 4. Reject H 0 at level α if the cumulative probability of 3 is less than or equal to α. This is known as the multinomial test of goodness of fit, and the necessary calculations must be carried out on a computer. Even then, this is only practicable if n and k are small. Fortunately, for large n, there are ways around the distributional problem. Likelihood ratio theory can be invoked; the likelihood ratio test* of H 0 : gX (j) = f j against the alternative H 1 : gX (j) = f j is formed from the  ratio  = kj=1 (nfj /Nj )Nj . It is known that −2 ln  = −2

k 

  Nj ln nfj − ln Nj

j=1 L

→ χ 2 (k − 1).

The null hypothesis is rejected at level α if the calculated value of −2ln exceeds χ 1−α (k − 1). A very old test dating back to the beginning of this century is based on X2 =

k 

(Nj − nfj )2 /(nfj ).

(3)

j=1

This is known as Pearson’s chi-square and has the same limiting distribution as (2). Since N j is the observed number of X i in the sample with X i = j, (Oj ), and E[N j ] = nf j = (Ej ), (3) is sometimes written in the form of a mnemonic, X2 =

k  (Oj − Ej )2 /Ej .

(4)

j=1

Not only do (2) and (3) share the same limiting central distribution, but they are also asmptotically equivalent in probability. However, since X 2 is a direct measure of agreement between observation and expectation under H 0 , it has some intuitive appeal not shared by (2). Both (2) and (3) give asymptotic tests which tend to break down if the nf j are too small. A common rule is that all nf j should be greater than 1 and that 80% of them should be greater than or equal to 5. These conditions are sometimes hard to meet in practice. For a general discussion and further references, see Horn (16). Radlow and Alf (29) point out that a direct comparison of X 2 with the multinomial test may be unjustified. The latter test orders experimental outcomes, m, in terms of P(m) instead of ordering them in terms of discrepancies from H 0 . It is suggested that X 2 should be compared with the following exact procedure: 1. Calculate P(m) for all possible outcomes m. 2. Calculate X 2 for each m based on H 0 , X 2 (m). 3. Sum P(m) for which X 2 (m) ≥ X 2 (n), the observed X 2 value. 4. Reject H 0 at level α if this sum exceeds α.

GOODNESS OF FIT

Numerical comparisons of this exact test with X 2 showed that the two agreed remarkably well, even for small n. The agreement of the exact multinomial test with X 2 , on the other hand, was poor. Thus care must be exercised in the assessment of the performances of large-sample tests using small n. Appropriate baseline exact tests must be used for comparisons. Another procedure is the discrete Kolmogorov–Smirnov* goodness-of-fit test (5). Let D− = maxx (FX (x) − Gn (x)),

The philosophy adopted in the preceding subsection for rejecting H 0 is used here, the 2 (k − q − 1). critical level being χ1−α Although (4) and (8) are the standard tests recommended in textbooks and have received the most attention by practitioners and theoreticians, there are others. For example, a general class of goodness-of-fit tests can be based on quadratic form* theory for multinormally distributed random variables. Under sufficient regularity conditions the following results can be established by routine methods of probability calculus. Put

D+ = maxx (Gn (x) − FX (x)), −

N = (N1 , N2 , . . . , Nk ),

+

D = max(D , D ); then D− , D+ , and D test, respectively, H 1 : GX (x) ≤ F X (x), H 1 : GX (x) ≥ F X (x), and H 1 : GX (x) = F X (x). A discussion of the application of these statistics is given in Horn (16), where their efficiency relative to X 2 is discussed and a numerical example given. For more recent asymptotic results, see Wood and Altavela (39). 1.2 Composite H0 When H 0 : GX (x) = F X (x, θ ), θ ∈
, is to be tested, θ must be estimated and the theory becomes more elaborate. However, provided that asymptotically efficient estimators θˆ n are used, tests (2) and (3) extend in a natural way and continue to be equivalent in probability. More specifically, since X i is assumed discrete, put H 0 : gX (j) = f j (θ ) and H 1 : gX (j) = f j (θ ), θ ∈
. Let θˆ n be as above; e.g., θˆ n could be the maximum likelihood estimator* (MLE) for θ under H 0 . Then under H 0 , −2 ln (θˆ n ) = −2

k  j=1

→ χ 2 (k − q − 1), X 2 (θˆ n ) =

k 

(Nj − nfj (θˆ n ))2 /(nfj (θˆ n ))

j=1 L

Ni = Ni /n, (f(θ)) = (f1 (θ), f2 (θ), . . . , fk (θ )); then

√

L

n(N − f (θ)) → N(0, V), where V = [υij ], υii = fi (θ )(1 − fi (θ )), υij = −fi (θ)fj (θ ), i = j,

and rank (V) = k − 1. Now suppose that θ ∗n is any estimator for θ which can be expressed in the locally, suitably regular functional form θ ∗n = g(N). Let D = [dij ], dij = ∂gi (N)/∂Nj , i = 1, 2, . . . , q; j = 1, 2, . . . , k, Q = [qrs ], qrs = ∂fr (θ )/∂θs , r = 1, 2, . . . , k; s = 1, 2, . . . , q. Then √ P √ n(N − f (θ ∗n )) → n(I − QD)(N − f (θ )) L

→ N(0, ),

Nj [ln nfj (θˆ n ) − ln Nj ]

L

→ χ 2 (k − q − 1).

3

where  = (I − QD)V(I − QD) . If  g is any generalized inverse* of  (i.e.,  g  = ), it then follows that, under H 0 , Qn (θ ∗n ) = n(N − f (θ ∗n ))  g (N − f (θ ∗n )) L

→ χ 2 (k − q − 1).

4

GOODNESS OF FIT

The power of tests such as (5), which include (8) as a special case, can also be examined using sequences of local alternatives, similar arguments, and the noncentral chi-square* distribution. However, such studies are of limited use, as tests of fit should detect broad alternatives, a performance feature that can be checked only by computer simulations. Note that  g may be a function of θ , which can be replaced by θ ∗n or any other consistent estimator for θ without affecting the asymptotic distribution of Qn (θ ∗n ). For an early, rigorous account of the theory of chi-square tests*, see Cram´er (6). A modern and comprehensive treatment of this theory with many ramifications is given by Moore and Spruill (24). Their paper is technical and covers cases where the Xi are random vectors and θ is estimated by a variety of methods. An easy-to-read over-view of this work is given by Moore (23). 2

CONTINUOUS RANDOM VARIABLES

The testing of goodness-of-fit hypotheses when f X (x, θ ) is a continuous function of x introduces features not exhibited by the discrete tests discussed in the section ‘‘Discrete Random Variables.’’ However, by suitable constructions some of the latter tests remain useful. These points will be expanded on below. 2.1 Simple H0 In order to apply results of the section ‘‘Discrete Random Variables,’’ partition the real line into k ≥ 2 sets: I1 = (−∞, a1 ],

Ij is used in the tests. The k classes are usually chosen to keep the npj acceptably high. In order to achieve some standardization, it seems reasonable to use pj = k−1 and to determine the ai by the equations F X (a1 ) = k−1 , F X (a2 ) − F X (a1 ) = k−1 , etc. (see Kendall and Stuart (18)). Nevertheless, there remains an essential nonuniqueness aspect to the tests. Given the same set of data, different statisticians can reach different conclusions using the same general procedures. In fact, these tests condense the data and examine whether or not gX (x) is a member of the particular class of density functions with given content pj for Ij . Despite these drawbacks, the approach outlined above has enjoyed wide support and is most commonly used in practice. The method of condensation of data presented above when X is a continuous random variable may also have to be practiced when X is discrete. In this case subsets of X are used in place of individual elements to achieve cell expectations sufficiently large to render the asymptotic distribution theory valid. A useful way of visually checking the adequacy of H 0 is to examine the order statistics X  1 , X  2 , . . . , X  n . Since f X (x) is fully specified, E[X  i ] = ηi can be calculated and plotted against X  i . If H 0 holds, this plot should be roughly linear. There are analytical counterparts to the simple order statistics plots. Let 0 < λ1 < λ2 < · · · < λk < 1, ni = [nλi ] + 1, where [x] is the greatest integer less than or equal to x, and consider Xn i , i = 1, 2, . . . , k. Under suitable regularity conditions on f X (x),

I2 = (a1 , a2 ], . . . ,

Ik = (ak−1 , ∞). To test H 0 : gX (x) = f X (x), let N j be the number of X i ∈ Ij in the sample and put pj = Ij f X (x)dx. Then, under H 0 , the N j have a multinomial distribution* with parameters
pj , n = kj=1 Nj and any of the tests from the discussion of simple H 0 in the section ‘‘Discrete Random Variables’’ can be applied. Clearly, making a situation discrete which is essentially continuous leads to a loss of precision. The actual values of the X i are suppressed and only their relationship with the

Y2 = n

k  {[FX (Xn i ) − FX (Xn i−1 )] − pi }2 p−1 i i=1

L

→ χ 2 (k − 1),

where pi = λi − λi−1 (4). This bypasses the problem of constructing intervals Ij and uses part of the natural ordering of the sample. A number of tests make specific use of the sample CDF, Gn (x), for testing H 0 : GX (x) = F X (x). Some of these are now listed.

GOODNESS OF FIT

2.1.1 Kolmogorov–Smirnov* Dn − , Dn + , and Dn . Define

Statistics,

D+ n

Wn2 = n

= supx [Gn (x) − FX (x)],

Dn = max(Dn − , Dn + ). Then Dn − and Dn + can be used to test H 0 against the one-sided alternatives H 1 : GX (x) ≤ F X (x) and H 1 : GX (x) ≥ F X (x), respectively, while Dn tests H 1 : GX (x) = F X (x). The CDFs of the three statistics are known exactly and are independent of F X (x) (8). To see this, let U = F X (X); then Dn + = sup0≤u≤1 Gn (u) − u, etc. The most useful set of tables is given by Pearson and Hartley (25), who also include some numerical examples. A derivation of the asymptotic distributions of Dn + and Dn can be based on the stochastic process yn (t) =

√ n(Gn (t) − t),

0 ≤ t ≤ 1,

(5)

which has zero mean and C[y(s), y(t)] = min(s, t) − st,

0 ≤ s, t ≤ 1. (6) The central limit theorem* ensures that [yn (t1 ), yn (t2 ), . . . , yn (tk )] is asymptotically multinormal with null mean vector and the above covariance structure. Thus the finitedimensional distributions of Y n (t) converge to those of y(t), tied-down Brownian motion*. Intuitively, the distributions of supt yn (t) and supt —yn (t)— will tend to those of supt y(t) and supt —y(t)—. This can be verified using the theory of weak convergence. The two crossing problems thus generated can be solved to yield the desired limiting CDFs (8). For a different approach, see Feller (13). It is interesting that these investigations show 4n(Dn + )2 to be asymptotically distributed as χ 2 (2). 2.1.2 Cram´er–von Mises Test*. Let  ∞ [Gn (X) − FX (x)]2 dx; Wn2 = n

By means of the probability transformation U = F X (X), (8) can be written 

D− n = supx [FX (x) − Gn (x)],

(7)

−∞

then Wn2 is a measure of the agreement between Gn (x) and F X (x) for all x and is known as the Cram´er–von Mises statistic.

5

1

[Gn (u) − u]2 du,

(8)

0

emphasizing that this test is also distribution-free. The CDF of Wn2 is not known for all n but has been approximated; the asymptotic distribution is derived in Durbin (8). For easy-to-use tables, see Pearson and Hartley (25). 2.1.3 Tests Related to the Cram´er–von Mises Test. Various modifications of Wn2 are used for specific purposes. For instance, a weight  1 function ψ(t) can be introduced to give 0 [Gn (t) − t]2 ψ(t) dt as a test statistic. When ψ(t) = [t(1 − t)]−1 , the resulting statistic is called the Anderson–Darling statistic, A2n , and leads to the Anderson–Darling test*. Since E[n[Gn (t) − t]2 ] = [t(1 − t)], this weights discrepancies by the reciprocal of their standard deviations and puts more weight in the tails of the distribution, a feature that may be important. The same remarks made for Wn2 apply to A2n . A number of scientific investigations yield data in the form of directions and it may be required to test the hypothesis that these are orientated at random. Since each direction is represented by an angle measured from a fixed position P, such data can be represented as points on a unit circle. The test then concerns the randomness of the distribution of the points on this circle. Watson (38) introduced the statistic  Un2 = n

1

[Gn (t) − t − Gn (t) − t]2 dt,

(9)

0

1 where 0 [Gn (t) − t]dt = Gn (t) − t. It can be shown that Un2 is independent of the choice of P. The asymptotic distribution of Un2 is known (8) and appropriate tables may be found in Pearson and Hartley (25). Under H 0 : GX (x) = F X (x), the variables U i = F X (X  1i ) are distributed as U [0, 1]. Hence Gn = 0 Gn (u) du = 1 − U, has expectation 12 , variance (12n)−1 and tends rapidly to normality. This provides a direct large-sample test

6

GOODNESS OF FIT

of H 0 , although exact significance points are available (34). Tests related to Dn − , Dn + , Dn , and Wn2 have been proposed by Riedwyl (31). He defines the ith discrepancy as di = F(X  i )  ) and examines tests based on − F n (X i
n
n 2
n d , etc. 1 i 1 |di |, 1 di , maxi di , maxi |di |, Some pertinent exact and asymptotic results are given. Hegazy and Green (15) considered tests
based on the forms T1 = n−1 n1 |Xi − νi | and
T2 = n−1 n1 (Xi − νi )2 , where ν i = ηi and ν i = ξ i , the mode of X  i . Tests of the hypothesis H 0 : GX (x) = F X (x) can be reduced as shown above to testing whether or not U i = F X (X i ) is distributed U [0, 1]. Thus ηi = i/(n + 1) and ξ i = (i − 1)/(n − 1). The powers of these T tests were examined against normal*, Laplace*, exponential*, and Cauchy* alternatives and compared with the powers of other tests. The conclusion was that T 1 and T 2 have similar performances and that it is slightly better to use ξ i than ηi . These T statistics generally compare favorably with the tests just described, or minor modifications of them. Hegazy and Green (15) provide an extensive bibliography of other studies of power of goodness-of-fit tests. 2.2 Composite H0 The most common hypothesis that requires testing is H 0 : GX (x) = F X (x, θ ) for some θ ∈
. The introduction of nuisance parameters* creates new technical difficulties which can only be touched on briefly here. In general, however, the same form of tests as those just presented are used, with modifications. In order to make use of the results in the discussion of composite H 0 in the section ‘‘Discrete Random Variables,’’ k intervals are introduced as in the preceding subsection. The interval contents are functions of θ ,  pj (θ ) = I f (x, θ ) dx and if N j is the number j of X i in Ij , a multinomial system is generated, the parameters being functions of the unknown θ . The whole problem may now be treated by the methods of the section on discrete variables, and the same comment concerning loss of information and nonuniqueness due to grouping applies.

A number of special points need emphasis. The estimation of θ must be made from the data in the grouped state if the distribution theory of the section on discrete variables is to hold. For instance, θ estimated from the X i and f (x, θ ) should not be used in the X 2 (θ ) statistic. Doing so results in a limiting CDF which depends on θ and a conservative test if 2 (k − q − 1) significance level is used. the χ1−α Since θ is not known, there is some difficulty defining the intervals Ij . In general, the boundaries of the intervals are functions of θ ; Moore and Spruill (24) have shown that, provided that consistent estimators of the boundary values are used, the asymptotic results (4), (8), and (5) remain valid if the random intervals are used as if they were the true ones. For example, reconsider the problem of testing H 0 : GX (x) = N X (x, θ ). Consistent
estimators of θ 1 = µ and θ 2 = σ 2 are X = n1 Xi /n
and S2 = n1 (Xi − X)2 /(n − 1) and it is appropriate that the Ij be constructed with X and S in place of µ and σ to ensure approximate contents of k−1 . Using these estimated intervals, the procedure requires that µ and σ 2 be estimated efficiently, by maximum likelihood, for instance, and the tests applied in the usual way. A test developed by Moore (22) and suggested by Rao and Robson (30) has interesting flexibility and power potential. Let Vn (θ ) be a k-vector with ith component (Ni − nfi (θ))/ nfi (θ), B(θ ) a k × q matrix with elements pi (θ )−1/2 ∂pi (θ )/∂θ j , and J(θ ) the usual information matrix for F X (x, θ ). Define the statistic Tn (θˆ n ) = Vn (θˆ n )[I − B(θˆ n )J−1 (θˆ n ){B(θˆ n )} ]−1 ×Vn (θˆ n ), where θˆn is the ungrouped MLE for θ ; then L Tn (θˆ n ) → χ 2 (k − 1). The problem of estimating intervals can be bypassed by the use of quantile* statistics. Define λi and pi as for (6), and the statistic Yn2 (θ ) = n

k  {[FX (Xn i , θ ) 1

−FX (Xn i−1 , θ)] − pi }2 p−1 i .

GOODNESS OF FIT L If θ = θ n minimizes Yn2 (θ), then Y 2 (θ˜ n ) → χ 2 (k − q − 1). Alternatively, the following test is available. Put Xni /n = N∗i and let N* be the (k × 1) vector of the N∗i ; then it is well known that √ L n(N∗ − ν) → N(0, V), where ν i is defined by F X (ν i ) = λi and υ ij = λi (1 − λj )[f X (ν i )f X (ν j )]−1 , i ≤ j. In general, both ν and V are functions of unknown θ , so define

An (θ ) = n(N∗ − ν(θ)) V−1 (θ ) ×(N∗ − ν(θ)) and choose θ = θ ∗n to minimize An (θ ). Then L

An (θ ∗n ) → χ 2 (k − q), k > q. If q = 2, θ 1 and θ 2 are location and scale parameters, respectively, and an explicit expression exists for θn∗ . The matrix V for the standardized variable (X − θ 1 )/θ 2 can be used in (12) and a single matrix inversion is needed to complete the test (37). The tests described in the discussion of simple H 0 do not extend readily to composite hypotheses. In general, for the cases considered and reported in the literature to date, the resulting tests are not distribution-free but depend on F(x, θ ) and on the method used ˆ This is because the CDF to estimate θ , θ. has a different limiting distribution when the parameters are estimated to that which results when the null hypothesis is simple (9). Hence tables of critical values constructed for simple hypothesis cases cannot be used for testing composite hypotheses. In fact, different critical values are needed for each hypothesis tested; the tests are carried out ˆ in the expressions replacing F X (x) by F(x, θ) of the preceding section. 2.2.1 Kolmogorov–Smirnov Statistics Dn − , A technique for obtaining exact critical values was developed by Durbin (10), who applied it to obtain values for testing the composite hypothesis of exponentiality

Dn + , Dn .

H0 : f (x, θ ) = θ −1 exp(−x/θ ), 0 < x, θ ∈ (0, ∞). The technique is complicated, however, and has not been applied to other cases.

7

By a variety of techniques, including Monte Carlo methods*, Stephens (35) has given procedures for finding accurate critical values for testing composite hypotheses involving the normal and the exponential distributions. These procedures are also described by Pearson and Hartley (25). For a treatment of this problem using sufficient statistics*, see Kumar and Pathak (20). 2.2.2 Cram´er–von Mises Statistic∗ Wn2 . No technique is yet available for obtaining exact critical values of Wn2 for testing composite hypotheses. The first accurate calculations of asymptotic significance points for testing exponentiality and normality were made by Durbin et al. (12). Further extensions and related results were given by Stephens (36). Again, methods of obtaining good approximations to finite-sample critical values for tests of exponentiality and normality are given by Stephens (35). 2.2.3 Tests Related to the Cram´er–von Mises Tests. Similar treatments to those of Wn2 are given to A2n and Un2 by Stephens (35,36) and Durbin et al. (12) for testing exponentiality and normality. In summary, then, the development of tests of fit for composite hypotheses using the sample CDF has centered largely on the exponential and the normal distributions. The most useful reference for the practitioner is Stephens (35), where tables cater for most of the common tests when the hypothesis is simple, and for the composite hypothesis cases mentioned above. If the data are censored*, see Pettitt and Stephens (27) and Dufour and Maag (7). 3

FURTHER TESTS AND CONSIDERATIONS

In this final section a few other special tests and some additional ideas impinging on goodness of fit will be mentioned. 3.1 Two Sample Tests Let X 1 , X 2 , . . . , X m and Y 1 , Y 2 , . . . , Y n , m ≤ n, be random samples from two different populations with continuous CDFs F X (x) and F Y (y) and sample CDFs FXm (t) and FYn (t).

8

GOODNESS OF FIT

In analogy with the discussion on the Kolmogorov–Smirnov statistics Dn − , Dn + , and Dn pertaining to simple H 0 in the section ‘‘Continuous Random Variables,’’ the hypothesis H 0 : F X (t) = F Y (t) can be tested against the alternatives H 1 : F X (t) ≤ F Y (t), H 1 : F X (t) ≥ F Y (t), and H 1 : F X (t) = F Y (t) by the respective statistics

the power properties. Nevertheless, a general pattern emerged; singly, the two statistics are useful for detecting departures from specific types of alternatives, and in combination they are reasonably robust against a large variety of alternatives.

n m D− mn = supt [FY (t) − FX (t)],

There are fruitful extensions to the technique of plotting order statistics* against expectation as introduced in the discussion of simple H 0 in the section ‘‘Continuous Random Variables.’’ Let the hypothesis be

m n D+ mn = supt [FX (t) − FY (t)], + Dmn = max[D− mn , Dmn ].

The exact distributions of these statistics are known; for finite sample critical points of Dmn , see Pearson and Hartley (25). For further references to tabulations, see Steck (33). If the statistics above are multiplied by [mn(m + n)−1 ]1/2 limiting distributions √ exist n Dn − , which are the same as those for √ √ + n Dn , and n Dn . Similar modifications can be made to the Cram´er–von Mises statistic to cater for two sample tests; again, see Durbin (8). 3.2 Tests of Departure from Normality* In view of the central role of the normal distribution in statistical theory and practice, a great deal of effort has been spent in developing tests of normality (see the first section of this article). Some of these tests have been dealt with in previous sections; only special tests tailored for the normal distribution will be covered here.
Let mr = n−1 n1 (Xi −√X)r and S2 = nm2 (n − 1)−1 ; then statistics b1 = m3 /S3 and b2 = m4 /S4 measure skewness and kurtosis in the sample. If the population √ from which the sample is drawn is normal, b1 and b2 should be near 0 and 3, respectively, and departure from these values is evidence to the contrary. √ Both b1 and b2 have been examined separately, jointly, and as a linear combination (see Pearson and Hartley (25) and Pearson et al. (26)). Both were compared with other tests for power; a variety of skewed and leptokurtic distributions were used as alternatives. The picture is somewhat confused, due in part to the wide spectrum of alternative distributions used and to the use of small numbers of Monte Carlo trials to establish

3.3 Wilks–Francia Test*

H0 : G(x, θ ) = F((x − θ1 ) θ2 ),

(10)

i.e., G is determined up to a location and scale parameter. Then a plot of X  i against the expectation of the standardized order statistics, ηi , should lie near the line θ 1 + θ 2 ηi under H 0 . Now, the unweighted esti
least-squares
mator for θ 2 is θ 2 = n1 Xi ηi / n1 ηi2 and the residual sum of squares is R2n =

n 

(Xi − X − θ 2 ηi )2

1

=

n 

 (Xi

− X) − 2

1

n 

2 bi Xi

,

1


where bi = ηi /( ni ηi2 )1/2 . Dividing both sides
n  2 by 1 (Xi − X) to remove the scale effect yields  n 

−1 (Xi

1

=1−

− X) 

2

n  1

R2n 2

bi Xi

n



(Xi − X)2

1

= 1 − Wn . Then W  n is the Wilks–Francia test statistic and measures the departure of the order statistics from their expectations; it has been used to test normality specifically, but it clearly enjoys a wider application. To carry out the test, tables of ηi are required as well as critical points; reference is made to the original paper by Shapiro and

GOODNESS OF FIT

Francia (32). Note that small values of W  n are significant and that the test has been shown to be consistent. An asymptotic distribution for the test has been established (see Durbin (11), where further tests of fit using order statistics are discussed). For completeness, it is pointed out that there exist goodness-of-fit procedures using the differences, or spacings*, between successive order statistics. Some of these tests are reviewed by Pyke (28), who developed the limiting distributions of functions of spacings and certain general limit theorems. More recent work in this area is reported by Kale (17) and Kirmani and Alam (19). 4

FINAL REMARKS

A few general considerations are worth raising. The single sample procedures outlined in previous sections deal with the problem of testing H 0 : GX (x) = F X (x, θ ), where θ is either fixed or is specified only up to a set
. If θ is not fixed, an estimator θˆ is substituted for it in F and the concordance of this estimated model with the data assessed. It is important in carrying out tests of fit not to lose sight of the fundamental purpose of the exercise. For example, tests of normality are often required as an intermediate step to further analyses. Alternatively, the performance of specific statistical processes, such as a random number generator, may need to be checked against specification. In these instances, the philosophy of using F, or a good estimate of it, to test against available data seems entirely reasonable. A different situation is generated if predictions* are required. In this case an estimate of F is to be used to predict future outcomes of the random variable X. It is possible that FX (x, θˆ ) may allow satisfactory predictions to be made, especially if the model was appropriate and θˆ based on a large sample. But there may be other candidates which would do a better job of ˆ such as to set up prediction than FX (x, θ), a measure of divergence of one PDF from another (see Ali and Silvey (3)) and then to try to find that PDF, based on the data, which comes closest to the estimated PDF. This treatment may need Bayesian arguments to construct predictive densities (2).

9

More specifically, let f X (x, θ ) be the density which is to be estimated and introduce the weight function p(θ —z) on
based on data z. Put  p(θ |z)fX (x, θ ) dθ ; (11) qX (x|z) =


then qX is called a predictive density for f X . On the other hand, for any estimator for θ based on z, θˆ (z), fX (x, θˆ (z)) is called an estimative density. Using the Kullback–Leibler directed measure of divergence, Aitchison (2) showed that qX (x—z) is optimal in the sense that it is closer to f X than any other competing density, in particular fX (x, θˆ (z)). Although this result may depend on the divergence measure used, it shows that fX (x, θˆ (z)) may not always be the appropriate estimator for f X (x, θ ). A chi-squared type of goodness-of-fit test for the predictive density has been developed by Guteman (14). REFERENCES 1. References are classified as follows: (A), applied; (E), expository; (R), review; (T), theoretical. 2. Aitchison, J. (1975). Biometrika, 62, 547–554. (T) 3. Ali, S. M. and Silvey, S. D. (1966). J. R. Statist. Soc. B, 28, 131–142. (T) 4. Bofinger, E. (1973). J. R. Statist. Soc. B, 35, 277–284. (T) 5. Conover, W. J. (1972). J. Amer. Statist. Ass., 67, 591–596. (T) 6. Cram´er, H. (1945). Mathematical Methods of Statistics. Princeton University Press, Princeton, N.J. (E) 7. Dufour, R. and Maag, U. R. (1978). Technometrics, 20, 29–32. (A) 8. Durbin, J. (1973). Distribution Theory for Tests Based on the Sample Distribution Function. Reg. Conf. Ser. Appl. Math. SIAM, Philadelphia. (T, R) 9. Durbin, J. (1973). Ann. Statist., 1, 279–290. (T) 10. Durbin, J. (1975). Biometrika, 62, 5–22. (T) 11. Durbin, J. (1977). Goodness-of-fit tests based on the order statistics. Trans. 7th Prague Conf. Inf. Theory, Statist. Decision Functions, Random Processes / 1974 Eur. Meet. Statist., Prague, 1974, Vol. A, 109–118. (R)

10

GOODNESS OF FIT

12. Durbin, J., Knott, M., and Taylor, C. C. (1975). J. R. Statist. Soc. B, 37, 216–237. (T) 13. Feller, W. (1948). Ann. Math. Statist., 19, 177. (T) 14. Guteman, I. (1967). J. R. Statist. Soc. B, 29, 83–100. (T) 15. Hegazy, Y. A. S. and Green, J. R. (1975). Appl. Statist., 24, 299–308. (A) 16. Horn, S. D. (1977). Biometrics, 33, 237–248. (A, R) 17. Kale, B. K. (1969). Sankhya¯ A, 31, 43–48. (T) 18. Kendall, M. G. and Stuart, A. (1973). The Advanced Theory of Statistics, Vol. 2: Inference and Relationship, 3rd ed. Hafner Press, New York. (E) 19. Kirmani, S. N. U. A. and Alam, S. N. (1974). Sankhya¯ A, 36, 197–203. (T) 20. Kumar, A. and Pathak, P. K. (1977). Scand. Statist. J., 4, 39–43. (T) 21. Lindgren, B. W. (1976). Statistical Theory, 3rd ed. Macmillan, New York. (E) 22. Moore, D. S. (1977). J. Amer. Statist. Ass., 72, 131–137. (T) 23. Moore, D. S. (1979). In Studies in Statistics, R. V. Hogg, ed. Mathematical Association of America, Washington, D.C., pp. 66–106. (T, E) 24. Moore, D. S. and Spruill, M. C. (1975). Ann. Statist., 3, 599–616. (T) 25. Pearson, E. S. and Hartley, H. O. (1972). Biometrika Tables for Statisticians, Vol. 2. Cambridge University Press, Cambridge. (E) 26. Pearson, E. S., D’Agostino, R. B., and Bowman, K. O. (1977). Biometrika, 64, 231–246. (A) 27. Pettitt, A. N. and Stephens, M. A. (1976). Biometrika, 63, 291–298. (T) 28. Pyke, R. (1965). J. R. Statist. Soc. B, 27, 395–436. (T) 29. Radlow, R. and Alf, E. F. (1975). J. Amer. Statist. Ass., 70, 811–813. (A) 30. Rao, K. C. and Robson, D. S. (1974). Commun. Statist., 3, 1139–1153. (T) 31. Riedwyl, H. (1967). J. Amer. Statist. Ass., 62, 390–398. (A) 32. Shapiro, S. S. and Francia, R. S. (1972). J. Amer. Statist. Ass., 67, 215–216. (A) 33. Steck, G. P. (1969). Ann. Math. Statist., 40, 1449–1466. (T) 34. Stephens, M. A. (1966). Biometrika, 53, 235–240. (T) 35. Stephens, M. A. (1974). J. Amer. Statist. Soc., 69, 730–743. (A, E)

36. Stephens, M. A. (1976). Ann. Statist., 4, 357–369. (T) 37. Tallis, G. M. and Chesson, P. (1976). Austr. J. Statist., 18, 53–61. (T) 38. Watson, G. S. (1961). Biometrika, 48, 109–114. (T) 39. Wood, C. L. and Altavela, M. M. (1978). Biometrika, 65, 235–239. (T)

GROUP-RANDOMIZED TRIALS

variance of any group-level statistic beyond what would be expected with random assignment of members to conditions. Moreover, with a limited number of groups, the degrees of freedom (df) available to estimate grouplevel statistics are limited. Any test that ignores either the extra variation or the limited df will have a Type I error rate that is inflated (3). This problem will only worsen as the ICC increases (4–6). As a result of these problems, RCTs are preferred over GRTs whenever randomization of individual participants is possible. However, individual randomization is not always possible, especially for many public health interventions that operate at a group level, manipulate the physical or social environment, or cannot be delivered to individuals. Just as the RCT is the gold standard in public health and medicine when allocation of individual participants is possible, the GRT is the gold standard when allocation of identifiable groups is necessary. The purpose of this article is to put GRTs in context in terms of other kinds of designs and in terms of the terminology used in other fields, to summarize their development in public health, to characterize the range of public health research areas that now employ GRTs, to characterize the state of practice with regard to the design and analysis of GRTs, to consider their future in public health research, and to review the steps required to plan a new GRT.

DAVID M. MURRAY The Ohio State University Division of Epidemiology School of Public Health Columbus, Ohio

1

INTRODUCTION

Group-randomized trials (GRTs) are comparative studies used to evaluate interventions that operate at a group level, manipulate the physical or social environment, or cannot be delivered to individuals (1). Examples include school-, worksite-, and communitybased studies designed to improve the health of students, employees, or residents. Four characteristics distinguish the GRT from the more familiar randomized clinical trial (RCT) (1). First, the unit of assignment is an identifiable group; such groups are not formed at random, but rather through some physical, social, geographic, or other connection among their members. Second, different groups are assigned to each condition, creating a nested or hierarchical structure for the design and the data. Third, the units of observation are members of those groups so that they are nested within both their condition and their group. Fourth, only a limited number of groups assigned to each condition usually exists. Together, these characteristics create several problems for the design and analysis of GRTs. The major design problem is that a limited number of often heterogeneous groups makes it difficult for randomization to distribute potential sources of confounding evenly in any single realization of the experiment, which increases the need to employ design strategies that will limit confounding and analytic strategies to deal with confounding where it is detected. The major analytic problem is that an expectation exists for positive intraclass correlation (ICC) among observations on members of the same group (2). That ICC reflects an extra component of variance attributable to the group above and beyond the variance attributable to its members. This extra variation will increase the

2 GROUP-RANDOMIZED TRIALS IN CONTEXT GRTs represent a subset of a larger class of designs often labeled nested, hierarchical, multilevel, or clustered designs. Units of observation are nested within identifiable groups or clusters, which are in turn nested within study conditions. This description defines a hierarchy of at least three levels in the design: units of observation, units of assignment, and study conditions. More complex designs may have even more levels. For example, in cohort or repeated measures designs, repeat observations are further nested within units of observation.

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

GROUP-RANDOMIZED TRIALS

As used here, the label group-randomized trial refers to a design in which identifiable groups are assigned to study conditions for the express purpose of assessing the impact of one or more interventions on one or more endpoints. The terms nested, hierarchical, multilevel, and clustered designs can be used more broadly to refer to any dataset that has a hierarchical structure, and these more general terms are often used to characterize observational studies as well as comparative studies. Many examples of observational and comparative hierarchical designs can be found in education, where students are nested within classrooms, which are nested within schools, which are nested within school districts, which are nested within communities, and so on. Investigators in education often refer to such designs as hierarchical or multilevel designs (7–10). Other examples can be found in survey sampling, and in disciplines that employ surveys, such as epidemiology, sociology, and demography. In these disciplines, cluster sampling is a commonly used technique (2). Cluster-sampling designs can be a good way to limit cost when the investigator lacks a complete enumeration of the population of interest and does not want to expend the resources required to generate such an enumeration. As simple random sampling is impossible without a complete enumeration, clusters such as blocks or neighborhoods or other identifiable groups are enumerated and sampled in a first stage, followed by enumeration and sampling of individuals within the selected clusters in a second stage. Applied properly, cluster-sampling methods can yield unbiased estimates of population rates or means at a lower cost than would have been the case with simple random sampling. Unfortunately, cluster sampling invariably leads to increased variation and often to limited degrees of freedom. These problems are well known to survey-sampling statisticians (2, 11–13). Biostatisticians often use the term clusterrandomization study to refer to a grouprandomized trial (14, 15). This terminology is based on the fact that an identifiable group is a cluster. It borrows from the terminology of survey sampling. With the

broad definition given to ‘‘group’’ in this text, the phrases cluster-randomization study and group-randomized trial are equivalent. Epidemiologists have often used the terms community trial and community-intervention trial (16–18). These terms emerged from the community-based heart disease prevention studies of the late 1970s and the 1980s (19–23). None of those studies were randomized trials, but all involved whole communities as the unit of assignment with collection of data from individuals within those communities. Community trial is an attractive label, because it includes both randomized designs and nonrandomized designs. However, it is often thought to refer only to studies that involve whole communities (e.g., Reference 24), and so creates confusion when applied to studies involving other identifiable groups. 3 THE DEVELOPMENT OF GROUP-RANDOMIZED TRIALS IN PUBLIC HEALTH GRTs gained attention in public health in the late 1970s with the publication of a symposium on coronary heart disease prevention trials in the American Journal of Epidemiology (3, 21, 25–27). Cornfield’s paper in particular has become quite famous among methodologists working in this area, as it identified the two issues that have vexed investigators who employ GRTs from the outset: extra variation and limited degrees of freedom. The last 25 years have witnessed dramatic growth in the number of GRTs in public health and dramatic improvements in the quality of the design and analysis of those trials. Responding directly to Cornfield’s warning, Donner and colleagues at the University of Western Ontario published a steady stream of papers on the issues of analysis facing group-randomized trials beginning in the early 1980s. Murray and colleagues from the University of Minnesota began their examination of the issues of design and analysis in group-randomized trials in the mid-1980s. Other investigators from the National Institutes of Health, the University of Washington, the New England Research Institute, and elsewhere added to

GROUP-RANDOMIZED TRIALS

this growing literature in public health, especially in the 1990s. By the late 1980s and early 1990s, many group-randomized trials were under way that were of very high quality in terms of their methods of design and analysis. Examples include the Community Intervention Trial for Smoking Cessation (COMMIT) (28), the Working Well Trial (29), and the Child and Adolescent Trial for Cardiovascular Health (CATCH) (30). These improvements occurred as investigators and reviewers alike gradually came to understand the special issues of design and analysis that face grouprandomized trials and the methods required to address them. Unfortunately, the improvements were not represented in all group-randomized trials. Even in the 1990s, grants were funded and papers were published based on poor designs and poor analyses. Simpson et al. (31) reviewed GRTs that were published between 1990 and 1993 in the American Journal of Public Health (AJPH) and in Preventive Medicine (Prev Med). They reported that fewer than 20% dealt with the design and analytic issues adequately in their sample size or power analysis and that only 57% dealt with them adequately in their analysis. In 1998, the first textbook on the design and analysis of GRTs appeared (1). It detailed the design considerations for the development of GRTs, described the major approaches to their analysis both for Gaussian data and for binary data, and presented methods for power analysis applicable to most GRTs. The second textbook on the design and analysis of GRTs appeared in 2000 (15). It provided a good history on GRTs, examined the role of informed consent and other ethical issues, focused on extensions of classical methods, and included material on regression models for Gaussian, binary, count, and time-to-event data. Other textbooks on analysis methods germane to GRTs appeared during the same period (10, 32, 33). Murray et al. recently reviewed a large number of articles on new methods relevant to the design and analysis of GRTs published between 1998 and 2003 (34). Of particular importance for investigators planning GRTs has been the increased availability of estimates of ICC for a variety of

3

endpoints in a variety of groups. Donner and Klar reported a number of ICCs in their text (15). Murray and Blitstein (35) identified more than 20 papers published before 2002 that reported ICCs, whereas Murray et al. identified a similar number of papers published between 1998 and 2003 that reported ICCs (34). Murray and Blitstein also reported on a pooled analysis of ICCs from worksite, school, and community studies. They confirmed earlier reports that the adverse impact of positive ICC can be reduced either by regression adjustment for covariates or by taking advantage of over-time correlation in a repeated-measures analysis. More recently, Janega et al. provided additional documentation that standard errors for intervention effects from end-of-study analyses that reflect these strategies are often different from corresponding standard errors estimated from baseline analyses that do not (36, 37). As the ICC of concern in any analysis of an intervention effect is the ICC as it operates in the primary analysis (1), these findings reinforce the need for investigators to use estimates in their power analysis that closely reflect the endpoints, the target population, and the primary analysis planned for the trial. 4

THE RANGE OF GRTS IN PUBLIC HEALTH

Varnell et al. examined GRTs published between 1998 and 2002 in the AJPH and in Prev Med (38). They found 58 GRTs published in just those two journals during that 5-year period. That rate of 11.6 articles per year was double the publication rate observed for GRTs in the same journals between 1990 and 1993 in the earlier review by Simpson et al. (31). Those trials were conducted for a variety of primary outcomes. The largest fraction (25.9%) targeted smoking prevention or cessation, with trials focused on dietary representing the next largest fraction (20.7%). Studies designed to evaluate screening programs, particularly for cancer, represented the next largest fraction (12.1%). Other studies focused on alcohol or drug use, or on a combination of tobacco, alcohol, and drug use (8.6%); on sun protection (5.2%); on physical or sexual abuse (3.4%); on physician preventive practices (3.4%); on work place health

4

GROUP-RANDOMIZED TRIALS

and safety measures (3.4%); or on multiple health outcomes (8.6%). Those trials were also conducted in a variety of settings. The largest fraction (29.3%) were conducted in schools or colleges. Worksites accounted for 19.0% of the trials. Medical practices accounted for 15.5%, whereas communities accounted for 12.1%. Trials were also conducted in housing projects (5.2%) or churches (5.2%), and 13.8% were conducted in other settings. The size of the group used in these GRTs varied considerably. Almost 30% were conducted in groups having more than 100 members, 24.1% in groups having 51 to 100 members, 32.8% in groups having 10 to 50 members, and 13.8% in groups having fewer than 10 members. Most of these trials employed a pretest-post-test design (55.2%), whereas 13.8% relied on a post-test-only design; 29.3% employed designs having three time points; and 12% involved more than three time points. Most of the trials employed cohort designs (63.8%), whereas 20.7% were crosssectional and 15.5% used a combination of cohort and cross-sectional designs. As this pattern suggests, GRTs are applicable to a wide variety of problems within public health and may be used in a wide variety of settings. It remains the best design available whenever the intervention operates at a group level, manipulates the physical or social environment, or cannot be delivered to individuals. 5 CURRENT DESIGN AND ANALYTIC PRACTICES IN GRTS IN PUBLIC HEALTH As noted above, two textbooks now exist that provide guidance on the design and analysis of GRTs (1, 15), as well as more recent summary papers focused on these issues (39–43). These sources identify several analytic approaches that can provide a valid analysis for GRTs. In most, the intervention effect is defined as a function of a condition-level statistic (e.g., difference in means, rates, or slopes) and assessed against the variation in the corresponding grouplevel statistic. These approaches included mixed-model ANOVA/ANCOVA for designs

having only one or two time intervals, random coefficient models for designs having three or more time intervals, and randomization tests as an alternative to the modelbased methods. Other approaches are generally regarded as invalid for GRTs because they ignore or misrepresent a source of random variation, which include analyses that assess condition variation against individual variation and ignore the group, analyses that assess condition variation against individual variation and include the group as a fixed effect, analyses that assess the condition variation against subgroup variation, and analyses that assess condition variation against the wrong type of group variation. Still other strategies may have limited application for GRTs. Application of fixed-effect models with post hoc correction for extra variation and limited df assumes that the correction is based on an appropriate ICC estimate. Application of survey-based methods or generalized estimating equations (GEE) and the sandwich method for standard errors requires a total of 40 or more groups in the study, or a correction for the downward bias in the sandwich estimator for standard errors when fewer than 40 groups exist in the study (34). Varnell et al. recently reported on the state of current practice in GRTs with regard to design and analytic issues for the 58 GRTs published between 1998 and 2002 in the AJPH and Prev Med (38). They reported that only 15.5% provided evidence that they dealt with the design and analytic issues adequately in their sample size or power analysis, either in the published paper or in an earlier background paper. To qualify as providing evidence, the paper or background paper had to report the ICC estimate expected to apply to the primary analysis in the study, the variance components used to calculate that ICC, or a variance inflation factor calculated from that ICC. The vast majority of GRTs published in these two journals did not provide such evidence. More surprising, the proportion reporting such evidence was actually somewhat lower than reported by Simpson et al. when they reviewed the same journals for the period between 1990 and 1993 (31). Varnell et al. noted that it was possible that many of

GROUP-RANDOMIZED TRIALS

these studies performed such power calculations but did not report them. However, 27 (46%) of the reviewed studies had fewer than 10 groups per condition, and of that number, only one reported evidence of estimating sample size using methods appropriate for GRTs. Varnell et al. concluded that it was likely that many investigators reporting small GRTs planned their study without considering issues of sample size adequately. Varnell et al. reported that only 54.4% of the GRTs published between 1998 and 2002 in the AJPH and Prev Med dealt with design and analytic issues adequately in their analysis (38). Most of the studies that reported using only appropriate methods used mixedmodel regression methods (61.2%), whereas others used two-stage methods (35.4%) or GEE with more than 40 groups (6.4%). Of the remaining studies, 26.3% reported a combination of appropriate and inappropriate methods, whereas 19.3% reported using only inappropriate methods. The most widely used inappropriate methods were analysis at an individual level, ignoring group-level ICC; analysis at a subgroup level, ignoring grouplevel ICC; and use of GEE or another asymptotically robust method with fewer than 40 groups and no correction for the downward bias identified in those methods under those conditions. Varnell et al. reported appreciable differences between the AJPH and Prev Med with regard to these patterns (38). For example, among the 27 GRTs published in AJPH, 66.7% reported only analyses taking ICC into account properly, compared with 43.3% in Prev Med. At the same time, only 14.5% of the GRTs published in AJPH reported only inappropriate analyses, compared with 23.3% in Prev Med. The range of practices across other journals is likely to be even wider. Particularly surprising in this review was that 33.3% of the studies reviewed reported analyses considered inappropriate well before the studies were published, which suggests that some investigators, reviewers, and journal editors have not yet taken to heart the long-standing warnings against analysis at an individual or subgroup level that ignores the group-level ICC, or an analysis that includes group as a fixed effect.

5

Indeed, investigators continued to put forward false arguments to justify the use of inappropriate methods, including claims that appropriate methods could obscure important clinical changes, or that the observed ICCs were small and therefore ignorable. Such arguments ignore the methodological realities of the GRT and represent exactly the kind of self-deception that Cornfield warned against more than 25 years ago (3). 6 THE FUTURE OF GROUP-RANDOMIZED TRIALS Publication of disappointing results for several large GRTs in the mid-1990s, such as the Stanford Five City Project (44), the Minnesota Heart Health Program (45), the Pawtucket Heart Health Program (20), and COMMIT (46, 47), led some to question the value of GRTs in general and of community trials in particular. As noted at the time, it is both shortsighted and impractical to question GRTs in general based on disappointing results even from a number of large and expensive trials (48). Whenever the investigator wants to evaluate an intervention that operates at a group level, manipulates the social or physical environment, or cannot be delivered to individuals, the GRT is the best comparative design available. However, many challenges exist to the design and analysis of GRTs, and the investigator must take care to understand those challenges and the strategies that are available to meet them: The question is not whether to conduct GRTs, but rather how to do them well (1). Certainly, no decline in the number of GRTs has occurred since the mid-1990s. As reported by Varnell et al., the annual rate of publication of GRTs in AJPH and Prev Med during 1998 to 2002 was double that for the period 1990 to 1993. The number of GRTs proposed to NIH has also increased considerably over the past 10 to 15 years, and a study section now exists at NIH that counts GRT applications as one of its most common design types (Community Level Health Promotion, formerly known as SNEM-1). Even so, many challenges facing GRTs remain. For example, no question exists that

6

GROUP-RANDOMIZED TRIALS

it is harder to change the health behavior and risk profile of a whole community than it is to make similar changes in smaller identifiable groups such as those at worksites, physician practices, schools, and churches. And although no quantitative analysis has been published, it seems that the magnitude of the intervention effects reported for GRTs has been greater for trials that involved smaller groups than for trials involving such large aggregates as whole communities. With smaller groups, it is possible to include more groups in the design, thereby improving the validity of the design and the power of the trial. With smaller groups, it is easier to focus intervention activities on the target population. With smaller groups, the cost and difficulty of the implementation of the study generally are reduced. For these and similar reasons, future group-randomized trials may do well to focus on more and smaller identifiable groups rather than on whole cities or larger aggregates. Indeed, that pattern is evidenced in a number of recent trials (49–54). At the same time, positive effects have been reported for some studies involving very large aggregates. An example is the ASSIST project from the National Cancer Institute (55). Although this trial was not randomized, 17 states participated in the ASSIST intervention to reduce smoking, and the remaining 33 states plus the District of Columbia served as control sites. As such, the units of assignment in the ASSIST trial were even larger aggregates than are used in most GRTs. The encouraging results from that trial confirm that it is possible to successfully deliver an intervention even in very large aggregates. Another challenge is simply the difficulty in developing interventions strong enough to change the health behaviors of the target populations. This point is not new, but it is one that has been well known to investigators working in GRTs for some time (56–58). The methods for the design and analysis of GRTs have evolved considerably from the 1970s and 1980s; however, interventions continue to be employed that often prove ineffective. One of the problems for some time has been that interventions are proposed that lack even preliminary evidence of efficacy (42).

Efficacy trials in health promotion and disease prevention often are begun without the benefit of prototype studies, and often even without the benefit of adequate pilot studies, which has happened in large part because the funding agencies have been reluctant to support pilot and prototype studies, preferring instead to fund efficacy and effectiveness trials. Unfortunately, the interventions that lead to GRTs tend to be more complicated than those in other areas or those that lead to clinical trials. As such, it is even more important to subject them to adequate testing in pilot and prototype studies. These earlier phases of research can uncover important weaknesses in the intervention content or implementation methods. Moving too quickly to efficacy trials risks wasting substantial time and resources on interventions that could have been substantially improved through the experience gained in those pilot and prototype studies. Hopefully, the funding agencies will recognize this point and begin to provide better support for pilot and prototype studies. The R21 mechanism at NIH is wellsuited for that kind of pilot and feasibility work. Prototype studies will typically be based on only one or two groups per condition, and so are particularly problematic if the investigator wants to make causal inferences relating the intervention as delivered to the outcomes as observed. Studies based on only one group per condition cannot estimate variation because of the group independent of variation due to condition (59). Studies based on only a few groups per condition cannot estimate that component of variance with much accuracy. Some of the methods described earlier as having limited application to GRTs can be used, but all would require strong assumptions. For example, application of a post hoc correction would require the strong assumption that the external estimate of ICC is valid for the data at hand. Application of the subgroup or batch analysis would require the strong assumption that the subgroup captures the group variance. Does this mean that prototype studies should not be conducted? The answer is clearly no, and, more to the point, the answer must be that such studies simply should not be used to make causal inferences relating

GROUP-RANDOMIZED TRIALS

the intervention and the outcome. Studies involving only one or two groups per condition are prototype studies, not efficacy trials, and must be analyzed with that in mind. With only one or two groups per condition, the investigator can estimate the magnitude of the intervention effect, but it will not be possible to estimate a standard error for that effect with any degree of accuracy. As a result, it will not be possible to put a narrow confidence bound around the intervention effect or to draw any conclusions about the statistical significance of that effect. Even so, if the effect is much smaller than expected, that finding should push the investigators to rework their intervention, as it is not likely that a reasonably sized efficacy trial would show such a small intervention effect to be significant. If the effect is as large as expected or larger, that finding should provide good support to the investigator in seeking support for the efficacy trial required to establish the causal link between the intervention and the outcome. Methodological challenges remain as well. For example, a number of recent studies have documented the downward bias in the sandwich estimator used in GEE when fewer than 40 groups exist in the study (4, 60, 61). Some of these studies, and others, have proposed corrections for that estimator (61–66). Unfortunately, none of these corrections appear in the standard software packages, so they are relatively unavailable to investigators who analyze GRTs. Absent an effective correction, the sandwich estimator will have an inflated Type I error rate in GRTs having less than 40 groups, and investigators who employ this approach continue to risk overstating the significance of their findings. As another example, a number of recent studies have proposed methods for survival analysis that could be applied to data from GRTs (15, 67–71). Some of these methods involved use of the sandwich estimator, and so would be subject to the same concern as noted above for GEE. None of the methods appear in the standard software packages, so they also are relatively unavailable to investigators who analyze GRTs. As a third example, permutation tests have been advocated over model-based methods because they require fewer assumptions.

7

At the same time, they tend to have lower power. To overcome this problem, Feng et al. developed an optimal randomization test that had nominal size and better power than alternative randomization tests or GEE, although it was still not as powerful as the model-based analysis when the model was specified correctly (72). Additional research is needed to compare Braun and Feng’s optimal randomization test and model-based methods under model misspecification. Every reason exists to expect that continuing methodological improvements will lead to better trials. Evidence also exists that better trials tend to have more satisfactory results. For example, Rooney and Murray presented the results of a meta-analysis of group-randomized trials in the smokingprevention field (73). One of the findings was that stronger intervention effects were associated with greater methodological rigor. Stronger intervention effects were reported for studies that planned from the beginning to employ the unit of assignment as the unit of analysis, that randomized a sufficient number of assignment units to each condition, that adjusted for baseline differences in important confounding variables, that had extended follow-up, and that had limited attrition. One hopes that such findings will encourage use of good design and analytic methods. A well-designed and properly executed GRT remains the method of choice in public health and medicine when the purpose of the study is to establish the efficacy or effectiveness of an intervention that operates at a group level, manipulates the social or physical environment, or cannot be delivered to individuals. However, efficacy and effectiveness trials should occur in proper sequence. They should occur only after pilot studies have established the feasibility and acceptability of the materials and protocols for the intervention and evaluation. They should occur only after prototype studies have shown that the magnitude of the intervention effect is large enough to warrant the larger trials. Efficacy and effectiveness trials should be large enough to ensure sufficient power, with groups assigned at random from within wellmatched or stratified sets to protect against bias. Investigators should measure exposure

8

GROUP-RANDOMIZED TRIALS

and other process variables in all conditions. They should select a model for the analysis that reflects the design of the study and the nature of the endpoints. Importantly, investigators should be cautious of strategies that appear to easily solve or avoid the design and analytic problems that are inherent in group-randomized trials, for those methods are likely to prove to be inappropriate. 7 PLANNING A NEW GROUP-RANDOMIZED TRIAL The driving force behind any GRT must be the research question. The question will be based on the problem of interest and will identify the target population, the setting, the endpoints, and the intervention. In turn, those factors will shape the design and analytic plan. Given the importance of the research question, the investigators must take care to articulate it clearly. Unfortunately, that does not always happen. Investigators may have ideas about the theoretical or conceptual basis for the intervention, and often even clearer ideas about the conceptual basis for the endpoints. They may even have ideas about intermediate processes. However, without very clear thinking about each of these issues, the investigators may find themselves at the end of the trial unable to answer the question of interest. To put themselves in a position to articulate their research question clearly, the investigators should first document thoroughly the nature and extent of the underlying problem and the strategies and results of previous efforts to remedy that problem. A literature review and correspondence with others working in the field are ingredients essential to that process, as the investigators should know as much as possible about the problem before they plan their trial. Having become experts in the field, the investigators should choose the single question that will drive their GRT. The primary criteria for choosing that question should be: (1) Is it important enough to do?, and (2) Is this the right time to do it? Reviewers will ask both questions, and the investigators must be able to provide well-documented answers.

Most GRTs seek to prevent a health problem, so that the importance of the question is linked to the seriousness of that problem. The investigators should document the extent of the problem and the potential benefit from a reduction in that problem. The question of timing is also important. The investigators should document that the question has not been answered and that the intervention has a good chance to improve the primary endpoint in the target population, which is most easily done when the investigators are thoroughly familiar with previous research in the area; when the etiology of the problem is well known; when a theoretical basis exists for the proposed intervention; when preliminary evidence exists on the feasibility and efficacy of the intervention; when the measures for the dependent and mediating variables are well-developed; when the sources of variation and correlation as well as the trends in the endpoints are well understood; and when the investigators have created the research team to carry out the study. If that is not the state of affairs, then the investigators must either invest the time and energy to reach that state or choose another question. Once the question is selected, it is very important to put it down on paper. The research question is easily lost in the dayto-day details of the planning and execution of the study, and because much time can be wasted in pursuit of issues that are not really central to the research question, the investigators should take care to keep that question in mind. 7.1 The Research Team Having defined the question, the investigators should determine whether they have sufficient expertise to deal with all the challenges that are likely to develop as they plan and execute the trial. They should identify the skills that they do not have and expand the research team to ensure that those skills are available. All GRTs will need expertise in research design, data collection, data processing and analysis, intervention development, intervention implementation, and project administration. As the team usually will need to convince a funding agency that they are appropriate

GROUP-RANDOMIZED TRIALS

for the trial, it is important to include experienced and senior investigators in key roles. No substitute exists for experience with similar interventions, in similar populations and settings, using similar measures, and similar methods of data collection and analysis. As those skills are rarely found in a single investigator, most trials will require a team, with responsibilities shared among its members. Most teams will remember the familiar academic issues (e.g., statistics, data management, intervention theory), but some may forget the very important practical side of trials involving identifiable groups. However, to forget the practical side is a sure way to get into trouble. For example, a schoolbased trial that does not include on its team someone who is very familiar with school operations is almost certain to get into trouble with the schools. A hospital-based trial that does not include on its team someone who is very familiar with hospital operations is almost certain to get into trouble with the hospitals. And the same can be said for every other type of identifiable group, population, or setting that might be used. 7.2 The Research Design The fundamentals of research design apply to GRTs as well as to other comparative designs. As they are discussed in many familiar textbooks (11, 74–77), they will be reviewed only briefly here. Additional information may be found in two recent textbooks on the design and analysis of GRTs (1, 15). The goal in the design of any comparative trial is to provide the basis for valid inference that the intervention as implemented caused the result(s) as observed. Overall, three elements are required: (1) control observations must exist, (2) a minimum of bias must exist in the estimate of the intervention effect, and (3) sufficient precision for that estimate must exist. The nature of the control observations and the way in which the groups are allocated to treatment conditions will determine in large measure the level of bias in the estimate of the intervention effect. Bias exists whenever the estimate of the intervention effect is different from its true value. If that bias is substantial, the investigators will be misled

9

about the effect of their intervention, as will the other scientists and policy makers who use their work. Even if adequate control observations are available so that the estimate of the intervention effect is unbiased, the investigator should know whether the effect is greater than would be expected by chance, given the level of variation in the data. Statistical tests can provide such evidence, but their power to do so will depend heavily on the precision of the intervention effect estimate. As the precision improves, it will be easier to distinguish true effects from the underlying variation in the data.

7.3 Potential Design Problems and Methods to Avoid them For GRTs, the four sources of bias that are particularly problematic and should be considered during the planning phase are selection, differential history, differential maturation, and contamination. Selection bias refers to baseline differences among the study conditions that might explain the results of the trial. Bias because of differential history refers to some external influence that operates differentially among the conditions. Bias because of differential maturation reflects uneven secular trends among the groups in the trial favoring one condition or another. These first three sources of bias can either mask or mimic an intervention effect, and all three are more likely given either nonrandom assignment of groups or random assignment of a limited number of groups to each condition. The first three sources of bias are best avoided by randomization of a sufficient number of groups to each study condition, which will increase the likelihood that potential sources of bias are distributed evenly among the conditions. Careful matching or stratification can increase the effectiveness of randomization, especially when the number of groups is small. As a result, all GRTs planned with fewer than 20 groups per condition would be well served to include careful matching or stratification before randomization.

10

GROUP-RANDOMIZED TRIALS

The fourth source of bias is caused by contamination, which occurs when interventionlike activities find their way into the comparison groups; it can bias the estimate of the intervention effect toward the null hypothesis. Randomization will not protect against contamination; although investigators can control access to their intervention materials, they can often do little to prevent the outside world from introducing similar activities into their control groups. As a result, monitoring exposure to activities that could affect the trial’s endpoints in both the intervention and comparison groups is especially important in GRTs, which will allow the investigators to detect and respond to contamination if it occurs. Objective measures and evaluation personnel who have no connection to the intervention are also important strategies to limit bias. Finally, analytic strategies, such as regression adjustment for confounders, can be very helpful in dealing with any observed bias. 7.4 Potential Analytic Problems and Methods to Avoid them The two major threats to the validity of the analysis of a GRT that should be considered during the planning phase are misspecification of the analytic model and low power. Misspecification of the analytic model will occur if the investigator ignores or misrepresents a measurable source of random variation, or misrepresents the pattern of any over-time correlation in the data. To avoid model misspecification, the investigator should plan the analysis concurrent with the design, plan the analysis around the primary endpoints, anticipate all sources of random variation, anticipate the error distribution for the primary endpoint, anticipate patterns of over-time correlation, consider alternate structures for the covariance matrix, consider alternate models for time, and assess potential confounding and effect modification. Low power will occur if the investigator employs a weak intervention, has insufficient replication, has high variance or ICC in the endpoints, or has poor reliability of intervention implementation. To avoid low power,

investigators should plan a large enough study to ensure sufficient replication, choose endpoints with low variance and ICC, employ matching or stratification before randomization, employ more and smaller groups instead of a few large groups, employ more and smaller surveys or continuous surveillance instead of a few large surveys, employ repeat observations on the same groups or on the same groups and members, employ strong interventions with good reach, and maintain the reliability of intervention implementation. In the analysis, investigators should employ regression adjustment for covariates, model time if possible, and consider post hoc stratification. 7.5 Variables of Interest and their Measures The research question will identify the primary and secondary endpoints of the trial. The question may also identify potential effect modifiers. It will then be up to the investigators to anticipate potential confounders and nuisance variables. All these variables must be measured if they are to be used in the analysis of the trial. In a clinical trial, the primary endpoint is a clinical event, chosen because it is easy to measure with limited error and is clinically relevant (75). In a GRT, the primary endpoint need not be a clinical event, but it should be easy to measure with limited error and be relevant to public health. In both RCTs and GRTs, the primary endpoint, together with its method of measurement, must be defined in writing before the start of the trial. The endpoint and its method of measurement cannot be changed after the start of the trial without risking the validity of the trial and the credibility of the research team. Secondary endpoints should have similar characteristics and also should be identified before the start of the trial. In a GRT, an effect modifier is a variable whose level influences the effect of the intervention. For example, if the effect of a schoolbased drug-use prevention program depends on the baseline risk level of the student, then baseline risk is an effect modifier. Effect modification can be seen intuitively by looking at separate intervention effect estimates for the levels of the effect modifier. If they differ to

GROUP-RANDOMIZED TRIALS

a meaningful degree, then the investigator has evidence of possible effect modification. A more formal assessment is provided by a statistical test for effect modification, which is accomplished by including an interaction term between the effect modifier and condition in the analysis and testing the statistical significance of that term. If the interaction is significant, then the investigator should present the results separately for the levels of the effect modifier. If not, the interaction term is deleted and the investigator can continue with the analysis. Proper identification of potential effect modifiers comes through a careful review of the literature and from an examination of the theory of the intervention. Potential effect modifiers must be measured as part of the data-collection process so that their role can later be assessed. A confounder is related to the endpoint, not on the causal pathway, and unevenly distributed among the conditions; it serves to bias the estimate of the intervention effect. No statistical test for confounding exists; instead, it is assessed by comparing the unadjusted estimate of the intervention effect to the adjusted estimate of that effect. If, in the investigator’s opinion, a meaningful difference exists between the adjusted and unadjusted estimates, then the investigator has an obligation to report the adjusted value. It may also be appropriate to report the unadjusted value to allow the reader to assess the degree of confounding. The adjusted analysis will not be possible unless the potential confounders are measured. Proper identification of potential confounders also comes through a careful review of the literature and from an understanding of the endpoints and the study population. The investigators must take care in the selection of potential confounders to select only confounders and not mediating variables. A confounder is related to the endpoint and unevenly distributed in the conditions, but is not on the causal pathway between the intervention and the outcome. A mediating variable has all the characteristics of a confounder, but is on the causal pathway. Adjustment for a mediating variable, in the false belief that it is a confounder, will bias the estimate of the intervention effect toward the null hypothesis.

11

Similarly, the investigator must take care to avoid selecting as potential confounders variables that may be affected by the intervention. Such variables will be proxies for the intervention itself, and adjustment for them will also bias the estimate of the intervention effect toward the null hypothesis. An effective strategy to avoid these problems is to restrict confounders to variables measured at baseline. Such factors cannot be on the causal pathway, nor can their values be influenced by an intervention that has not been delivered. Investigators may also want to include variables measured after the intervention has begun, but will need to take care to avoid the problems described above. Nuisance variables are related to the endpoint, not on the causal pathway, but evenly distributed among the conditions. They cannot bias the estimate of the intervention effect, but they can be used to improve precision in the analysis. A common method is to make regression adjustment for these factors during the analysis so as to reduce the standard error of the estimate of the intervention effect, thereby improving the precision of the analysis. Such adjustment will not be possible unless the nuisance variables are measured. Proper identification of potential nuisance variables also comes from a careful review of the literature and from an understanding of the endpoint. The cautions described above for the selection of potential confounding variables apply equally well to the selection of potential nuisance variables. 7.6 The Intervention No matter how well designed and evaluated a GRT may be, strengths in design and analysis cannot overcome a weak intervention. Although the designs and analyses employed in GRTs were fair targets for criticism during the 1970s and 1980s, the designs and analyses employed more recently have improved, with many examples of very well-designed and carefully analyzed trials. Where intervention effects are modest or short-lived, even in the presence of good design and analytic strategies, investigators must take a hard look at the intervention and question whether it was strong enough.

12

GROUP-RANDOMIZED TRIALS

One of the first suggestions for developing the research question was that the investigators become experts on the problem that they seek to remedy. If the primary endpoint is cigarette smoking among ninth graders, then the team should seek to learn as much as possible about the etiology of smoking among young adolescents. If the primary endpoint is obesity among Native American children, then the team should seek to learn as much as possible about the etiology of obesity among those young children. If the primary endpoint is delay time in reporting heart attack symptoms, then the team should seek to learn as much as possible about the factors that influence delay time. And the same can be said for any other endpoint. One of the goals of developing expertise in the etiology of the problem is to identify points in that etiology that are amenable to intervention. Critical developmental stages may exist, or critical events or influences that trigger the next step in the progression, or it may be possible to identify critical players in the form of parents, friends, coworkers, or others who can influence the development of that problem. Without careful study of the etiology, the team will largely be guessing and hoping that their intervention is designed properly. Unfortunately, guessing and hoping rarely lead to effective interventions. Powerful interventions are guided by good theory on the process for change, combined with a good understanding of the etiology of the problem of interest. Poor theory will produce poor interventions and poor results, which was one of the primary messages from the community-based heart disease prevention studies, where the intervention effects were modest, generally of limited duration, and often within chance levels. Fortmann et al. noted that one of the major lessons learned was how much was not known about how to intervene in whole communities (56). Theory that describes the process of change in individuals may not apply to the process of change in identifiable groups. If it does, it may not apply in exactly the same way. Good intervention for a GRT will likely need to combine theory about individual change with theory about group processes and group change.

A good theoretical exposition will also help identify channels for the intervention program. For example, strong evidence exists that recent immigrants often look to longterm immigrants of the same cultural group for information on health issues. This fact has led investigators to try to use those longterm immigrants as change agents for the more recent immigrants. A good theoretical exposition will often indicate that the phenomenon is the product of multiple influences and so suggest that the intervention operate at several different levels. For example, obesity among schoolchildren appears to be influenced most proximally by their physical activity levels and by their dietary intake. In turn, their dietary intake is influenced by what is served at home and at school and their physical activity is influenced by the nature of their physical activity and recess programs at school and at home. The models provided by teachers and parents are important both for diet and physical activity. This multilevel etiology suggests that interventions be directed at the school foodservice, physical education, and recess programs; at parents; and possibly at the larger community. As noted earlier, GRTs would benefit by following the example of clinical trials, where some evidence of feasibility and efficacy of the intervention is usually required before launching the trial. When a study takes several years to complete and costs hundreds of thousands of dollars or more, that expectation seems very fair. Even shorter and less expensive GRTs would do well to follow that advice. 7.7 Power A detailed exposition on power for GRTs is beyond the scope of this article. Excellent treatments exist, and the interested reader is referred to those sources for additional information. Chapter 9 in the Murray text provides perhaps the most comprehensive treatment of detectable difference, sample size, and power for GRTs (1). Even so, a few points bear repeating here. First, the increase in between-group variance because of the ICC in the simplest analysis is calculated as 1 + (m − 1)ICC,

GROUP-RANDOMIZED TRIALS

where m is the number of members per group; as such, ignoring even a small ICC can underestimate standard errors if m is large. Second, although the magnitude of the ICC is inversely related to the level of aggregation, it is independent of the number of group members who provide data. For both of these reasons, more power is available given more groups per condition with fewer members measured per group than given just a few groups per condition with many members measured per group, no matter the size of the ICC. Third, the two factors that largely determine power in any GRT are the ICC and the number of groups per condition. For these reasons, no substitute exists for a good estimate of the ICC for the primary endpoint, the target population, and the primary analysis planned for the trial, and it is unusual for a GRT to have adequate power with fewer than 8 to 10 groups per condition. Finally, the formula for the standard error for the intervention effect depends on the primary analysis planned for the trial, and investigators should take care to calculate that standard error, and power, based on that analysis.

8

ACKNOWLEDGEMENT

The material presented here draws heavily on work published previously by David M. Murray and his colleagues (1, 34, 38, 43). Readers are referred to those sources for additional information.

REFERENCES 1. D. M. Murray, Design and Analysis of GroupRandomized Trials. New York: Oxford University Press, 1998. 2. L. Kish, Survey Sampling. New York: John Wiley & Sons, 1965.

13

5. D. M. Murray and R. D. Wolfinger, Analysis issues in the evaluation of community trials: progress toward solutions in SAS/STAT MIXED. J. Community Psychol. CSAP Special Issue 1994: 140–154. 6. D. M. Zucker, An analysis of variance pitfall: the fixed effects analysis in a nested design. Educ. Psycholog. Measur. 1990; 50: 731–738. 7. A. S. Bryk and S. W. Raudenbush, Hierarchical Linear Models: Applications and Data Analysis Methods. Newbury Park, CA: Sage Publications, 1992. 8. H. Goldstein, Multilevel Models in Educational and Social Research. New York: Oxford University Press, 1987. 9. I. Kreft and J. De Leeuw, Introducing Multilevel Modeling. London: Sage Publications, 1998. 10. S. W. Raudenbush and A. S. Bryk, Hierarchical Linear Models, 2nd ed. Thousand Oaks, CA: Sage Publications, 2002. 11. L. Kish, Statistical Design for Research. New York: John Wiley & Sons, 1987. 12. E. L. Korn and B. I. Graubard, Analysis of Health Surveys. New York: John Wiley & Sons, 1999. 13. C. J. Skinner, D. Holt, and T. M. F. Smith, Analysis of Complex Surveys. New York: John Wiley & Sons, 1989. 14. A. Donner, N. Birkett, and C. Buck, Randomization by cluster: sample size requirements and analysis. Amer. J. Epidemiol. 1981; 114(6): 906–914. 15. A. Donner and N. Klar, Design and Analysis of Cluster Randomization Trials in Health Research. London: Arnold, 2000. 16. COMMIT Research Group, Community Intervention Trial for Smoking Ccessation (COMMIT): summary of design and intervention. J. Natl. Cancer Inst. 1991; 83(22): 1620–1628. 17. A. Donner, Symposium on community intervention trials. Amer. J. Epidemiol. 1995; 142(6): 567–568. 18. R. V. Luepker, Community trials. Prevent. Med. 1994; 23: 602–605.

3. J. Cornfield, Randomization by group: a formal analysis. Amer. J. Epidemiol. 1978; 108(2): 100–102.

19. H. Blackburn, Research and demonstration projects in community cardiovascular disease prevention. J. Public Health Policy 1983; 4(4): 398–420.

4. D. M. Murray, P. J. Hannan, and W. L. Baker, A Monte Carlo study of alternative responses to intraclass correlation in community trials: is it ever possible to avoid Cornfield’s penalties? Eval. Rev. 1996; 20(3): 313–337.

20. R. A. Carleton et al., The Pawtucket Heart Health Program: community changes in cardiovascular risk factors and projected disease risk. Amer. J. Public Health 1995; 85(6): 777–785.

14

GROUP-RANDOMIZED TRIALS

21. J. W. Farquhar, The community-based model of life style intervention trials. Amer. J. Epidemiol. 1978; 108(2): 103–111.

36.

22. J. W. Farquhar et al., The Stanford five-city project: design and methods. Amer. J. Epidemiol. 1985; 122(2): 323–324. 23. D. R. Jacobs et al., Community-wide prevention strategies: evaluation design of the Minnesota Heart Health Program. J. Chronic Dis. 1986; 39(10): 775–788. 24. D. E. Lilienfeld and P. D. Stolley, Foundations of Epidemiology, 3rd ed. New York: Oxford University Press, 1994. 25. S. B. Hully, Symposium on CHD prevention trials: design issues in testing life style intervention. Amer. J. Epidemiol. 1978; 108(2): 85–86. 26. R. Sherwin, Controlled trials of the diet-heart hypothesis: some comments on the experimental unit. Amer. J. Epidemiol. 1978; 108(2): 92–99. 27. S. L. Syme, Life style intervention in clinicbased trials. Amer. J. Epidemiol. 1978; 108(2): 87–91.

37.

38.

39.

40.

41.

42.

28. M. H. Gail et al., Aspects of statistical design for the Community Intervention Trial for Smoking Cessation (COMMIT). Controlled Clin. Trials 1992; 13: 6–21. 29. D. B. Abrams et al., Cancer Control at the Workplace: The Working Well Trial. Prevent. Med. 1994; 23: 15–27. 30. D. M. Zucker et al., Statistical Design of the Child and Adolescent Trial for Cardiovascular Health (CATCH): implication of cluster randomization. Controlled Clin. Trials 1995; 16: 96–118. 31. J. M. Simpson, N. Klar, and A. Donner, Accounting for cluster randomization: a review of Primary Prevention Trials, 1990 through 1993. Amer. J. Public Health 1995; 85(10): 1378–1383. 32. H. Brown and R. Prescott, Applied Mixed Models in Medicine. Chichester, UK: John Wiley & Sons, Inc., 1999. 33. C. E. McCulloch and S. R. Searle, Generalized, Linear and Mixed Models. New York: John Wiley & Sons, 2001. 34. D. M. Murray, S. P. Varnell, and J. L. Blitstein, Design and analysis of grouprandomized trials: a review of recent methodological developments. Amer. J. Public Health 2004; 94(3): 423–432. 35. D. M. Murray and J. L. Blitstein, Methods to reduce the impact of intraclass correlation

43.

44.

45.

46.

47.

48.

in group-randomized trials. Eval. Rev. 2003; 27(1): 79–103. J. B. Janega et al., Assessing intervention effects in a school-based nutrition intervention trial: which analytic model is most powerful? Health Educ. Behav. 2004; 31(6): 756–774. J. B. Janega et al., Assessing the most powerful analysis method for school-based intervention studies with alcohol, tobacco and other drug outcomes. Addict. Behav. 2004; 29(3): 595–606. S. Varnell et al., Design and analysis of grouprandomized trials: a review of recent practices. Amer. J. Public Health 2004; 94(3): 393–399. Z. Feng et al., Selected statistical issues in group randomized trials. Annu. Rev. Public Health 2001; 22: 167–187. Z. Feng and B. Thompson, Some design issues in a community intervention trial. Controlled Clin. Trials 2002; 23: 431–449. N. Klar and A. Donner, Current and future challenges in the design and analysis of cluster randomization trials. Stat. Med. 2001; 20: 3729–3740. D. M. Murray, Efficacy and effectiveness trials in health promotion and disease prevention: design and analysis of group-randomized trials. In: N. Schneiderman et al., (eds.), Integrating Behavioral and Social Sciences with Public Health. Washington, DC: American Psychological Association, 2000, pp. 305–320. D. M. Murray, Statistical models appropriate for designs often used in group-randomized trials. Stat. Med. 2001; 20: 1373–1385. J. W. Farquhar et al., Effects of communitywide education on cardiovascular disease risk factors: The Stanford Five-City Project. JAMA 1990; 264(3): 359–365. R. V. Luepker et al., Community education for cardiovascular disease prevention: risk factor changes in the Minnesota Heart Health Program. Amer. J. Public Health 1994; 84(9): 1383–1393. COMMIT Research Group, Community Intervention Trial for Smoking Cessation (COMMIT): I. Cohort results from a four-year community intervention. Amer. J. Public Health 1995; 85(2): 183–192. COMMIT Research Group, Community Intervention Trial for Smoking Cessation (COMMIT): II. Changes in adult cigarette smoking prevalence. Amer. J. Public Health 1995; 85(2): 193–200. M. Susser, Editorial: the tribulations of trials—intervention in communities. Amer. J. Public Health 1995; 85(2): 156–158.

GROUP-RANDOMIZED TRIALS 49. M. Ausems et al., Short-term effects of a randomized computer-based out-of-school smoking prevention trial aimed at elementary schoolchildren. Prevent. Med. 2002; 34: 581–589. 50. D. Lazovich et al., Effectiveness of a worksite intervention to reduce an occupational exposure: the Minnesota Wood Dust Study. Amer. J. Public Health 2002; 92(9): 1498–1505. 51. J. A. Mayer et al., Promoting skin cancer prevention counseling by pharmacists. Amer. J. Public Health 1998; 88(7): 1096–1099. 52. M. J. Rotheram-Borus, M. B. Lee, M. Gwadz, and B. Draimin, An intervention for parents with Aids and their adolescent children. Amer. J. Public Health 2001; 91(8): 1294–1302. 53. J. Segura et al., A randomized controlled trial comparing three invitation strategies in a breast cancer screening program. Prevent. Med. 2001; 33: 325–332. 54. L. I. Solberg, T. E. Kottke, and M. L. Brekke, Will primary care clinics organize themselves to improve the delivery of preventive services? A randomized controlled trial. Prevent. Med. 1998; 27: 623–631. 55. F. A. Stillman et al., Evaluation of the American Stop Smoking Intervention Study (ASSIST): a report of outcomes. J. Amer. Cancer Inst. 2003; 95(22): 1681–1691. 56. S. P. Fortmann et al., Community intervention trials: reflections on the Stanford FiveCity Project experience. Amer. J. Epidemiol. 1995; 142(6): 576–586. 57. T. D. Koepsell et al., Invited commentary: symposium on community intervention trials. Amer. J. Epidemiol. 1995; 142(6): 594–599. 58. D. M. Murray, Design and analysis of community trials: lessons from the Minnesota Heart Health Program. Amer. J. Epidemiol. 1995; 142(6): 569–575. 59. S. Varnell, D. M. Murray, and W. L. Baker, An evaluation of analysis options for the one group per condition design: can any of the alternatives overcome the problems inherent in this design? Eval. Rev. 2001; 25(4): 440–453. 60. Z. Feng, D. McLerran, and J. Grizzle, A comparison of statistical methods for clustered data analysis with Gaussian error. Stat. Med. 1996; 15: 1793–1806. 61. M. D. Thornquist and G. L. Anderson, Small sample properties of generalized estimating equations in group-randomized designs with gaussian response. Presented at the 120th Annual APHA Meeting, Washington, DC, 1992.

15

62. R. M. Bell and D. F. McCaffrey, Bias reduction in standard errors for linear regression with multi-stage samples. Survey Methodol. 2002; 28(2): 169–181. 63. M. Fay and P. Graubard, Small-sample adjustments for Wald-type tests using sandwich estimators. Biometrics 2001; 57: 1198–1206. 64. G. Kauermann and R. J. Carroll, A note on the efficiency of sandwich covariance matrix estimation. J. Amer. Stat. Assoc. 2001; 96(456): 1387–1396. 65. L. A. Mancl and T. A. DeRouen, A covariance estimator for GEE with improved small-sample properties. Biometrics 2001; 57: 126–134. 66. W. Pan and M. M. Wall, Small-sample adjustments in using the sandwich variance estimator in generalized estimating equations. Stat. Med. 2002; 21: 1429–1441. 67. T. Cai, S. C. Cheng, and L. J. Wei, Semiparametric mixed-effects models for clustered failure time data. J. Amer. Stat. Assoc. 2002; 95(458): 514–522. 68. R. J. Gray and Y. Li, Optimal weight functions for marginal proportional hazards analysis of clustered failure time data. Lifetime Data Anal. 2002; 8: 5–19. 69. D. Hedeker, O. Siddiqui, and F. B. Hu, Random-effects regression analysis of correlated grouped-time survival data. Stat. Meth. Med. Res. 2000; 9: 161–179. 70. M. R. Segal, J. M. Neuhaus, and I. R. James, Dependence estimation for marginal models of multivariate survival data. Lifetime Data Anal. 1997; 3: 251–268. 71. K. K. Yau, Multilevel models for survival analysis with random effects. Biometrics 2001; 57: 96–102. 72. T. Braun and Z. Feng, Optimal permutation tests for the analysis of group randomized trials. J. Amer. Stat. Assoc. 2001; 96: 1424–1432. 73. B. L. Rooney and D. M. Murray, A Metaanalysis of smoking prevention programs after adjustments for errors in the unit of analysis. Health Educ. Quart. 1993; 23(1): 48–64. 74. R. E. Kirk, Experimental Design: Procedures for the Behavioral Sciences, 2nd ed. Belmont, CA: Brooks/Cole Publishing Company, 1982. 75. C. L. Meinert, Clinical Trials. New York: Oxford University Press, 1986. 76. W. R. Shadish, T. D. Cook, and D. T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference.

16

GROUP-RANDOMIZED TRIALS

Boston, MA: Houghton Mifflin Company, 2002. 77. B. J. Winer, D. R. Brown, and K. Michels, Statistical Principles in Experimental Design. New York: McGraw-Hill, 1991.

GROUP SEQUENTIAL DESIGNS

interim analysis is performed in which a decision is made to stop or to continue the trial. Particularly, a trial can be stopped with the early rejection of a hypothesis H0 . A trial can also be stopped for futility if the data indicate that H 0 probably cannot be rejected at the planned end of the trial. As a result of the repeated testing at the interim analyses, the Type I error rate α will be inflated. Group sequential designs offer the possibility to account for this kind of multiplicity and provide test procedures that control the Type I error rate. In the simplest case, the maximum number of stages, K, to be performed and the number of observations per stage n1 , . . ., nK are fixed in advance. For example, in a two-stage design (i.e., K = 2), a prespecified number n1 of subjects are observed in the first stage. If the P-value for testing H0 falls below a specified boundary α1
, H 0 can be rejected and no further subjects need to be recruited. If the P-value is too large (e.g., greater than 0.50), it is unlikely to obtain a statistically significant result after the observation of the second-stage data. Hence, further recruitment of patients will be stopped and, of course, H0 cannot be rejected. If it is decided to continue the trial, in the second stage a prespecified number of subjects n2 are observed. If the P-value calculated from the total dataset (i.e., n1 + n2 observations) falls below a boundary α2
, H 0 is rejected, otherwise H 0 is not rejected. As a result of the interim look, a false rejection of H0 can occur in the interim as well as in the final analysis and two tests of significance are repeatedly applied to the accumulating data. Hence, if the P-values obtained from the first and from both stages, respectively, are compared with the unadjusted level α, the overall Type I error rate exceeds α. If α1
= α2
= α/2, the procedure controls α because of the Bonferroni correction. The choice of more sophisticated boundaries is one important and fundamental topic in the theory of group sequential test designs. Peter Armitage was the first who adopted the sequential testing approach to clinical research. The first edition of his book Sequential Medical Trials was published in

GERNOT WASSMER Institute for Medical Statistics, Informatics, and Epidemiology University of Cologne Cologne, Germany

1

INTRODUCTION

In clinical research there is a great interest in interim analyses for ethical, economical, and organizational or administrative reasons. Interim analyses are performed within sequential designs that offer the possibility to stop a trial early with a statistically significant test result. This trial will possibly need less patients than the trial with a fixed sample size where a test decision can be made only at the end of the trial. A therapy which was shown to be superior can be applied earlier and the inferior therapy can be replaced by the better one. Furthermore, in interim analyses the quality of the performance of the trial can be assessed and possibly be improved when necessary. Interim analyses are also carried out for the assessment of the safety of the treatments under consideration, and the observation of serious adverse events can lead to an early stopping of the trial. The appointment of an Independent Data and Safety Monitoring Board (DSMB) in a clinical trial is a recommended constitution to perform these issues according to generally accepted standards of Good Clinical Practice (GCP) (1–3). Another issue is the redesign of the trial (e.g., sample size reassessment), but it was not intended originally nor is it generally possible with ‘‘classical’’ group sequential designs (see Flexible Design, Interim Analysis, Stopping Boundaries). Sequential designs were originally developed for the analysis of observations as soon as they become available. In group sequential designs, statistical inference is performed after the observation of groups of subjects, which is appealing, especially in medical research, because such a trial does not need to be continuously monitored, but can be organized in several stages. After each stage, an

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

GROUP SEQUENTIAL DESIGNS

1960 (4). The concept of repeated significance tests and its application in clinical trials was initiated by him. Armitage et al. (5) introduced the recursive integration formula, which enables the implementation of the repeated significance testing approach. Pocock (6) and O’Brien and Fleming (7), on the other hand, gave the major impetus for the development of group sequential test procedures that are widely used today, especially in clinical research. For a short historical review of the early work about group sequential procedures, see Jennison and Turnbull (References 8 and 9, Section 1.3) Ghosh (10) provides a history of sequential analysis in general, taking into account the developments beginning in the seventeenth century. The development of group sequential designs was, for the most part, within the repeated significance testing approach. This approach is conceptually different from the ‘‘purely’’ sequential approach. Applying the latter, both types of error probabilities are under control and used to determine the stopping boundaries of the sequential test procedure. The comprehensive theoretical development of these procedures owes much to the optimality of the sequential probability ratio test (SPRT), and to the derivation of analytic expressions for the decision regions and certain test characteristics. The SPRT was initiated by Abraham Wald (11). It is an optimum test in the sense that the expected sample size is minimum under both the null hypothesis and the alternative hypothesis. Several textbooks contain the theoretical developments in sequential designs (e.g., References 12–15). Theoretical research on repeated significance tests was also conducted. A series of papers were concerned with finding a bound for the critical value and with an approximation for the power and the expected sample size. Much of this work is a result of the research group of David Siegmund, in which many of the theoretical results were obtained from renewal theory. Essential developments and the mathematical background are provided in the book by Siegmund (13). An important part in the development of sequential designs was concerned with ‘‘triangular plans,’’ in which the stopping region for the sum of the observations is defined

by two straight lines that cross at the end of the trial. These are ‘‘closed’’ or ‘‘truncated’’ plans in which a decision to stop the trial is fixed at some maximum amount of information. Much of the theoretical development is concerned with the overshooting problem that occurs when the test statistic is not on the boundary. Groups of observations as well as continuous monitoring can be taken into account. Mathematical sophistication has led to the ‘‘Christmas tree correction,’’ which turns out to be quite accurate in many cases of practical importance (16). Whitehead (15) provides a comprehensive overview of sequential methods with special reference to the triangular plans in clinical trials. In a review article, Whitehead (17) argues against the distinction of group sequential methods from the wider family of sequential methods. This is certainly true, but the development and investigation of group sequential designs was in some sense separated from the rigorous mathematical derivation of results within the sequential theory, which is also a result of the rapid development of computer power that made the computations of the recursive numerical integral introduced by Armitage et al. (5) possible. Easy-to-use computer programs are available today to investigate the characteristics of the procedures numerically. The group sequential designs introduced by Pocock and O’Brien and Fleming stand for the ‘‘classical group sequential designs.’’ The basic notions of these designs and some of their generalizations are discussed in the following section. 2 CLASSICAL DESIGNS Pocock (6) proposed two-sided tests for normal responses with known variance assuming that the number of observations is equal between the K stages (i.e., n1 = . . .= nK = n, where nk denotes the sample size per stage of the trial). This situation applies to testing the difference of means in a parallel group design as well as to testing a single mean in a one-sample design (or a design with paired observations). In a group sequential test design, the hypothesis can be rejected

GROUP SEQUENTIAL DESIGNS

at the kth stage and the trial is terminated after that stage if the P-value is smaller or equal than a value αk
. The adjusted nominal significance levels αk
, k = 1, . . . , K, are determined such that the overall probability of a false rejection of H0 in the sequential scheme does not exceed α, which can also be defined in terms of the adjusted critical bounds uk , k = 1, . . . , K, for the standardized test statistic ∗ Zk of the accumulating data. Any sequence of critical bounds u1 , . . . , uK that fulfills ∗

∗

PH0 (|Z1 | ≥ u1 or . . . or |ZK | ≥ uK ) = α

(1)

defines a valid level-α test procedure for the two-sided test design. The statistics ∗ ∗ ∗ Z1 , . . . , ZK are correlated as each Zk incorporates all observations from the first k stages, k = 1, . . . , K. Therefore, the probability on the left-hand side in Equation (1) must in principal be computed with the multivariate normal integral, and the critical values u1 , . . . , uK are found by ∗ a numerical search. Each statistic Zk (k > 1), however, can be written as a sum of independent statistics. As a result of this independent increments structure, one can use the recursive integration formula of Armitage et al. (5) to facilitate the computation of the multiple integral and the determination of the critical values. Pocock’s design is characterized by assuming constant critical values u1 = . . .= uk = u or, equivalently, constant adjusted nominal significance levels α1
= . . . = αk
= α
over the stages of the trial. O’Brien and Fleming’s design, on the other hand, is defined through monotonically decreasing critical values uk √ given by uk = c/ k, where c is a suitable constant such that Equation (1) holds. Therefore, this approach postulates more conservative nominal significance levels for early stages, which yields an increasing sequence of adjusted significance levels. In Table 1, the nominal adjusted levels αk
along with the corresponding critical values uk for the standardized test statistic are presented for both designs for α = 0.05 and K = 2, 3, 4, 5. These choices of K are in common use and the figures in the table can be immediately used for application. Figure 1 graphically illustrates the decision regions of a four-stage

3

O’Brien and Fleming’s and Pocock’s design in terms of the critical values uk . As K increases, the necessary adjustment becomes stronger. For Pocock’s design, the improvement of using an adjusted bound satisfying Equation (1) in place of the simple Bonferroni correction is obvious: Instead of using α/K at each stage of the test procedure, one can use the less stringent adjusted significance levels αk
of Table 1. In O’Brien and Fleming’s design, strong evidence of an effect is required to stop the trial at early stages, but it is easier to reject H0 later on. As a consequence, the last-stage critical value is near to the critical value of the two-sided fixed sample size design (i.e., no interim analysis). For α = 0.05, the latter is given by zα/2 = 1.96, where zα/2 denotes the upper α/2 quantile of the standard normal distribution. In other words, the price to pay in terms of having to use a more conservative level of the Type I error rate is low for interim looks when using O’Brien and Flemings test design. Even stricter criteria for interim looks were earlier independently proposed by Haybittle (18) and Peto et al. (19). They suggested using u1 = . . .= uK−1 = 3 and uK = zα/2 . The actual Type I error rate of this procedure exceeds the nominal level α, but the excess is small, and one can simply adopt the approach to adjust the critical value uK such that the Type I error rate of the procedure is maintained. For example, for K = 5 and α = 0.05, the adjusted critical level for the kth stage is given by 1.990, which only slightly exceeds 1.960 and is considerably smaller than the values 2.413 and 2.040 required by the Pocock and the O’Brien and Fleming design (Table 1). Different group sequential test designs can be compared with respect to the necessary maximum sample size and the average sample size or average sample number (ASN). Usually, one fixes an effect size ϑ* and looks for the sample size such that the probability to reject H 0 is equal to some desired power 1 − β. Given ϑ*, the probability to reject H 0 increases as the sample sizes n per stage increase. Hence, in a two-sided design with K stages, the sample size per stage, n, to achieve power 1 − β for a group sequential design at a prespecified effect size ϑ* is found

4

GROUP SEQUENTIAL DESIGNS Table 1. Adjusted Nominal Significance Levels α
k and Critical Values uk , k = 1, . . ., K for Pocock’s and O’Brien and Fleming’s Design (Two-Sided α = 0.05)

Z*k

Pocock

O’Brien and Fleming αk
uk

K

k

αk


2

1 2

0.02939 0.02939

2.178 2.178

0.00517 0.04799

2.797 1.977

3

1 2 3

0.02206 0.02206 0.02206

2.289 2.289 2.289

0.00052 0.01411 0.04507

3.471 2.454 2.004

4

1 2 3 4

0.01821 0.01821 0.01821 0.01821

2.361 2.361 2.361 2.361

0.00005 0.00420 0.01942 0.04294

4.049 2.863 2.337 2.024

5

1 2 3 4 5

0.01581 0.01581 0.01581 0.01581 0.01581

2.413 2.413 2.413 2.413 2.413

0.00001 0.00126 0.00845 0.02256 0.04134

4.562 3.226 2.634 2.281 2.040

uk

4 OBF

reject H0

3 P 2

continue trial

1 0 −1

continue trial

−2

P

−3

reject H0

OBF

−4

1

3

2 stage

4

Figure 1. Decision regions for a two-sided four-stage group sequential design according to O’Brien and Fleming (OBF) and Pocock (P), respectively (α = 0.05).

numerically by solving ASNϑ ∗ = n + nPϑ ∗ (|Z1 | < u1 ) ∗

∗

Pϑ ∗ (|Z1 | ≥ u1 or . . . or |ZK | ≥ uK ) = 1 − β (2)

for n. The maximum sample size is N = Kn. The ASN given ϑ* becomes

+ nPϑ ∗ (|Z1 | < u1 , |Z2 | < u2 ) + . . . + nPϑ ∗ (|Z1 | < u1 , . . . , |ZK−1 | < uK−1 ) (3) The ASN can also be calculated under some other effect size. If the parameter specified in the alternative is true, ASNϑ ∗ specifies

GROUP SEQUENTIAL DESIGNS

the expected sample size if the assumption made in the sample size calculation is true. In case of normally distributed observations, it can be shown that N and the corresponding ASNϑ ∗ are inversely proportional to ϑ ∗2 in analogy to the fixed sample size design. Particularly, it suffices to provide tables that specify the maximum and average sample size as compared with the fixed sample size design. In Table 2, the values of the relative change are provided for two-sided α = 0.05 and different values of K and 1 − β. The factor N/nf is called the inflation factor of a particular design because it specifies how much the maximum sample size in the group sequential design must be inflated relative to the fixed sample size design. Comprehensive tables for the inflation factor in different designs can be found in Reference 9. The inflation factor can be used to perform sample size calculations for a group sequential test design in very different situations. For example, let the necessary sample size in a fixed sample design at given α = 0.05 and 1 − β = 0.80 be calculated as, say, nf = 54. If one wants to use a five-stage Pocock design, the maximum sample size necessary to achieve power 80% is the next integer larger than 1.228 × 54 = 66.3. That is, 67 observations need to be suitably distributed among the stages of the trial. Clearly, the assumption of equal sample sizes between the stages cannot exactly be fulfilled in most cases. It can be shown, however, that a slight departure from this assumption has virtually no effect on the test characteristics. That is, it might be reasonable in this case to start with n1 = 14 observations, and the subsequent sample sizes are n2 = 14, n3 = n4 = n5 = 13. The average sample size of this design is 0.799 × 54 = 43.1, which means that an essential reduction in sample size as compared with the fixed sample design is expected in this trial. A further important test characteristic is the probability of rejecting the null hypothesis at a specific stage of the trial. In this example, if the alternative is true, the probabilities to reject H0 in the first, second, third, fourth, and fifth stage are given by 0.153, 0.205, 0.187, 0.148, and 0.107, respectively. That is, some chance exists to stop the trial with the rejection of H0 at very early stages, saving time and money. On the other

5

hand, if one wants to use an O’Brien and Fleming design, the maximum and average sample size are 1.028 × 54 = 55.5 and 0.818 × 54 = 44.2, respectively. The corresponding probabilities to reject H 0 in one of the five stages of the trial are 0.001, 0.076, 0.259, 0.276, and 0.188, respectively. That is, it is very unlikely with this design to reject H0 at early stages, but the maximum sample size is only slightly above the sample size in a fixed sample design. One must balance the reasons for choosing a design also including other aspects than reducing the sample size. It is an essential characteristic of the classical design, however, that a particular design chosen must be thoroughly discussed before the start of the trial. In the ongoing trial, this design cannot be changed (see Flexible Designs). For given power, the maximum sample size in O’Brien and Fleming’s design is only slightly larger than the sample size necessary in a fixed sample size design. On the other hand, the ASN under the specified alternative, ASNϑ ∗ , in Pocock’s design is smaller. For increasing number of stages, K, Table 2 shows that ASNϑ ∗ decreases and the maximum number of observations increases. Although generally not true (in fact, the ASNϑ ∗ in Pocock’s design starts slightly increasing again for very large K), the most relevant reduction in ASNϑ ∗ is reached for K = 5. This result was first shown in References 20 and 21 and provides additional reasoning for not performing too many stages. From an organizational perspective, even the choice of less than four, one or two, say, interim analyses might be preferred. Pocock’s and O’Brien and Fleming’s designs form the milestone in the development of many other types of designs that were developed during the last two decades. In a subsequent article, Pocock (21) found optimum critical values that, given K, α, and 1 − β, minimize the expected sample size under the alternative. These critical values were found by a K-dimensional grid search. For example, if K = 5, α = 0.05, and 1 − β = 0.95, the optimum critical values are given

6

GROUP SEQUENTIAL DESIGNS Table 2. Average Sample Size ASNϑ ∗ and Maximum Sample Size N Relative to the Sample Size nf in a Fixed Sample Size Design for Pocock’s and O’Brien and Fleming’s Design (Two-Sided α = 0.05) Pocock N/nf ASNϑ ∗ /nf

O’Brien and Fleming ASNϑ ∗ /nf N/nf

K

1−β

2

0.80 0.90 0.95

0.853 0.776 0.718

1.110 1.100 1.093

0.902 0.851 0.803

1.008 1.007 1.007

3

0.80 0.90 0.95

0.818 0.721 0.649

1.166 1.151 1.140

0.856 0.799 0.751

1.017 1.016 1.015

4

0.80 0.90 0.95

0.805 0.697 0.619

1.202 1.183 1.170

0.831 0.767 0.716

1.024 1.022 1.021

5

0.80 0.90 0.95

0.799 0.685 0.602

1.228 1.206 1.191

0.818 0.750 0.696

1.028 1.026 1.025

by (u1 , u2 , u3 , u4 , u5 ) = (2.446, 2.404, 2.404, 2.404, 2.396) For K = 5, α = 0.05, and 1 − β = 0.50, these values are given by (u1 , u2 , u3 , u4 , u5 ) = (3.671, 2.884, 2.573, 2.375, 2.037) That is, for high power values, a design with constant critical levels over the stages is (nearly) optimum with respect to the average sample size. For low power values, on the other hand, a design with decreasing critical values, like O’Brien and Fleming’s design, should be used. Wang and Tsiatis (22) suggested that the optimum critical values can approximately be found within the -class of critical values uk given by uk = c(K, α, )k−0.5 , k = 1, . . . , K

(4)

where c(K, α, ) is a suitably chosen constant. They showed that the optimized ASNϑ ∗ is only negligibly larger when searching within the one-parameter family of critical values defined through Equation (4) as compared with the K-dimensional search. For  = 0

and  = 0.5, one obtains O’Brien and Fleming’s and Pocock’s design, respectively. The Wang and Tsiatis -class family offers a wide class of boundaries with different shapes. The critical values and the properties of the tests with intermediate values of  were extensively tabulated (9, 23) or can be found using a suitable software package (24, 25). One-sided test designs were also considered in the literature. However, the issues involved in the question of whether a test should be two-sided or one-sided are getting more complex when considering multistage designs (26). Nevertheless, it is conceptually straightforward and occasionally reasonable to define one-sided tests in the context of group sequential designs. A set of critical values u1 , . . . , uK satisfying ∗

∗

PH0 (Z1 ≥ u1 or . . . or ZK ≥ uK ) = α

(5)

defines a one-sided group sequential test design. The power and the ASN are defined analogously. It is interesting that the critical values defined through Equation (5) are virtually (but not exactly) identical to the two-sided critical values defined through Equation (1) at significance level 2α, which is analogous to the fixed sample design. As the difference of the exact and the approximate critical values is less than 0.000001 for all

GROUP SEQUENTIAL DESIGNS

practically relevant situations, it suffices to consider only one case. DeMets and Ware (27, 28) considered the issue of stopping the trial for futility, which becomes evident in the one-sided setting. If, for example, the one-sided P-value at some stage k is greater than 0.50, the effect is directed opposed to the alternative. Hence, no reasonable chance to obtain a significant result at the end of the trial may exist. In the planning phase of the trial, it can be decided that in this case the trial will be stopped. Taking into account the stopping for futility option, the critical values are different as compared with the original design defined through Equation (5). Indeed, the critical values are somewhat smaller and, most importantly, if the null hypothesis is true, the ASN reduces considerably. DeMets and Ware considered various choices of stopping for futility boundaries, including the constant boundary (i.e. stopping the trial if the P-value exceeds a constant α 0 > α). Taking into account the possibility of stopping the trial in favor of H0 was also considered in the context of two-sided tests. Emerson and Fleming (29) proposed one-sided and two-sided group sequential designs in which both error probabilities are under control. These designs were termed ‘‘symmetric designs’’ and were generalized by Pampallona and Tsiatis (30). Within the -class of boundaries, they found designs that are optimal with respect to the average sample size under the alternative hypothesis. The application of the group sequential test design discussed so far is also possible for other designs. In simulation studies (6), it was shown that the use of the critical values derived for normally distributed observations with a known variance provide sufficient control of the nominal level if the variance is unknown and the t-test situation applies. Furthermore, in designs with binary or exponential response or many other types of response variables, the boundaries derived for the normal case can be applied. Generally, if the sample size is not too small, the Type I error rate is approximately equal to α, if the increments are independent and approximately normal, which is also (approximately) true for studies with a survival endpoint, certain longitudinal designs, and

7

for nonparametric testing situations, which offers a wide application of group sequential designs in clinical trials (e.g., References 31–34). Some work was also done on the exact calculation for group sequential trials in other designs such as trials comparing two treatments with dichotomous response (35); exact calculations for the t-test, χ 2 , and Ftest situation (36); or exact group sequential permutational tests (37). 3

THE α-SPENDING FUNCTION APPROACH

The test procedures discussed in the last section were originally designed for a fixed maximum number of stages and equally sized groups of observations per stage. In practice, however, the latter assumption is rarely exactly fulfilled. The schedule of interim analyses can at best ensure that the stage sizes are roughly the same. In practice of clinical trials, the strict assumption of equal sample sizes may prove impractical. The size and power requirement are not greatly affected, however, if this assumption is not grossly violated. Nevertheless, situations exist with a substantial effect on size and power (38). Hence, the importance of more flexible procedures is obvious. One can obtain valid tests relaxing the strict assumption of equal sample sizes between the stages. First of all, one may prespecify sample sizes n1 , . . ., nK , which are not necessarily equal. For example, consider a four-stage design in which the first interim analysis should take place after observation of 10% of all planned sampling units, the second interim after 40%, and the third interim analysis after 70% of observations. With these so-called information rates
tk = ki=1 ni /N specified in advance, it is possible to derive the maximum sample size, N, such that the procedure has desired power 1 − β, given K, α, and the desired boundary shape. The critical values and the test characteristics differ from those derived with equally spaced stages. One can even optimize a group sequential plan with regard to, say, the ASN under the alternative considering varying information rates and varying boundary shapes (39, 40). The α-spending function or use function approach, proposed by Lan and DeMets (41)

8

GROUP SEQUENTIAL DESIGNS

and extended by Kim and DeMets (42), is conceptually different. The idea is to specify the amount of significance level spent up to an interim analysis rather than the shape of the adjusted critical levels. One uses a function α*(tk ) that specifies the cumulative Type I error rate spent at a time point tk of the kth analysis. α*(tk ) is a non-decreasing function with α*(0) = 0 and α*(1) = α that must be fixed before the start of the trial. As above, tk represents the information rate that develops from the specific course of the study (43). The information rates need not be prespecified before the actual course of the trial but must be chosen independently of the observed effect sizes. Consequently, the number of observations at the kth analysis and the maximum number K of analyses are flexible. In the two-sided case, the critical value for the first analysis is given by u1 = −1 (1 − α*(t1 )/2), where t1 = n1 /N and −1 (·) refers to the inverse normal cumulative distribution function. The critical values for the remaining stages are then computed successively. That is, once uk−1 is calculated, the critical value at stage k is calculated through ∗

∗

πk = PH0 (|Z1 | < u1 , . . . , |Zk−1 | < uk−1 , ∗

|Zk | ≥ uk ) ∗

∗

= α (tk ) − α (tk−1 )

(6)

π k denotes the probability of a Type I error
at stage k and, therefore, K k=1 πk = α . The one-sided case is treated analogously. In this way, the sequence of critical values u1 , . . . , uK that defines a group sequential test is not specified in advance but results from the information rates observed during the course of the trial. The use function approach is particularly attractive if the interim analyses are planned at specific time points rather than after a specific number of evaluable observations has been obtained. Particularly, the sample size per stage may be an unknown quantity that will be observed when the interim analysis is performed. With this approach, even the number of interim analyses must not be fixed in advance. Instead, a maximum amount of information must be specified. In the simplest case, it is the maximum sample size N of

the trial. Through the use of a specified use function, the significance level spent up to this information is fixed in advance, which enables the calculation of the adjusted levels. The overall significance level α is maintained if the study stops whenever tk = tK = 1. The actually observed number of observations, however, may fall short of or exceed N. To account for the latter case ∗ (i.e., tK >1), the α-spending function α˜ (tk ) = ∗ min(α, α (tk )) might be used to account for random overrunning. If the study stops with a smaller maximum sample size than antic∗ ipated (i.e., tK < 1), then setting α˜ (tK ) = α forces the procedure to fully exhaust the level α up to the last stage (44). An important application of group sequential designs is in trials where the endpoint is the time to an event (e.g., survival data). It was shown by Tsiatis (34) that the usual logrank test can be embedded into the group sequential design. The information is the observed number of events (i.e., deaths), and the use function approach turns out to be a very useful and flexible instrument for analyzing such trials in a sequential design (32, 45). A number of proposals were designed in the literature for the form of the function α*(tk ). The α-spending functions ∗

(7) α1 (tk ) = α ln(1 + (e − 1) · tk ) and  √ 2(1 − (zα/2 / tk )) (one-sided case) ∗ α2 (tk ) = √ 4(1 − (zα/4 / tk )) (two-sided case) (8) approximate Pocock’s and O’Brien and Fleming’s group sequential boundaries, respectively. Kim and DeMets (42) proposed a one parameter family of α-spending functions ∗

ρ

α3 (ρ, tk ) = αtk

(9)

The spending functions in Equations (7)–(9) are illustrated in Fig. 2. Notice that the use of constant adjusted significance levels (i.e., Pocock’s design) does not correspond to a linear α-spending function. Instead, assuming a linear α-spending function (i.e., ∗ using α3 (ρ) with ρ = 1) and equally spaced

GROUP SEQUENTIAL DESIGNS a*(tk ) 0.05 0.04 a*3 (2)

0.03

a*3 (1.5) a*3 (1) a*1

0.02

a*2

0.01 0.00 0.0

0.2

0.4

0.6

0.8

1.0

tk

Figure 2. Examples of α-spending functions. α1∗ and α2∗ approximate Pocock’s and O’Brien and Fleming’s group sequential boundaries, respectively. Kim and DeMets α-spending function class α3∗ (ρ) is shown for ρ = 1, 1.5, 2.

stage sizes yields somewhat decreasing critical values. Hwang et al. (46) introduced a one parameter family of α-spending functions and showed that the use of this family yields approximately optimal plans similar to the class of Wang and Tsiatis (22). Optimum tests adapting the use function approach were also found by Jennison (47). Similar approaches were proposed by Fleming et al. (48) and Slud and Wei (33). They are defined in terms of the Type I error rates π k in Equation (6) by specifying π 1 , . . ., π K summing to α, which is different from the spending function approach that specifies π k at some information rate tk . Nevertheless, it requires the same technique for computing the decision regions. It is tempting to use the results of an interim analysis to modify the schedule of interim looks, which is particularly true for the α-spending function approach because the maximum number of analyses must not be prespecified. For example, if the test result is very near to showing significance, it could be decided to conduct the next interim analysis earlier than originally planned. However, a data-driven analysis strategy is not allowed for the α-spending function approach. Cases exist in which the Type I error rate is seriously inflated, as was shown by several

9

authors (8, 38). In this case, therefore, one should use adaptive or flexible designs that are designed specifically for a data-driven analysis strategy and offer an even larger degree of flexibility (see Flexible Designs). The α-spending function approach can be generalized in several ways. First, it is easy to implement an asymmetric procedure for the determination of an upper and a lower bound in a two-sided test situation. Then, two αspending functions must be given specifying the Type I error rate for the lower and the upper bound, respectively. Second, for planning purposes, it is quite natural to consider a function describing how the power of the procedure should be spent during the stages of the study, known as the power spending function. Approaches that use the power spending function are described in References 49–51 . 4 POINT ESTIMATES AND CONFIDENCE INTERVALS An important field of research in group sequential designs was concerned with the parameter estimation. Through the use of a stopping rule (i.e., the possibility of stopping a trial early with the rejection (or acceptance) of the null hypothesis), point estimates that are derived for the fixed sample size case (e.g., maximum likelihood estimates) are biased. In the long run, hence, one is faced with the over-estimation or underestimation of the true parameter. Point estimates were proposed that correct for the (overall) estimation bias through numerical methods (e.g., References 52–60). Two conceptually different methods for the calculation of confidence intervals were considered in the literature. The first method enables the calculation after the trial has stopped and a final decision of rejection or acceptance of the null hypothesis was reached (e.g., References 59, 61–66). This approach requires the strict adherence to the stopping rule and depends on the ordering in the sample space, which means that it has to be decided if, for example, an observed effect leading to the rejection of the null hypothesis in the first interim is ‘‘more extreme’’ than an effect that is larger but observed in the second interim analysis. If so, the resulting ordering rates effects obtained in earlier

10

GROUP SEQUENTIAL DESIGNS

stages of the trial are stronger than effects obtained later on. However, other orderings exist that are reasonable choices too, and no general agreement exists over which ordering is the best. Generally, it is possible to derive confidence intervals based on such orderings with confidence level exactly equal to 1 − α. Bias adjusted point estimates can be derived as the 50% confidence limit yielding median unbiased estimates. Furthermore, dependent on the ordering of the sample space, it is also possible to obtain overall P-values after completion of the trial (67). The second method merely takes into account the multiplicity that develops from the repeated looks at the data. The resulting intervals are called Repeated Confidence Intervals (RCIs). They are defined as a sequence of intervals Ik for which the coverage probability is simultaneously fulfilled, i.e., Pϑ (ϑ ∈ Ik , k = 1, . . . , K) = 1 − α where ϑ is the unknown parameter. This concept was introduced by Jennison and Turnbull (68, 69) [see also Lai (70)]. The calculation of RCIs is straightforward. When considering the family of group sequential tests for testing H 0 : ϑ = ϑ 0 , one simply has to find, at given stage k, the values ϑ 0 that do not lead to a rejection of H0 . Typically, these values are easy to find, and closed form analytical expressions exist in many cases. In more complicated situations, the RCI can be found iteratively. For example, when considering the mean of normally distributed observations, the sequence of RCIs is given by the sequence of intervals Ik = [xk − uk SEM; xk + uk SEM], k = 1, . . . , K where xk is the observed mean of observations up to stage k, SEM denotes the standard error of the mean, and u1 , . . ., uK are the adjusted critical values corresponding to a particular group sequential test design. The RCIs are wider than the confidence interval in a fixed sample design because they take into account the repeated looks at the accumulating data. For a design with larger critical values at early stages (e.g., O’Brien and Fleming’s design), these intervals are

fairly wide in the first stage, but become narrower as the study progresses. The RCIs can be calculated at each interim analysis. Based on the RCI, it might be decided to terminate or to continue the trial. For example, consider testing the hypothesis H 0 : ϑ = 0. H 0 can be rejected at stage k if the RCI at stage k does not contain ϑ = 0. But also if, in this case, it is decided to continue the trial, the calculation of RCIs in subsequent stages is possible and might be used for the decision process. The RCIs are independent of the adherence to the stopping rule and can therefore be calculated also if the study is going on. As a consequence, these intervals are conservative if not all stages of the trial were actually performed. 5 SUPPLEMENTS Several important issues are not addressed in this article and are referred to further study of the literature. Incorporating covariate information is considered in References 71–73 . In trials with multiple endpoints or in multi-armed trials, such as dose-response trials, the multiplicity occurs in two dimensions. First, the study is performed sequentially and one has to account for the multiple looks effect. Second, at each stage of the procedure, the need exists to perform multiple tests on several hypotheses concerning single endpoints or one wants to perform multiple comparisons between the treatment groups. Furthermore, it should be possible to drop inferior treatments in a multi-armed trial. In multi-armed trials, group sequential methods can be applied, too, and some current work exists in this area (e.g., References 74–80). Parametric and nonparametric group sequential tests for multiple endpoints were proposed in, for example, References 81–86. Last, group sequential designs in equivalence trials were among other designs discussed in References 40, 87–90. Group sequential designs require the need of specialized software. Software packages are available that are specifically designed for planning and analyzing a sequentially planned trial (24, 25, 91), which helps for the correct implementation of a group sequential design and ensures the accurate analysis.

GROUP SEQUENTIAL DESIGNS

For advanced users, it is also possible to perform the relevant calculations using standard statistical software packages. SAS/IML (92) as well as S-PLUS (93) provide modules for the calculation and the assessment of group sequential test designs.

REFERENCES 1. P. Armitage, Interim analysis in clinical trials. Stat. Med. 1991; 10: 925–937. 2. K. McPherson, Sequential stopping rules in clinical trials. Stat. Med. 1990; 9: 595–600. 3. A. J. Sankoh, Interim analysis: an update of an FDA reviewer’s experience and perspective. Drug Inf. J. 1999; 33: 165–176. 4. P. Armitage, Sequential Medical Trials. 2nd ed. Oxford: Blackwell, 1975. 5. P. Armitage, C. K. McPherson, and B. C. Rowe, Repeated significance tests on accumulating data. J. Roy. Stat. Soc. A 1969; 132: 235–244. 6. S. J. Pocock, Group sequential methods in the design and analysis of clinical trials. Biometrika 1977; 64: 191–199. 7. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics 1979; 35: 549–556. 8. C. Jennison and B. W. Turnbull, Group sequential tests and repeated confidence intervals. In: B. K. Ghosh and P. K. Sen (eds.), Handbook of Sequential Analysis. New York: Marcel Dekker, 1991, pp. 283–311. 9. C. Jennison and B. W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. Boca Raton, FL: Chapman & Hall, 2000.

11

16. J. Whitehead and I. Stratton, Group sequential clinical trials with triangular rejection regions. Biometrics 1983; 39: 227–236. 17. J. Whitehead, Sequential methods. In: C. K. Redmond and T. Colton (eds.), Biostatistics in Clinical Trials. Chichester: Wiley, 2001, pp. 414–422. 18. J. L. Haybittle, Repeated assessments of results in clinical trials of cancer treatment. Brit. J. Radiol. 1971; 44: 793–797. 19. R. Peto, M. C. Pike, P. Armitage, N. E. Breslow, D. R. Cox, S. V. Howard, N. Mantel, K. McPherson, J. Peto, and P. G. Smith, Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Brit. J. Cancer 1976; 34: 585–612. 20. K. McPherson, On choosing the number of interim analyses in clinical trials. Stat. Med. 1982; 1: 25–36. 21. S. J. Pocock, Interim analyses for randomized clinical trials: the group sequential approach. Biometrics 1982; 38: 153–162. 22. S. K. Wang and A. A. Tsiatis, Approximately optimal one-parameter boundaries for group sequential trials. Biometrics 1987; 43: 193–199. 23. G. Wassmer, Statistische Testverfahren fur ¨ gruppensequentielle und adaptive Plane ¨ in klinischen Studien. Theoretische Konzepte und deren praktische Umsetzung mit SAS. K¨oln: Verlag Alexander M¨onch, 1999. 24. Cytel Software Corporation, EaSt 2000: Software for the Design and Interim Monitoring of Group Sequential Clinical Trials. Cambrigde, MA: Cytel, 2000. 25. G. Wassmer and R. Eisebitt, ADDPLAN: Adaptive Designs - Plans and Analyses. Cologne: ADDPLAN GmbH, 2002.

10. B. K. Ghosh, A brief history of sequential analysis. In: B. K. Ghosh and P. K. Sen (eds.), Handbook of Sequential Analysis. New York: Marcel Dekker, 1991, pp. 1–19.

26. P. C. O’Brien, Data and safety monitoring. In: P. Armitage and T. Colton (eds.), Encyclopedia of Biostatistics. Chichester: Wiley, 1998, pp. 1058–1066.

11. A. Wald, Sequential Analysis. New York: Wiley, 1947.

27. D. L. DeMets and J. H. Ware, Group sequential methods for clinical trials with a one-sided hypothesis. Biometrika 1980; 67: 651–660.

12. B. K. Ghosh, Sequential Tests of Statistical Hypotheses. Reading, MA: Addison-Wesley, 1970. 13. D. Siegmund, Sequential Analysis. New York: Springer, 1985.

28. D. L. DeMets and J. H. Ware, Asymmetric group sequential boundaries for monitoring clinical trials. Biometrika 1982; 69: 661–663.

14. G. B. Wetherill, Sequential Methods in Statistics. London: Chapman and Hall, 1975.

29. S. S. Emerson and T. R. Fleming, Symmetric group sequential test designs. Biometrics 1989; 45: 905–923.

15. J. Whitehead, The Design and Analysis of Sequential Clinical Trials, rev. 2nd ed. Chichester: Wiley, 1997.

30. S. Pampallona and A. A. Tsiatis, Group sequential designs for one-sided and two-sided hypothesis testing with provision for early

12

GROUP SEQUENTIAL DESIGNS stopping in favour of the null hypothesis. J. Stat. Plan. Inf. 1994; 42: 19–35.

31. D. L. DeMets and M. H. Gail, Use of logrank tests and group sequential methods at fixed calender times. Biometrics 1985; 41: 1039–1044. 32. K. K. G. Lan and J. M. Lachin, Implementation of group sequential logrank tests in a maximum duration trial. Biometrics 1990; 46: 759–770. 33. E. Slud and L. J. Wei, Two-sample repeated significance tests based on the modified Wilcoxon statistic. J. Amer. Stat. Ass. 1982; 77: 862–868. 34. A. A. Tsiatis, Repeated significance testing for a general class of statistics used in censored survival analysis. J. Amer. Stat. Ass. 1982; 77: 855–861. 35. D. Y. Lin, L. J. Wei, and D. L. DeMets, Exact statistical inference for group sequential trials. Biometrics 1991; 47: 1399–1408. 36. C. Jennison and B. W. Turnbull, Exact calculations for sequential t, chi-square and F tests. Biometrika 1991; 78: 133–141. 37. C. R. Mehta, N. Patel, P. Senchaudhuri, and A. A. Tsiatis, Exact permutational tests for group sequential clinical trials. Biometrics 1994; 50: 1042–1053. 38. M. A. Proschan, D. A. Follmann, and M. A. Waclawiw, Effects on assumption violations on type I error rate in group sequential monitoring. Biometrics 1992; 48: 1131–1143. 39. E. H. Brittain and K. R. Bailey, Optimization of multistage testing times and critical values in clinical trials. Biometrics 1993; 49: 763–772.

45. K. Kim and A. A. Tsiatis, Study duration for clinical trials with survival response and early stopping rule. Biometrics 1990; 46: 81–92. 46. I. K. Hwang, W. J. Shih, and J. S. DeCani, Group sequential designs using a family of Type I error probability spending functions. Stat. Med. 1990; 9: 1439–1445. 47. C. Jennison, Efficient group sequential tests with unpredictable group sizes. Biometrika 1987; 74: 155–165. 48. T. R. Fleming, D. P. Harrington, and P. C. O’Brien, Designs for group sequential trials. Contr. Clin. Trials 1984; 5: 348–361. 49. P. Bauer, The choice of sequential boundaries based on the concept of power spending. Biom. Inform. Med. Biol. 1992; 23: 3–15. 50. M. N. Chang, I. K. Hwang, and W. J. Shih, Group sequential designs using both type I and type II error probability spending functions. Comm. Stat. Theory Meth. 1998; 27: 1323–1339. 51. S. Pampallona, A. A. Tsiatis, and K. Kim, Interim monitoring of group sequential trials using spending functions for the type I and type II error probabilities. Drug Inf. J. 2001; 35: 1113–1121. 52. S. S. Emerson, Computation of the uniform minimum variance unbiased estimator of the normal mean following a group sequential trial. Comput. Biomed. Res. 1993; 26: 68–73. 53. S. S. Emerson and T. R. Fleming, Parameter estimation following group sequential hypothesis testing. Biometrika 1990; 77: 875–892. 54. S. S. Emerson and J. M. Kittelson, A computationally simpler algorithm for the UMVUE of a normal mean following a sequential trial. Biometrics 1997; 53: 365–369.

¨ ¨ 40. H-H. Muller and H. Schafer, Optimization of testing times and critical values in sequential equivalence testing. Stat. Med. 1999; 18: 1769–1788.

55. K. Kim, Point estimation following group sequential tests. Biometrics 1989; 45: 613–617.

41. K. K. G. Lan and D. L. DeMets, Discrete sequential boundaries for clinical trials. Biometrika 1983; 70: 659–663.

56. A. Liu and W. J. Hall, Unbiased estimation following a group sequential test. Biometrika 1991; 86: 71–78.

42. K. Kim and D. L. DeMets, Design and analysis of group sequential tests based on the Type I error spending rate function. Biometrika 1987; 74: 149–154.

57. J. C. Pinheiro and D. L. DeMets, Estimating and reducing bias in group sequential designs with Gaussian independent increment structure. Biometrika 1997; 84: 831–845.

43. K. K. G. Lan and D. L. DeMets, Group sequential procedures: calender versus information time. Stat. Med. 1989; 8: 1191–1198.

58. E. Skovlund and L. Walløe, Estimation of treatment difference following a sequential clinical trial. J. Amer. Stat Ass. 1989; 84: 823–828.

44. K. Kim, H. Boucher, and A. A. Tsiatis, Design and analysis of group sequential logrank tests in maximum duration versus information trials. Biometrics 1995; 51: 988–1000.

59. S. Todd, J. Whitehead, and K. M. Facey, Point and interval estimation following a sequential clinical trial. Biometrika 1996; 83: 453–461.

GROUP SEQUENTIAL DESIGNS 60. J. Whitehead, On the bias of maximum likelihood estimation following a sequential test. Biometrika 1986; 73: 573–581. 61. M. N. Chang and P. C. O’Brien, Confidence intervals following group sequential tests. Contr. Clin. Trials 1986; 7: 18–26. 62. D. S. Coad and M. B. Woodroofe, Corrected confidence intervals after sequential testing with applications to survival analysis. Biometrika 1996; 83: 763–777. 63. D. E. Duffy and T. J. Santner, Confidence intervals for a binomial parameter based on multistage tests. Biometrics 1987; 43: 81–93. 64. K. Kim and D. L. DeMets, Confidence intervals following group sequential tests in clinical trials. Biometrics 1987; 43: 857–864. 65. G. L. Rosner and A. A. Tsiatis, Exact confidence intervals following a group sequential trial: a comparison of methods. Biometrika 1988; 75: 723–729. 66. A. A. Tsiatis, G. L. Rosner, and C. R. Mehta, Exact confidence intervals following a group sequential test. Biometrics 1984; 40: 797–803. 67. K. Fairbanks and R. Madsen, P values for tests using a repeated significance test design. Biometrika 1982; 69: 69–74. 68. C. Jennison and B. W. Turnbull, Repeated confidence intervals for group sequential clinical trials. Contr. Clin. Trials 1984; 5: 33–45. 69. C. Jennison and B. W. Turnbull, Interim analysis: the repeated confidence interval approach. J. R. Statist. Soc. B 1989; 51: 305–361. 70. T. L. Lai, Incorporating scientific, ethical and economic considerations into the design of clinical trials in the pharmaceutical industry: a sequential approach. Comm. Stat. Theory Meth. 1984; 13: 2355–2368. 71. C. Jennison and B. W. Turnbull, Group sequential analysis incorporating covariate information. J. Amer. Stat. Ass. 1997; 92: 1330–1341. 72. D. O. Scharfstein, A. A. Tsiatis, and J. M. Robins, Semiparametric efficiency and its implication on the design and analysis of group-sequential studies. J. Amer. Stat. Ass. 1997; 92: 1342–1350. 73. A. A. Tsiatis, G. L. Rosner, and D. L. Tritchler, Group sequential tests with censored survival data adjusting for covariates. Biometrika 1985; 365–373. 74. D. A. Follmann, M. A. Proschan, and N. L. Geller, Monitoring pairwise comparisons in multi-armed clinical trials. Biometrics 1994; 50: 325–336.

13

75. N. L., Geller M. A. Proschan, and D. A. Follmann, Group sequential monitoring of multiarmed clinical trials. Drug Inf. J. 1995; 29: 705–713. 76. M. Hellmich, Monitoring clinical trials with multiple arms. Biometrics 2001; 57: 892–898. 77. M. Hughes, Stopping guidelines for clinical trials with multiple treatments. Stat. Med. 1993; 12: 901–915. 78. W. Liu, A group sequential procedure for allpairwise comparisons of k treatments based on range statistics. Biometrics 1995; 51: 946–955. 79. K. J. Lui, A simple generalization of the O’Brien and Fleming group sequential test procedure to more than two treatment groups. Biometrics 1993; 49: 1216–1219. 80. M. A. Proschan, D. A. Follmann, and N. L. Geller, Monitoring multi-armed trials. Stat. Med. 1994; 13: 1441–1452. 81. C. Jennison and B. W. Turnbull, Group sequential tests for bivariate response: interim analyses of clinical trials with both efficacy and safety endpoints. Biometrics 1993; 49: 741–752. 82. J. M. Lachin, Group sequential monitoring of distribution-free analyses of repeated measures. Stat. Med. 1997; 16: 653–668. 83. S. J. Lee, K. Kim, and A. A. Tsiatis, Repeated significance testing in longitudinal clinical trials. Biometrika 1996; 83: 779–789. 84. J. Q. Su and J. M. Lachin, Group sequential distribution-free methods for the analysis of multivariate observations. Biometrics 1992; 48: 1033–1042. 85. D-I. Tang, C. Gnecco, and N. L. Geller, Design of group sequential clinical trials with multiple endpoints. J. Amer. Stat. Ass. 1989; 84: 776–779. 86. S. Todd, Sequential designs for monitoring two endpoints in a clinical trial. Drug Inf. J. 1999; 33: 417–426. 87. S. Durrlemann and R. Simon, Planning and monitoring of equivalence studies. Biometrics 1990; 46: 329–336. 88. C. Jennison and B. W. Turnbull, Sequential equivalence testing and repeated confidence intervals, with application to normal and binary responses. Biometrics 1993; 49: 31–43. 89. J. M. Kittelson and S. S. Emerson, A unifying family of group sequential test designs. Biometrics 1999; 55: 874–882. 90. J. Whitehead, Sequential designs for equivalence studies. Stat. Med. 1996; 15: 2703–2715.

14

GROUP SEQUENTIAL DESIGNS

91. MPS Research Unit, PEST 4: Operating Manual. Reading, UK: University of Reading, 2000. 92. SAS Institute, Inc., SAS/IML Software: Changes and Enhancements through Release 6. 11. Cary, NC: SAS Institute, Inc., 1995. 93. Mathsoft Inc., S-Plus 2000. Seattle, WA: MathSoft, 2000.

HAZARD RATE MITCHELL H. GAIL National Cancer Institute Bethesda, MD, USA

The hazard rate at time t of an event is the limit λ(t) = limit↓0 −1 Pr (t  T≺ t + |t  T), where T is the exact time to the event. Special cases and synonyms of hazard rate, depending on the event in question, include force of mortality (where the event is death), instantaneous incidence rate, incidence rate, and incidence density (where the event is disease occurrence). For events that can only occur once, such as death or first occurrence of an illness, the probability that the event occurs
t in the interval [0, t) is given by 1 − exp (− 0 λ(u)du). The
t quantity 0 λ(u)du is known as the cumulative hazard. Often, the theoretical hazard rate λ(u) is estimated by dividing the number of events that arise in a population in a short time interval by the corresponding person-years at risk. The various terms, hazard rate, force of mortality, incidence density, person–years incidence rate, and incidence rate are often used to denote estimates of the corresponding theoretical hazard rate.

1

HAZARD RATIO CLAUDIA SCHMOOR ERIKA GRAF

risk ratio is illustrated by comparing two theoretical populations. Then, using data from a hypothetical clinical trial, the calculation of various estimators is demonstrated for the hazard ratio and comments are made on their mathematical properties. Some references for further reading conclude the article.

and

University Hospital Freiburg Center of Clinical Trials Freiburg, Germany

1

2

INTRODUCTION

DEFINITIONS

Mathematically, the hazard rate of a population is defined as

Hazard ratios are useful measures to compare groups of patients with respect to a survival outcome. In the context of clinical trials ‘‘survival outcome’’ or ‘‘survival time’’ are used generically to denote the time ‘‘survived’’ from a specified time origin, like diagnosis of cancer, start of treatment, or first infarction, to a specific event of interest, like death, recurrence of cancer, or reinfarction (see survival analysis). Broadly speaking, the hazard rate quantifies the risk of experiencing an event at a given point in time. The hazard ratio at that time is the ratio of this risk in one population divided by the risk in a second population chosen as reference group. The hazard ratio between two populations can vary over time. For a given point in time, hazard ratios below 1.0 indicate that the risk of the event is lower in the first group than the risk in the reference group, whereas a hazard ratio of 1.0 indicates equal risks, and values above 1.0 imply a higher risk in the first group compared with the reference population. Often, it is plausible to assume proportional hazards (see Cox’s proportional hazards model) (i.e., to assume that although the respective hazard rates in each group may change in time, their ratio is constant over time). In this case, the constant hazard ratio adequately summarizes the relative survival experience in the two groups over the whole time axis. It describes the size of the difference between groups, similarly as a risk ratio or relative risk, which is used to compare proportions. However, the interpretation is different, as will be explained later. In the sequel, a formal definition of the hazard ratio is first provided, and the interpretation of hazard rate, hazard ratio, and

λ(t) = −

d d 1 · S(t) = − (log S(t)) S(t) dt dt

(1)

where S(t) denotes the survival function of the population (i.e., the proportion surviving beyond time t). Thus, λ(t) is the slope or instantaneous rate at which the population diminishes in the time point t, rescaled by the factor 1/S(t) to the surviving proportion S(t) of the population. Therefore, λ(t) is also called ‘‘instantaneous failure rate.’’ ‘‘Failure rate,’’ ‘‘force of mortality,’’ ‘‘hazard (rate) function,’’ or ‘‘hazard’’ have also been used to denote λ(t). λ(t) is approximately equal to the proportion of individuals that will die within one time unit from t, out of the subgroup of those that are still alive in t. Either S(t) or λ(t) may be used to characterize the survival distribution of a population, so that if one is known, the other is automatically determined. If two populations A and B with hazard rates λA (t) and λB (t) are now considered, their hazards may be compared by forming the so-called relative hazard function θ (t) =

λB (t) λA (t)

(2)

θ (t) is the hazard ratio (or relative hazard) of population B compared with population A at time t. In cases where the relative hazard function θ (t) of the two populations does not depend on t, the constant ratio θ (t) = θ is called the hazard ratio (or relative hazard) of population B compared to population A. If the hazard ratio θ (t) = θ is constant, a relationship can be established between

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

HAZARD RATIO

the hazard ratio and the two survival functions: Using Equation (1), it can be shown by mathematical transformations that for all t θ=

log[SB (t)] log[SA (t)]

(3)

Conversely, SB (t) can then be expressed as a function of SA (t) and the constant hazard ratio θ : SB (t) = SA (t)θ

for all

t,

(4)

following Equation (3). In contrast, the risk ratio (or relative risk) of population B versus population A at time t, RR(t) =

1 − SB (t) , 1 − SA (t)

(5)

compares cumulative risks of dying up to time t. 3 ILLUSTRATION OF HAZARD RATE, HAZARD RATIO, AND RISK RATIO To illustrate the interpretation of survival functions, hazard rates, hazard ratios, risk ratios (or relative risks), and the relationship between them, two hypothetical populations A and B are considered. Assume that the hypothetical control population A has a survival rate that can be described by the function SA (t) = exp( − (0.3 t)1.3 ). The corresponding survival rates of this population at yearly intervals are displayed in Table 1: 81.1% survive the first year from diagnosis, 59.8% survive the first two years, and so on, until after 10 years nearly the entire population has died. Now assume a new treatment is introduced, to the effect of reducing, at all time points t, the instantaneous death rate by 50% in the corresponding population B, which implies a constant hazard ratio of θ = 0.5 for population B versus population A as in Equation (3), so that SB (t) = SA (t)0.5 , following Equation (4). As a result, survival improves to 81.1%0.5 = 90.1% after 1 year, to 59.8%0.5 = 77.3% after 2 years, and so on. The hazard rates are also shown in Table 1. Note that although they increase over time, their ratio for population B versus population

A is indeed constant over time and equal to θ = 0.5. In contrast, the risk ratio displayed in the last column of Table 1 strongly depends on t. Here, the absolute risks of dying up to time t are being compared for population B versus population A. For example, after 4 years from diagnosis, 100% − 28.2% = 71.8% have died in the control population A, compared with 100% − 53.1% = 46.9% in population B, yielding a risk ratio of 46.9/71.8 = 0.653 for treatment versus control at t = 4. In other words, treatment decreases the risk of dying within 4 years from diagnosis by a factor of 0.653. This risk ratio is higher than the hazard ratio of 0.5, and it has a different interpretation. Recall that at all times t the hazard rate can roughly be approximated by the proportion of patients dying within 1 year among the patients still alive at the beginning of the year. S(t) − S(t + 1) is calculated to see how many patients die within 1 year from t, and the result is divided by S(t) to rescale this rate to the patients still alive at time t. For example, to approximate the hazard rate at t = 4 in the control population A, note that out of the 28.2% who survive to the beginning of year 4, SA (4)-SA (5) = 9.8% die in the following year, so that λA (4) ≈ 9.8/28.2 = 0.347, as a crude approximation (the true value is 0.412, see Table 1). Obviously, this method is too inaccurate to be of any practical relevance. More precise approximations could be obtained by using smaller time units  like weeks, days, or seconds instead of years, so that as  → 0, the approximation would yield the true hazard rate in t. However, the rough approximation approach illustrates that at all times t the hazard ratio compares the two populations with respect to the instantaneous risks of death in t only within the individuals surviving up to t. At each t, the current death rate is calculated with respect to a new reference population, consisting of the individuals still at risk of death at time t. In contrast, risk ratios compare cumulative risks of dying up to time t. Although the hazard ratio function of two populations can either vary in time or remain constant over time, the risk ratio necessarily has to vary over time. From Equation (5), one can see that a constant risk ratio (RR(t) =

HAZARD RATIO

3

Table 1. Survival and Hazard Rates in Two Hypothetical Populations A and B: Comparing Survival Experience Based on Hazard Ratio and Risk Ratio Years from diagnosis t 0 1 2 3 4 5 6 7 8 9 10

Proportion surviving t SB (t) = SA (t)0.5 SA (t) = exp(−(0.3t)1.3 ) 100.0% 81.1% 59.8% 41.8% 28.2% 18.4% 11.7% 7.3% 4.4% 2.6% 1.5%

100.0% 90.1% 77.3% 64.7% 53.1% 42.9% 34.2% 26.9% 21.0% 16.2% 12.4%

Hazard rate in t λA (t) λB (t) 0 0.272 0.335 0.378 0.412 0.440 0.465 0.487 0.507 0.525 0.542

Hazard ratio in t θ(t)

Risk ratio in t RR(t)

0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5

0.526 0.564 0.607 0.653 0.700 0.745 0.788 0.826 0.860 0.889

0 0.136 0.167 0.189 0.206 0.220 0.233 0.244 0.254 0.263 0.271

Table 2. Comparison of Hazard Ratio and Risk Ratio Topic

Hazard ratio

Risk ratio

Interpretation of the quantities compared in the ratio?

instantaneous risk of death in t among those surviving up to time t yes or no

cumulative risk of death up to time t no

yes, if roughly constant in time t

no

no

yes

Ratio constant function of time t? Useful summary measure across time axis? Useful to compare survival rates at a specific time t?

RR) would imply that the proportions surviving in the two populations are multiples of each other over the whole time axis: SB (t) = RR SA (t), which is virtually impossible to occur in real life examples unless the survival curves are identical (RR = 1), because it would mean that one of the two populations cannot start with 100% of the individuals at risk in t = 0. Therefore, in survival analysis, estimation of the hazard ratio will provide a suitable summary measure of the relative survival experiences in two populations over the whole time axis whenever the assumption of a roughly constant relative hazard function seems plausible. Estimation of a risk ratio only makes sense if, for some reason, the populations should be compared with each other at one particular point in time. Table 2 summarizes the properties of hazard ratio and risk ratio.

4 EXAMPLE ON THE USE AND USEFULNESS OF HAZARD RATIOS The estimation of the hazard ratio is illustrated by means of the example of a clinical trial (hypothetical data) introduced in a seminal paper by Peto et al. (1). In this hypothetical example, 25 patients were randomized between two treatment arms A and B, and the effect of treatment on the survival time of the patients shall be analyzed. Additionally, as a covariate, the renal function of the patients at the time of randomization is known, which may also have an influence on survival. Not all patients were observed until death and therefore their survival time was censored at the end of follow-up (censoring). The survival time is calculated as number of days from randomization to death if the patient died during follow-up, or as number of days from randomization to the end

4

HAZARD RATIO

Table 3. Data of Patients of Hypothetical Study Patient number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 1 2 3 4

Treatment1

Renal function2

Survival time3

Survival time censored?4

A B B A A B A A B B B A B B A B B A A B A A B A B

I N N N I N N I N N N N I N N I N N N N I N N N I

8 180 632 852 52 2240 220 63 195 76 70 8 13 1990 1976 18 700 1296 1460 210 63 1328 1296 365 23

N N N Y N Y N N N N N N N Y Y N N Y Y N N Y N Y N

Treatment group (A or B). Renal function (N: normal or I: impaired). Time between randomization and death or end of follow-up (survival time) in days. Indicator whether the patient was observed until death (survival time censored = N) or not (survival time censored = Y).

of follow-up. Additionally, an indicator is necessary giving the information whether the calculated survival time is censored or uncensored. Table 3 shows the data for the 25 study patients. 5 AD HOC ESTIMATOR OF THE HAZARD RATIO The effect of treatment on the survival time of the patients can be characterized quantitatively through an estimate of the hazard ratio between the treatment groups A and B. In doing so, it is assumed implicitly that the hazard ratio is constant in time. At first, the recorded survival times in both treatment groups are reviewed (Table 4). For a calculation of the hazard ratio estimator, the survival times have to be arranged in the following way: For each time point,

when one or more deaths occurred (death times = uncensored survival times), the number of patients who died at that time and the number of patients alive just before this time point are needed, separately for both treatment groups. This information is displayed in Table 5. In total, the observed number of deaths in treatment group i is then given by (i = A, B)

Oi =

J


dij

j=1

Under the hypothesis that the hazard or instantaneous risk of dying in treatment group A is identical to the instantaneous risk of dying in treatment group B at all times, the expected number of deaths in treatment

HAZARD RATIO

group i is given by (i = A, B) Ei =

J


dj

j=1

which may be interpreted as the ratio of the relative death rates in the treatment groups (1, 2). In this example, this estimator gives a 6/8.338 = 0.5666 (i.e., hazard ratio of HR1 = 11/8.662 the hazard rate or instantaneous risk of dying in treatment group A is estimated as about 57% of that in treatment group B).

nij nj

The reasoning behind this calculation is as follows: If the hazards are identical in both groups and a total of dj deaths are observed at time point tj , then one expects dj nij /nj out of these deaths to come from group i, because nij /nj is the proportion of patients of group i among the patients at risk of death at time point tj . The total number in treatment group i is obtained by summation over all time points. Table 6 shows the required arrangement of the data and the calculations for the example, which results in OA = 6, OB = 11, EA = 8.338, and EB = 8.662. Note that the sum of observed deaths and the sum of expected deaths is the same: OA + OB = EA + EB . This relationship provides a simple check for the correctness of the calculations. An ad hoc estimator of the hazard ratio between treatment group A and treatment group B is then given by the ratio of the two relative proportions of observed to expected deaths HR1 =

5

6 CONFIDENCE INTERVAL OF THE AD HOC ESTIMATOR If no difference existed in the risk of dying between the treatment groups, the hazard ratio would be equal to 1. In order to give information about the precision of the estimator, and in order to judge whether the observed difference between treatment groups is a result of chance, a confidence interval for the hazard ratio should be calculated. With this confidence interval, also the hypothesis θ = 1 of no treatment effect (i.e., no difference of the hazard functions of treatment groups A and B) may be tested. This test may be performed at significance level α by calculating a 100(1 − α)% confidence interval. If this interval excludes 1, it may be concluded that at level α a significant difference between treatment groups exists.

OA /EA OB /EB

Table 4. Survival Times by Treatment Group (+: Censored Survival Time) Treatment Group

Survival time (days)

A B

8, 8, 52, 63, 63, 220, 365+, 852+, 1296+, 1328+, 1460+, 1976+ 13, 18, 23, 70, 76, 180, 195, 210, 632, 700, 1296, 1990+, 2240+

Table 5. Arrangement of Data for Hazard Ratio Estimation Death time t1 t2 t3 ... tJ total

Group A

Number of deaths Group B

total

Group A

dA1 dA2 dA3 ... dAJ OA

dB1 dB2 dB3 ... dBJ OB

d1 d2 d3 ... dJ OA + OB

nA1 nA2 nA3 ... nAJ

Number at risk Group B nB1 nB2 nB3 ... nBJ

tj = ordered death times (uncensored survival times) in ascending order; j = 1, . . . , J dij = observed number of deaths at time tj in treatment group i; i = A, B; j = 1, . . . , J dj = observed total number of deaths at time tj ; j = 1, . . . , J nij = number of patients being alive in treatment group i just before time tj (called ‘number at risk’); i = A, B; j = 1, nj = total number of patients being alive just before time tj ; j = 1, . . . , J J= number of death times

total n1 n2 n3 ... nJ

...,

J

6

HAZARD RATIO

A confidence interval for the hazard ratio may be calculated by considering the logarithm of the hazard ratio, because this is approximately normally distributed. A 100(1 − α)% confidence interval for the logarithm of the hazard ratio is given by

is equal to 0.4852, and the 95% confidence interval of the hazard ratio estimator HR1 is equal to [exp(−0.5681 − 1.96 × 0.4852), exp(−0.5681 + 1.96 × 0.4852)] = [0.219, 1.467]

[log(HR1 ) − z1−α/2 SE(log(HR1 )), log(HR1 ) + z1−α/2 SE(log(HR1 ))] 7 AD HOC ESTIMATOR STRATIFIED FOR THE COVARIATE RENAL FUNCTION

where z1−α/2 is the (1 − α/2)-quantile of the standard normal distribution. For a 95% confidence interval, this quantity is equal to 1.96 (3). SE(log (HR1 )) is an estimator for the standard error of the logarithm of the estimated hazard ratio and given by SE(log(HR1 )) =



In the previous estimation of the hazard ratio, a simple two-group comparison of treatment A and B was performed without taking into account covariate information of the patients (i.e. in this example the renal function). Although the patients were assigned randomly to the treatment groups, the proportion of patients with impaired renal function is not exactly identical in treatment groups A and B (group A: 4 out of 12 (33%) patients with impaired renal function; group B: 3 out of 13 (23%) patients with impaired renal function). Thus, if the covariate renal function had an influence on survival, it should be taken into account in the analysis of the effects of treatment on survival.

1/EA + 1/EB

A 100(1 − α)% confidence interval for the hazard ratio is then given by [exp(log(HR1 ) − z1−α/2 SE(log(HR1 ))), exp(log(HR1 ) + z1−α/2 SE(log(HR1 )))] In this example, the logarithm of the hazard ratio is equal to −0.5681, the standard error of the logarithm of the estimated hazard ratio

Table 6. Arrangement of Data of the Hypothetical Study for Hazard Ratio Estimation

Death time

Number of deaths Group A Group B total

Number, at risk’ Group A Group B total

Expected number of deaths Group A Group B nAj

dAj

dBj

dj

nAj

nBj

nj

dj

8 13 18 23 52 63 70 76 180 195 210 220 632 700 1296

2 0 0 0 1 2 0 0 0 0 0 1 0 0 0

0 1 1 1 0 0 1 1 1 1 1 0 1 1 1

2 1 1 1 1 2 1 1 1 1 1 1 1 1 1

12 10 10 10 10 9 7 7 7 7 7 7 5 5 4

13 13 12 11 10 10 10 9 8 7 6 5 5 4 3

25 23 22 21 20 19 17 16 15 14 13 12 10 9 7

0.960 0.435 0.455 0.476 0.500 0.947 0.412 0.438 0.467 0.500 0.538 0.583 0.500 0.556 0.571

1.040 0.565 0.545 0.524 0.500 1.053 0.588 0.562 0.533 0.500 0.462 0.417 0.500 0.444 0.429

Total

6

11

17

8.338

8.662

nj

dj

nBj

tj

nj

HAZARD RATIO

7

Table 7. Arrangement of Data of the Hypothetical Study for Hazard Ratio Estimation Stratified for Renal Function Death time Renal function normal

impaired

tj

Number of deaths Group A Group B total

Number, at risk’ Group A Group B total

Expected number of deaths Group A Group B nAj

dBj

dj

nAj

nBj

nj

dj

8

1

0

1

8

10

18

0.444

0.556

70 76 180 195 210 220 632 700 1296

0 0 0 0 0 1 0 0 0

1 1 1 1 1 0 1 1 1

1 1 1 1 1 1 1 1 1

7 7 7 7 7 7 5 5 4

10 9 8 7 6 5 5 4 3

17 16 15 14 13 12 10 9 7

0.412 0.438 0.467 0.500 0.538 0.583 0.500 0.556 0.571

0.588 0.562 0.533 0.500 0.462 0.417 0.500 0.444 0.429

Total

2

8

10

5.009

4.991

8

1

0

1

4

3

7

0.571

0.429

13 18 23 52 63

0 0 0 1 2

1 1 1 0 0

1 1 1 1 2

3 3 3 3 2

3 2 1 0 0

6 5 4 3 2

0.500 0.600 0.750 1.000 2.000

0.500 0.400 0.250 1.000 0.000

Total

4

3

7

5.421

1.579

For an analysis of the effect of the renal function on survival, one may use the same methods as described above. A simple twogroup comparison of patients is performed with impaired and with normal renal function by estimating the hazard ratio with corresponding 95% confidence interval. The calculations are not outlined in detail, but the result is simply shown. Using the notation Oi and Ei , i = I,N, for the observed and expected number of deaths in patients with impaired and normal renal function, the results are as follows: OI = 7, ON = 10, EI = 1.6, and EN = 15.4, which results in an estimated hazard ratio of patients with impaired renal function versus patients with normal renal function of 6.7375 with a 95% confidence interval of [1.3227, 34.319]. So, a strong negative influence of an impaired renal function exists on survival, because the confidence interval is far from 1.

nj

dj

nBj

dAj

nj

This result indicates that the analysis of the effect of treatment on survival should be corrected for the renal function of the patients. For this purpose, a so-called stratified analysis will be performed, which means the calculations of observed and expected numbers of deaths in treatment groups A and B described above are done separately in patients with impaired and normal renal function and combined thereafter to get a hazard ratio estimator of treatment group A versus B. The calculations are shown in Table 7. The stratified calculation leads to the following numbers for observed and expected numbers in treatment groups A and B: OA = 2 + 4 = 6, OB = 8 + 3 = 11, EA = 5.009 + 5.421 = 10.43, and EB = 4.991 + 1.579 = 6.57. Inserting these numbers in the same formulas as used for the unstratified estimator (see above), the hazard ratio estimator of treatment group A versus treatment group

8

HAZARD RATIO

B stratified for renal function is then HR2 = 6/10.43 11/6.57 = 0.3436 with a 95% confidence interval of [0.129, 0.912]. With the stratified analysis, one may now conclude that treatment A is superior to treatment B with respect to survival, because the confidence interval does not include 1. As mentioned above, a strong effect of the renal function exists on survival, and the proportion of patients with impaired renal function is slightly higher in treatment group A than in treatment group B. The stratified analysis adjusts for this imbalance and, therefore, is able to show more clearly the superiority of treatment A as compared with treatment B with respect to survival. 8

PROPERTIES OF THE AD HOC ESTIMATOR

Although not having optimal statistical properties, the ad hoc estimator of the hazard ratio above has gained popularity because of its simplicity. One should remark that this estimate is not ‘‘consistent’’ for a true hazard ratio different from 1. With increasing sample size, this estimate does not necessarily converge to the true hazard ratio, which is an important property usually required for a good statistical estimator. Thus, this ad hoc estimate should only be used for getting a rough impression of the size of the difference between treatment groups and better estimators with better mathematical properties should be applied, generally. One such class of improved estimators are the so-called generalized rank estimators. 9 CLASS OF GENERALIZED RANK ESTIMATORS OF THE HAZARD RATIO The class of so-called generalized rank estimators of the hazard ratio is defined by J 

HRGR =

j=1 J  j=1

d

wj nAj Aj

d

wj nBj Bj

which is the ratio of the sum of weighted death rates between treatment groups, weighted by some known factor wj . The construction of a statistical test based on the

class of generalized rank estimators leads to the class of so-called generalized rank tests. For different choices of the weight factor wj , different well-known two-sample tests for survival data result. Different weights can be used to weight deaths differently according to whether they occur early or late in time. Therefore, the resulting estimators have different statistical properties depending on the form of the true hazard functions. In particular, the weight factor

wj =

nAj nBj nj

results in the well-known logrank test. Under the proportional hazard assumption (i.e., an underlying constant hazard ratio), the logrank test is the most efficient test in the class of generalized rank tests, which means the logrank test is the most powerful test in this class. The generalized rank estimator with this so-called logrank weight factor will then be denoted by HR3 and can be expressed as J 

HR3 =

j=1 J  j=1

nAj nBj dAj nj nAj nAj nBj dBj nj nBj

J 

=

j=1 J  j=1

nBj nj dAj nAj nj dBj

For this example, the necessary calculations are displayed in Table 8. The generalized rank estimator with logrank weights results in HR3 =

3.009 = 0.5627 5.347

The calculation of a confidence interval for the hazard ratio based on this estimator is not as easy as for the ad hoc estimator given above. The formula is rather complicated, so it is dropped here, and only the result is presented [0.214, 1.477]. It is also possible, in accordance to the stratified ad hoc estimator, to calculate a generalized rank estimator with logrank weights that is stratified for renal function. This calculation is not presented here but another possibility for stratified estimation is presented later in this article.

HAZARD RATIO

As mentioned above, under the proportional hazard assumption, the logrank test is the most efficient test in the class of generalized rank tests. Accordingly, the corresponding hazard ratio estimator with logrank weights is the most efficient estimator in the class of generalized rank estimators, which means it has smaller variance than any other generalized rank estimator. Another estimator of the hazard ratio that has even better mathematical properties than the logrank estimator is the estimator from the Cox’s proportional hazard model.

10 ESTIMATION OF THE HAZARD RATIO WITH COX’S PROPORTIONAL HAZARD MODEL Two reasons exist to prefer Cox’s proportional hazard model (Cox model) for estimation of the hazard ratio over the methods presented so far. The most important reason is the fact that a regression model like the Cox model allows one to analyze the effect of different factors on survival simultaneously. The Cox model is the most prominent multiple regression model in survival analysis.

9

The use of multiple regression models is necessary whenever the effects of other patient characteristics besides treatment, so-called prognostic factors or covariates, are of additional interest. Reasons for this interest may either stem from the necessity of adjusting the effect of treatment for other factors, as in the example above, or from the desire to study the effect of these factors on survival per se. Another reason is that the estimator of the hazard ratio derived from the Cox model has the best mathematical properties if the proportional hazards assumption is true. It is the most efficient estimator (i.e., it has the smallest variance). In the Cox model, treatment and other factors are assumed to affect the hazard function in a multiplicative way, which implies the proportional hazards assumption (i.e., a constant hazard ratio over time). For an illustration of the Cox model with the example of the hypothetical study, an indicator X 1 for the randomized treatment (X 1 = 1, if treatment A is allocated, X 1 = 0, if treatment B is allocated) and an indicator X 2 for the factor renal function (X 2 = 1, if renal

Table 8. Calculation of the Generalized Rank Estimator with Logrank Weights HR3 for the Hazard Ratio Between Treatment Groups Death time tj 8 13 18 23 52 63 70 76 180 195 210 220 632 700 1296 total

Weight

Death rate Group A Group B

Nominator

Denominator

nBj

nAj nBj dAj nj nAj

nAj nBj dBj nj nBj

0 0.077 0.083 0.091 0 0 0.100 0.111 0.125 0.143 0.167 0 0.200 0.250 0.333

1.042 0 0 0 0.500 1.053 0 0 0 0 0 0.417 0 0 0

0 0.435 0.455 0.476 0 0 0.412 0.438 0.467 0.500 0.539 0 0.500 0.556 0.571

3.009

5.347

nAj nBj

dAj

dBj

nj

nAj

6.240 5.652 5.455 5.238 5.000 4.737 4.118 3.938 3.733 3.500 3.231 2.917 2.500 2.222 1.714

0.167 0 0 0 0.100 0.222 0 0 0 0 0 0.143 0 0 0

10

HAZARD RATIO

function is impaired, and X 2 = 0, if renal function is normal) are introduced. In the simplest situation of a Cox model, the hazard function depends on only one factor. If it is assumed that the hazard function only depends on the randomized treatment, the Cox model is formulated as

approximately normally distributed. A 100(1 − α)% confidence interval for the hazard ratio is then, as in the above situation of the ad hoc estimation, given by

λ(t|X1 ) = λ0 (t) · exp(β1 X1 )

where z1−α/2 is again the (1 − α/2)-quantile of the standard normal distribution (for a 95% confidence interval equal to 1.96). SE(log (HR4 )) is an estimator for the standard error of the logarithm of the estimated hazard ratio. In the Cox model, no explicit formula can be given for this quantity, and it has to be calculated with appropriate computer software, too. In the example, the estimated standard error is SE(log (HR4 )) = 0.5096 resulting in a 95% confidence interval of the hazard ratio estimator HR4 of [0.208, 1.531]. If one takes the renal function into account in the analysis of the treatment effect, similarly to the calculation of the ad hoc estimator stratified for renal function shown above, it can be done very easily with the Cox model. Depending on the randomized treatment and on the renal function, using the indicators X 1 and X 2 , the hazard function is now formulated as

The so-called regression coefficient β 1 represents the effect of X 1 on the hazard function. With the notation used in the introduction, this can easily be seen by calculating the ratio of the hazard functions θ (t) of patients with X 1 = 1 and X 1 = 0: θ (t) =

λA (t) λ(t|X1 = 1) λ0 (t) · exp(β1 ) = = λB (t) λ(t|X1 = 0) λ0 (t)

= exp(β1 ) which shows that the hazard ratio of treatment group A versus treatment group B is equal to exp(β 1 ) in the formulation of the Cox model, and reflects the fact that the hazard ratio in the Cox model is constant over time t. Thus, a Cox model including only one factor yields—as an alternative to the ad hoc estimator HR1 and the generalized rank estimator with logrank weights HR3 —another possibility for estimation of the hazard ratio between treatment groups. One drawback of using the Cox model for estimation of the hazard ratio is that the calculations cannot be performed by hand. The regression coefficient β 1 is estimated from empirical patient data by the so-called maximum partial likelihood procedure. This method requires complicated mathematical iteration procedures, which can only be done by appropriate computer software like the procedure PHREG of the Statistical Analysis System (SAS). In the example, the calculation results in a value of −0.5728 for β 1 , and consequently the hazard ratio of treatment group A versus treatment group B from the Cox model is estimated as HR4 = exp(−0.5728) = 0.5639. A confidence interval for the hazard ratio based on the Cox model can again be calculated by considering the logarithm of the hazard ratio log(HR4 ) = β 1 , because it is

[exp(log(HR4 ) − z1−α/2 SE(log(HR4 ))), exp(log(HR4 ) + z1−α/2 SE(log(HR4 )))]

λ(t|X1 , X2 ) = λ0 (t) · exp(β1 X1 + β2 X2 ) The unknown regression coefficient β 1 again represents the effect of X 1 on the hazard function, as can be seen easily by calculating the ratio of the hazard functions θ (t) of patients with X 1 = 1 and X 1 = 0: θ (t) = =

λA (t) λ(t|X1 = 1, X2 ) = λB (t) λ(t|X1 = 0, X2 ) λ0 (t) · exp(β1 + β2 X2 ) = exp(β1 ) λ0 (t) · exp(β2 X2 )

Again, one can see that the hazard ratio of treatment group A versus treatment group B is equal to exp(β 1 ). Similarly, the ratio of the hazard functions of patients with an impaired renal function versus patients with a normal renal function is given by λ(t|X1 , X2 = 1) λ0 (t) · exp(β1 X1 + β2 ) = λ(t|X1 , X2 = 0) λ0 (t) · exp(β1 X1 ) = exp(β2 )

HAZARD RATIO

11

Table 9. Result of Simultaneous Analysis of Effects of Treatment and Renal Function on Survival with the Cox Model Factor

log(HR)

SE(log(HR))

Hazard Ratio

95% confidence interval

Treatment (A vs. B) Renal function (impaired vs. normal)

−1.2431 4.1055

0.5993 1.1645

0.2885 60.673

[0.089, 0.934] [6.191,594.62]

Table 10. Overview of the Results of the Different Estimators for the Hazard Ratio of Treatment Group A vs. Treatment Group B Estimator Ad hoc estimator Ad hoc estimator stratified for renal function Generalized rank estimator with logrank weights Cox model Cox model adjusted for renal function

Accounts for renal function

Notation

Hazard Ratio

95% confidence interval

no yes

HR1 HR2

0.5666 0.3436

[0.219, 1.467] [0.129, 0.912]

no

HR3

0.5627

[0.214, 1.477]

no yes

HR4 HR5

0.5639 0.2885

[0.208, 1.531] [0.089, 0.934]

showing that the hazard ratio of patients with impaired renal function versus patients with normal renal function is equal to exp(β 2 ). The big difference to the model above, where X 2 was not included, is that the regression coefficients β 1 and β 2 will now be estimated simultaneously from the data. In the Cox model, this analysis of the treatment effect is called adjusted for the renal function. Besides the stratified ad hoc estimator HR2 and the stratified generalized rank estimator, this Cox model estimator of the treatment effect represents another possibility for taking renal function of the patients into account when estimating the hazard ratio between treatment groups. While in the stratified approaches renal function is only included in order to enhance the estimator of the treatment effect, in the Cox model, one has the additional benefit that its effect on survival can be studied simultaneously with the effect of treatment. Table 9 shows the result of this analysis. The estimator of the hazard ratio between treatment groups from this analysis is then HR5 = exp(− 1.2431) = 0.2885 with a 95% confidence interval of [0.089, 0.934]. Table 10 gives an overview of the results of the different estimators for the hazard ratio of treatment group A versus treatment group B presented in this article.

The estimators taking the renal function of the patients into account, HR2 and HR5 , come to the same conclusion that treatment A is superior to treatment B with respect to survival, because the confidence intervals do not include 1. 11

FURTHER READING

Definition and estimation of hazard ratios belong to the broader subject of survival analysis. In a series of 4 papers, Clark et al. (4–7) give an excellent, comprehensive introduction with great emphasis on clinical applications. Girling et al. (2) give a more basic introduction in the general context of clinical trials in their chapter 9.3.4, and a simple formula for sample size calculation in chapter 5.4.3. For those that aim to understand the subject in depth, including the more technical aspects even with limited statistical experience, the monograph by Marubini and Valsecchi (8) is recommended. REFERENCES 1. R. Peto et al., Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. Analysis and examples. Brit. J. Cancer 1977; 35: 1–39.

12

HAZARD RATIO

2. D. J. Girling et al., Clinical Trials in Cancer: Principles and Practice. Oxford: Oxford University Press, 2003. 3. D. G. Altman, Practical Statistics for Medical Research. London: Chapman & Hall, 1991. 4. T. G. Clark, M. J. Bradburn, S. B. Love, and D. G. Altman, Survival analysis part I: basic concepts and first analyses. Brit. J. Cancer 2003; 89: 232–238. 5. M. J. Bradburn, T. G. Clark, S. B. Love, and D. G. Altman, Survival analysis part II: multivariate data analysis–an introduction to concepts and methods. Brit. J. Cancer 2003; 89: 431–436. 6. M. J. Bradburn, T. G. Clark, S. B. Love, and D. G. Altman, Survival analysis part III: multivariate data analysis—choosing a model and assessing its adequacy and fit. Brit. J. Cancer 2003; 89: 605–611. 7. T. G. Clark, M. J. Bradburn, S. B. Love, and D. G. Altman, Survival analysis part IV: further concepts and methods in survival analysis. Brit. J. Cancer 2003; 89: 781–786. 8. E. Marubini and M. G. Valsecchi, Analysing Survival Data from Clinical Trials and Observational Studies. Chichester: John Wiley & Sons, 1995.

Heritability Before discussing what genetic heritability is, it is important to be clear about what it is not. For a binary trait, such as whether or not an individual has a disease, heritability is not the proportion of disease in the population attributable to, or caused by, genetic factors. For a continuous trait, genetic heritability is not a measure of the proportion of an individual’s score attributable to genetic factors. Heritability is not about cause per se, but about the causes of variation in a trait across a particular population.

Definitions Genetic heritability is defined for a quantitative trait. In general terms it is the proportion of variation attributable to genetic factors. Following a genetic and environmental variance components approach, let Y have a mean µ and variance σ 2 , which can be partitioned into genetic and environmental components of variance, such as additive genetic variance σa2 , dominance genetic variance σd2 , common environmental variance σc2 , individual specific environmental variance σe2 , and so on. Genetic heritability in the narrow sense is defined as σa2 , (1) σ2 while genetic heritability in the broad sense is defined as σg2 , (2) σ2 where σg2 includes all genetic components of variance, including perhaps components due to epistasis (gene–gene interactions; see Genotype) [3]. In addition to these random genetic effects, the total genetic variation could also include that variation explained when the effects of measured genetic markers are modeled as a fixed effect on the trait mean. The concept of genetic heritability, which is really only defined in terms of variation in a quantitative trait, has been extended to cover categorical traits by reference to a genetic liability model. It is assumed that there is an underlying, unmeasured continuous “liability” scale divided into categories by “thresholds”. Under the additional assumption that

the liability follows a normal distribution, genetic and environmental components of variance are estimated from the pattern of associations in categorical traits measured in relatives. The genetic heritability of the categorical trait is then often defined as the genetic heritability of the presumed liability (latent variable), according to (1) and (2).

Comments There is no unique value of the genetic heritability of a characteristic. Heritability varies according to which factors are taken into account in specifying both the mean and the total variance of the population under consideration. That is to say, it is dependent upon modeling of the mean, and of the genetic and environmental variances and covariances (see Genetic Correlations and Covariances). Moreover, the total variance and the variance components themselves may not be constants, even in a given population. For example, even if the genetic variance actually increased with age, the genetic heritability would decrease with age if the variation in nongenetic factors increased with age more rapidly. That is to say, genetic heritability and genetic variance can give conflicting impressions of the “strength of genetic factors”. Genetic heritability will also vary from population to population. For example, even if the heritability of a characteristic in one population is high, it may be quite different in another population in which there is a different distribution of environmental influences. Measurement error in a trait poses an upper limit on its genetic heritability. Therefore traits measured with large measurement error cannot have substantial genetic heritabilities, even if variation about the mean is completely independent of environmental factors. By the definitions above, one can increase the genetic heritability of a trait by measuring it more precisely, for example by taking repeat measurements and averaging, although strictly speaking the definition of the trait has been changed also. A trait that is measured poorly (in the sense of having low reliability) will inevitably have a low heritability because much of the total variance will be due to measurement error (σe2 ). However, a trait with relatively little measurement error will have a high heritability if all the nongenetic factors are known and taken into account in the modeling of the mean.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

2

Heritability

Fisher [1] recognized these problems and noted that whereas . . . the numerator has a simple genetic meaning, the denominator is the total variance due to errors of measurement [including] those due to uncontrolled, but potentially controllable environmental variation. It also, of course contains the genetic variance . . . Obviously, the information contained in [the genetic variance] is largely jettisoned when its actual value is forgotten, and it is only reported as a ratio to this hotch-potch of a denominator.

and Vandenburg’s F = 1/[1 − σa2 /σ 2 )] [6]. Furthermore, the statistical properties of these estimators do not appear to have been studied.

References [1] [2]

[3] [4]

Historically, other quantities have also been termed heritabilities, but it is not clear what parameter is being estimated, e.g. Holzinger’s H = (rMZ − rDZ ) (the correlation between monozygotic twins minus the correlation between dizygotic twins) (see Twin Analysis) [2], Nichol’s H R = 2(rMZ − rDZ )/rMZ [5], the E of Neel & Schull [4] based on twin data alone,

[5]

[6]

Fisher, R.A. (1951). Limits to intensive production in animals, British Agricultural Bulletin 4, 217–218. Holzinger, K.J. (1929). The relative effect of nature and nurture influences on twin differences, Journal of Educational Psychology 20, 245–248. Lush, J.L. (1948). Heritability of quantitative characters in farm animals, Suppl. Hereditas 1948, 256–375. Neel, J.V. & Schull, W.J. (1954). Human Heredity. University of Chicago Press, Chicago. Nichols, R.C. (1965). The National Merit twin study, in Methods and Goals in Human Behaviour Genetics, S.G. Vandenburg, ed. Academic Press, New York. Vandenberg, S.G. (1966). Contributions of twin research to psychology, Psychological Bulletin 66, 327–352.

JOHN L. HOPPER

HISTORICAL CONTROL

Numerous articles focus on HCT designs, especially in the area of oncology. Some authors avoid explicit discussion of historical controls in their studies by comparing their results with an (assumed) standard response rate without acknowledgment of how the reference rate was established. Articles with references to the methodological literature have previously appeared in other encyclopedia articles by Gehan (3) as well as by Beach and Baron (4). Other common references to the comparative observational study literature are Cook and Campbell (5), Rosenbaum (6), and Rothman and Greenland (7).

NEAL THOMAS Pfizer Inc., Global Research and Development, New London, Connecticut

1

HISTORICAL CONTROLS AND BIAS

The use of historical data from previously completed clinical trials, epidemiological surveys, and administrative databases to estimate the effect of placebo or standard-of-care treatment is one approach for assessing the benefit of a new therapy. Study designs without collection of concurrent control data have been largely replaced by randomized clinical trials (RCTs) with concurrently collected control data in many clinical settings. Studies comparing a new treatment with historical experience are frequently criticized and regarded by many researchers as unreliable for any purpose other than hypothesis generation because the historical controlled trial (HCT) may be subject to unknown sources of bias such as follows:

2 METHODS TO IMPROVE THE QUALITY OF COMPARISONS WITH HISTORICAL DATA There has been more acceptance of HCTs in therapeutic areas in which it is difficult to enroll patients in RCTs. HCTs are common in early development trials in oncology. The Simon two-stage design is an example of a commonly used HCT design with tumor response as the primary endpoint (8). Statisticians and other authors involved in these therapeutic areas are more positive regarding the value of HCTs, provided that they are well designed (3). Pocock (9) notes several features of a welldesigned HCT:

1. Changes in the patients over time due to changing treatment patterns. Improvements in diagnostic tests may also change the staging of patients receiving treatment.

1. Control group has received the precisely defined treatment in a recent previous study. 2. Criteria for eligibility and evaluation must be the same. 3. Prognostic factors should be known and be the same for both treatment groups. 4. No unexplained indications lead one to expect different results.

2. Changes in the measurement of the disease. These changes could be due to evolving laboratory tests or shifts in clinical rating assessments. 3. Changes in the use of concomitant medications and treatments. 4. The lack of blinding when collecting new clinical data for comparison with a known target (control) performance is another source of potential bias. Because a new treatment is typically thought to be superior to merit evaluation, the lack of blinding may be partially responsible for the perception that HCTs are often biased in favor of new treatments.

Covariate adjustment methods [Snedecor and Cochran (10)] can be applied in HCTs provided appropriately detailed patient-level data are available from earlier trials. If a large pool of patient-level data is available, matching methods (6,7) may also be used. These adjustments are examples of direct standardization in the observational study

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

HISTORICAL CONTROL

literature. When patient-level data are not available, it may still be possible to assess the potential for bias using summaries of baseline covariate information. Specialized methods for covariate adjustments and differences in exposure and follow-up time have been developed for time-to-event endpoints (11). 3 BAYESIAN METHODS TO COMBINE CONCURRENT AND HISTORICAL CONTROLS Bayesian statistical inference provides a method to combine data from historical and concurrently randomized controls (9,12). Within the Bayesian framework, it is possible to make explicit judgments about the relevance of past data accounting for potential bias. For example, a 1000-patient historical study may be regarded as equivalent to 100 concurrently randomized patients due to potentially large biases. This approach is ethically appealing because it can reduce the number of patients receiving placebo, while retaining the features of a randomized, blinded clinical trial. There has been extensive methodological development and applications of this approach in preclinical studies using animal laboratory data 13–15. Because the animals are bred and maintained in relatively homogeneous conditions for several generations, it is possible to both limit and estimate the variability between cohorts over time. The variability between cohorts provides a more objective basis for weighting data from past cohorts with concurrently randomized controls. 4 EFFECTS BASED ON TIMES-SERIES ESTIMATES Times series of event rates in large populations, often based on large administrative or federal epidemiological data collections, are another source of historical control information. Because detailed data about treatments and baseline conditions are most likely missing, time series are the least reliable source of historical control information. Using time series statistical modeling (5,6), the response to prevailing treatments during a past time period can be projected to more recent time

periods presuming no change in treatments has occurred. The projections are then compared with recent response rates after major changes in common treatments or policies. This methodological approach is common in marketing research. There are also applications for assessing the effects of medical treatments and environmental exposures. One famous example is the increasing rates of lung cancer that were associated with increased smoking rates in the early part of the twentieth century, which was cited as one source of evidence for the effect of cigarette smoking on lung cancer (16). REFERENCES 1. W. Cochran and D. Rubin, Controlling bias in observational studies: A review, Sankya, Series A 1973; 35: 417–446. 2. R. Rosenbaum and D. Rubin, The central role of the propensity score in observational studies for causal effects. Biometrika, 1983; 70: 41–55. 3. E. Gehan, Historical controls. In: Encyclopedia of Statistical Sciences, vol. 3. New York: Wiley, 1983. 4. M. Beach and J. Baron, Bias from historical controls. In: Encyclopedia of Biostatistics, vol. 1. New York: Wiley, 1998. 5. T. Cook and D. Campbell, QuasiExperimentation. Boston, MA: HoughtonMifflin, 1979. 6. P. Rosenbaum, Observational Studies. New York: Springer-Verlag, 1995. 7. K. Rothman and S. Greenland, Modern Epidemiology, 2nd ed. Philadelphia, PA: Lippincott-Raven, 1998. 8. R. Simon, Optimal two-stage designs for phase II clinical Trials. Controll. Clin. Trials 1989; 10: 1–10. 9. S. Pocock, The combination of randomized and historical controls in clinical trials. J. Chron. Dis. 1976; 29: 175–188. 10. G. Snedecor and W. Cochran, Statistical Methods, 8th ed. Ames, IA: Iowa State University Press, 1989. 11. N. Keiding, Historical controls in survival analysis. In: Encyclopedia of Biostatistics, vol. 3, New York: Wiley, 1998. 12. D. Berry and D. Strangl, Bayesian Biostatistics, New York: Marcel Dekker, 1996. 13. A. Dempster, M. Selwyn, and W. Weeks, Combining historical and randomized controls

HISTORICAL CONTROL assessing trends in proportions. J. Am. Stat. Assoc. 1983; 78: 221–227. 14. R. Tamura and S. Young, The incorporation of historical control information in tests of proportions: simulation study of Tarone’s procedure. Biometrics 1988; 42: 221–227.

3

15. R. Tarone, The use of historical control information in testing for a trend in proportions. Biometrics 1982; 38: 215–220. 16. R. Doll and B. Hill, Smoking and carcinoma of the lung. Brit. Med. J. 1950: 739–747.

HYPOTHESIS

to the discovery of Pluto in 1864, which supported Newton’s theory of gravitation. The theory, however, failed to explain irregularities of the orbit of Mercury, which could finally be understood using Einstein’s theory of relativity. According to Popper (2) and Hempel (3), the process of falsifying hypotheses has to be deductive rather than inductive. Only a deductive approach enables science to sort out incorrect hypotheses and to keep only those that passed multiple attempts to falsify them. The falsification process works in principle as follows: Let H denote a hypothesis and Ii a series of implications such that

GERD ROSENKRANZ Biostatistics and Statistical Reporting Novartis Pharma AG Basel Switzerland,

1

SCIENTIFIC HYPOTHESES

A hypothesis is ‘‘an idea or a suggestion put forward as a starting point for reasoning and explanation’’ (1). Hypotheses cannot be generated in a mechanistic fashion from empirical data: their invention is a creative process. In the course of this process, one tries to discover regularities or structures in a set of observations in an attempt to explain or organize the findings. This process is therefore inductive in nature (i.e., an attempt to derive general rules from one or more individual cases). The verification of hypotheses, however, needs a different approach. Popper (2) claimed that a necessary criterion for hypotheses to be scientific is that they are testable. Only those hypotheses that have stood up against relevant tests can be assigned a certain degree of validity and credibility. In fact, he emphasized that, strictly speaking, scientific hypotheses can only be falsified, but never become truly verified: the possibility always exists that future observations contradict a hypothesis. An example of a scientific hypothesis that passed several tests before it was eventually falsified is Newton’s theory of gravitation. It is based on the hypothesis that the gravitational forces between two bodies are proportional to the product of their masses divided by the square of their distance. One implication of this theory is that planets follow elliptical orbits on their way around the sun. When irregularities of the orbit of Neptune were observed in the nineteenth century, it first looked as if this phenomenon could not be explained by Newton’s theory. However, if one postulates the existence of another (at that point in time) unknown planet that disturbs the movements of Neptune, Newton’s theory would apply again. This reasoning led

H −→ I1 −→ . . . −→ Ik where ‘‘−→’’ stands for ‘‘implies.’’ If any of the implications turn out to be incorrect, then H is incorrect as well on purely logical grounds. In the contrary case, if Ik is true, then it cannot be concluded that H is true as well. It is easily seen that true implications can be derived from a wrong hypothesis, but not the other way a round. In this sense, deductive reasoning provides firm criteria for the verification of scientific hypotheses. The deductive method also helps to identify potential traps in hypothesis testing. Two examples are considered. First, assume that I is an implication of hypothesis H if a further assumption A holds; for example, H ∧ A −→ I. The symbol ‘‘∧’’ stands for the logical ‘‘and.’’ If I can be proven to be incorrect, one can only conclude that H or A are not both correct. In particular, the falsification of H is only possible if assumption A is known to be true. Often, two or more competing hypotheses exist. In such a situation, it would be desirable to decide which one is correct. Unfortunately, this decision is not possible in general: Either one can falsify one or both, but the falsification of one does not imply the correctness of the other with the exception of the entirely trivial case in which one claims exactly the contrary of the other.

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

2

HYPOTHESIS

STATISTICAL HYPOTHESES

Pr[k|H, n] =


 n k p (1 − p0 )n−k k 0

(1)

2.1 An Introductory Example The deductive framework has been successful in testing (i.e., falsifying) hypotheses that claim to be correct under all circumstances (like the theory of gravitation). Such hypotheses are called general or universal. However, hypotheses exist that are not universal but probabilistic in nature. These hypotheses will be called statistical hypotheses. As an example, consider the hypothesis that a vaccine is effective. Although an effective vaccine increases the likelihood of being protected against a specific infection, a vaccine can never be completely protective. As a consequence, if some vaccinees become infected despite vaccination, it is generally not possible to claim that the vaccine is not protective at all: It can, nevertheless, protect the majority of vaccinees. This example demonstrates that, unlike with universal hypothesis, probabilistic hypotheses cannot be falsified by some counter examples. A way to approach testability of statistical hypotheses is to make use of the law of large numbers: If the proportion of infected individuals among a large number of vaccinees is small, the hypothesis that the vaccine is protective is supported. If, on the contrary, the proportion of infected individuals is large, the hypothesis might be regarded as not acceptable. It needs to be emphasized, however, that regardless of how many vaccinated persons have been monitored to detect an infection, no complete certainty exists that the results look the same in the non-observed individuals. To look at the matter more closely, let E be an event of interest, like the breakthrough of an infection in a vaccinated individual, and let p = Pr[E], the probability that E occurs. Consider the hypothesis that this probability equals some number 0 < p0 < 1, or formally, H : p = p0 . To test this hypotheses, consider performing n experiments or randomly sampling n objects and to count the number of events. If H is true and the experiments are independent (i.e., the occurrence of an event in one experiment is not predictive of an experiment in another), the probability to observe k events is given by

In the context of the vaccination example, one could determine a critical number 0 ≤ c(H, n) ≤ n such that if k ≥ c(H, n), one would reject the hypothesis because too many vaccinees would become infected. Otherwise, the hypothesis would have passed the test. The number c(H, n) can be reasonably defined such that the probability of the number of events to exceed c(H, n) is small (e.g., less than some 0 < α < 1) if H is true. Hence, c(H, n) is defined such that

Pr[reject H|H ] =


 n  n j p0 (1 − p0 )n−j ≤ α j

j=c(H,n)

(2) Although a test was found to reject H, the criterion derived above has been developed under additional assumptions: One assumption is that the experiments are independent. Another requirement is that the experiments should be identically distributed (i.e., come from a single, common distribution), otherwise it is not clear whether Pr[E] = p0 holds for all experiments. Hence, strictly speaking, if k ≥ c(H, n) holds, one has only found that Pr[E] = p0 , the repeatability and the independence of the experiments cannot be true at the same time. The design and conduct of the experiments have to ensure that the assumptions of identically distributed and independent experiments are met in order to draw conclusions about the hypothesis of interest. The basic idea of statistical hypothesis tests is to identify events that are unlikely under a hypothesis H and to reject H when such events occur. It remains to be shown whether this approach delivers optimal tests or whether tests exist that enable H to be rejected with a higher probability given the same information. The key for answering this question is to compare the hypothesis H with another hypothesis K, which cannot be true at the same time; for example, K : p = p1 for some p1 > p0 . The probability to reject H if K

HYPOTHESIS

is true is then given by Pr[reject H|K ] =


 n  n j p1 (1 − p1 )n−j j

j=c(H,n)

(3) Although this probability depends on p1 , the test based on k ≥ c(H, n) can be shown to be optimal for all p1 > p0 . 2.2 The Structure of a Statistical Test In the previous section, the idea of a statistical hypothesis and a test to reject it was described for a specific situation. It is worthwhile to discuss some general ideas of hypothesis testing before moving on to more complex hypotheses. The first step in constructing a statistical test is to summarize the information contained in data x = (x1 , . . . , xn ) in an adequate way by some function T(x). In the previous section, the data were 0 or 1 depending on whether an event occurred and the statistic n was the number of events (i.e., T(x) = i=1 xi ). Next, one has to identify a rejection region S such that the hypothesis H will be rejected if T ∈ S. In the example above, S = {k; k ≥ c(H, n)}. T and S have to fulfill the following requirements. To reduce the risk to reject H in case it is correct, one requests Pr[T ∈ S|H ] ≤ α

(4)

for some small 0 < α < 1, which is called the level of the test. Second, under no alternative K should the probability of rejection be less than α, otherwise alternatives would exist under which the rejection of H is less likely than under H itself: Pr[T ∈ S|K ] ≥ α

(5)

Such tests are called unbiased. Ideally, one would also like to control the risk to fail to reject the hypothesis if it is incorrect, which is generally not possible. Instead one aims at finding an optimal test in the sense that for an alternative K, Pr[T ∈ S|K ] = max

(6)

3

Tests that fulfill Equation (6) for all alternatives K are called uniformly most powerful (UMP). As shown above, the test based on k ≥ c(H, n) is UMP for testing H : p = p0 against K : p = p1 for all p1 > p0 . UMP tests are always unbiased and exist for a wide variety of hypotheses (see Reference 4). Just as a test for a universal hypothesis aims to falsify the hypothesis, a test of a statistical hypothesis intends to reject it. However, some differences exist. A falsified universal hypothesis is falsified once for all, whereas some uncertainty about the correctness of the decision to reject a statistical hypothesis always exists. Second, a universal hypothesis can, in principle, be falsified by a single counterexample. To reject a statistical hypothesis, sufficient evidence is needed that contradicts the hypothesis, the degree of which is determined by the level of a statistical test. The choice of the level is often a matter of debate. When tables of critical values were produced in the beginning of the last century, α = 0.05 or α = 0.01 were used to limit the number of tables. Although these selections are somewhat arbitrary, α = 0.05 has been widely accepted. If the statistical test is in one direction, as in the example above, a level of α = 0.025 is often required. Applied to clinical studies, it would allow one out of 40 inefficacious drugs to pass clinical testing successfully. As this result does not look acceptable to health authorities, they tend to require two successful clinical studies (at least for certain indications) before a drug can become approved. For a level of 2.5% for an individual study, this result would correspond to α = 0.000625 if one big study is conducted instead of two separate trials (see Reference 5). Discussions can also develop concerning the size of the power of a statistical test. It was said above that, if possible, one would perform an optimal test (i.e., one that maximizes the probability to reject H if a specific alternative K is true). Apart from further assumptions, the power of a test depends on K and on the sample size (i.e., the number of independent data points). Theoretically, the probability to reject H can be made close to one for a specific K by increasing the sample size, as can be concluded from Equations

4

HYPOTHESIS

(2) and (3) for the vaccination example. This is often not feasible from a resource point of view, nor is it desirable from a scientific point of view to avoid that alternatives that differ from H in an irrelevant manner lead to rejection of H. 3 SPECIFIC HYPOTHESES IN CLINICAL STUDIES As explained above, the intention behind testing of statistical hypotheses is to reject a specific hypothesis under certain criteria that limit the uncertainty that the decision taken is correct. Hypothesis testing is therefore primarily applied in phase III studies. The role of these studies in the clinical development process is to confirm the results of previous findings by rejecting testable hypotheses. Dose finding studies or studies that intend to verify the biological concept behind a potential effect need different statistical methods. In the following, some aspects of hypotheses testing in confirmatory clinical studies will be addressed. 3.1 Superiority, Equivalence, and Non-Inferiority The formulation of an appropriate statistical hypothesis depends on the objective of the clinical study. If, for example, a clinical study is conducted to demonstrate the superiority of a new treatment over a comparator, it is most reasonable to consider testing the hypothesis that the new treatment is worse or at best equally efficacious. This approach enables one to control the risk to reject the hypothesis if it is true by selecting the level of the test. In the following, it is assumed that the difference in effect between two treatments can be described in terms of a parameter θ , which is zero in case of equal effects and positive in case of superiority. For example, one can think of θ to represent the difference in the reduction of blood pressure or the log-odds ratio of two success rates. Demonstrating superiority then means to be able to reject H : θ ≤ 0 in favor of K : θ > 0

(7)

Circumstances exist where the intention is not to demonstrate superiority but to provide evidence for similarity. If a new formulation of a drug becomes available, it can become necessary to figure out whether the bioavailability of the new formulation is comparable with that of the existing one. If so, doses and dose regimen of the existing formulation would also apply for the new formulation. In this situation, it is preferable to try to reject the hypothesis that the bioavailability of the two formulations is different. Ideally, one would like to have a level α test of H : θ = 0 versus K : θ = 0 that would only reject with a probability >α if θ is exactly zero. Unfortunately, it is not possible to construct such a test. A solution is to select an equivalence margin δ > 0 and to aim to reject H : θ ∈ (−δ, δ) in favor of K : θ ∈ (−δ, δ) (8) Another case of interest is the development of a new drug that is not necessarily more efficacious than an existing one but has other advantages, for example, a better safety profile. In this case, one would like to demonstrate that the efficacy of the new drug is comparable with the one that exists without excluding the possibility that it can be even better. At first glance, one would like to reject H : θ < 0 if K : θ ≥ 0 is true. The test would reject for values of θ being 0 or greater. Although this test looks very desirable, it is generally not possible to construct a test with a level α for all θ < 0 and >α for θ ≥ 0. As before, it is necessary to select a δ > 0, the non-inferiority margin, and to set out to reject H : θ ≤ −δ in favor of K : θ ≥ −δ (9)

Although for both hypotheses, Equation (8) and Equation (9) UMP unbiased tests exist in many relevant situations, some new issues coming up for bioequivalence and non-inferiority hypotheses exist. First, the equivalence or non-inferiority margin has to defined. For bioequivalence studies, where the ratio of pharmacokinetic parameters like area under the concentration curve or maximum concentration is of concern, general agreement exists to define δ such that ratios

HYPOTHESIS

between 0.80 and 1.25 (or the logarithms of the ratio between −0.22314 and 0.22314) belong to K. The selection of a non-inferiority margin for clinical studies is more difficult for several reasons. A study that compares a new drug with an active drug with the intention to demonstrate non-inferiority of the new one should also demonstrate that the new drug is superior to placebo, which is not an issue when placebo, the new compound, and the active comparator are tested in the same study in which both superiority over placebo and non-inferiority compared with an active compound can be demonstrated. However, in disease areas where placebo is not an option, only indirect comparisons with placebo are possible. Under these circumstances, the effect of the active comparator over placebo has to be determined from historical data through a meta-analysis. As a consequence, doubts might develop about whether the active comparator would still be superior to placebo in the setting of the present trial. If these doubts are justified, the study runs the risk to demonstrate noninferiority with respect to placebo. To some extent, this concern can be met by designing the non-inferiority study such that it is similar in design to previous studies of the active comparator in regard to patient population, co-medication, doses and regimen, and so on. Although a straightforward proposal is to set δ to a fraction of the difference between an active comparator and placebo, a non-inferiority margin needs to be justified on a case-by-case basis. An intrinsic issue of equivalence or noninferiority studies is that their credibility cannot be protected by methods like randomization and blinding (6). For example, a transcription error in the randomization code would make results under treatment and control look more similar. If this happens too often in a superiority study, the objective of the trial would be compromised, whereas these errors would favor the desired outcome in an non-inferiorty study. The impact of these intrinsic problems is somewhat lowered by the fact that clinical studies have usually more than one objective. Non-inferiority studies need to demonstrate advantages of a new drug in other areas like safety or tolerability,

5

otherwise little reason would exist to prefer the new medication over an existing one. To achieve these additional objectives, correct experimentation is required to maximize the chance of demonstrating an improvement. 3.2 Multiple Hypotheses Clinical studies often have more than one objective so that more than one hypothesis is to be tested. In the context of non-inferiority, it was already mentioned that basically two hypotheses are of interest: superiority over placebo and non-inferiority in regard to an active comparator. In other cases, more than one dose of a new drug is to be compared with a control. In many studies, a series of secondary objectives follow the primary objective or more than one primary objective is of concern. A general issue with multiple questions is that if more than one hypothesis is to be tested, more than one can be falsely rejected. As a consequence, the level set for a test of an individual hypothesis needs to be smaller to keep the overall error rate for a set of hypotheses. If the combination of hypotheses is what really matters, the definition of the level of a hypothesis test and, consequently, its power need some reconsideration. Instead of going through a general discussion of the multiplicity problem, a series of examples that frequently occur in clinical trials is presented. Interested readers are referred to Refences 7 and 8. First, consider a study that is to compare two doses of a new drug against placebo. Let θi denote the differences over placebo for dose i = 1, 2, Hi : θi ≤ 0 and Ki : θi > 0 and Ti the corresponding level α tests with critical regions Si . The objective is to demonstrate that at least one dose is superior over placebo to claim efficacy of the new drug. In statistical terms, one would like to reject the hypothesis that neither dose is superior over placebo or to reject H = H1 ∧ H2 in favor of K = K1 ∨ K2 (10) Note that the symbol ‘‘∨’’ stands for ‘‘one or the other or both.’’ For the level λ = Pr[T1 ∈ S1 ∨ T2 ∈ S2 |H1 ∧ H2 ] of the corresponding

6

HYPOTHESIS

test, α ≤ λ ≤ Pr[T1 ∈ S1 |H1 ] + Pr[T2 ∈ S2 |H2 ] = 2α (11) Hence, the level of the overall test can be somewhere between α and 2α. The simplest way to remedy the situation is to make T i level α/2 tests. Alternatively, one can apply some more powerful multiple comparison methods (see References 7–8). Combinations of several drugs are often required for the treatment of chronic diseases like hypertension and diabetes. To demonstrate the usefulness of combinations, one has to demonstrate that the efficacy of a combination is superior to the efficacy of each of its components. The hypothesis to be rejected is that the combination is not better than the best of the individual components against the alternative that it is better than each of them. Using notation from above, one intends to reject H = H1 ∨ H2 in favor of K = K1 ∧ K2 (12) As a win only exists if both hypotheses can be rejected, no adjustment of the level of the individual tests is required for Pr[T1 ∈ S1 ∧ T2 ∈ S2 |H ] ≤ Pr[T1 ∈ S1 |H1 ] = α Unfortunately, it is not possible to exhaust the significance level any better without further assumptions. One plausible assumption could be that the combination cannot be much better than one component but worse than the other. However, as pointed out earlier, if H is rejected under the additional assumption only, one could only conclude that H and the assumption cannot be correct simultaneously. Sometimes, a hierarchy exists between two objectives of a clinical study such that the achievement of one is only worthwhile if the other one can be achieved as well. An example is a non-inferiority study that intends to claim a superior safety profile for the new drug. In this setting, the better safety profile would be irrelevant if efficacy of the new compound would be inferior to that of an active comparator. If H1 stands for the noninferiority hypothesis and H2 for the safety

hypothesis, one tries first to reject H1 . Only if H 1 has been rejected, one attempts to reject H 2 , which implies λ = Pr[T1 ∈ S1 |H1 ] + Pr[T2 ∈ S2 ∧ T1 ∈ S1 |H ] = Pr[T1 ∈ S1 |H1 ] = α

(13)

Hence, an overall level α test can be obtained if the tests of the individual hypotheses have level α. In fact, Equation (13) shows that the only situation under which the test T1 ∈ S1 ∨ T2 ∈ S2 has level α is that T2 can only reject if T1 can. (Of course, the roles of T1 and T2 can be reversed.) The same reasoning can be applied if H2 is the hypothesis of no difference in efficacy. Thus, a test for noninferiority can be followed by a level α test for superiority after non-inferiority has been demonstrated without affecting the overall level. The downside of the hierarchical approach is that if the first hypothesis cannot be rejected, no way to reject the second one exists. In the absence of a clear hierarchy among hypotheses, like in a study that compares two doses against a control, hierarchical testing should be avoided. A final remark is on the power of tests of multiple hypotheses. For the two doses against a control study [Equation (10)], the power is given by Pr[T1 ∈ S1 ∨ T2 ∈ S2 |K ], which is usually greater than the power of the individual tests. Hence, even if one halves the level of the individual tests to achieve an acceptable overall level, a loss in power does not necessarily occur. Only if one is also interested in achieving a certain power for each of the individual hypotheses, the sample size of the study has to be increased. For the combination study, the power of T1 ∈ S1 ∧ T2 ∈ S2 is clearly smaller than the power of each individual test. However, the power of a combination test is slightly higher than the product of the power of the individual tests because they are always correlated since the data from the combination are used in both. For the hierarchical procedure, the power of the first test is unaffected, but some loss in power for the second test exist because the second hypothesis will only be considered if

HYPOTHESIS

the first could be rejected: Pr[T2 ∈ S2 |K ] = Pr[T2 ∈ S2 |T1 ∈ S1 , K ]Pr[T1 ∈ S1 |K ] ≤ Pr[T1 ∈ S1 |K ] The worst case obtains when both tests are independent. REFERENCES 1. A. S. Hornby, E. V. Gatenby, H. Wakefield, The Advanced Learner’s Dictionary of Current English. London: Oxford University Press, 1968. 2. K. Popper, The Logic of Scientific Discovery. London: Routledge, 2002. 3. C. G. Hempel, Philosophy of Natural Sciences. Englewood Cliffs, NJ: Prentice Hall, 1969. 4. E. L. Lehmann, Testing Statistical Hypotheses, 2nd ed. New York: Springer, 1986. 5. L. Fisher, One large, well-designed, multicenter study as an alternative to the usual FDA paradigm. Drug Inform. J. 1999; 33: 265–271. 6. S. Senn, Inherent difficulties with active controlled equivalence trials. Stat. Med. 1993; 12: 2367–2375. 7. Y. Hochberg and A. C. Tamhane, Multiple Comparison Procedures. New York: Wiley, 1987. 8. P. H. Westfall and S. S. Young, ResamplingBased Multiple Testing. New York: Wiley, 1993.

7

HYPOTHESIS TESTING

hypothesis testing is the process of evaluating the sample data to determine whether the null hypothesis is false. If based on data from the clinical trial the null hypothesis is shown to be improbable, then one concludes that the evidence supports the research hypothesis. The null hypothesis is denoted by H0 , whereas the alternative hypothesis is denoted by Ha or H1 (1, 2). Hypothesis tests can be used to test hypotheses about a single population parameter, or they can be used to compare parameters from two or more populations. Hypothesis tests can also be one-sided or two-sided depending on the nature of the alternative hypothesis. For example, a one-sided test regarding a single population mean would be specified as

NICOLE M. LAVALLEE PROMETRIKA, LLC, Cambridge, MA

MIGANUSH N. STEPANIANS PROMETRIKA, LLC, Cambridge, MA

Hypothesis testing is one of the two main areas of statistical inference, the other being parameter estimation. Although the objective of parameter estimation is to obtain point estimates or interval estimates (i.e., confidence intervals) for population parameters, hypothesis testing is used when the objective is to choose between two competing alternative theories or hypotheses regarding population parameters (e.g., population means, population proportions, population standard deviations, etc.). For example, the makers of a new antihypertensive drug believe that their medication will decrease systolic blood pressure, on average, by at least 20 mmHg in patients with moderate hypertension. The possible outcomes to this question can be specified as dichotomous, either it does or it does not. To test this claim, a statistical hypothesis test can be performed. To establish firm evidence of safety or efficacy of a new drug, clinical development plans ordinarily include at least one clinical trial that is designed to test prespecified hypotheses about population parameters that reflect the safety and/or efficacy of that drug. 1

H0 : µ ≤ µ0 versus Ha : µ > µ0 or H0 : µ ≥ µ0 versus Ha : µ < µ0 Whereas a two-sided test would be specified as H0 : µ = µ0 versus Ha : µ = µ0 Similarly, a one-sided test comparing two population means would be specified as H0 : µ1 ≤ µ2 versus Ha : µ1 > µ2 or H0 : µ1 ≥ µ2 versus Ha : µ1 < µ2 Whereas a two-sided test would be specified as

SPECIFICATION OF THE HYPOTHESES

H0 : µ1 = µ2 versus Ha : µ1 = µ2 .

The two competing theories being compared in a hypothesis test are referred to as the null hypothesis and the alternative hypothesis. The alternative hypothesis, which is also called the research hypothesis, is the statement that the researcher is trying to prove, for example, that the new drug provides greater efficacy than a placebo. The null hypothesis is the antithesis of the research hypothesis (e.g., the new drug provides no greater efficacy than placebo). Statistical

The choice between a one-sided and twosided alternative hypothesis depends on what the researcher is trying to demonstrate. For example, if the objective of a given trial is to show that there is a decrease in systolic blood pressure from baseline to the end of the trial, then a one-sided test regarding the mean change from baseline in systolic blood pressure will be performed. The one-sided

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

HYPOTHESIS TESTING

alternative hypothesis can be stated as Ha : µ < 0, where µ is the population mean change from baseline in systolic blood pressure. Similarly, if the objective is to show that mean systolic blood pressure for patients receiving a given medication (µ1 ) is lower than the mean for those receiving placebo (µ2 ), then you would choose a one-sided alternative hypothesis (Ha : µ1 < µ2 ). If the objective is simply to show a difference between the two treatments without specifying which is better, then you would choose a two-sided alternative (Ha : µ1 = µ2 ). It should be noted that although a onesided alternative hypothesis appears appropriate when the objective is to show efficacy relative to placebo, regulatory guidelines and industry conventions are that the test of superiority of an active drug over placebo should be specified as a two-sided test. This is done to make the test consistent with the twosided confidence intervals, which are appropriate for assessing the difference between two treatments (3). 2

ERRORS IN HYPOTHESIS TESTING

To reject the null hypothesis, we want to have a reasonable level of certainty that the alternative hypothesis is true. Since we can never know with complete certainty whether the alternative hypothesis is true, two types of errors can be made in hypothesis testing. One mistake would be to reject H0 when it is true, which is referred to as a Type I error (or false positive). A different mistake would be to fail to reject H0 when the alternative is true, which is referred to as a Type II error (or false negative). The possible results of a hypothesis test are summarized in Table 1. Consider the following example. The efficacy of a new drug is being assessed in a placebo-controlled trial. The null hypothesis is that there is no difference between the new drug and the placebo, on average, with respect to a particular outcome measure. The alternative hypothesis is that the new drug is different from the placebo with respect to the particular outcome measure. A Type I error would occur if it was concluded that there was a difference between the new drug and the placebo, when in fact there was not. A Type

II error would occur if it was concluded that there was no difference between the new drug and the placebo, when there actually was. A Type I error is generally considered more serious than a Type II error. This is particularly true in clinical trials, because it is important to guard against making incorrect claims regarding the potential benefit of a new treatment. Therefore, hypothesis tests are constructed to ensure a low probability of making a Type I error. The probability of a Type I error is called the significance level of the test and is denoted by α. Most often in clinical trials the significance level is set at α = 0.05, so that there is only a 5% chance of making a Type I error. The probability of a Type II error is denoted by β. Ideally, one would want to limit both the probability of a Type I error (α) and the probability of a Type II error (β). However, for any sample of a given size, the Type I error and the Type II error are inversely related; as the risk of one decreases, the risk of the other increases. The probability of a Type II error, β, can be reduced while maintaining the probability of a Type I error at a fixed level of α by increasing the sample size of the experiment. Increasing the sample size reduces the sampling variability, and therefore, it increases the probability that the test will reject the null hypothesis when the alternative hypothesis is true. This probability is referred to as the power of the test and is equal to 1 −β. The goal of hypothesis testing is to determine whether there is sufficient evidence to reject the null hypothesis in favor of the alternative hypothesis at a specific significance level (i.e., with a specified probability of committing a Type I error). If there is insufficient evidence, then we fail to reject the null hypothesis. Failing to reject the null hypothesis does not mean that the null hypothesis is accepted, however, only that there was not sufficient evidence from the sample to disprove it. For a given hypothesis test, α is always known, since it is preset at a certain level; however, the determination of β is complex, and one cannot readily specify the probability of committing a Type II error. Hence, it is recommended to reserve judgment and to avoid declaring that the null

HYPOTHESIS TESTING

3

Table 1. Possible Hypothesis Test Outcomes Truth

Conclusion from Hypothesis Test

Do Not Reject H0 Reject H0

hypothesis is accepted unless the probability of a Type II error can be provided. 3 TEST STATISTICS AND DETERMINATION OF STATISTICAL SIGNIFICANCE The decision of whether to reject the null hypothesis is made based on the evidence provided from the sample data. A test statistic is a value calculated from the sample data that summarizes the information available about the population parameter(s), and it is used to decide whether to reject the null hypothesis. The choice of the test statistic depends on the hypotheses of interest, the sample size, and the assumed distribution of the underlying population. For example, if we are performing a test regarding the mean of a normally distributed population with known variance, the appropriate test statistic is a function of the sample mean X as well as of the sample size n and the population standard deviation σ . Consider a hypothesis test of the form H0 : µ ≤ µ0 versus Ha : µ > µ0. It can be shown that the test statistic Z=

X − µo √ σ/ n

has a standard normal distribution under the null hypothesis. When the population variance is unknown but the sample size is large (a common threshold used is n > 30), test statistics of the above form can be calculated by substituting the sample standard deviation s for σ and can be assumed to have a standard normal distribution even if the underlying population is not normally distributed. The basic premise of a hypothesis test is that we will assume H0 is true unless the data provide sufficient evidence to the contrary.

H0 is true

H0 is false

Correct Decision Type I Error

Type II Error Correct Decision

However, we must specify what constitutes sufficient evidence before performing the test. Intuitively, for the above hypotheses, a sample mean, X, that is much larger than µ0 would provide evidence in favor of Ha . Under the assumption that µ = µ0 , the test statistic Z would have a positive value and fall in the right tail of the standard normal distribution. The probability of obtaining a test statistic as extreme or more extreme than the observed value calculated under the assumption that H0 is true is referred to as the P-value. Intuitively, if the P-value is very small, then it can be concluded that the data support Ha . Based on knowledge of the distribution of the test statistic, we can obtain the P-value. The smaller the P-value, the stronger the evidence that H0 is unlikely to be true. In hypothesis testing, we reject the null hypothesis if the P-value of the test is less than the prespecified significance level α. In clinical trials, when the null hypothesis is rejected in favor of the research hypothesis of interest, then it is common to state that statistical significance has been reached. The set of values for the test statistic that would cause the null hypothesis to be rejected is called the critical region. The boundary of the critical region is called the critical value. The test statistic must be more extreme than this value (i.e., further in the tail of the distribution of the test statistic) in order to reject the null hypothesis. The critical value(s) depends on the significance level for the test and whether the test is one-sided or two-sided. For one-sided hypothesis tests, there will be one critical value in the lower or upper tail of the distribution. For two-sided tests, there will be two critical values, marking the rejection regions in both tails of the distribution (provided that the test statistic has a symmetric distribution under the null hypothesis).

4

4

HYPOTHESIS TESTING

EXAMPLES

We illustrate the concepts and techniques of hypothesis testing by two examples. The first example demonstrates how to perform a test on a single population mean, and the second example compares the means from two independent samples. 4.1 One-Sample Test A new drug has been developed to treat hypertension. A Phase II trial is conducted to test whether treatment with the new drug will lower systolic blood pressure (SBP) by more than 20 mmHg, on average, in patients with moderate hypertension who are treated for two months. A random sample of 50 patients from the population the drug is intended to treat is enrolled in the trial. For each patient, the change in SBP from baseline to the end of the treatment period is determined. This is the outcome measure on which the test will be conducted. The null and alternative hypotheses can be stated as H0 : µ ≤ 20

4.2 Two-Sample Test Now suppose that instead of testing the within-patient change in SBP, the primary objective of another study is to test whether patients treated with the new drug experience a significantly greater decrease in SBP than patients treated with placebo. The null and alternative hypotheses can be stated as H0 : µ1 = µ2 or equivalently, µ1 − µ2 = 0

Ha : µ > 20

Ha : µ1 = µ2 or equivalently, µ1 − µ2 = 0

The significance level of the test is specified as α = 0.05. The sample mean and standard deviation are determined to be 26 and 13, respectively. The test statistic is equal to Z=

test, corresponding to 5% of the area under the standard normal curve. H0 will be rejected for all test statistics that are greater than 1.645. Since Z > 1.645, we reject H0 at a 0.05 level of significance and conclude that the new drug will lower SBP by more than 20 mmHg, on average, in moderate hypertensive patients. The probability of obtaining a test statistic of this magnitude or larger, under the assumption that H0 is true, is P = 0.00055. The P-value can be calculated using a statistical software package and is equal to the area under the standard normal probability density function corresponding to Z ≥ 3.264.

X − µo 26 − 20 √ = √ = 3.264 S/ n 13/ 50

where µ1 represents the mean change for the population of patients treated with the new drug and µ2 represents the mean change for the population of patients treated with placebo. For a test of two independent means, it can be shown that the test statistic

Since under the null hypothesis, this test statistic follows a standard normal distribution, the critical value is z0.05 = 1.645. Figure 1 depicts the critical region for this

X1 − X2 Z =
  S22 S21 n1 + n2

0.05

0

1.645

Figure 1. Standard normal curve with one-sided rejection region.

HYPOTHESIS TESTING

where X1 and X2 are the sample means corresponding to µ1 and µ2 , respectively, and S1 and S2 are the corresponding sample standard deviations. The test statistic Z has a standard normal distribution under the null hypothesis given that both n1 and n2 are large. For this trial, 80 patients are enrolled and randomized in equal numbers to the active drug and placebo. At the end of the trial, the sample means and standard deviations are calculated for each group. The test statistic is calculated as 22 − 16 X1 − X2 Z =
  =
 2  = 2.145 2 2 13 122 S2 S1 + + n1 n2 40 40 has a standard normal distribution. The critical value for this test is z0.025 = 1.96. Figure 2 depicts the critical region for this test, corresponding to 2.5% of the area under the standard normal curve in each tail. H0 will be rejected for all test statistics that are less than −1.96 or greater than 1.96. Since Z > 1.96, we reject H0 at a 0.05 level of significance and conclude that the new drug lowers SBP significantly more than the placebo after two months of treatment. This result is significant with a P-value of 0.0320. The P-value is equal to the area under the standard normal probability density function corresponding to the combined areas for Z > 2.145 and Z < −2.145. For the above examples of one and two sample tests of means, each had sufficiently large sample sizes, so that a test statistic could be constructed that followed a standard normal distribution. Note that in the case of

small sample sizes (<30 per group), hypothesis tests of means can be constructed where the test statistic follows a t-distribution under the null hypothesis. REFERENCES 1. R. J. Larsen and M. L. Marx, An Introduction to Mathematical Statistics and Its Applications, 3rd ed. Englewood Cliffs, NJ: Prentice-Hall, 2000. 2. J. Neter, W. Wasserman, and G. A. Whitmore, Applied Statistics, 4th ed. Newton, MA: Allyn and Bacon, 1992. 3. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline. Statistical Principles for Clinical Trials: E9. 1998.

FURTHER READING M. Bland, An Introduction to Medical Statistics, 3rd ed. New York: Oxford University Press, 2000. G. Casella and R. L. Berger, Statistical Inference, 2nd ed. Belmont, CA: Wadsworth, 2001. R. V. Hogg and A. T. Craig, Introduction to Mathematical Statistics, 6th ed. Englewood Cliffs, NJ: Prentice-Hall, 2004. L. Ott and W. Mendenhall, Understanding Statistics, 6th ed. Boston, MA: Duxbury Press, 1994. S. J. Pocock, Clinical Trials, A Practical Approach. New York: Wiley, 1983. G. W. Snedecor and W. G. Cochran, Statistical Methods, 8th ed. Ames, IA: Iowa State University Press, 1991. G. A. Walker, Common Statistical Methods for Clinical Research with SAS Examples. San Diego, CA: Collins-Wellesley Publishing, 1996.

0.025 Figure 2. Standard normal curve with two-sided rejection region.

−1.96

5

0

1.96

6

HYPOTHESIS TESTING

CROSS-REFERENCES Critical Value One-sided Test vs. Two-sided Test P-value Type I Error, Type II Error

IDENTIFYING THE MOST SUCCESSFUL DOSE (MSD) IN DOSE-FINDING STUDIES

of dose finding in which there are two correlated outcomes we wish to control for has been studied by Braun (7). This differs from the main objective here where we wish to consider simultaneously consider two outcomes, one of which is viewed as positive, one as negative, and both are believed to increase with increasing dose.

JOHN O’QUIGLEY Universite Pierre it Marie Curie Paris VI, 75005 Paris, France

SARAH ZOHAR

2 DESIGNS

CIC 9504 INSERM Centre d’Investigation Clinique, Hˆopital Saint-Louis Paris, France U717 INSERM Paris, France

1

We take Y and V to be binary random variables (0,1), where Y = 1 denotes a toxicity, Y = 0 a nontoxicity, V = 1 a response, and V = 0 a nonresponse. The dose level of a new drug is to be taken from a panel of k discrete dose levels, {d1 , . . . , dk }, with actual toxicity rates assumed to be monotonic and increasing. The dose level assigned to the jth patient (j ≤ n with n the total number of patients), X j , can be viewed as a random variable taking discrete values xj , where xj
{d1 , . . . , dk }. The probability of toxicity at the dose level Xj = xj is defined by R(xj ) = Pr (Y j = 1|X j = xj ). The probability of response given no toxicity at dose level Xj = xj is defined by Q(xj ) = Pr(Vj = 1|X = xj , Yj = 0) so that P(di ) = Q(di ){1 − R(di )} is the probability of success. It is this quantity P that we would like to maximize over the range of doses available in the study and a successful trial identifies the dose level l such that P(dl ) > P(di ) (for all i where i = l). We call such a dose the most successful dose (MSD), and our purpose is, rather than find the MTD, to find the MSD. The relationship between toxicity and dose (xj ) and the relationship between response given no toxicity and dose are modeled through the use of two one-parameter models (1). The reasoning behind the use of underparameterized models is given in Shen and O’Quigley (8). Roughly the idea is that, at the MTD itself, only a single parameter is required to specify the probability of toxicity. Since sequential designs using updating will tend to settle on a single level, it is usually enough to work with a model only rich enough to characterize correctly the probabilities at a single level. Underparameterization, on the

MOTIVATION

O’Quigley et al. (1) proposed designs that take account of efficacy of treatment and toxic side effects simultaneously. In HIV studies, the toxicity is often the ability to tolerate the treatment, whereas in the cancer setting, this is more likely to be the presence or not of serious side effects. For HIV, efficacy will be seen readily and is most often a measure of the impact of treatment on the viral load. For cancer, it may take longer to see any effects of efficacy, although it is becoming increasingly common to make use of biological markers and other pharmacological parameters that can be indicative of a positive effect of treatment. Non-biological measurements may also be used, and in the light of developments in assessing the extent of tumor regression, for example, it is sometimes possible to use measures that previously were considered to be less reliable. Several articles (2–4) take a Bayesian approach to the question and base their findings on an efficacy–toxicity tradeoff. The approach of O’Quigley et al. can also take advantage of any prior information that is available. To do this, we would need to put prior distributions on both the toxicity and efficacy parameters and then use the Bayes formula in much the same way as used by Whitehead et al. (5). Ivanova (6) considered the algorithms of O’Quigley et al. (1), together with some further modification. The problem

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

378

IDENTIFYING THE MOST SUCCESSFUL DOSE (MSD) IN DOSE-FINDING STUDIES

other hand, means that extrapolation can be poor. However, since extrapolation is mostly only local and only used to provide rough guidance, this is a lesser concern to that of accurate local estimation. The same ideas carry through when modeling the most successful dose. Note that R(di ) and Q(di ) refer to exact, usually unknown, probabilities. We use the model based equivalents of these as ψ and φ, respectively, with these quantities coinciding when the working model is correct. Generally though we are interested in the case where the working model is only a first approximation to a more complex reality so that we do not expect R(di ) and ψ to coincide at more than one of the available levels. The same applies to Q(di ) and φ Equality will hold though for, at least, one level. Understanding the equality sign in this sense, i.e., in the sense of a working model, we can then write R(di ) = ψ(di , a) = αia and Q(di ) = φ(di , b) = β bi where 0 < α 1 < · · · < α k < 1, 0 < a < ∞, 0 < β 1 < · · · < β k < 1 and 0 < b < ∞. After the inclusion of j patients, R(di ), Q(di ), and P(di ) are estimated using maximum likelihood (11). The methods are robust to the choice of the α i (and, by extension, the choice of the β i so that these arbitrary choices have a very weak impact on operating characteristics. For large samples, under very broad conditions (9), the choice will have no impact at all. Our practical suggestion would be to divide roughly the interval (0,1) into comparable segments. For four dose levels, for example, we could choose both the α i and the β i to be 0.2, 0.4, 0.6, and 0.8. Note that, in all events, the designs remain invariant to power transformations on these values so that the operating characteristics for small samples based on the above model are exactly identical to, say, the square of the model, i.e., taking α i and β i to be equal to 0.04, 0.16, 0.36, and 0.64. The (j + 1)th patient will be included at the dose level xj , which maximizes the probˆ i ) with ˆ j+1 ) > P(d ability of success, so that P(x i = 1, . . . , k; xj+1 = di (1). To maximize the log likelihood function, we must have heterogeneity in patient observation in terms of toxicity and response. Some initial escalating scheme is then required, and here, we base

379

this on a simple up-and-down algorithm that includes patients in groups of 3 at a time. If all three experience no toxicity, then the dose level is escalated. If a single toxicity is observed, then we remain at the same dose level. If two or more toxicities are observed, then the dose level is lowered. We continue in this way until the first response is encountered. As soon as we have both a response and a toxicity, we are in a position to fit the parameters of the underparameterized model. The estimated parameters enable us to reconstruct the working dose toxicity and dose response relationships. From these we can target, for each subsequent inclusion or group of inclusions, the dose believed to be the most successful (MSD). Usually, from this point, patients are then allocated one at a time although, of course, it is possible to continue with grouped inclusion, based on the model, if desired. In estimating the MSD, all available information is taken into account via the use of our two-parameter model. Apart from unusual situations, this model will be underparameterized in practice. This approach, for an operational standpoint, represents the most obvious extension to the usual CRM strategy for dealing with toxicity alone. It works well as judged by the simulations of Zohar and O’Quigley (11) across a broad range of situations. Instead of the above prescription, O’Quigley et al. (1) suggested a compromise solution in which a CRM (10) model is used to target the rate of toxicity alone. This rate is no longer fixed, however, and can be changed in the course of the study. The solution is likely to be of greatest utility in those cases where we anticipate little or negligible toxicity. This is often the case for HIV studies and is becoming more the case for cancer studies where we work with non-cytotoxic agents. Initially we target a low rate of toxicity, keeping track of the overall success rate. Should the overall success rate be below some aimed for minimum amount, then we will allow the targeted rate of toxicity to be increased. We continue in this way until we either find some level with an overall success rate that we can take to be acceptable or until all levels have been eliminated. Specifically, a likelihood CRM (10,12) approach is used to find the dose level associated with the target probability of toxicity

380

IDENTIFYING THE MOST SUCCESSFUL DOSE (MSD) IN DOSE-FINDING STUDIES

θ . After the inclusion of j − 1 patients, the jth patient will be treated at the dose level xj , which is associated with the estimated toxicity probability closest to the target, such that |ψ(xj , aˆ j ) − θ | < |ψ(di , aˆ j ) − θ | with xj
{d1 , . . . , dk } and ψ(di , a)= α ai . Again, we use a maximum likelihood approach, and so, in order to maximize the log likelihood function, we must have heterogeneity among the toxicity observations, i.e., at least one toxicity together with at least one nontoxicity. An upand-down dose allocation procedure is used until this heterogeneity is observed. Unlike the approach based on an underparameterized model in which, before the model is fitted, we need both heterogeneity in the toxicities and in the responses, here we only need heterogeneity in the toxicities. We do not need to consider any de-escalation, and in the particular simulation here, we use inclusion of three patients at a time until the first toxicity is encountered. Following this patients are allocated typically one at a time although, once again, if desired, grouped inclusions require no real additional work. This design uses the reestimated dose-toxicity relationship for the dose allocation scheme, but as the trial progresses and our inferences are updated, then more information about the rate of success becomes available. At each dose level, the
probability of success is
estimated by Pˆ i = l≤j vl I (xl = di , yl = 0)/ l≤j I(xj =di ) As inclusions increase, we can base a decision on whether to continue, stop and recommend the current level or to reject all levels on the value of Pˆ i . . 2.1 Criteria for Stopping As the trial proceeds and information is gathered about success at the dose level di , a decision is taken based on a sequential probability ratio test (13). The hypotheses to be compared are naturally composite, but for operational simplicity, we compare two point hypotheses-modifying the parameters to these in order to obtain desirable operating characteristics. The first composite hypothesis is that the success rate is greater than p1 at di . The second composite hypothesis is that the success rate is lower than p0 at this dose level. A conclusion in favor of

a success rate greater than p1 leads the trial to end with di recommended. By contrast, a conclusion in favor of a success rate lower than p0 at di leads us to remove that dose level and lower ones, and at the same time, the target toxicity probability is increased from θ to θ + θ (until some upper maximum target toxicity rate is reached). The trial then continues using those dose levels that have not been eliminated. The sequential probability ratio test (SPRT) when treating these as point hypotheses can be expressed as   p < (ni − wi ) log p1 0     1−pi 1−
2 + wi log 1−p < log


log


2 1−
1



0

1

where ni is the number of patients included at dose level di wi is the number of successes (response with no toxicity) at di . The sequential Type I and Type II error rates are
1 and
2 . For fixed sample size, the dose level recommended at the end of the trial for all approaches is the dose level that would have been given to the (n + 1)th patient. 3 GENERAL OBSERVATIONS Identification of the MSD, as opposed to the MTD, provides a step for the statistical methodology toward current clinical practice. Here, even if not always well used, it is now common to record information of toxicity at the same time as that on toxicity. Many potential new treatments can fail because either they showed themselves to be too toxic or, at an acceptable level of toxicity, they failed to show adequate response. The classic format for a dose finding study in cancer has been to ignore information relating to response or efficacy. It has been argued that, in some sense, toxicity itself is serving a role as a surrogate for efficacy— the more toxic the treatment, then the more effect we anticipate seeing. However, this is clearly limited. We might see quite a lot of toxicity and yet little effect. We might also be able to achieve a satisfactory effect at lower rates of toxicity than are commonly used in this setting. Some of the newer biological formulations

IDENTIFYING THE MOST SUCCESSFUL DOSE (MSD) IN DOSE-FINDING STUDIES

may not even be well described by the simple paradigm-more toxicity equates to more effect. Before seeing any toxicity, it might be required to increase the given levels of dose by orders of magnitude so that it becomes increasingly likely that the methodology of dose finding will need to include more and more information relating to the desired outcome and not just the toxic side effects. To give the best chance to any promising new therapy, it becomes vital for the investigators to be able to identify, during the early development, the level of drug, the MSD, which the evidence shows provides the best chance of overall success. The work presented here is broad enough for many situations that can originate in the cancer context. Nonetheless it would be desirable, before undertaking any study using these methods, to carry out simulations of operating characteristics at the kind of sample sizes that can be realistically envisioned. This is not difficult and can also provide an informative background to the necessary discussions that will take place among the clinicians, the pharmacologists, the nursing staff, and the statisticians involved. In the context studied here, that of jointly finding a dose based on both toxicity and response, it is clear that the concept of the maximum tolerated dose (MTD) is not fully adequate. Attention instead focuses on the MSD, and our belief is that the MSD is likely to become of increasing interest in the early development of new therapeutics. In situations where we expect to see very little or almost no toxicity or in situations where we expect to see very little or response rates no greater than, say, 5%, then current designs focusing on only one endpoint are likely to perform quite adequately. The greatest gains with the suggested methods are likely to be found when there are non-negligible effects of both toxicity and efficacy and, in particular, where we weight them in a similar way; i.e., we are most interested in whether a treatment can be considered a failure-either due to toxicity or due to lack of efficacy. Then it makes sense to look for the dose that maximizes overall success rate (MSD). The proposed methods can be considered in conjunction with several others, currently available. The choice of the most appropriate method depends, as always, on the precise

381

context under investigation. Although, for the situations described in the above paragraph, existing designs would seem to require no modification, it could still be argued that by including a toxicity criterion in an efficacy dose finding design, we add a safeguard at little extra cost. One difficulty, not addressed here, is that there may be different precisions in our measurement of efficacy when compared with our measurement of toxicity, the latter tending to be typically more objective. REFERENCES 1. J. O’Quigley, M. D. Hughes, and T. Fenton, Dose-finding designs for HIV studies. Biometrics 2001; 57(4): 1018–1029. 2. P. F. Thall and K. E. Russell, A strategy for dose-finding and safety monitoring based on efficacy and adverse outcomes in phase I/II clinical trials. Biometrics 1998; 54(1): 251–264. 3. P. F. Thall, E. H. Estey, and H. G. Sung, A new statistical method for dose-finding based on efficacy and toxicity in early phase clinical trials. Invest New Drugs 1999; 17(2): 155–167. 4. P. F. Thall and J. D. Cook, Dose-finding based on efficacy-toxicity trade-offs. Biometrics 2004; 60(3): 684–693. 5. J. Whitehead, S. Patterson, D. Webber, S. Francis, and Y. Zhou, Easy-to-implement Bayesian methods for dose-escalation studies in healthy volunteers. Biostatistics 2001; 2(1): 47–61. 6. A. Ivanova, A new dose-finding design for bivariate outcomes. Biometrics 2003; 59(4): 1001–1007. 7. T. M. Braun, The bivariate continual reassessment method. Extending theCRM to phase I trials of two competing outcomes. Control Clin. Trials 2002; 23(3): 240–256. 8. L. Z. Shen and J. O’Quigley, Using a oneparameter model to sequentially estimate the root of a regression function. Computat. Stat. Data Anal. 2000; 34(3): 357–369. 9. L. Z. Shen and J. O’Quigley, Consistency of continual reassessment method under model misspecification. Biometrika 1996; 83: 395–405. 10. J. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: A practical design for phase 1 clinical trials in cancer. Biometrics 1990; 46(1): 33–48.

382

IDENTIFYING THE MOST SUCCESSFUL DOSE (MSD) IN DOSE-FINDING STUDIES

11. S. Zohar and J. O’Quigley, Identifying the most successful dose (MSD) in dose-finding studies in cancer. Pharm. Stat. 2006; 5(3): 187–199. 12. J. O’Quigley and L. Z. Shen, Continual reassessment method: A likelihood approach. Biometrics 1996; 52(2): 673–684. 13. G. B. Wetherill and K. D. Glazebrook, Sequential methods in Statistics, 3th ed. London, U.K.: Chapman and Hall, 1986.

IMAGING SCIENCE IN MEDICINE

at all levels from the atomic through molecular, cellular, and tissue to the whole body and the social influences on disease patterns. At present, a massive research effort is focused on acquiring knowledge about genetic coding (the Human Genome Project) and the role of genetic coding in human health and disease. This effort is progressing at an astounding rate and gives rise to the belief among many medical scientists that genetics and bioinformatics (mathematical modeling of biological information, including genetic information) are the major research frontiers of medical science for the next decade or longer. The human body is an incredibly complex system. Acquiring data about its static and dynamic properties yields massive amounts of information. One of the major challenges to researchers and clinicians is the question of how to acquire, process, and display vast quantities of information about the body, so that the information can be assimilated, interpreted, and used to yield more useful diagnostic methods and therapeutic procedures. In many cases, the presentation of information as images is the most efficient approach to this challenge. As humans, we understand this efficiency; from our earliest years we rely more heavily on sight than on any other perceptual skill in relating to the world around us. Physicians also increasingly rely on images to understand the human body and intervene in the processes of human illness and injury. The use of images to manage and interpret information about biological and medical processes is certain to continue to expand in clinical medicine and also in the biomedical research enterprise that supports it. Images of a complex object such as the human body reveal characteristics of the object such as its transmissivity, opacity, emissivity, reflectivity, conductivity, and magnetizability, and changes in these characteristics with time. Images that delineate one or more of these characteristics can be analyzed to yield information about underlying properties of the object, as depicted in Table 1. For example, images (shadowgraphs) created by X rays transmitted through a region of the body

WILLIAM R. HENDEE Medical College of Wisconsin Milwaukee, WI

1

INTRODUCTION

Natural science is the search for ‘‘truth’’ about the natural world. In this definition, truth is defined by principles and laws that have evolved from observations and measurements about the natural world that are reproducible through procedures that follow universal rules of scientific experimentation. These observations reveal properties of objects and processes in the natural world that are assumed to exist independently of the measurement technique and of our sensory perceptions of the natural world. The purpose of science is to use these observations to characterize the static and dynamic properties of objects, preferably in quantitative terms, and to integrate these properties into principles and, ultimately, laws and theories that provide a logical framework for understanding the world and our place in it. As a part of natural science, human medicine is the quest for understanding one particular object, the human body, and its structure and function under all conditions of health, illness, and injury. This quest has yielded models of human health and illness that are immensely useful in preventing disease and disability, detecting and diagnosing conditions of illness and injury, and designing therapies to alleviate pain and suffering and restore the body to a state of wellness or, at least, structural and functional capacity. The success of these efforts depends on our depth of understanding of the human body and on the delineation of effective ways to intervene successfully in the progression of disease and the effects of injuries. Progress in understanding the body and intervening successfully in human disease and injury has been so remarkable that the average life span of humans in developed countries is almost twice that expected a century ago. Greater understanding has occurred

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

IMAGING SCIENCE IN MEDICINE

reveal intrinsic properties of the region such as its effective atomic number Z, physical density (grams/cm3 ) and electron density (electrons/cm3 ). Nuclear medicine images, including emission computed tomography (ECT) where pharmaceuticals release positrons [positron emission tomography (PET)] and single photons [single photon emission computed tomography (SPECT)], reveal the spatial and temporal distribution of target-specific pharmaceuticals in the human body. Depending on the application, these data can be interpreted to yield information about physiological processes such as glucose metabolism, blood volume, flow and perfusion, tissue and organ uptake, receptor binding, and oxygen utilization. In ultrasonography, images are produced by capturing energy reflected from interfaces in the body that separate tissues that have different acoustic impedances, where the acoustic impedance is the product of the physical density and the velocity of ultrasound in the tissue. Magnetic resonance imaging (MRI) of relaxation characteristics following magnetization of tissues can be translated into information about the concentration, mobility, and chemical bonding of hydrogen and, less frequently, other elements present in biological tissues. Maps of the electrical field (electroencephalography) and the magnetic field (magnetoencephalography) at the surface of the skull can be analyzed to identify areas of intense electrical activity in the brain. These and other techniques that use

the energy sources listed in Table 1 provide an array of imaging methods useful for displaying structural and functional information about the body that is essential to improving human health by detecting and diagnosing illness and injury. The intrinsic properties of biological tissues that are accessible by acquiring and interpreting images vary spatially and temporally in response to structural and functional changes in the body. Analysis of these variations yields information about static and dynamic processes in the human body. These processes may be changed by disease and disability, and identification of the changes through imaging often permits detecting and delineating the disease or disability. Medical images are pictures of tissue characteristics that influence the way energy is emitted, transmitted, reflected, etc. by the human body. These characteristics are related to, but not the same as, the actual structure (anatomy), composition (biology and chemistry), and function (physiology and metabolism) of the body. Part of the art of interpreting medical images is to bridge the gap between imaging characteristics and clinically relevant properties that aid in diagnosing and treating disease and disability. 2 ADVANCES IN MEDICAL IMAGING Advances in medical imaging have been driven historically by the ‘‘technology push’’ principle. Especially influential have been

Table 1. Energy Sources and Tissue Properties Employed in Medical Imaging Image Sources

Image Influences

Image Properties

• X rays • γ rays • Visible light • Ultraviolet light • Annihilation radiation • Electric fields • Magnetic fields • Infrared • Ultrasound • Applied voltage

• Mass Density • Electron density • Proton density • Atomic number • Velocity • Pharmaceutical location • Current flow • Relaxation • Blood volume/flow • Oxygenation level of blood • Temperature • Chemical state

• Transmissivity • Opacity • Emissivity • Reflectivity • Conductivity • Magnetizability • Resonance absorption

IMAGING SCIENCE IN MEDICINE

imaging developments in other areas, particularly in the defense and military sectors, that have been imported into medicine because of their potential applications in detecting and diagnosing human illness and injury. Examples include ultrasound developed initially for submarine detection (sonar), scintillation detectors and reactorproduced isotopes (including 131 I and 60 Co) that emerged from the Manhattan Project (the United States World War II effort to develop the atomic bomb), rare-earth fluorescent compounds synthesized initially in defense and space research laboratories, electrical conductivity detectors for detecting rapid blood loss on the battlefield, and the evolution of microelectronics and computer industries from research funded initially for security and surveillance, defense, and military purposes. Basic research laboratories have also provided several imaging technologies that have migrated successfully into clinical medicine. Examples include reconstruction mathematics for computed tomographic imaging and nuclear magnetic resonance techniques that evolved into magnetic resonance imaging and spectroscopy. The migration of technologies from other arenas into medicine has not always been successful. For example, infrared detection devices developed for night vision in military operations have so far not proven useful in medicine, despite early enthusiasm for infrared thermography as an imaging method for early detection of breast cancer. Today the emphasis in medical imaging is shifting from a ‘‘technology push’’ approach toward the concept of ‘‘biological/clinical pull.’’ This shift in emphasis reflects a deeper understanding of the biology underlying human health and disease and a growing demand for accountability and proven usefulness of technologies before they are introduced into clinical medicine. Increasingly, unresolved biological questions important in diagnosing and treating human disease and disability are used as an incentive for developing new imaging methods. For example, the function of the human brain and the causes and mechanisms of various mental disorders such as dementia, depression, and schizophrenia are among the greatest biological enigmas that confront biomedical

3

scientists and clinicians. A particularly fruitful method for penetrating this conundrum is the technique of functional imaging that employs tools such as ECT and MRI. Functional magnetic resonance imaging (fMRI) is especially promising as an approach to unraveling some of the mysteries of human brain function in health and in various conditions of disease and disability. Another example is the use of X-ray computed tomography and magnetic resonance imaging as feedback mechanisms to shape and guide the optimized deployment of radiation beams for cancer treatment. The growing use of imaging techniques in radiation oncology reveals an interesting and rather recent development. Until about three decades ago, the diagnostic and therapeutic applications of ionizing radiation were practiced by a single medical specialty. In the late 1960s, these applications began to separate into distinct medical specialties, diagnostic radiology and radiation oncology, that have separate training programs and clinical practices. Today, imaging is used extensively in radiation oncology to characterize the cancers to be treated, design the plans of treatment, guide the delivery of radiation, monitor the response of patients to treatment, and follow patients over the long term to assess the success of therapy, the occurrence of complications, and the frequency of recurrence. The process of accommodating this development in the training and practice of radiation oncology is encouraging a closer working relationship between radiation oncologists and diagnostic radiologists. 3 EVOLUTIONARY DEVELOPMENTS IN IMAGING Six major developments are converging today to raise imaging to a more prominent role in biological and medical research and in the clinical practice of medicine (1): • The ever-increasing sophistication of the

biological questions that can be addressed as knowledge expands and understanding grows about the complexity of the human body and its static and dynamic properties.

4

IMAGING SCIENCE IN MEDICINE • The ongoing evolution of imaging tech-

•

•

•

•

nologies and the increasing breadth and depth of the questions that these technologies can address at ever more fundamental levels. The accelerating advances in computer technology and information networking that support imaging advances such as three- and four-dimensional representations, superposition of images from different devices, creation of virtual reality environments, and transportation of images to remote sites in real time. The growth of massive amounts of information about patients that can best be compressed and expressed by using images. The entry into research and clinical medicine of young persons who are amazingly facile with computer technologies and comfortable with images as the principal pathway to acquiring and displaying information. The growing importance of images as an effective means to convey information in visually oriented developed cultures.

A major challenge confronting medical imaging today is the need to exploit this convergence of evolutionary developments efficiently to accelerate biological and medical imaging toward the realization of its true potential. Images are our principal sensory pathway to knowledge about the natural world. To convey this knowledge to others, we rely on verbal communications that follow accepted rules of human language, of which there are thousands of varieties and dialects. In the distant past, the acts of knowing through images and communicating through languages were separate and distinct processes. Every technological advance that brought images and words closer, even to the point of convergence in a single medium, has had a major cultural and educational impact. Examples of such advances include the printing press, photography, motion pictures, television, video games, computers, and information networking. Each of these technologies has enhanced the shift from using words to communicate information toward a more efficient synthesis of images to provide insights and words

to explain and enrich insights (2). Today, this synthesis is evolving at a faster rate than ever before, as evidenced, for example, by the popularity of television news and documentaries and the growing use of multimedia approaches to education and training. A two-way interchange of information is required to inform and educate individuals. In addition, flexible means are needed for mixing images and words and their rate and sequence of presentation to capture and retain the attention, interest, and motivation of persons engaged in the educational process. Computers and information networks provide this capability. In medicine, their use in association with imaging technologies greatly enhances the potential contribution of medical imaging to resolving patient problems in the clinical setting. At the beginning of the twenty-first century, the six evolutionary developments listed before provide the framework for major advances in medical imaging and its contributions to improvements in the health and well-being of people worldwide. 3.1 Molecular Medicine Medical imaging has traditionally focused on acquiring structural (anatomic) and, to a lesser degree, functional (physiological) information about patients at the organ and tissue levels. This focus has nurtured the correlation of imaging findings with pathological conditions and led to enhanced detection and diagnosis of human disease and injury. At times, however, detection and diagnosis occur at a stage in the disease or injury where radical intervention is required and the effectiveness of treatment is compromised. In many cases, detection and diagnosis at an earlier stage in the progression of disease and injury are required to improve the effectiveness of treatment and enhance the well-being of patients. This objective demands that medical imaging refocus its efforts from the organ and tissue levels to the cellular and molecular levels of human disease and injury. Many scientists believe that medical imaging is well positioned today to experience this refocusing as a benefit of knowledge gained at the research frontiers of molecular biology and genetics. This benefit is

IMAGING SCIENCE IN MEDICINE

often characterized as the entry of medical imaging into the era of molecular medicine. Examples include the use of magnetic resonance to characterize the chemical composition of cancers, emission computed tomography to display the perfusion of blood in the myocardium, and microfocal X-ray computed tomography to reveal the microvasculature of the lung. Contrast agents are widely employed in X ray, ultrasound, and magnetic resonance imaging techniques to enhance the visualization of properties correlated with patient anatomy and physiology. Agents in wide use today localize in tissues either by administration into specific anatomic compartments such as the gastrointestinal or vascular systems or by reliance on nonspecific changes in tissues such as increased capillary permeability or alterations in the extracellular fluid space. These localization mechanisms frequently do not yield a sufficient concentration differential of the agent to reveal subtle tissue differences associated with the presence of an abnormal condition. New contrast agents are needed that exploit growing knowledge about biochemical receptor systems, metabolic pathways, and ‘‘antisense’’ (variant DNA) molecular technologies to yield concentration differentials sufficient to reveal subtle variations among various tissues that may reflect the presence of pathological conditions. Another important imaging application of molecular medicine is using imaging methods to study cellular, molecular, and genetic processes. For example, cells may be genetically altered to attract metal ions that (1) alter the magnetic susceptibility, thereby permitting their identification by magnetic resonance imaging techniques; or (2) are radioactive and therefore can be visualized by nuclear imaging methods. Another possibility is to transect cells with genetic material that causes expression of cell surface receptors that can bind radioactive compounds (3). Conceivably this technique could be used to tag affected cells and monitor the progress of gene therapy. Advances in molecular biology and genetics are yielding new knowledge at an astonishing rate about the molecular and genetic infrastructure that underlie the static and

5

dynamic processes of human anatomy and physiology. This new knowledge is likely to yield increasingly specific approaches to using imaging methods to visualize normal and abnormal tissue structure and function at increasingly microscopic levels. These methods will in all likelihood lead to further advances in molecular medicine. 3.2 Human Vision Images are the product of the interaction of the human visual system with its environment. Any analysis of images, including medical images, must include at least a cursory review of the process of human vision. This process is outlined here; a more detailed treatment of the characteristics of the ‘‘end user’’ of images is provided in later sections of this Encyclopedia. 3.2.1 Anatomy and Physiology of the Eye. The human eye, diagrammed in Fig. 1, is an approximate sphere that contains four principal features: the cornea, iris, lens, and retina. The retina contains photoreceptors that translate light energy into electrical signals that serve as nerve impulses to the brain. The other three components serve as focusing and filtering mechanisms to transmit a sharp, well-defined light image to the retina. 3.2.1.1 Tunics. The wall of the eye consists of three layers (tunics) that are discontinuous in the posterior portion where the optic nerve enters the eye. The outermost tunic is a fibrous layer of dense connective tissue that includes the cornea and the sclera. The cornea comprises the front curved surface of the eye, contains an array of collagen fibers and no blood vessels, and is transparent to visible light. The cornea serves as a coarse focusing element to project light onto the observer’s retina. The sclera, or white of the eye, is an opaque and resilient sheath to which the eye muscles are attached. The second layer of the wall is a vascular tunic termed the uvea. It contains the choroid, ciliary body, and iris. The choroid contains a dense array of capillaries that supply blood to all of the tunics. Pigments in the choroid reduce internal light reflection that would otherwise blur the images. The ciliary body

6

IMAGING SCIENCE IN MEDICINE

its thickness. Accommodation is accompanied by constriction of the pupil, which increases the depth of field of the eye. The lens loses its flexibility from aging and is unable to accommodate, so that near objects can be focused onto the retina. This is the condition of presbyopia in which reading glasses are needed to supplement the focusing ability of the lens. Clouding of the lens by aging results in diminution of the amount of light that reaches the retina. This condition is known as a lens cataract; when severe enough it makes the individual a candidate for surgical replacement of the lens, often with an artificial lens. 3.2.1.3 Retina. The innermost layer of the eye is the retina, which is composed of two components, an outer monolayer of pigmented cells and an inner neural layer of photoreceptors. Because considerable processing of visual information occurs in the retina, it often is thought of more as a remote part of the brain rather than as simply another component of the eye. There are no photoreceptors where the optic nerve enters the eye, creating a blind spot. Near the blind spot is the mucula lutae, an area of about 3 mm2 over which the retina is especially thin. Within the macula lutae is the fovea centralis, a slight depression about 0.4 mm in diameter. The fovea is on the optical axis of the eye and is the area where the visual cones are concentrated to yield the greatest visual acuity.

FP

O

contains the muscles that support and focus the lens. It also contains capillaries that secrete fluid into the anterior segment of the eyeball. The iris is the colored part of the eye that has a central aperture termed the pupil. The diameter of the aperture can be altered by the action of muscles in the iris to control the amount of light that enters the posterior cavity of the eye. The aperture can vary from about 1.5–8 mm. 3.2.1.2 Chambers and Lens. The anterior and posterior chambers of the eye are filled with fluid. The anterior chamber contains aqueous humor, a clear plasma-like fluid that is continually drained and replaced. The posterior humor is filled with vitreous humor, a clear viscous fluid that is not replenished. The cornea, aqueous and vitreous humors, and the lens serve collectively as the refractive media of the eye. The lens of the eye provides the fine focusing of incident light onto the retina. It is a convex lens whose thickness can be changed by action of the ciliary muscles. The index of refraction of the lens is close to that of the surrounding fluids in which it is suspended, so it serves as a fine-focusing adjustment to the coarse focusing function of the cornea. The process of accommodation by which near objects are brought into focus is achieved by contraction of the ciliary muscles. This contraction causes the elastic lens to bow forward into the aqueous humor, thereby increasing

Figure 1. Horizontal section through the human eye (from Ref. 4, with permission).

IMAGING SCIENCE IN MEDICINE

The retina contains two types of photoreceptors, termed rods and cones. Rods are distributed over the entire retina, except in the blind spot and the fovea centralis. The retina contains about 125 million rods, or about 105 /mm2 . Active elements in the rods (and in the cones as well) are replenished throughout an individual’s lifetime. Rods have a low but variable threshold to light and respond to very low intensities of incident light. Vision under low illumination levels (e.g., night vision) is attributable almost entirely to rods. Rods contain the light-sensitive pigment rhodopsin (visual purple) which undergoes chemical reactions (the rhodopsin cycle) when exposed to visible light. Rhodopsin consists of a lipoprotein called opsin and a chromophore (light-absorbing chemical compound called 11-cis-retinal) (5). The chemical reaction begins with the breakdown of rhodopsin and ends with the recombination of the breakdown products into rhodopsin. The recovery process takes 20–30 minutes, which is the time required to accommodate to low levels of illumination (dark adaptation). The process of viewing with rods is known as ‘‘scotopic’’ vision. The rods are maximally sensitive to light of about 510 nm in the blue–green region of the visible spectrum. Rods have no mechanisms to discriminate different wavelengths of light, and vision under low illumination conditions is essentially ‘‘colorblind.’’ More than 100 rods are connected to each ganglion cell, and the brain has no way of discriminating among these photoreceptors to identify the origin of an action potential transmitted along the ganglion. Hence, rod vision is associated with relatively low visual acuity in combination with high sensitivity to low levels of ambient light. The retina contains about 7 million cones that are packed tightly in the fovea and diminish rapidly across the macula lutae. The density of cones in the fovea is about 140,000/mm2 . Cones are maximally sensitive to light of about 550 nm in the yellow–green portion of the visible spectrum. Cones are much (1/104 ) less sensitive to light than rods, but in the fovea there is a 1:1 correspondence between cones and ganglions, so that

7

visual acuity is very high. Cones are responsible for color vision through mechanisms that are imperfectly understood at present. One popular theory of color vision proposes that three types of cones exist; each has a different photosensitive pigment that responds maximally to a different wavelength (450 nm for ‘‘blue’’ cones, 525 nm for ‘‘green’’ cones, and 555 nm for ‘‘red’’ cones). The three cone pigments share the same chromophore as the rods; their different spectral sensitivities result from differences in the opsin component. 3.2.2 Properties of Vision. For two objects to be distinguished on the retina, light rays from the objects must define at least a minimum angle as they pass through the optical center of the eye. The minimum angle is defined as the visual angle. The visual angle, expressed in units of minutes of arc, determines the visual acuity of the eye. A rather crude measure of visual acuity is provided by the Snellen chart that consists of rows of letters that diminish in size from top to bottom. When viewed from a distance of 20 feet, a person who has normal vision can just distinguish the letters in the eighth row. This person is said to have 20:20 vision (i.e., the person can see from 20 feet what a normal person can see at the same distance). At this distance, the letters on the eighth row form a visual angle of 1 minute of arc. An individual who has excellent vision and who is functioning in ideal viewing conditions can achieve a visual angle of about 0.5 minutes of arc, which is close to the theoretical minimum defined by the packing density of cones on the retina (6). A person who has 20:100 vision can see at 20 feet what a normal person can see at 100 feet. This individual is considered to have impaired visual acuity. Other more exact tests administered under conditions of uniform illumination are used for actual clinical diagnosis of visual defects. If the lettering of a Snellen chart is reversed (i.e., white letters on a black chart, rather than black letters on a white chart), the ability of observers to recognize the letters from a distance is greatly impaired. The eye is extremely sensitive to small amounts of light. Although the cones do not

8

IMAGING SCIENCE IN MEDICINE

respond at illumination levels below a threshold of about 0.001 cd/m2 , rods are much more sensitive and respond to just a few photons. For example, as few as 10 photons can generate a visual stimulus in an area of the retina where rods are at high concentration (7). Differences in signal intensity that can just be detected by the human observer are known as just noticeable differences (JND). This concept applies to any type of signal, including light, that can be sensed by the observer. The smallest difference in signal that can be detected depends on the magnitude of the signal. For example, we may be able to discern the brightness difference between one and two candles, but we probably cannot distinguish the difference between 100 and 101 candles. This observation was quantified by the work of Weber who demonstrated that the JND is directly proportional to the intensity of the signal. This finding was quantified by Fechner as
dS = k

dI I

 ,

(1)

where I is the intensity of stimulus, dS is an increment of perception (termed a limen), and k is a scaling factor. The integral form of this expression is known as the Weber–Fechner law: S = k (log I) + C, (2) or, by setting C = −k(logI0 ),
S=k

log I I0

 .

(3)

This expression states that the perceived signal S varies with the logarithm of the relative intensity. The Weber–Fechner law is similar to the equation for expressing the intensity of sound in decibels and provides a connection between the objective measurement of sound intensity and the subjective impression of loudness. A modification of the Weber–Fechner law is known as the power law (6). In this expression, the relationship between a stimulus and the perceived signal can be stated as
 dI dS , =n S I

(4)

which, when integrated, yields log S = n(log I) + K

(5)

and, when K is written as −n[logI0 ],
S=

I I0

n ,

(6)

where I0 is a reference intensity. The last expression, known as the power law, states that the perceived signal S varies with the relative intensity raised to the power n. The value of the exponent n has been determined by Stevens for a variety of sensations, as shown in Table 2. 3.3 Image Quality The production of medical images relies on intercepting some form of radiation that is transmitted, scattered, or emitted by the body. The device responsible for intercepting the radiation is termed an image receptor (or radiation detector). The purpose of the image receptor is to generate a measurable signal as a result of energy deposited in it by the intercepted radiation. The signal is often, but not always, an electrical signal that can be measured as an electrical current or voltage pulse. Various image receptors and their uses in specific imaging applications are described in the following sections. In describing the properties of a medical image, it is useful to define certain image characteristics. These characteristics and their definitions change slightly from one type of imaging process to another, so a model is needed to present them conceptually. X-ray projection imaging is the preferred model because this process accounts for more imaging procedures than any other imaging method used in medicine. In X-ray imaging, photons transmitted through the body are intercepted by an image receptor on the side of the body opposite from the X-ray source. The probability of interaction of a photon of energy E in the detector is termed the quantum detection efficiency η (8). This parameter is defined as η = 1 − e−µ(E)t ,

(7)

IMAGING SCIENCE IN MEDICINE

9

Table 2. Exponents in the Power Law for a Variety of Psychophysical Responses∗ Perceived Quantity Loudness Brightness Smell Taste Temperature Vibration Duration Pressure Heaviness Electric shock ∗ From

Exponent

Stimulus

0.6 0.5 0.55 1.3 1.0 0.95 1.1 1.1 1.45 3.5

Binaural Point source Coffee Salt Cold on arm 60 Hz on finger White noise stimulus Static force on palm Lifted weight 60 Hz through fingers

Ref. 7.

where µ(E) is the linear attenuation coefficient of the detector material that intercepts X rays incident on the image receptor and t is the thickness of the material. The quantum detection efficiency can be increased by making the detector thicker or by using materials that absorb X rays more readily (i.e., have a greater attenuation coefficient µ(E) because they have a higher mass density or atomic number. In general, η is greater at lower Xray energies and decreases gradually with increasing energy. If the absorbing material has an absorption edge in the energy range of the incident X rays, however, the value of η increases dramatically for X-ray energies slightly above the absorption edge. Absorption edges are depicted in Fig. 2 for three detectors (Gd2 O2 S, YTaO4 , and CaW04 ) used in X-ray imaging. 3.3.1 Image Noise. Noise may be defined generically as uncertainty in a signal due to random fluctuations in the signal. Noise is present in all images. It is a result primarily of forming the image from a limited amount of radiation (photons). This contribution to image noise, referred to as quantum mottle, can be reduced by using more radiation to form the image. However, this approach also increases the exposure of the patient to radiation. Other influences on image noise include the intrinsically variable properties of the tissues represented in the image, the type of receptor chosen to acquire the image, the image receptor, processing and display electronics, and the amount of scattered radiation that contributes to the image. In most

instances, quantum mottle is the dominant influence on image noise. In an image receptor exposed to N 0 photons, the image is formed with ηN 0 photons, and the photon image noise σ can be estimated as σ =(ηN 0 )1/2 . The signal-to-noise ratio (SNR) is ηN0 (ηN0 )1/2

(8)

= (ηN0 )1/2

(9)

SNR =

A reduction in either the quantum detection efficiency η of the receptor or the number of photons N 0 used to form the image yields a lower signal-to-noise ratio and produces a noisier image. This effect is illustrated in Fig. 3. A complete analysis of signal and noise propagation in an imaging system must include consideration of the spatialfrequency dependence of both the signal and the noise. The propagation of the signal is characterized by the modulation transfer function (MTF), and the propagation of noise is described by the Wiener (noise power) spectrum W(f ). A useful quantity for characterizing the overall performance of an imaging system is its detective quantum efficiency DQE(f ), where (f ) reveals that DQE depends on the frequency of the signal. This quantity describes the efficiency with which an imaging system transfers the signal-to-noise ratio of the radiative pattern that emerges from the patient into an image to be viewed by an observer. An ideal imaging system has

IMAGING SCIENCE IN MEDICINE

FP

O

10

Figure 2. Attenuation curves for three materials used in X-ray intensifying screens (from Ref. 9, with permission).

a DQE(f ) = η at all spatial frequencies. In actuality, DQE(f ) is invariably less than η, and the difference between DQE(f ) and η becomes greater at higher spatial frequencies. If DQE = 0.1η at a particular frequency, then the imaging system performs at that spatial frequency as if the number of photons were reduced to 1/10. Hence, the noise would increase by 101/2 at that particular frequency. 3.3.2 Spatial Resolution. The spatial resolution of an image is a measure of the smallest visible interval in an object that can be seen in an image of the object. Greater spatial resolution means that smaller intervals can be visualized in the image, that is, greater spatial resolution yields an image that is sharper. Spatial resolution can be measured and expressed in two ways: (1) by a test object

that contains structures separated by various intervals, and (2) by a more formal procedure that employs the modulation transfer function (MTF). A simple but often impractical way to describe the spatial resolution of an imaging system is by measuring its point-spread function (PSF; Fig. 4). The PSF(x,y) is the acquired image of an object that consists of an infinitesimal point located at the origin, that is, for an object defined by the coordinates (0,0). The PSF is the function that operates on what would otherwise be a perfect image to yield an unsharp (blurred) image. If the extent of unsharpness is the same at all locations in the image, then the PSF has the property of being ‘‘spatially invariant,’’ and the relationship of the image to the object (or

11

FP

O

IMAGING SCIENCE IN MEDICINE

FP O

Figure 3. Illustration of quantum mottle. As the illumination of the image increases, quantum mottle decreases and the clarity of the image improves, as depicted in these classic photographs (from Ref. 10).

Figure 4. The point-spread function PSF (x,y).

perfect image) is Image(x, y) = PSF(x, y) ⊗ object(x, y),

(10)

where the ‘‘⊗’’ indicates a mathematical operation referred to as ‘‘convolution’’ between the two functions. This operation can be stated as Image(x, y) = PSF(x − u, y − v) object(u, v) du dv. (11)

The convolution effectively smears each value of the object by the PSF to yield the image. The convolution (blurring) operation can be expressed by a functional operator, S[ . . . ], such that PSF(x, y) = S[point(x, y)]

(12)

where S[ . . . ] represents the blurring operator, referred to as the linear transform of the system.

12

IMAGING SCIENCE IN MEDICINE

The modulation transfer function MTF (m,n) is obtained from the PSF(x,y) by using the two-dimensional Fourier transform F MTF(m, n) = F[PSF(x, y)],

(or line source) is defined mathematically as +∞ point(x, y) dy. line(x) =

(13)

where (m,n) are the conjugate spatial frequency variables for the spatial coordinates (x,y). This expression of MTF is not exactly correct in a technical sense. The Fourier transform of the PSF is actually termed the system transfer function, and the MTF is the normalized absolute value of the magnitude of this function. When the PSF is real and symmetrical about the x and y axes, the absolute value of the Fourier transform of the PSF yields the MTF directly (11). MTFs for representative X-ray imaging systems (intensifying screen and film) are shown in Fig. 5. The PSF and the MTF, which is in effect the representation of the PSF in frequency space, are important descriptors of spatial resolution in a theoretical sense. From a practical point of view, however, the PSF is not very helpful because it can be generated and analyzed only approximately. The difficulty with the PSF is that the source must be essentially a singular point (e.g., a tiny aperture in a lead plate exposed to X rays or a minute source of radioactivity positioned at some distance from a receptor). This condition allows only a few photons (i.e., a very small amount of radiation) to strike the receptor, and very long exposure times are required to acquire an image without excessive noise. In addition, measuring and characterizing the PSF present difficult challenges. One approach to overcoming the limitations of the PSF is to measure the line-spread function (LSF). In this approach, the source is represented as a long line of infinitesimal width (e.g., a slit in an otherwise opaque plate or a line source of radioactivity). The LSF can be measured by a microdensitometer that is scanned across the slit in a direction perpendicular to its length. As for the point-spread function, the width of the line must be so narrow that it does not contribute to the width of the image. If this condition is met, the width of the image is due entirely to unsharpness contributed by the imaging system. The slit

(14)

−∞

The line-spread function LSF(x) results from the blurring operator for the imaging system operating on a line source (Fig. 6), that is,  +∞   LSF(x) = S[line(x)] = S  point(x, y) dy −∞

+∞ = S[point(x, y)] dy

(15)

−∞

LSF(x) =

+∞ PSF(x, y) dy,

(16)

−∞

that is, the line-spread function LSF is the point-spread function PSF integrated over the y dimension. The MTF of an imaging system can be obtained from the Fourier transform of the LSF. F[LSF (x)] +∞ LSF(x) exp(−2π imx) dx =

(17) (18)

−∞

  +∞ +∞  PSF(x, y) exp(−2π imx) dy dx (19) = −∞

−∞

 +∞ +∞   = PSF(x, y) exp[−2π i(mx  −∞

−∞

 +my)] dy dy n = 0

= F[PSF(x, y)]n=0 = MTF(m, 0),

(20) (21)

that is, the Fourier transform of the linespread function is the MTF evaluated in one dimension. If the MTF is circularly symmetrical, then this expression describes the MTF completely in the two-dimensional frequency plane. One final method of characterizing the spatial resolution of an imaging system is by using the edge-response function STEP (x,y).

FP

O

IMAGING SCIENCE IN MEDICINE

Figure 6. The line-spread function is the image of an ideal line object, where S represents the linear transform of the imaging system (from Ref. 11, with permission).

FP

O

Figure 5. Point-spread (top) and modulation-transfer (bottom) functions for fast and medium-speed CaWO4 intensifying screens (from Ref. 9, with permission).

13

14

IMAGING SCIENCE IN MEDICINE

In this approach, the imaging system is presented with a source that transmits radiation on one side of an edge and attenuates it completely on the other side. The transmission is defined as: STEP(x, y) = 1 if x0, and 0 if x < 0

(22)

This function can also be written as x STEP(x, y) =

line(x) dx

(23)

−∞

The edge-spread function ESF (x) can be computed as ESF(x) = S[STEP(x, y)]  x   = S  line(x) dx

(24)

−∞

x =

S[line(x)] dx

(25)

LSF(x) dx

(26)

−∞

x = −∞

This relationship, illustrated in Fig. 7, shows that the LSF (x) is the derivative of the edgeresponse function ESF (x). LSF(x) =

d [ESF(x)] dx

(27)

3.3.3 Contrast. Image contrast refers conceptually to the difference in brightness or darkness between a structure of interest and the surrounding background in an image. Usually, information in a medical image is presented in shades of gray (levels of ‘‘grayness’’). Differences in gray shades are used to distinguish various types of tissue, analyze structural relationships, and sometimes quantify physiological function. Contrast in an image is a product of both the physical characteristics of the object being studied and the properties of the image receptor used to form the image. In some cases, contrast can be altered by the exposure conditions chosen for the examination [for example, selection of the photon energy (kVp ) and use of a contrast agent in X-ray imaging]. Image contrast is also influenced by perturbing factors such as scattered radiation and the presence of extraneous light in the detection and viewing systems. An example of the same image at different levels of contrast is shown in Fig. 8. In most medical images, contrast is a consequence of the types of tissue represented in the image. In X-ray imaging, for example, image contrast reveals differences in the attenuation of X rays among various regions of the body, modified to some degree by other factors such as the properties of the image receptor, exposure technique, and the presence of extraneous (scattered) radiation. A simplified model of the human

Figure 7. The edge-spread function is derived from the image of an ideal step function, where S represents the linear transform of the imaging system (from Ref. 11, with permission).

FP

O

The relationship between the ESF and the LSF is useful because one can obtain a microdensitometric scan of the edge to yield an

edge-spread function. The derivative of the ESF yields the LSF, and the Fourier transform of the LSF provides the MTF in one dimension.

15

FP

O

IMAGING SCIENCE IN MEDICINE

Figure 8. Different levels of contrast in an image (from Ref. 12, with permission)

body consists of three different body tissues: fat, muscle, and bone. Air is also present in the lungs, sinuses, and gastrointestinal tract, and a contrast agent may have been used to accentuate the attenuation of X rays in a particular region. The chemical composition of the three body tissues, together with their percentage mass composition, are shown in Table 3. Selected physical properties of the tissues are included in Table 4, and the mass attenuation coefficients for different tissues as a function of photon energy are shown in Fig. 9. In Table 4, the data for muscle are also approximately correct for other soft tissues such as collagen, internal organs (e.g., liver

and kidney), ligaments, blood, and cerebrospinal fluid. These data are very close to the data for water, because soft tissues, including muscle, are approximately 75% water, and body fluids are 85% to nearly 100% water. The similarity of these tissues suggests that conventional X-ray imaging yields poor discrimination among them, unless a contrast agent is used to accentuate the differences in X-ray attenuation. Because of the presence of low atomic number (low Z) elements, especially hydrogen, fat has a lower density and effective atomic number compared with muscle and other soft tissues. At less than 35 keV, X

Table 3. Elemental Composition of Tissue Constituents∗ % Composition (by Mass)

Adipose Tissue

Hydrogen Carbon Nitrogen Oxygen Sodium Magnesium Phosphorus Sulfur Potassium Calcium ∗ From

11.2 57.3 1.1 30.3

0.06

Muscle (Striated) 10.2 12.3 3.5 72.9 0.08 0.02 0.2 0.5 0.3 0.007

water 11.2

88.8

Bone (Femur) 8.4 27.6 2.7 41.0 7.0 7.0 0.2 14.7

Ref. 11, with permission.

Table 4. Properties of Tissue Constituents of the Human Body∗ Material Air Water Muscle Fat Bone ∗ From

Effective Atomic Number

Density (kg/m3 )

7.6 7.4 7.4 5.9–6.3 11.6–13.8

1.29 1.00 1.00 0.91 1.65–1.85

Ref. 11, with permission.

Electron Density (electrons/kg) 3.01 × 1026 3.34 × 1026 3.36 × 1026 3.34–3.48 × 1026 3.00–3.10 × 1026

IMAGING SCIENCE IN MEDICINE

FP O

16

rays interact in fat and soft tissues predominantly by photoelectric interactions that vary with Z3 of the tissue. This dependence provides higher image contrast among tissues of slightly different composition (e.g., fat and muscle) when low energy X rays are used, compared with that obtained from higher energy X rays that interact primarily by Compton interactions that do not depend on atomic number. Low energy X rays are used to accentuate subtle differences in soft tissues (e.g., fat and other soft tissues) in applications such as breast imaging (mammography), where the structures within the object (the breast) provide little intrinsic contrast. When images are desired of structures of high intrinsic contrast (e.g., the chest where bone, soft tissue, and air are present), higher energy X rays are used to suppress X-ray attenuation in bone which otherwise would create shadows in the image that could hide underlying soft-tissue pathology. In some accessible regions of the body, contrast agents can be used to accentuate tissue contrast. For example, iodine-containing contrast agents are often injected into the circulatory system during angiographic imaging of blood vessels. The iodinated agent is watersoluble and mixes with the blood to increase its attenuation compared with surrounding soft tissues. In this manner, blood vessels can be seen that are invisible in X-ray images without a contrast agent. Barium is another element that is used to enhance contrast, usually in studies of the gastrointestinal (GI) tract. A thick solution of a barium-containing

Figure 9. Mass attenuation coefficient of tissues.

compound is introduced into the GI tract by swallowing or enema, and the solution outlines the borders of the GI tract to permit visualization of ulcers, polyps, ruptures, and other abnormalities. Contrast agents have also been developed for use in ultrasound (solutions that contain microscopic gas bubbles that reflect sound energy) and magnetic resonance imaging (solutions that contain gadolinium that affects the relaxation constants of tissues). 3.3.4 Integration of Image Noise, Resolution and Contrast—The Rose Model. The interpretation of images requires analyzing all of the image’s features, including noise, spatial resolution, and contrast. In trying to understand the interpretive process, the analysis must also include the characteristics of the human observer. Collectively, the interpretive process is referred to as ‘‘visual perception.’’ The study of visual perception has captured the attention of physicists, psychologists, and physicians for more than a century—and of philosophers for several centuries. A seminal investigation of visual perception, performed by Albert Rose in the 1940s and 1950s, yielded the Rose model of human visual perception (13). This model is fundamentally a probabilistic analysis of detection thresholds in low-contrast images. Rose’s theory states that an observer can distinguish two regions of an image, called ‘‘target’’ and ‘‘background,’’ only if there is enough information in the image to permit making the distinction. If the signal is

IMAGING SCIENCE IN MEDICINE

assumed to be the difference in the number of photons used to define each region and the noise is the statistical uncertainty associated with the number of photons in each region, then the observer needs a certain signal-tonoise ratio to distinguish the regions. Rose suggested that this ratio is between 5 and 7. The Rose model can be quantified by a simple example (11) that assumes that the numbers of photons used to image the target and background are Poisson distributed and that the target and background yield a lowcontrast image in which N = number of photons that define the target ∼ number of photons that define the background N = signal = difference in the number of photons that define target and background A = area of the target = area of background region C = contrast of the signal compared with background

of photons that define the target and the background: N (28) C= N Signal = N = CN

SNR =

CN signal = noise (N)1/2

= C(N)1/2 = C(A)1/2

(30)

where  is the photon fluence (number of photons detected per unit area) and A is the area of the target or background. Using the experimental data of his predecessor Blackwell, Rose found that the SNR has a threshold in the range of 5–7 for differentiating a target from its background. The Rose model is depicted in Fig. 3. A second approach to integrating resolution, contrast, and noise in image perception involves using contrast-detail curves. This method reveals the threshold contrast needed to perceive an object as a function of its diameter. Contrast-detail curves are shown in Fig. 10 for two sets of images; one was acquired at a relatively high signal-to-noise ratio (SNR), and the other at a lower SNR. The curves illustrate the intuitively obvious

FP

Figure 10. Results of tests using the contrastdetail phantom of Fig. 2–14 for high-noise and low-noise cases. Each dashed line indicates combinations of size and contrast of objects that are just barely visible above the background noise (from Ref. 9, with permission).

(29)

For Poisson-distributed events, noise=(N)1/2 , and the signal-to-noise ratio (SNR) is

O

The contrast between target and background is related to the number of detected photons N and the difference N between the number

17

18

IMAGING SCIENCE IN MEDICINE

conclusion that images of large objects can be seen at relatively low contrast, whereas smaller objects require greater contrast to be visualized. The threshold contrast curves begin in the upper left corner of the graph (large objects [coarse detail], low contrast) and end in the lower right corner (small objects [fine detail], high contrast). Contrast-detail curves can be used to compare the performance of two imaging systems or the same system under different operating conditions. When the performance data are plotted, as shown in Fig. 10, the superior imaging system is one that encompasses the most visible targets or the greatest area under the curve. 3.4 Image Display/Processing Conceptually, an image is a two-(or sometimes three-) dimensional continuous distribution of a variable such as intensity or brightness. Each point in the image is an intensity value; for a color image, it may be a vector of three values that represent the primary colors red, green, and blue. An image includes a maximum intensity and a minimum intensity and hence is bounded by finite intensity limits as well as by specific spatial limits. For many decades, medical images were captured on photographic film, which provided a virtually continuous image limited only by the discreteness of image noise and film granularity. Today, however, many if not most medical images are generated by computer-based methods that yield digital images composed of a finite number of numerical values of intensity. A two-dimensional medical image may be composed of J rows of K elements, where each element is referred to as a picture element or pixel, and a threedimensional image may consist of L slices, where each slice contains J rows of K elements. The three-dimensional image is made up of volume elements or voxels; each voxel has an area of one pixel and a depth equal to the slice thickness. The size of pixels is usually chosen to preserve the desired level of spatial resolution in the image. Pixels are almost invariably square, whereas the depth of a voxel may not correspond to its width and length. Interpolation is frequently used

to adjust the depth of a voxel to its width and length. Often pixels and voxels are referred to collectively as image elements or elements. Digital images are usually stored in a computer so that each element is assigned to a unique location in computer memory. The elements of the image are usually stored sequentially, starting with elements in a row, then elements in the next row, etc., until all of the elements in a slice are stored; then the process is repeated for elements in the next slice. There is a number for each element that represents the intensity or brightness at the corresponding point in the image. Usually, this number is constrained to lie within a specific range starting at 0 and increasing to 255 [8 bit (1 byte) number], 65,535 [16 bit (2 byte) number], or even 4,294,967,295 [32 bit (4 byte) number]. To conserve computer memory, many medical images employ an 8-bit intensity number. This decision may require scaling the intensity values, so that they are mapped over the available 256 (0–255) numbers within the 8-bit range. Scaling is achieved by multiplying the intensity value Ii in each pixel by 255/Imzx to yield an adjusted intensity value to be stored at the pixel location. As computer capacity has grown and memory costs have decreased, the need for scaling to decrease memory storage has become less important and is significant today only when large numbers of high-resolution images (e.g., X-ray planar images) must be stored. To display a stored image, the intensity value for each pixel in computer memory is converted to a voltage that is used to control the brightness at the corresponding location on the display screen. The intensity may be linearly related to the voltage. However, the relationship between voltage and screen brightness is a function of the display system and usually is not linear. Further, it may be desirable to alter the voltage: brightness relationship to accentuate or suppress certain features in the image. This can be done by using a lookup table to adjust voltage values for the shades of brightness desired in the image, that is, voltages that correspond to intensity values in computer memory are adjusted by using a lookup table to other voltage values that yield the desired distribution of image brightness. If a number of lookup

IMAGING SCIENCE IN MEDICINE

Figure 11. (a) A linear display mapping; (b) a nonlinear display to increase contrast (from Ref. 6, with permission).

O

FP

O

3.4.1 Image Processing. Often pixel data are mapped onto a display system in the manner described before. Sometimes, however, it is desirable to distort the mapping to accentuate certain features of the image. This process, termed image processing, can be used to smooth images by reducing their

noisiness, accentuate detail by sharpening edges in the image, and enhance contrast in selected regions to reveal features of interest. A few techniques are discussed have as examples of image processing. In many images, the large majority of pixels have intensity values that are clumped closely together, as illustrated in Fig. 12a. Mapping these values onto the display, either directly or in modified form as described earlier is inefficient because there are few bright or dark pixels to be displayed. The process of histogram equalization improves the efficiency by expanding the contrast range within which most pixels fall, and by compressing the range at the bright and dark ends where few pixels have intensity values. This method can make subtle differences in intensity values among pixels more visible. It is useful when the pixels at the upper and lower ends of the intensity range are not important. The process of histogram equalization is illustrated in Fig. 12b. Histogram

FP

tables are available, the user can select the table desired to illustrate specific features in the image. Examples of brightness: voltage curves obtained from different lookup tables are shown in Fig. 11. The human eye can distinguish brightness differences of about 2%. Consequently, a greater absolute difference in brightness is required to distinguish bright areas in an image compared with dimmer areas. To compensate for this limitation, the voltage applied to the display screen may be modified by the factor ekV to yield an adjusted brightness that increases with the voltage V. The constant k can be chosen to provide the desired contrast in the displayed images.

19

Figure 12. (a) Representative image histogram; (b) intensity- equalized histogram.

IMAGING SCIENCE IN MEDICINE

FP O

20

equalization is also applicable when the pixels are clumped at the high or low end of intensity values. All images contain noise as an intrinsic product of the imaging process. Features of an image can be obscured by noise, and reducing the noise is sometimes desired to make such features visible. Image noise can be reduced by image smoothing (summing or averaging intensity values) across adjacent pixels. The selection and weighting of pixels for averaging can be varied among several patterns; representative techniques are included there as examples of image smoothing. In the portrayal of a pixel and its neighbors shown in Fig. 13, the intensity value of the pixel (j,k) can be replaced by the average intensity of the pixel and its nearest neighbors (6). This method is then repeated for each pixel in the image. The nearest neighbor approach is a ‘‘filter’’ to reduce noise and yield a smoothed image. The nearest neighbor approach is not confined to a set number of pixels to arrive at an average pixel value; for example, the array of pixels shown in Fig. 13 could be reduced from 9 to 5 pixels or increased from 9 to 25 pixels, in arriving at an average intensity value for the central pixel. An averaging of pixel values, in which all of the pixels are averaged by using the same weighting to yield a filtered image, is a simple approach to image smoothing. The averaging process can be modified so that the intensity values of some pixels are weighted more

Figure 13. The nearest neighbor approach to image smoothing.

heavily than others. Weighted filters usually emphasize the intensity value of the central pixel (the one whose value will be replaced by the averaged value) and give reduced weight to the surrounding pixels in arriving at a weighted average. An almost universal rule of image smoothing is that when smoothing is employed, noise decreases, but unsharpness increases (i.e., edges are blurred). In general, greater smoothing of an image to reduce noise leads to greater blurring of edges as a result of increased unsharpness. Work on image processing often is directed at achieving an optimum balance between increased image smoothing and increased image unsharpness to reveal features of interest in specific types of medical images. The image filtering techniques described before are examples of linear filtering. Many other image-smoothing routines are available, including those that employ ‘‘nonlinear’’ methods to reduce noise. One example of a nonlinear filter is replacement of a pixel intensity by the median value, rather than the average intensity, in a surrounding array of pixels. This filter removes isolated noise spikes and speckle from an image and can help maintain sharp boundaries. Images can also be smoothed in frequency space rather than in real space, often at greater speed. 3.4.2 Image Restoration. Image restoration is a term that refers to techniques to remove or reduce image blurring, so that the image is ‘‘restored’’ to a sharper condition that is more representative of the object. This

IMAGING SCIENCE IN MEDICINE

technique is performed in frequency space by using Fourier transforms for both the image and the point-spread function of the imaging system. The technique is expressed as O(j, k) =

I(j, k) P(j, k)

(31)

where I (j,k) is the Fourier transform of the image, P (j,k) is the Fourier transform of the point-spread function, and O (j,k) is the Fourier transform of the object (in threedimensional space, a third spatial dimension l would be involved). This method implies that the unsharpness (blurring) characteristics of the imaging device can be removed by image processing after the image has been formed. Although many investigators have pursued image restoration with considerable enthusiasm, interest has waned in recent years because two significant limitations of the method have surfaced (6). The first is that the Fourier transform P (j,k) can be zero for certain values of (j,k), leading to an undetermined value for O (j,k). The second is that image noise is amplified greatly by the restoration process and often so overwhelms the imaging data that the restored image is useless. Although methods have been developed to reduce these limitations, the conclusion of most attempts to restore medical images is that it is preferable to collect medical images at high resolution, even if sensitivity is compromised, than to collect the images at higher sensitivity and lower resolution, and then try to use image-restoration techniques to recover image resolution. 3.4.3 Image Enhancement. The human eye and brain act to interpret a visual image principally in terms of boundaries that are presented as steep gradients in image brightness between two adjacent regions. If an image is processed to enhance these boundary (edge) gradients, then image detail may be more visible to the observer. Edgeenhancement algorithms function by disproportionately increasing the high-frequency components of the image. This approach also tends to enhance image noise, so that edgeenhancement algorithms are often used together with an image-smoothing filter to suppress noise.

4

21

CONCLUSION

The use of images to detect, diagnose, and treat human illness and injury has been a collaboration among physicists, engineers, and physicians since the discovery of X rays in 1895 and the first applications of X-ray images to medicine before the turn of the twentieth century. The dramatic expansion of medical imaging during the past century and the ubiquitous character of imaging in all of medicine today have strenghthened the linkage connecting physics, engineering, and medical imaging. This bond is sure to grow even stronger as imaging develops as a tool for probing the cellular, molecular, and genetic nature of disease and disability during the first few years of the twenty-first century. Medical imaging offers innumerable challenges and opportunities to young physicists and engineers interested in applying their knowledge and insight to improving the human condition. REFERENCES 1. W. R. Hendee, Rev. Mod. Phys. 71(2), Centenary, S444–S450 (1999). 2. R. N. Beck, in W. Hendee and J. Trueblood, eds., Digital Imaging, Medical Physics, Madison, WI, 1993, pp. 643–665. 3. J. H. Thrall, Diagn. Imaging (Dec.), 23–27 (1997). 4. W. R. Hendee, in W. Hendee and J. Trueblood, eds., Digital Imaging, Medical Imaging, Madison, WI, 1993, pp. 195–212. 5. P. F. Sharp and R. Philips, in W. R. Hendee and P. N. T. Wells, ed., The Perception of Visual Information, Springer, NY, 1997, pp. 1–32. 6. B. H. Brown et al., Medical Physics and Biomedical Engineering, Institute of Physics Publishing, Philadelphia, 1999. 7. S. S. Stevens, in W. A. Rosenblith, ed., Sensory Communication, MIT Press, Cambridge, MA, 1961, pp. 1–33. 8. J. A. Rowlands, in W.R. Hendee, ed., Biomedical Uses of Radiation, Wiley-VCH, Weinheim, Germany, 1999, pp. 97–173. 9. A. B. Wolbarst, Physics of Radiology, Appleton and Lange, Norwalk, CT, 1993. 10. A. Rose, Vision: Human and Electronic, Plenum Publishing, NY, 1973.

22

IMAGING SCIENCE IN MEDICINE

11. B. H. Hasegawa, The Physics of Medical X-Ray Imaging, 2nd ed., Medical Physics, Madison, WI, 1991, p. 127. 12. W. R. Hendee, in C. E. Putman and C. E. Ravin, eds., Textbook of Diagnostic Imaging, 2nd ed., W.B. Saunders, Philadelphia, 1994, pp. 1–97. 13. A. Rose, in Proc. Inst. Radioengineers 30, 293–300 (1942).

IMPUTATION

analysis to only those subjects with no missing data) is generally acceptable. Missing data are missing at random (MAR) if missingness depends only on observed values of variables and not on any missing values; for example, if the value of blood pressure at time two is more likely to be missing when the observed value of blood pressure at time one was normal, regardless of the value of blood pressure at time two. If missingness depends on the values that are missing, even after conditioning on all observed quantities, the missing data mechanism is not missing at random (NMAR). Missingness must then be modeled jointly with the data—the missingness mechanism is ‘‘nonignorable.’’ The specific imputation procedures described here are most appropriate when the missing data are MAR and ignorable (see References 2 and 3 for details). Imputation (multiple) can still be validly used with nonignorable missing data, but it is more challenging to do it well.

SAMANTHA R. COOK and DONALD B. RUBIN Harvard University

1

INTRODUCTION

Missing data are a common problem with large databases in general and with clinical and health-care databases in particular. Subjects in clinical trials may fail to provide data at one or more time points or may drop out of a trial altogether, for reasons including lack of interest, untoward side effects, and change of geographical location. Data may also be ‘‘missing’’ due to death, although the methods described here are generally not appropriate for such situations. An intuitive way to handle missing data is to fill in (i.e., impute) plausible values for the missing values, thereby creating completed datasets that can be analyzed using standard complete-data methods. The past 25 years have seen tremendous improvements in the statistical methodology for handling incomplete datasets. After briefly discussing missing data mechanisms, the authors present some common imputation methods, focusing on multiple imputation (1). They then discuss computational issues, present some examples, and conclude with a short summary.

2

3

SINGLE IMPUTATION

Single imputation refers to imputing one value for each missing datum. Singly imputed datasets are straightforward to analyze using complete-data methods, which makes single imputation an apparently attractive option with incomplete data. Little and Rubin (3, p. 72) offer the following guidelines for creating imputations. They should be (1) conditional on observed variables; (2) multivariate, to reflect associations among missing variables; and (3) drawn from predictive distributions rather than set equal to means, to ensure that variability is reflected. Unconditional mean imputation, which replaces each missing value with the mean of the observed values of that variable, meets none of the three guidelines listed above. Conditional mean imputation can satisfy the first guideline by filling in missing values with means calculated within cells defined by variables such as gender and/or treatment arm, but it does not meet the second or third guidelines. Regression imputation can satisfy the first two guidelines by replacing

MISSING DATA MECHANISMS

A missing data mechanism is a probabilistic rule that governs which data will be observed and which will be missing. Rubin (2) and Little and Rubin (3) describe three types of missing data mechanisms. Missing data are missing completely at random (MCAR) if missingness is independent of both observed and missing values of all variables. MCAR is the only missing data mechanism for which ‘‘complete case’’ analysis (i.e., restricting the

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

IMPUTATION

the missing values for each variable with the values predicted from a regression (e.g., least squares) of that variable on other variables. Stochastic regression imputation adds random noise to the value predicted by the regression model, and when done properly can meet all three guidelines. Hot deck imputation replaces each missing value with a random draw from a donor pool of observed values of that variable; donor pools are selected, for example, by choosing individuals with complete data who have ‘‘similar’’ observed values to the subject with missing data, e.g., by exact matching or using a distance measure on observed variables to define similar. Hot deck imputation, when done properly, can also satisfy all three of the guidelines listed above. Even though analyzing a singly imputed dataset with standard techniques is easy, such an analysis will nearly always result in estimated standard errors that are too small, confidence intervals that are too narrow, and P-values that are too significant, regardless of how the imputations were created. Thus, single imputation is almost always statistically invalid. However, the multiple version of a single imputation method will be valid if the imputation method is ‘‘proper.’’ Proper imputations satisfy the three criteria of Little and Rubin.

4 PROPERLY DRAWN SINGLE IMPUTATIONS Let Y represent the complete data; i.e., all data that would be observed in the absence of missing data, and let Y = {Y obs ,Y mis }, the observed and missing values, respectively. For notational simplicity, assume ignorability of the missing data mechanism. Also let θ represent the (generally vector-valued) parameter associated with an appropriate imputation model, which consists of both a sampling distribution on Y governed by θ , p(Y|θ ), and a prior distribution on θ , p(θ ). A proper imputation is often most easily obtained as a random draw from the posterior predictive distribution of the missing data given the observed data, which formally

can be written as
p(Ymis |Yobs ) = p(Ymis , θ |Yobs ) dθ
=

p(Ymis |Yobs , θ )p(θ |Yobs ) dθ.

If the missing data follow a monotone pattern (defined below), this distribution is straight-forward to sample from. When missing data are not monotone, iterative computational methods are generally necessary, as described shortly. 5 PROPERLY DRAWN SINGLE IMPUTATIONS—THEORY WITH MONOTONE MISSINGNESS A missing data pattern is monotone if the rows and columns of the data matrix can be sorted in such a way that an irregular staircase separates Y obs and Y mis . Figures 1 and 2 illustrate monotone missing data patterns. Let Y 0 represent fully observed variables, Y 1 the incompletely observed variable with the fewest missing values, Y 2 the variable with the second fewest missing values, and so on. Proper imputation with a monotone missing data pattern begins by fitting an appropriate model to predict Y 1 from Y 0 and then using this model to impute the missing values in Y 1 . For example, fit a regression of Y 1 on Y 0 using the units with Y 1 observed, draw the regression parameters from their posterior distribution, and then draw the missing values of Y 1 given these parameters and Y 0 . Next impute the missing values for Y 2 using Y 0 and the observed and imputed values of

X

Y(1)

X

Y(0)

? ? Treated

Control

Figure 1. Pattern of missing data for intergel trial.

IMPUTATION X

Y(1)

X

Y(0)

? Treated

Control

Figure 2. Pattern of missing data for genzyme trial.

Y 1 . Continue in this manner until all missing values have been imputed. The collection of imputed values is a proper imputation of the missing data, Y mis , and the collection of univariate prediction models is the implied full imputation model. When missing data are not monotone, this method of imputation as described cannot be used. 6 PROPERLY DRAWN SINGLE IMPUTATIONS—THEORY WITH NONMONOTONE MISSINGNESS Creating imputations when the missing data pattern is nonmonotone generally involves iteration because the distribution p(Ymis |Yobs ) is often difficult to draw from directly. However, the data augmentation algorithm [DA; Tanner and Wong (4)] is often straightforward to implement. Briefly, DA involves iterating between sampling missing data given a current draw of the model parameters and sampling model parameters given a current draw of the missing data. The draws of Y mis form a Markov Chain whose stationary distribution is p(Ymis |Yobs ). Thus, once the Markov Chain has reached approximate convergence, a draw of Y mis from DA is effectively a proper single imputation of the missing data from the target distribution p(Ymis |Yobs ). Many programs discussed in Section (9) use DA to impute missing values. Other algorithms that use Markov Chain Monte Carlo methods for imputing missing values include Gibbs sampling and Metropolis–Hastings. See Gelman et al. (5, Chapter 11) for more details. As discussed, analyzing a singly imputed dataset using complete-data methods usually leads to anticonservative results because

3

imputed values are treated as if they were known, thereby underestimating uncertainty. Multiple imputation corrects this problem, while retaining the advantages of single imputation. 7

MULTIPLE IMPUTATION

Described in detail in Rubin (6), multiple imputation (MI) is a Monte Carlo technique that replaces the missing values Y mis with m > 1 plausible values, {Y mis,1 , . . . ,Y mis,m }, and therefore reveals uncertainty in the imputed values. Each set of imputations creates a completed dataset, thereby creating m completed datasets: Y (1) = {Y obs , Y mis,1 }, . . ,. Y (l) = {Y obs , Y mis,l }, . . ,.Y (m) = {Y obs , Y mis,m }. Typically m is fairly small: m = 5 is a standard number of imputations to use. Each m completed dataset is then analyzed as if there were no missing data and the results combined using simple rules described shortly. Obtaining proper MIs is no more difficult than obtaining a single proper imputation—the process for obtaining a proper single imputation is simply repeated independently m times. 8 COMBINING RULES FOR PROPER MULTIPLE IMPUTATIONS As in Rubin (6) and Schafer (7), let Q represent the estimated of interest (e.g., the mean of a variable, a relative risk, the intentionto-treat effect, etc.), let Qest represent the complete-data estimator of Q (i.e., the quantity calculated treating all imputed values of Y mis as known observed data), and let U represent the estimated variance of Qest − Q. Let Qest,l be the estimate of Q based on the lth imputation of Y mis with associated variance U l ; that is, the estimate of Q and associated variance are based on the complete-data analysis of the lth completed dataset, Y l = {Y obs , Y mis,l }. The MI estimate of Q is simply  the average of the m estimates: QMIest = m l=1 Qest,l /m . The estimated variance of QMIest − Q is found by combining between and within imputation variance, as with the analysis of variance: T = U ave + (1 + m−1 )B, where Uave =  m l=1 Ul /m is the within imputation variance

4

IMPUTATION

 2 and B = m l=1 (Qest,l − QMIest ) /(m − 1) is the between imputation variance. The quantity T −1/2 (Q − QMIest ) follows an approximate tν distribution with degrees of freedom ν = (m − 1)(1 + U ave /((1 + m−1 )B))2 . See Rubin and Schenker (8) for additional methods for combining vector-valued estimates, significance levels, and likelihood ratio statistics and Barnard and Rubin (9) for an improved expression for ν with small complete data sets. 9 COMPUTATION FOR MULTIPLE IMPUTATION Many standard statistical software packages now have built-in or add-on functions for MI. The S-plus libraries NORM, CAT, MIX, and PAN, for analyzing continuous, categorical, mixed, and panel data, respectively, are freely available [(7), http://www.stat.psu.edu /∼jls/], as is MICE [(10), http://web.inter.nl. net/users/S.van.Buuren/mi/html/mice.htm], which uses regression models to impute all types of data. SAS now has procedures PROC MI and PROC MIANALYZE; in addition IVEwear (11) is freely available and can be called using SAS (http://support.sas.com/rnd/ app/da/new/dami.html). New software packages have also been developed specifically for multiple imputation, for example, the commercially available SOLAS (http://www.statsol.ie/solas/solas.htm), which has been available for years and is most appropriate for datasets with a monotone or nearly monotone pattern of missing data, and the freely available NORM, a standalone Windows version of the S-plus function NORM (http://www.stat. psu.edu/∼jls/). For more information, see www.multiple-imputation.com or Horton and Lipsitz (12). 10

EXAMPLE: LIFECORE

Intergel is a solution developed by Lifecore Corporation to prevent surgical gynecological adhesions. A double-blind, multicenter randomized trial was designed for the U.S. Food and Drug Administration (FDA) to determine whether Intergel significantly reduces the formation of adhesions after a first surgery. The data collection procedure for this study

was fairly intrusive: Patients had to undergo a minor abdominal surgery weeks after the first surgery in order for doctors to count the number of gynecological adhesions. The trial, therefore, suffered from missing data because not all women were willing to have another surgery, despite having initially agreed to do so. The original proposal from FDA for imputing the missing values was to fill in the worst possible value (defined to be 32 adhesions—most patients with observed data had 10 or fewer adhesions) for each missing datum, which should lead to ‘‘conservative’’ results because there were more missing data in the treatment arm than in the placebo arm. This method ignores observed data when creating imputations; for example, one woman in the treatment group refused the second look surgery because she was pregnant, which would have been impossible with more than a few gynecological adhesions. Furthermore, because the imputed values are so much larger than the observed values, the standard errors based on these worst-possible value imputations were inflated, making it unlikely to be able to get significant results even when the two treatments were significantly different. Figure 1 displays the general pattern of monotone missing data in this case, with X representing covariates, Y(0) outcomes under placebo, and Y(1) outcomes under Intergel. The question marks represent missing values. Colton et al. (13) instead used an MI hot deck procedure to impute missing values. Donor pools were created within cells defined by treatment group, treatment center, and baseline seriousness of adhesions, which were observed for all patients: For each patient whose outcome was missing, the donor pool consisted of the two patients in the same treatment group and treatment center who had the closest baseline adhesion scores. Each value in the donor pool was used as an imputation. Formally this method is improper, but the limited donor pools should still make the method conservative because the matches are not as close as they would be with bigger sample sizes.

IMPUTATION

11

EXAMPLE: GENZYME

Fabrazyme is a drug developed by Genzyme Corporation to treat Fabry’s disease, a rare and serious genetic disease caused by an inability to metabolize creatinine. Preliminary results from a Phase 3 FDA trial of Fabrazyme versus placebo showed that the drug appeared to work well in patients in their 30s, who were not yet severely ill, in the sense that it lowered their serum creatinine substantially. A similar Phase 4 trial involved older patients who were more seriously ill. As there is no other fully competitive drug, it was desired to make Fabrazyme commercially available early, which would allow patients randomized to placebo to begin taking Fabrazyme, but would create missing outcome data among placebo patients after they began taking Fabrazyme. The study had staggered enrollment, so that the number of monthly observations of serum creatinine for each placebo patient depended on his time of entry into the study. Figure 2 illustrates the general pattern of monotone missing data with the same length follow-up for each patient. Again, X represents baseline covariates, Y(0) represents repeated measures of serum creatinine for placebo patients, and Y(1) represents repeated measures of serum creatinine for Fabrazyme patients. To impute the missing outcomes under placebo, a complex hierarchical Bayesian model was developed for the progression of serum creatinine in untreated Fabry patients. In this model, inverse serum creatinine varies linearly and quadratically in time, and the prior distribution for the quadratic trend in placebo patients is obtained from the posterior distribution of the quadratic trend in an analogous model fit to a historical database of untreated Fabry patients. Thus, the historical patients’ data only influence the imputations of the placebo patients’ data subtly—via the prior distribution on the quadratic trend parameters. Although the model fitting algorithm is complex, it is straightforward to use the algorithm to obtain draws from p(θ |Yobs ) for the placebo patients, and then draw Y mis conditional on the drawn value of θ , where θ represents all model parameters. Drawing

5

the missing values in this way creates a sample from p(Y mis |Y obs ) and thus an imputation for the missing values in the placebo group. The primary analysis will consider the time to an event, defined as either a clinical event (e.g., kidney dialysis, stroke) or a substantial increase in serum creatinine relative to baseline. The analysis will be conducted on each imputed dataset and the results will be combined (as outlined earlier) to form a single inference. 12

EXAMPLE: NMES

The National Medical Expenditure Survey (NMES) collects data on medical costs from a random sample of the U.S. population. The data include medical expenditures, background information, and demographic information. Multiple imputation for NMES was more complicated than in the previous two examples because the missing data pattern was not monotone. Figure 3 shows a simplification of the missing data pattern for NMES, where if Y 1 was fully observed, the missing data pattern would be monotone. Rubin (14) imputed the missing data in NMES by capitalizing on the simplicity of imputation for monotone missing data by first imputing the missing values that destroyed the monotone pattern (the ‘‘nonmonotone missing values’’) and then proceeding as if the missing data pattern was in fact monotone, and then iterating this process. More specifically, after choosing starting values for the missing data, iterate between the following two steps: (1) Regress each variable with any nonmonotone missing values (i.e., Y 1 ), on all other variables (i.e., Y 0 ,Y 2 ,Y 3 ), treating the current imputations as true values, but use this regression to impute only the

Y0 Y1 Y2 Y3 ?

?

? Figure 3. Illustrative display for type of pattern of missing data in NMES.

6

IMPUTATION

nonmonotone missing values. (2) Impute the remaining missing values in the monotone pattern; first impute the variable with the fewest missing values (Y 2 in Fig. 3), then the variable with the second fewest missing values (Y 3 in Fig. 3), and so on, treating the nonmonotone missing values filled in step 1 as known. This process was repeated five times to create five sets of imputations in the NMES example. 13

SUMMARY

Multiple imputation is a flexible tool for handling incomplete datasets. MIs are often straightforward to create using computational procedures such as DA or using special MI software now widely available. Moreover, the results from singly imputed datasets are easy to combine into a single MI inference. Although MI is Bayesianly motivated, many MI procedures have been shown to have excellent frequentist properties (15). In small samples, the impact of the prior distribution on conclusions can be assessed by creating MIs using several different prior specifications. Furthermore, although only MAR procedures have been considered here, missing data arising from an NMAR mechanism may be multiply imputed by jointly modeling the data and the missingness mechanism; in some cases, results are insensitive to reasonable missingness models and the missing data can then be effectively treated as being MAR (6). Rubin (6), Schafer (7), and Little and Rubin (3) are excellent sources for more detail on the ideas presented here, the last two being less technical and more accessible than the first. REFERENCES 1. D. B. Rubin, Multiple imputations in sample surveys: A phenomenological Bayesian approach to nonresponse (with discussion). ASA Proc. Section on Survey Research Methods, 1978: 20–34. 2. D. B. Rubin, Inference and missing data. Biometrika 1976; 63: 581–590. 3. R. J. A. Little, and D. B. Rubin, Statistical Analysis with Missing Data. New York: Wiley, 2002.

4. M. A. Tanner, and W. H. Wong, The calculation of posterior distributions by data augmentation (with discussion). J. Amer. Stat. Assoc. 1987; 82: 528–550. 5. A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin, Bayesian Data Analysis. London: Chapman & Hall, 1995. 6. D. B. Rubin, Multiple Imputation for Nonresponse in Surveys. New York: Wiley, 1987. 7. J. L. Schafer, Analysis of Incomplete Multivariate Data. London: Chapman & Hall, 1997. 8. D. B. Rubin, and N. Schenker, Multiple imputation in health-care databases: An overview and some applications. Stat. Med. 1991; 10: 585–598. 9. J. Barnard, and D. B. Rubin, Small-sample degrees of freedom with multiple imputation. Biometrika 1999; 86: 948–955. 10. S. van Buuren, H. C. Boshuizen, and D. L. Knook, Multiple imputation of missing blood pressure covariates in survival analysis. Stat. Med. 1999; 18: 681–694. 11. T. E. Raghunathan, J. M. Lepkowski, J. van Hoewyk, and P. Solenberger, A multivariate technique for multiply imputing missing values using a sequence of regression models. Survey Methodol. 2001; 27: 85–95. 12. N. J. Horton, and S. R. Lipsitz, Multiple imputation in practice: Comparison of software packages for regression models with missing variables. Amer. Statistician 2001; 55: 244–254. 13. T. Colton, S. Piantadosi, and D. B. Rubin, Multiple imputation for second-look variables based on Intergel pivotal trial data. Report submitted to FDA. 2001. 14. D. B. Rubin, Nested multiple imputation of NMES via partially incompatible MCMC. Statistica Neerlandica 2003; 57: 3–18. 15. D. B. Rubin, Multiple imputation after 18 + years (with discussion). J. Amer. Stat. Assoc. 1996; 91: 473–520.

INCOMPETENT SUBJECTS AND PROXY CONSENT

make a will), but this person is entirely competent to make a medical treatment decision because she understands the risks, benefits, and alternatives of the proposed medical procedure. Children are presumed to be incompetent, and important decisions, such as medical treatment decisions, are generally made on their behalf by their parents.

LEONARD H. GLANTZ Boston University School of Public Health Boston, Massachusetts

Research with subjects who are incompetent presents the most serious ethical challenges than any other category of research. By definition, incompetent research subjects cannot consent to their participation in research, and consent is an essential element of the protection of the rights of human subjects. The Nuremberg Code proclaims that the informed and voluntary consent of the human subject is ‘‘absolutely essential.’’ However, requiring consent of every human subject would preclude research on certain classes of people. 1

2 CONSIDERATIONS IN ENROLLING INCOMPETENT SUBJECTS The use of incompetent subjects in research is inherently suspect because they cannot consent voluntarily to participation, which is generally regarded as an essential ethical and legal condition for research participation. To avoid this problem, incompetent subjects should not be included in a research project unless it is impossible to conduct the research with competent subjects. For example, there is no reason to conduct research on a new antibiotic that will be used in the general population on incompetent subjects. However, research to test the efficacy of a treatment for the alleviation of serious dementia or developing a treatment for premature newborns could not use competent subjects. Even where good scientific reasons are provided for using incompetent subjects, serious ethical challenges remain. One can think of research as existing on a spectrum from pure research, in which the sole goal of the research is the creation of knowledge, to the last stages of clinical research, in which substantial data support the conclusion that the use of the test material holds out a realistic prospect of direct benefit to the subject. For example, substantial ethical differences exist between exposing an incompetent subject to the early toxicity testing of drug in which subjects are used to determine the negative effects of different doses, and administering that drug at the end of the testing process when it is designed to determine whether subjects will benefit from the use of the drug. Although research and treatment are never

DEFINITION OF INCOMPETENCE

Both legal and ethical principles presume that every adult is competent to make important decisions about themselves. This determination is essential to maximize the autonomy of individuals and enable them to act on what they believe is in their own best interests. This presumption is based on the belief that adults can weigh the benefits, risks, and alternatives to proposed actions and decisions. However, once it is determined that adults cannot weigh these considerations, they are deemed to be incompetent and are deprived of their decision-making capacity. Adults may be incompetent as a result of serious dementia, serous developmental disabilities, certain types of psychosis (not all psychotic people are incompetent), and the influence of mind-altering substances (once the influence of such substances disappear, the person will again be competent). Incompetence is determined by establishing the inability to understand the risks, benefits, and alternatives of particular activities. For example, a person may be incompetent to make a will because a mental illness makes them incapable of recognizing that they have children (a basic requirement to competently

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INCOMPETENT SUBJECTS AND PROXY CONSENT

the same thing, the closer the research resembles treatment the more permissible the use of incompetent subjects becomes. Although it is important for researchers to consider their own ethical obligations not to exploit the decisional incapacity of subjects, important considerations must be reviewed for surrogate decision makers who permit the use of their wards in research programs. Both parents and legal guardians are supposed to act in ways that protect their children or wards from unnecessary harm. Thus, some ethicists believe surrogate decision makers may never permit their wards to be subjects in research that presents risk of harm with no corresponding benefits. Other ethicists believe that surrogates may give such permission if the risks are trivial and benefits to knowledge are potentially important. For these ethicists, the issue is often how to define a trivial risk. Noninvasive interactions that have no risk for research subjects, such as taking hair clippings or weighing children, are not problematic for either group of ethicists. But drawing blood, conducting skin biopsies, or taking spinal fluid, although all perfectly routine in the clinical setting, are more problematic in the research setting in which the subject can receive no benefit from the physical intrusions. It is also important to recognize that investigators and surrogates are obligated to protect incompetent subjects from fear, distress, and discomforts. For example, whereas an MRI procedure may present no physical risks, many people find the noise and claustrophobia that are inherent in the process to be frightening. Taking a demented person from their home and subjecting them to an MRI can cause substantial fear and distress that may not be ethically justified. 3

SUBJECT ASSENT

Even in the instance where a person is incompetent to give consent, it is important to try to obtain an incompetent subjects assent to a procedure. Even where subjects cannot understand the reasons for a procedure, they may understand the procedure itself. For example, a person might not understand why a doctor wants to take blood from them, but

they can understand that it involves putting a needle in their arm. Although no benefit is provided to the person for blood withdrawal, it is widely accepted that the person’s permission for the intrusion should be obtained. The Federal research regulations require this permission for research in children when the Institutional Review Board (IRB) determines that the children in the study can give assent, and the research holds out no prospect of direct benefit to the child. 4 SUBJECT DISSENT For informed consent to be valid, it must be given by a competent person. However, an incompetent person can express dissent. Using the MRI procedure discussed above as an example, if the demented person seems to object to being placed in the device, either verbally or through her actions, it is widely accepted that such dissent should be respected. As a matter of ethics and fairness, there is a significant difference between performing a procedure on someone who does not consent but is compliant, and a subject who objects who is forcibly intruded on. 5 RESEARCH ADVANCED DIRECTIVES Because of the ethical issues presented by research with incompetent subjects, it has been suggested that potentially incompetent research subjects give advanced consent to being a research subject for specified classes of research. This consent is similar to the use of ‘‘living wills’’ or advanced directives that people execute to determine their medical care should they become incompetent to make medical decisions in the future. Research advance directives might prove to be useful where the condition being studied is progressive and will likely lead to incompetence in the future. For example, a person with the genetic markers for Huntington Disease might be willing to enroll in longitudinal research on the condition and may execute an advance directive that permits researchers to conduct physically intrusive procedures, such as spinal taps, when she becomes incompetent. As a result, the research that is conducted on this person when he/she becomes

INCOMPETENT SUBJECTS AND PROXY CONSENT

incompetent will be based on his/her consent, not the consent of a surrogate. Researchers have little experience with such documents, so one cannot know their effectiveness in addressing the difficult ethical questions presented by research on incompetent subjects. For example, if a person who has executed a research advanced directive dissents to participation in a procedure when he/she is incompetent, should the previous executed document or the current dissent govern? REFERENCES FURTHER READING G. J. Annas and L. H. Glantz, Rules for research in nursing homes. New Engl. J. Med. 1986; 315: 1157–1158. R. Dresser, Research involving persons with mental disabilities: a review of policy issues. In National Bioethics Advisory Commission, Research Involving Persons with Mental Disorders that May Affect Decisionmaking Capacity, Vol. II 1999, pp. 5–28. Available: http://bioethics.georgetown.edu/nbac/ pubs.html. M. J. Field and R. E. Behrman, eds. Ethical Conduct of Clinical Research Involving Children. Washington, DC: National Academy Press, 2004. L. H. Glantz, Nontherapeutic research with children; Grimes v. Kennedy Krieger Institute. Am. J. Public Health 2002; 92: 4–7. M. A. Grodin and L. H. Glantz, eds. Children as Research Subjects: Science, Ethics and Law. New York: Oxford U. Press, 1994. National Bioethics Advisory Commission, Research Involving Persons with Mental Disorders that May Affect Decisionmaking Capacity, Vol. 1, 1999. The Department of Health and Human Services, Protection of Human Subjects, Title 45 Code of Federal Regulations part 46 Subpart D, 1991.

CROSS-REFERENCES Informed Consent Process, Forms, and Assent

3

INDEPENDENT ETHICS COMMITTEE (IEC) An Independent Ethics Committee (IEC) is an independent body (a review board or a committee, institutional, regional, national, or supranational) that is composed of medical/scientific professionals and nonmedical/ nonscientific members. Their responsibility is to ensure the protection of the rights, safety, and well-being of human subjects involved in a trial and to provide public assurance of that protection, by, among other things, reviewing and approving/providing favorable opinion on the trial protocol, the suitability of the investigator(s), the facilities, and the methods and material to be used in obtaining and documenting informed consent of the trial subjects. The legal status, composition, function, operations, and regulatory requirements that pertain to Independent Ethics Committees may differ among countries, but they should allow the Independent Ethics Committee to act in agreement with Good Clinical Practice (GCP) as described in this guideline.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

1

inferences∗ and superpopulation inferences∗ to the extent that the validity of the claimed generality is model dependent∗ via its sensitivity to model misspecifications∗ . Also, it is possible to regard Bayesian inferences and superpopulation inferences as providing some unity to the role of design-based and model-based considerations. Thus a focus of distinction here between design-based and model-based inference is the population to which results are generalized rather than the nature of statistical methods. Models can be useful conceptually in either context; also they can shed light on the robustness∗ of inferences to their underlying assumptions. The related issue of external validity includes substantive justification for the area of application and statistical evaluation of the plausibility of model assumptions. For other pertinent discussion, see Deming [10, Chap. 7], Fisher [12], Godambe and Sprott [15], Johnson and Smith [18], Kempthorne and Folks [23, Chap. 17], Namboodiri [29], and INFERENCE, STATISTICAL. The distinctions between design-based inference and model-based inference can be expressed in clearest terms for comparative experimental studies (e.g., multicenter clinical trials∗ ). Typically, these involve a set of blocks (or sites) which are selected on a judgmental basis. Similarly, the experimental units may be included according to convenience or availability. Thus, these subjects constitute a fixed set of finite local study populations. When they are randomly assigned to two or more treatment groups, corresponding samples are obtained for the potential responses of all subjects under study for each of the respective treatments. By virtue of the research design, randomization model methods (e.g., Kruskal-Wallis tests∗ ) in CHI-SQUARE TEST —I can be used to obtain design-based inferences concerning treatment comparisons without any external assumptions. Illustrative examples are given in CHI-SQUARE TEST —I, and LOG-RANK SCORES. A limitation of design-based inferences for experimental studies is that formal conclusions are restricted to the finite population of subjects that actually received treatment.

INFERENCE, DESIGN-BASED VS. MODEL-BASED Design-based inference and model-based inference are alternative conceptual frameworks for addressing statistical questions from many types of investigations. These include: 1. Experimental studies of randomly allocated subjects 2. Historical (observational) and followup∗ studies of all subjects in a fortuitous, judgmental, or natural population 3. Sample surveys∗ of randomly selected subjects For these situations and others, there is interest in the extent of generality to which conclusions are expressed and the rationale by which they are justified. Some of the underlying inference issues can be clarified by directing attention at the sampling processes for data collection and the assumptions necessary for the data plausibly to represent a defined target population. A statistical analysis whose only assumptions are random selection of observational units or random allocation of units to experimental conditions may be said to generate design-based inferences; i.e., design-based inferences are equivalent to randomization∗ based inferences as discussed by Kempthorne [20–22], Kish [24, Chap. 14], Lehmann [27, pp. 55–57], and others. Also, such inferences are often said to have internal validity∗ (see Campbell and Stanley [3]) when the design is adequate to eliminate alternative explanations for the observed effects other than the one of interest. In this sense, internal validity requires only that the sampled population∗ and the target population∗ be the same. Alternatively, if assumptions external to the study design are required to extend inferences to the target population, then statistical analyses based on postulated probability distributional forms (e.g., binomial, normal, Poisson, Weibull, etc.) or other stochastic processes yield model-based inferences. These can be viewed as encompassing Bayesian

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

INFERENCE, DESIGN-BASED VS. MODEL-BASED

For agricultural crop studies and laboratory animal studies undertaken at local facilities, such issues merit recognition in a strict sense. However, for medical clinical trials∗ undertaken by multiple investigators at geographically diverse locations, it often may be plausible to view the randomized patients as conceptually representative of those with similar characteristics in some large target population of potential patients. In this regard, if sites and subjects had been selected at random from larger eligible sets, then models with random effects provide one possible way of addressing both internal and external validity considerations. However, such an approach may be questionable if investigators and/or patients were not chosen by a probability sampling mechanism. In this more common situation, one important consideration for confirming external validity is that sample coverage include all relevant subpopulations; another is that treatment differences be homogeneous across subpopulations. More formally, probability statements are usually obtained via assumptions that the data are equivalent to a stratified simple random sample from the partition of this population into homogeneous groups according to an appropriate set of explanatory variables. This stratification is necessary because the patients included in a study may overrepresent certain types and underrepresent others, even though those of each of the respective types might be representative of the corresponding target subpopulations. For categorical (or discrete) response measures, the conceptual sampling process described here implies the product multinomial distribution. As a result, model-based inferences concerning treatment comparisons and their interactions with the explanatory variable stratification can be obtained by using maximum likelihood or related methods as discussed in CHI-SQUARE TEST —I and LOGLINEAR MODELS IN CONTINGENCY TABLES. Illustrative examples are given in LOGISTIC REGRESSION. In a similar spirit, least-squares methods can be used for model-based inferences when continuous response variables have approximately normal distributions with common variance within the respective strata; and analogous procedures are applicable to other distributional structures (e.g.,

see Cox [9], McCullagh [28], and Nelder and Wedderburn [30]. The principal advantages of model-based inferences for such situations are their more general scope and the comprehensive information they provide concerning relationships of response measures to treatment and stratification variables. Contrarily, their principal limitation is that subjects in a study may not represent any meaningful population beyond themselves. See Fisher [13], Kempthorne [22], Neyman et al. [31], and Simon [37] for further discussion. For historical (observational) studies, model-based inferences are usually emphasized because the target population is more extensive than the fortuitous, judgmental, or naturally defined group of subjects included. Also, their designs do not involve either random allocation or random selection, as illustrated by the following examples: 1. A study of driver injury relative to vehicle size, vehicle age, and vehicle model year for all police-reported automobile accidents in North Carolina during 1966 or 1968–1972 (see Koch et al. [26]) 2. A nonrandomized prospective study to compare the experience of patients receiving a new treatment with that of a historical control population (see Koch et al. [26]) 3. A nonrandomized study to compare nine treatments for mastitis in dairy cows relative to their pretreatment status (see CHI-SQUARED TESTS —II). 4. Market research studies involving quota sampling∗ as opposed to random selection (see Kalton [19]) The assumptions by which the subjects are considered representative of the target population and the methods used for analysis are similar to those previously described for experimental studies. Otherwise, designbased inferences are feasible for historical studies through tests of randomization as a hypothesis in its own right, but their use should be undertaken cautiously; specific illustrations are given in CHI-SQUARE TEST —I, and Koch et al. [26]. More extensive discussion of various aspects of inference for observational studies appears in Anderson

INFERENCE, DESIGN-BASED VS. MODEL-BASED

et al. [1], Breslow and Day [2], Cochran [8], Fairley and Mosteller [11], and Kleinbaum et al. [25]. Design-based inferences are often emphasized for sample surveys because the target population is usually the same as that from which subjects have been randomly selected. They are obtained by the analysis of estimates for population averages or ratios and their estimated covariance matrix which are constructed by means of finite population sampling methodology. An illustrative example is given in CHI-SQUARE TEST —I. For sample surveys, the probabilistic interpretation of design-based inferences such as confidence intervals is in reference to repeated selection from the finite population via the given design. In constrast, model-based inferences are obtained from a framework for which the target population is a superpopulation with assumptions characterizing the actual finite population as one realization; and so their probabilistic interpretation is in reference to repetitions of the nature of this postulated sampling process. The latter approach can be useful for situations where the subjects in a sample survey are not necessarily from the target population of interest. For example, Clarke et al. [6] discuss the evaluation of several pretrial release programs for a stratified random sample of 861 defendants in a population of 2,578 corresponding to January-March 1973 in Charlotte, North Carolina. Since the entire population here is a historical sample, any sample of it is also necessarily a historical sample. Thus issues of model-based inference as described for historical studies would be applicable. Another type of example involves prediction to a date later than that at which the survey was undertaken; e.g., Cassel et al. [5] studied prediction of the future use of a bridge to be constructed in terms of number of vehicles. Otherwise, it can be noted that statistical methods for designbased inferences often are motivated by a linear model; e.g., a rationale for ratio estimates involves regression through the origin. A more general formulation for which a linear model underlies the estimator and its esti¨ mated variance is given in Sarndal [35,36]. Additional discussion concerning aspects of design-based or model-based approaches to

3

sample survey data or their combination is given in Cassel et al. [4], Cochran [7], Fuller [14], Hansen et al. [16], Hartley and Sielken [17], Royall [32], Royall and Cum¨ berland [33], Sarndal [34], Smith [38], and LABELS. The distinction between design-based inference and model-based inference may not be as clear cut as the previous discussion might have suggested. For example, some type of assumption is usually necessary in order to deal with missing data; and stratification undertaken purely for convenient study management purposes (rather than statistical efficiency) is sometimes ignored. Also, a model-based approach may be advantageous for estimation for subgroups with small sample sizes (i.e., small domain estimation; see Kalton [19]). For these and other related situations, the issue of concern is the robustness∗ of inferences to assumptions. In summary, design-based inferences involve substantially weaker assumptions than do model-based inferences. For this reason, they can provide an appropriate framework for policy-oriented purposes in an adversarial setting (e.g., legal evidence). A limitation of design-based inferences is that their scope might not be general enough to encompass questions of public or scientific interest for reasons of economy or feasibility. Of course, this should be recognized as inherent to the design itself (or the quality of its implementation) rather than the rationale for inference. In such cases, model-based inferences may provide relevant information given that the necessary assumptions can be justified. It follows that design-based inference and model-based inference need not be seen as competing conceptual frameworks; either they can be interpreted as directed at different target populations and thereby at different statistical questions (e.g., experimental studies), or their synthesis is important to dealing effectively with the target population of interest (e.g., sample surveys).

4

INFERENCE, DESIGN-BASED VS. MODEL-BASED

Acknowledgments The authors would like to thank Wayne Fuller, Peter Imrey, Graham Kalton, Oscar Kempthorne, ¨ Jim Lepkowski, Carl Sarndal, and Richard Simon for helpful comments relative to the preparation of this entry. It should be noted that they may not share the views expressed here. This research was partially supported by the U.S. Bureau of the Census through Joint Statistical Agreement JSA80-19; but this does not imply any endorsement.

REFERENCES 1. Anderson, S., Auquier, A., Hauck, W. W., Oakes, D., Vandaele, W., and Weisberg, H. I. (1980). Statistical Methods for Comparative Studies. Wiley, New York. 2. Breslow, N. E. and Day, N. E. (1980). Statistical Methods in Cancer Research, 1: The Analysis of Case Control Studies. International Agency for Research on Cancer, Lyon. 3. Campbell, D. T. and Stanley, J. C. (1963). Handbook on Research on Teaching, Rand McNally, Chicago, pp. 171–246. (Experimental and quasi-experimental designs for research on teaching.) ¨ 4. Cassel, C. M., Sarndal, C. E., and Wretman, J. H. (1977). Foundations of Inference in Survey Sampling. Wiley, New York. ¨ 5. Cassel, C. M., Sarndal, C. E., and Wretman, J. H. (1979). Scand. J. Statist., 6, 97–106. (Prediction theory for finite populations when model-based and design-based principles are combined.) 6. Clarke, S. H., Freeman, J. L., and Koch, G. G. (1976). J. Legal Stud., 5(2), 341–385. (Bail risk: a multivariate analysis.) 7. Cochran, W. G. (1946). Ann. Math. Statist., 17, 164–177. (Relative accuracy of systematic and stratified random samples for a certain class of populations.) 8. Cochran, W. G. (1972). Statistical Papers in Honor of George W. Snedecor, T. A. Bancroft, ed. Iowa State University Press, Ames, Iowa, pp. 77–90. 9. Cox, D. R. (1972). J. R. Statist. Soc. B, 34, 187–220. [Regression models and life tables (with discussion).] 10. Deming, W. E. (1950). Some Theory of Sampling. Wiley, New York. 11. Fairley, W. B. and Mosteller, F. (1977). Statistics and Public Policy. Addison-Wesley, Reading, Mass.

12. Fisher, R. A. (1925). Proc. Camb. Philos. Soc., 22, 700–725. (Theory of statistical estimation.) 13. Fisher, R. A. (1935). The Design of Experiments. Oliver & Boyd, Edinburgh (rev. ed., 1960). 14. Fuller, W. A. (1975). Sankhya C, 37, 117–132. (Regression analysis for sample survey.) 15. Godambe, V. P. and Sprott, D. A. (1971). Foundations of Statistical Inference. Holt, Rinehart and Winston, Toronto. 16. Hansen, M. H., Madow, W. G., and Tepping, B. J. (1978). Proc. Survey Res. Meth. Sec., Amer. Statist. Ass., pp. 82–107. [On inference and estimation from sample surveys (with discussion).] 17. Hartley, H. O. and Sielken, R. L. (1975). Biometrics, 31, 411–422. (A ‘‘super-population viewpoint’’ for finite population sampling.) 18. Johnson, N. L. and Smith, H., eds. (1969). New Developments in Survey Sampling. Wiley, New York. 19. Kalton, G. (1983). Bull. Int. Statist. Inst. (Models in the practice of survey sampling.) 20. Kempthorne, O. (1952). The Design and Analysis of Experiments. Wiley, New York. 21. Kempthorne, O. (1955). J. Amer. Statist. Ass. 50, 946–967. (The randomization theory of experimental inference.) 22. Kempthorne, O. (1979). Sankhya B, 40, 115–145. (Sampling inference, experimental inference, and observation inference.) 23. Kempthorne, O. and Folks, L. (1971). Probability, sity Press, Ames, Iowa. 24. Kish, L. (1965). Survey Sampling. Wiley, New York. 25. Kleinbaum, D. G., Kupper, L. L., and Morgenstern, H. (1982). Epidemiologic Research: Principles and Quantitative Methods. Lifetime Learning Publication, Belmont, Calif. 26. Koch, G. G., Gillings, D. B., and Stokes, M. E. (1980). Annu. Rev. Public Health, 1, 163–225. (Biostatistical implications of design, sampling, and measurement to health science data analysis.) 27. Lehmann, E. L. (1975). Nonparametrics: Statistical Methods Based on Ranks. Holden-Day, San Francisco. 28. McCullagh, P. (1980). J. R. Statist. Soc. B, 42, 109–142. (Regression models for ordinal data.) 29. Namboodiri, N. K. (1978). Survey Sampling and Measurement. Academic Press, New York. 30. Nelder, J. A. and Wedderburn, R. W. M. (1972). J. R. Statist. Soc. A, 135, 370–384.

INFERENCE, DESIGN-BASED VS. MODEL-BASED (Generalized linear models.) 31. Neyman, J., Iwaskiewicz, K., and Kolodziejczyk, S. (1935). J. R. Statist. Soc. (Suppl. 1), 2, 107–154. (Statistical problems in agricultural experimentation.) 32. Royall, R. M. (1976). Amer. J. Epidemiol., 104, 463–473. (Current advances in sampling theory: implications for human observational studies.) 33. Royall, R. M. and Cumberland, W. G. (1981). J. Amer. Statist. Ass., 76, 66–77. (An empirical study of the ratio estimator and estimators of its variance.) ¨ 34. Sarndal, C. E. (1978). Scand. J. Statist., 5, 27–52. (Design-based and model-based inference in survey sampling.) ¨ 35. Sarndal, C. E. (1980). Biometrika, 67, 639–650. (On π -inverse weighting vs. best linear unbiased weighting in probability sampling.) ¨ 36. Sarndal, C. E. (1982). J. Statist. Plann. Infer., 7, 155–170. 37. Simon, R. (1979). Biometrics, 35, 503–512. (Restricted randomization designs in clinical trials.) 38. Smith, T. M. F. (1976). J. R. Statist. Soc. A, 139, 183–204. [The foundations of survey sampling: a review (with discussion).]

GARY G. KOCH DENNIS B. GILLINGS

5

INFORMED CONSENT PROCESS, FORMS, AND ASSENT

1

THE PURPOSE FOR INFORMED CONSENT

The term informed consent would seem to be redundant. One may well ask how one can give consent to an undertaking about which they are not informed. The term is a response to the reality that before the introduction of the doctrine of informed consent patients, often gave their consent to surgeries or other medical interventions without being provided with information about the benefits, risks, and alternatives of an intervention a physician proposed to perform. In one of the earliest decisions that mandated physicians to obtain the informed consent of patients before performing surgery, the court explained that the purpose of the doctrine is ‘‘to enable the patient’s chart this course knowledgeably . . . ’’ (1) and, it found that to do so, the patient needed to have reasonable familiarity with the therapeutic alternatives to the proposed procedure and the inherent risks of the procedure and the alternatives. The court also explained that because the physician is an expert he has knowledge of the risks inherent in the proposed procedure and the probability of success. But that the weighing of the risks is not an expert skill, but a ‘‘nonmedical judgment reserved to the patient alone.’’ Similarly, the decision whether to become a research subject lies entirely in the hands of the potential subject, and without adequate information, which the potential subject can obtain only from the researcher, the subject’s consent would be uninformed. It is this reality that forms the foundation for both the ethical and legal obligation of the researcher to obtain a potential subject’s informed consent.

LEONARD H. GLANTZ Boston University School of Public Health, Boston, Massachusetts

Obtaining the informed consent of human subjects is an integral part of the overall obligations and duties inherent in the researcher role. Researchers are obliged to protect both the rights and the welfare of research subjects. The welfare of research subjects is protected by using the safest and least invasive methods possible to obtain the necessary information from subjects. For example, if the research involves obtaining blood from a subject, then the researchers should use the fewest needle sticks possible. Protecting the rights of research subjects requires empowering potential subjects to decide freely and intelligently whether they wish to become research subjects and to decide whether they wish to continue to be subjects once they have been enrolled. In this regard, the doctrine of informed consent was created to protect and enhance the potential subjects’ autonomy to decide whether to participate in a researcher’s project. Obtaining the truly informed consent of the potential subjects may well conflict with a researcher’s goals. Obviously, the researcher hopes to recruit a sufficient number of subjects in a timely manner, but fully informing potential subjects of the risks and discomforts, particularly if no direct benefit occurs for the subjects, may result in a potential subject’s refusal to participate. Indeed, if a researcher obtains a 100% participation rate, it may be the result of subjects not fully understanding the risks and discomforts that are an inherent part of the research. Because of this conflict, the potential subject’s informed consent is required to be obtained by law and regulation in addition to professional ethics. However, in obtaining the subject of informed consent, it is useful for the ethical researcher to be aware of this potential conflict.

2 THE NUREMBERG CODE AND INFORMED CONSENT In the research context, the clearest statement of the obligation to obtain consent is found in the Nuremberg Code. The Nuremberg Code was created as part of the judgment in the post-World War II criminal trials of Nazi physicians who conducted atrocious

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INFORMED CONSENT PROCESS, FORMS, AND ASSENT

experiments on concentration camp inmates. The first provision of the code states (2): The voluntary consent of the human subject is absolutely essential. This means that the person involved should have legal capacity to give consent; should be situated as to be able to exercise free power of choice, without the intervention of any element of force, fraud, deceit, duress, over-reaching, or other ulterior form of constraint or coercion, and should have sufficient knowledge and comprehension of the elements of the subject matter involved as to enable him to make an understanding and enlightened decision. This latter element requires that before the acceptance of an affirmative decision by the experimental subject there should be made known to him the nature, duration, and purpose of the experiment; the method and means by which it is to be conducted; all inconveniences and hazards reasonably to be expected; and the effects upon his health or person which may possibly come from his participation in the experiment.

This section of the code sets forth the goals and purposes of informed consent in the research context. It requires that a subject’s consent be both informed and voluntary, requires that both inconveniences and hazards that reasonably are to be expected be disclosed, and clearly states that the goal of the process is to enable the potential subjects to make ‘‘an understanding and enlightened decision.’’ The Nuremberg Code says nothing about consent forms and does not require them. Rather, it requires that the informed consent process be designed to enlighten potential subjects so they are in a position to decide voluntarily to enroll as a subject or to reject the investigator’s offer. It is the process of informed consent, not a consent form, that is essential to a well-informed decision. Indeed, the fact that the informed consent process and obtaining written informed consent are two separate matters is stated clearly in the federal research regulations. To approve research, the regulations require that the institutional review board (IRB) determine that ‘‘informed consent will be sought from each perspective subject or the subjects legally authorized representative’’ and separately requires the IRB to determine

that ‘‘informed consent will be appropriately documented . . . ’’ (3). 3 FEDERAL REGULATIONS AND INFORMED CONSENT Similar to the Nuremberg Code, the federal regulations require that that consent be obtained in circumstances that provide the potential subject ‘‘sufficient opportunity to consider whether or not to participate and that minimize the possibility of coercion or undue influence. Furthermore, the regulations require that the language used by the investigator when attempting to obtain a potential subject’s informed consent must be ‘understandable to the subject.’’’ The regulations, similar to the Nuremberg Code, require that the ‘‘basic elements of informed consent’’ include the following: 1. a statement that the study involves research, an explanation of the purposes of the research and the expected duration of the subject’s participation, a description of the procedures to be followed, and identification of any procedures that are experimental, 2. a description of any reasonably foreseeable risks or discomforts to the subject, 3. a description of any benefits to the subject or to others which reasonably may be expected from the research, 4. a disclosure of appropriate alternative procedures or courses of treatment, if any, that might be advantageous to the subject, 5. a statement describing the extent, if any, to which confidentiality of records identifying the subject will be maintained, 6. for research involving more than minimal risk, an explanation as to whether any compensation and an explanation as to whether any medical treatments are available if injury occurs, and, if so, what they consist of or where more information may be obtained, and ... 8. a statement that participation is voluntary, refusal to participate will involve

INFORMED CONSENT PROCESS, FORMS, AND ASSENT

no penalty or loss of benefits to which the subject is otherwise entitled, and the subject may discontinue participation at any time without penalty or loss of benefits to which the subject is otherwise entitled. Additional elements of informed consent are required in certain circumstances when appropriate. The federal rules also authorize the IRB to waive some or all elements of informed consent in limited circumstances when the investigator makes such a request. Such a waiver is authorized when ‘‘(1) the research involves no more than minimal risk to the subjects; (2) the waiver or alteration will not adversely affect the rights and welfare of the subjects; (3) the research could not practicably be carried out without the waiver or alteration; and (C) whenever appropriate, the subjects will be provided with additional pertinent information after participation.’’ 4 DOCUMENTATION OF INFORMED CONSENT A separate section of the regulations requires that informed consent be documented, with certain limited exceptions, and requires that the consent form, which must be signed by the subject or the subject’s legally authorized representative, contain the elements of the informed consent process. It is often not recognized or understood that obtaining a signature of a potential subject on an informed consent form does not mean that the person actually has been adequately informed and actually has given a knowledgeable and voluntary consent. The form is meant solely to memorialize the consent process, not substitute for that process. Unfortunately, over time, both investigators and institutional review boards have focused overly on the forms and have not emphasized the importance of the process itself. Several reasons for this are found. The first reason is that the only ‘‘contact’’ IRBs have with subjects is through the consent forms, and therefore they attempt to fulfill their obligation to protect the rights of subjects by focusing on the form. The second reason is

3

that a signed consent form is what investigators use to demonstrate that they have obtained the informed consent of subjects. Without somebody sitting in the room with the investigator and a potential subject, no other way is available for the investigator to demonstrate that she has met her obligation to obtain informed consent. Finally, when the federal regulatory authorities, the Office for Human Research Protection and the Food and Drug Administration, audit institutions and their IRBs for compliance with federal regulations, they focus almost entirely on the paperwork that is involved in approving research, including informed consent forms. Because of the concern that federal auditors might determine after the fact that a problem did occur with the informed consent form, the forms have become longer and more complicated so that they seem to be more complete. Furthermore, institutions have come to view these forms as ways of fending off potential litigation by subjects, even in the absence of any evidence that this fear is realistic. One major difficulty in creating a useful form is presenting complicated information in easyto-understand language. Indeed, other than consent forms, investigators never have to write in language a layperson can understand, and therefore they have little or no experience in doing so. It is not surprising that investigators tend to use the jargon and abbreviations of their professions that are so readily understood by them, their assistants, and their colleagues in the study. The length and complexity of the forms and the use of jargon raise questions as to the capacity of the investigator (and IRBs) to adequately explain the nature of the research to potential subjects in the context of the informed consent process. 5 IMPROVING THE PROCESS OF INFORMED CONSENT One challenge of the informed consent process is that it is neither medical nor scientific in nature. Rather, the informed consent process should be an educational undertaking; it requires careful thinking about what the ‘‘student’’ (subject) needs to know and how the ‘‘teacher’’ (investigator) can impart that

4

INFORMED CONSENT PROCESS, FORMS, AND ASSENT

knowledge in the best way. Teachers use just not reading material but also visual aids and assignments. For example, for certain types of research, the informed consent process can involve watching a video or even assigning potential subjects materials that they will discuss with the investigator when they come back at a later time. It has also been recommended that investigators test knowledge of potential subjects through the use of oral or written quizzes after the investigator informs the potential subject of the risks, benefits, and alternatives. If it is determined that the potential research subject is not aware of certain material matters, then the investigator can go back and explain those matters more until the potential subject is properly informed. If the informed consent form is going to be used as part of the teaching materials that will be provided to the potential subject, then it is essential that it be understandable to a layperson. Investigators will need to use common terms that may seem to them to be less precise but in fact will be more informative from the perspective of a layperson. For example, ‘‘catheters’’ are actually ‘‘thin tubes,’’ ‘‘extremities’’ are actually ‘‘arms and legs,’’ and ‘‘chemotherapy’’ is actually ‘‘medicines to try to treat your cancer.’’ In some studies, subjects may be given several drugs, and one often sees an exhaustive list of potential side effects for each drug. But, if the subject is receiving three drugs and all of them may cause nausea, then no need exists to say this three separate times—which drug causes the nausea in a particular subject is not of interest to the subject—it is the potential for nausea that matters. In sum, better consent processes will come about only if investigators seriously consider how to educate potential subjects fully about the risks, discomforts, potential benefits, and alternatives that might be derived from becoming a research subject. Instead of thinking about this as a technical matter, investigators might find it useful to think about how they would go about informing a close family member who is considering entering a research project. If investigators could accomplish this, then they would be in compliance with legal and ethical requirements.

6 ASSENT Assent is a term that is used in a situation where one cannot obtain true informed consent from the potential research subject because of their immaturity or incompetence. In such circumstances, the researcher needs to obtain the acquiescence of the subject to participate in research. The goal is to try to ensure that research is not conducted forcibly on objecting subjects. However, assent is more than the lack of objection. The federal regulations define assent as the ‘‘affirmative agreement to participate in research. Mere failure to object should not, absent affirmative agreement, be construed as assent.’’ In the context of the federal regulations, the term ‘‘assent’’ is used in the context of children who cannot give legally binding consent to become research subjects. Rather, the regulations state that if parents give ‘‘permission’’ and children ‘‘assent’’ to become a research subject, then the research may be conducted. Although no specific regulations have been developed on this topic, this process should also apply to subjects who are incompetent to provide consent. The incompetent person’s legally authorized representative would provide permission, and the incompetent person would give assent. When participation in research is likely to provide a direct benefit to an incompetent person, their lack of assent is not considered a barrier to their enrollment in the research. REFERENCES 1. Cobbs v. Grant, 8Cal. 3d 229 (California Supreme Court, 1972). 2. Trials of War Criminals before the Nuremberg Military Tribunals under Control Council Law No. 10, vol. 2. Nuremberg October 1946–April 1949. Washington, D.C.: U.S. Government Printing Office (n.d.), pp. 181–182. 3. The Department of Health and Human Services, Protection of Human subjects, Title 45 Code of Federal Regulations, part 46 (1991).

FURTHER READING G. J. Annas and M. A. Grodin (eds.), The Nazi Doctors and the Nuremberg Code. New York: Oxford University Press, 1992.

INFORMED CONSENT PROCESS, FORMS, AND ASSENT J. W. Berg, P. S. Appelbaum, L. S. Parker, and C. W. Lidz, Informed Consent: Legal Theory and Clinical Practice. New York: Oxford University Press, 2001. Office for Human Research Protections, Policy Guidances. Available: http://www.hhs.gov /ohrp/policy/index.html. The informed consent process. In D. D. Federman, K. E. Hanna, and L. L. Rodriguez (eds.), Responsible Research: A Systems Approach to Protecting Research Participants. Washington, D.C.: National Academy Press, 2002, pp. 119–127.

5

INSTITUTION

positions of leadership in multicenter settings. sponsor: 1. A person or agency that is responsible for funding a designated function or activity; sponsoring agency. 2. A person or agency that plans and carries out a specified project or activity. 3. The person or agency named in an Investigational New Drug Application or New Drug Application; usually a drug company or person at such a company, but not always (as with an INDA submitted by a representative of a research group proposing to carry out a phase III or phase IV drug trial not sponsored by a drug company). 4. A firm or business establishment marketing a product or service (5). study center: [trials] 1. data collection site; study clinic 2. Data collection or data generation site. 3. The center from which activities are directed; coordinating center; project office. 4. An operational unit in the structure of a study, especially a multicenter structure, separate and distinct from other such units in the structure, responsible for performing specified functions in one or more stages of the study (e.g., a clinical center or resource center) (5).

CURTIS L. MEINERT The Johns Hopkins University Bloomberg School of Public Health Center for Clinical Trials Baltimore, Maryland

institution: An established organization or corporation (as a college or university) especially of a public character (1). institution: Any public or private entity or agency (including federal, state, and other agencies) (2). institution: Any public or private entity or agency or medical or dental facility where clinical trials are conducted (3). institution: A place where a study is undertaken; usually a hospital or similar establishment (4). institution: An established organization, corporation, or agency, especially one that has a public character (5). institute: An organization for promotion of a cause; an educational institution and especially one devoted to technical fields (1). agency: an administrative unit through which power or influence is exerted to achieve some stated end or to perform some specified function (5). principal investigator (PI): [research] 1. The person having responsibility for conduct of the research proposed in a grant application submitted to the National Institutes of Health; such a person in any funding application submitted to the NIH, whether for grant or contract funding; such a person named on any funding proposal, regardless of funding source. 2. The person in charge of a research project; the lead scientist on a research project. 3. The head of a center in a multicenter study. 4. The chair of a multicenter study. 5. The head of a clinical center in a multicenter trial Usage note: Avoid in the sense of defn 2 in settings having multiple ‘‘principal investigators’’; use center director and chair of study to designate

In the parlance of clinical trials, most commonly used in reference to the corporate entity housing one or more study centers involved in carrying out a trial (e.g., a university housing such a study center), it is typically the fiscal officer of the institution who has legal authority for receiving and use of funds awarded to the institution for a study, consistent with the needs and dictates of the scientist responsible for the study center within the institution— usually referred to as principal investigator (see usage note for principal investigator). The most common usage in adjective form is in institutional review board (IRB). Most academic institutions engaged in research on human beings have one or more standing IRBs comprised of people from within and outside the institution. Investigators in the institution are obliged to submit all proposal

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

INSTITUTION

involving human beings to their IRB of record and may not initiate such research until or unless approved by that IRB and must thereafter maintain approval by submission of renewal requests not less frequently than annually, as dictated by the IRB. Typically, the term refers to boards within an investigators institution, but it is also used to refer to boards located outside the institution, including commercial IRBs. Virtually all multicenter studies are multiinstitutional in that they involve two institutions, such as recruiting and treating sites and a data center or coordinating center. REFERENCES 1. Merriam-Webster’s Collegiate Dictionary, 10th ed. Springfield, MA: Merriam-Webster, Inc., 2001. 2. Office for Human Research Protection, Code of Federal Regulations, Title 45: Public Welfare, Part 46: Protection of Human Subjects. Bethesda, MD: Department of Health and Human Services, National Institutes of Health, (revised) June 18, 1991. 3. International Conference on Harmonisation, E6 Good Clinical Practice. Washington, DC: U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), and the Center for Biologics Evaluation and Research (CBER), April 1996. 4. S. Day, Dictionary for Clinical Trials. Chichester: Wiley, 1999. 5. C. L. Meinert, Clinical Trials Dictionary: Terminology and Usage Recommendations. Baltimore, MD: The Johns Hopkins Center for Clinical Trials, 1996.

INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS

1.1 Federalwide Assurance (FWA) Although federal regulations guide IRB structure and function, these regulations are intended as a minimum standard rather than a maximum. Any IRB can have policies that exceed the minimum standard set by regulation. In fact, many IRBs have policies that apply to all human subjects research at its institution, whether the research is federally funded or FDA regulated. An FWA is an agreement between an institution and U.S. Department of Health and Human Services’ Office of Human Research Protection that specifies how the institution complies with applicable law and regulations. In this FWA, an institution may state that it applies federal regulations only to federally funded research, or it may state that it applies the regulations and its policies to all research on human subjects at its institution.

SUSAN S. FISH Boston University Schools of Public Health and Medicine Boston, Massachusetts

1

PURPOSE OF IRBs

The purpose of an IRB is to protect the rights and welfare of human research subjects (45 CFR 46, 21 CFR 56). An IRB provides external and independent review of research that involves human subjects, with a focus on the ethical and regulatory issues. The IRB reviews protocols prior to their implementation and during the course of the research. In the United States, independent review of research was first required in the mid1960s for research that was supported by the U.S. Public Health Service. However, the regulations that now guide clinical research review were initially promulgated in 1981 and have been amended periodically since then. Many institutions in the United States required independent review of research prior to requirements from various federal agencies, and some IRBs have been in existence since 1966. Currently IRB review is required for all studies that are either conducted or funded by various federal agencies as well as for all studies performed to support a marketing application to the Food and Drug Administration (FDA) for a drug, device or biologic agent. The regulations initially promulgated by the Department of Health and Human Services (45 CFR 46) were eventually adopted by 17 federal agencies, and have thus become known as the ‘‘Common Rule.’’ Both the Common Rule and the FDA regulations (21 CFR 50 and 56) regulate IRBs for the research for which their agencies are responsible. Although these two sets of regulations are different and occasionally in conflict with each other, the structure and function of IRBs described in both sets of regulations are, for the most part, identical.

2 STRUCTURE OF AN IRB 2.1 Membership of an IRB The structure of IRBs is guided by federal regulations, which describe the minimum requirements of membership. IRBs, by regulation, consist of at least five members. However, many IRBs have many more than five members; boards of 15–25 members are not uncommon. These members must have varying backgrounds so that they have the expertise to review the types of research submitted to that IRB. The membership must be diverse in terms of race, gender, cultural background, and profession. In addition, the membership must be familiar with and sensitive to community attitudes, communities in which the research is taking place, and from which research subjects are recruited, as well as the research community that is submitting the research for review. These sensitivities are intended to ‘‘promote respect for (the IRB’s) advice and counsel in safeguarding the rights and welfare of human subjects.’’ The membership should also include members with the knowledge and understanding of institutional commitments, applicable state and federal laws and regulations, and professional standards in the disciplines of research

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS

that is overseen by that IRB. The regulations go on to explain that if an IRB regularly reviews research that involves a certain category of subjects who might be vulnerable to coercion or undue influence concerning research participation, then the IRB membership should include at least one member who has knowledge about and experience in working with this group of people. An example of such a vulnerable group is children and adolescents. If an IRB regularly reviews biomedical studies in the pediatric population, then a pediatrician or pediatric nurse might be an IRB member who can meet this requirement and have the expertise with this population of subjects. If an IRB reviews educational studies that take place in the school system, then a teacher might meet this requirement. On any IRB, there must be at least one member whose ‘‘primary concerns’’ are in nonscientific areas and at least one member who focuses on scientific areas. In reality, most IRBs have many scientific members and a few nonscientific members. The nonscientific member(s) can provide a balancing perspective for the enthusiasm of the scientist members and can provide the perspective of the research subjects. Most IRB members are drawn from the institution whose research the IRB oversees. In its wisdom, regulations require that at least one member be unaffiliated with the institution, as a check and balance on potential institutional conflict of interest, who can bring to the table some awareness of broader community standards. When an IRB is faced with reviewing a study that requires expertise that is not possessed by its membership, the IRB may and should obtain consultation with one or more individuals who can supply that expertise. It may be that the consultation is in a scientific area (e.g., xenotransplantation), a methodologic area (e.g., qualitative research methodology), or a cultural setting in which the research may take place (e.g., orphans in rural Zambia). Although consultants may not vote as members of the IRB, their input can be invaluable to the understanding of the research by the IRB members who do vote. Consultation should be sought liberally by the IRB.

Membership on an IRB may be a volunteer activity or a paid activity. The duration of membership varies among IRBs. The IRB Chair facilitates meetings, conducts business, and has the authority and responsibility to act for the IRB in certain situations. What is consistent across IRBs is that IRB members are intelligent and dedicated people who take their role in protecting human research subjects quite seriously.

2.2 Conflict of Interest in the IRB The diversity of the IRB membership guards against the influence of potential conflicts of interest. Although IRB decisions are made by a majority vote of members present, most decisions are the result of near-consensus. It is unlikely that one person with an undisclosed conflict could single-handedly affect the outcome of a decision. However, an IRB must have a policy that defines a conflict of interest, and federal regulations prohibit an IRB member with a conflicting interest from participating in the review of a protocol. One conflict of interest that is almost universally recognized is that of being a researcher on a protocol under review. When an IRB member’s own protocol is being reviewed by the board, that member/researcher may not participate in the review except to provide information. In this case, the member/researcher must recuse him/herself from the discussion and vote and leave the meeting room during that time.

2.3 Authority of the IRB The IRB’s authority is assigned by the institution. Once that authority is assigned, it cannot be overridden by any part of the institution. IRB approval is necessary, but not sufficient, for conduct of a research study using human subjects. In other words, if an IRB approves a study, another body in the institution may prevent the project from being conducted. However, if an IRB does not approve a study, then it cannot be conducted. No body within an institution can overturn disapproval by an IRB.

INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS

3 ACCREDITATION OF HUMAN RESEARCH PROTECTION PROGRAMS An IRB is part of a larger human research protection program (HRPP), which also includes the institution, the researchers, the sponsors, and others involved in the clinical research enterprise. Voluntary accreditation of HRPPs has developed in the last decade as a means of identifying those programs that not only meet the minimal federal regulations, but also strive toward best practices in implementing human research ethics. Some IRBs are accredited whereas many others are not. 4

FUNCTION

IRB function is described in its policies and procedures, which are based on ethical principles and regulatory requirements. Each IRB will develop its own policies and procedures, and thus functioning differs among IRBs. However, some functions are common across IRBs. Research protocols that are not reviewed by expedited procedures (45 CFR 46.110) must be reviewed at a regular meeting of the members of the IRB. For official action to be taken at an IRB meeting, a quorum of members must be present, which includes at least one nonscientific member. According to regulations, the written procedures must contain, at a minimum, policies for the following: • Conducting initial review of a protocol • Conducting continuing review of a pro• • • • •

tocol Reporting IRB findings and actions to the investigator Reporting IRB findings and actions to the institution Determining which projects must be reviewed more often than annually Determining which projects should be audited Ensuring that proposed changes to the research be submitted and approved prior to their implementation, except as needed to protect a subject from immediate harm

3

• Ensuring prompt reporting to IRB, to

the Institution, and to funding agencies of unanticipated problems involving risks to subjects or others • Ensuring prompt reporting to IRB, to the Institution, and to funding agencies of serious or continuing noncompliance with federal regulations or policies of the IRB • Ensuring prompt reporting to the Institution, and to funding agencies of suspensions of terminations of IRB approval of a protocol The approval of a protocol by an IRB is required before a researcher may begin to conduct a study. The approval period for protocols may vary depending on the risks of the research and on other factors, but the maximum duration of approval is one year. Prior to the expiration of approval, the IRB must conduct continuing review of the ongoing protocol. The study cannot be continued beyond its approval expiration date unless the IRB has reapproved the study at its continuing review and issued a new approval expiration date. 4.1 Criteria for IRB Approval For an IRB to approve a research study, it must assure that certain criteria are met. Unless all criteria are met, the IRB cannot issue an approval. These criteria are derived directly from the three Belmont principles: respect for persons, beneficence, and justice (see ‘‘Belmont Report’’) Both initial review and continuing review require that these same criteria have been satisfied. As with all federal regulations, these are minimal criteria; any IRB may include additional criteria to be met. These minimal criteria are listed in the federal regulations (45 CFR 46.111; 21 CFR 56.111). 1. Risks to subjects are minimized: (1) By using procedures that are consistent with sound research design and that do not unnecessarily expose subjects to risk and (2) whenever appropriate, by using procedures already being performed on the subjects for diagnostic or treatment purposes.

4

INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS

2. Risks to subjects are reasonable in relation to anticipated benefits, if any, to subjects, and the importance of the knowledge that may reasonably be expected to result (see ‘‘Benefit-Risk Assessment’’ and ‘‘Risk-Benefit Analysis’’). 3. Selection of subjects is equitable. 4. Informed consent will be sought from each prospective subject or the subject’s legally authorized representative (see ‘‘Informed Consent,’’ ‘‘informed consent process,’’ ‘‘legally authorized representative,’’ and ‘‘assent’’). 5. Informed consent will be appropriately documented. (see ‘‘Informed Consent Form’’). 6. When appropriate, the research plan makes adequate provision for monitoring the data collected to ensure the safety of subjects (see ‘‘Data Safety Monitoring Board, DSMB’’ and ‘‘Data Monitoring Committee’’). 7. When appropriate, adequate provisions are implemented to protect the privacy of subjects and to maintain the confidentiality of data. 8. When some or all subjects are likely to be vulnerable to coercion or undue influence, additional safeguards have been included in the study to protect the rights and welfare of these subjects (see ‘‘Vulnerable Subjects’’). For an IRB to conduct a thorough review of the research once it has been implemented, the IRB has the authority to audit or observe the research, which includes the informed consent process (see ‘‘Informed Consent Process’’). Various IRBs perform these observations in different ways, but the authority for this activity exists for all IRBs. Researchers must be informed in writing as to the decision of the IRB concerning their protocol. If any changes to the protocol are required by the IRB so that it can approve the protocol, then those required changes must be specified in writing to the researcher. If a protocol is not approved, then the reasons for that decision must be communicated to the researcher in writing.

4.2 IRB Staff Most IRBs have staff support, although it can take a variety of forms. At one extreme is the IRB that has a part-time secretary/administrator who may be responsible for answering phones, typing letters to investigators, and supporting the IRB meeting by assuring that all members have the documents needed to review protocols. At the other extreme is an IRB office staffed with many certified IRB professionals who may review protocols for science, ethics, and regulatory compliance, as well as educate researchers about these same topics. 5 DIFFERENCES BETWEEN INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS Federal regulations proscribe much of the structure and function of IRBs. When these regulations were written in 1981, most research was conducted in a single academic medical center. In the subsequent years, many changes have occurred in the research environment, which include clinical research being conducted in free-standing clinics and private doctors’ offices that are not affiliated with academic medical centers. The need for review of studies conducted in these other sites generated independent review boards that are not affiliated with a particular institution. In addition, the complexity of multiple IRB reviews for a multicenter study, in which each IRB makes different demands of the protocol but for scientific validity a protocol must be conducted identically at each site, has led sponsors to attempt to undergo a single IRB review using an independent IRB whose approval applies to all sites. Other terms for these unaffiliated boards are commercial IRBs, central IRBs, for-profit IRBs, and noninstitutional IRBs. 5.1 Institutional Review Boards Institutional IRBs report to a high-ranking administrative official within the institution, which is usually a dean of a school or president of a hospital. Membership on the IRB consists mostly of members of the institution, with the exception of the nonaffiliated member(s) and possibly the nonscientist member(s). The members are usually volunteers,

INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS

5

and some may be minimally compensated for time or may be relieved from other responsibilities at the institution. The IRB may interact with other committees that review the research proposals, which include the scientific review committee, radiation safety committee, institutional biostafety committee, resource utilization committee, conflict of interest committee, grants administration, and others. Institutional IRBs have oversight for the research conducted at its institution(s) and by its investigators. Board members and IRB staff frequently know the researchers and work with them. Thus, the IRB must confront the issue of individual nonfinancial conflict of interest. Members and staff have relationships with the researchers, either positive or negative, that can affect the objectivity of their review and decision concerning a particular protocol. But the strength of the system is that the entire board can help minimize the impact of any potential relationship conflict; the entire board participates in the decision to approve or disapprove a protocol. Institutional IRBs are confronted with institutional conflict of interest issues. Research that is approved and conducted will benefit the institution of which many members are employees. Although rarely does pressure come from the institution to approve a particular study, the research effort at the institution benefits all members of that hospital or school, both in reputation and financially. Thus, a subconscious pressure may be placed on the board to approve protocols that must be guarded against.

on this committee is not part of their academic citizenship responsibility but rather is an extracurricular activity. Independent IRBs have oversight for a research protocol that may be conducted at many sites nationally or internationally. Board members and IRB staff do not work with the researchers, and frequently they are not familiar with the location or site at which the research will be conducted. Although the relationship conflicts of interest do not exist, the board members and staff are not familiar with the researcher and are not aware of the skills and ethics of a particular researcher beyond what might be available through review of curriculum vitae. The more subtle characteristics of the researcher are unknown to the IRB when it makes a decision to approve or disapprove a protocol. Independent IRBs must deal with a different type of institutional conflict of interest. These IRBs receive direct income from a given protocol review. Although the payment does not hinge on approval of the protocol, if an IRB repeatedly disapproves protocols from a given sponsor, then that sponsor is likely to use another independent IRB, which decreases the overall income of the IRB. Thus, a subconscious pressure may be placed on the board to approve protocols that must be guarded against.

5.2 Independent Review Boards

2. Code of Federal Regulations, Title 21, Chapter 1 [21 CFR 50], Food and Drug Administration, DHHS; Part 50—Protection of Human Subjects. Updated 1 April, 2007.

Independent IRBs traditionally are not affiliated with an academic institution or hospital performing the research, although they may be affiliated with a contract research organization or site management organization that performs research (see ‘‘Contract Research Organization’’). As suggested by their name, independent IRBs are usually independent of the researchers and independent of any other committees that may review the research. Their members are frequently paid, rather than being volunteers, because participation

REFERENCES 1. Code of Federal Regulations, Title 45A [45 CFR 46], Department of Health and Human Services; Part 46—Protection of Human Subjects. Updated 1 October, 1997.

FURTHER READING E. A. Bankert and R. J. Amdur, Institutional Review Board: Management and Function. Sudbury, MA: Jones and Bartlett Publishers, Inc., 2006. IRB: A Review of Human Subjects Research—a periodical dedicated to IRB issues. Published by The Hastings Center, Garrison, NY.

6

INSTITUTIONAL AND INDEPENDENT REVIEW BOARDS

IRB Forum, An internet discussion board focused on IRB issues. Available: http://www. irbforum.org/. USDHHS, Office of Human Research Protection Policy Guidance. Available: http://www. hhs.gov/ohrp/policy/index.html#topics. USFDA Guidances, Information Sheets, and Important Notices on Good Clinical Practice in FDA-Regulated Clinical Trials. Available: http://www.fda.gov/oc/gcp/guidance.html.

INSTITUTIONAL REVIEW BOARDS (IRB)

review of activities commonly conducted by research institutions. In addition to possessing the professional competence needed to review specific activities, an IRB must be able to ascertain the acceptability of applications and proposals in terms of institutional commitments and regulations, applicable law, standards of professional conduct and practice, and community attitudes. Therefore, IRBs must be composed of people whose concerns are in relevant areas. Currently, the FDA does not require IRB registration. The institutions where the study is to be conducted should be contacted to determine whether they have their own IRB. If the study is conducted at a site that does not have its own IRB, the investigators should be queried to see if they are affiliated with an institution with an IRB that would be willing to act as the IRB for that site in the study. Independent IRBs can be contracted to act as the IRB for a site without its own IRB. An IRB can be established in accordance with 21 Code of Federal Regulation (CFR) 56, and IRBs must comply with all applicable requirements of the IRB regulation and the Investigational Device Exemption (IDE) regulation (21 CFR Part 812) in reviewing and approving device investigations involving human testing. The sFDA does periodic inspections of IRB records and procedures to determine compliance with the regulations. The FDA Office of Health Affairs provides guidelines, including an ‘‘IRB Operations and Clinical Requirements’’ list, to help IRBs carry out their responsibilities for protection of research subjects. The topic of IRBs is also addressed in the Federal Register (March 13, 1975) and the Technical Amendments concerning ‘‘Protection of Human Subjects’’ (45 CFR Part 46).

Institutional Review Boards (IRBs) ensure the rights and the welfare of people who participate in clinical trials both before and during the trial. At hospitals and research institutions throughout the country, IRBs make sure that participants are fully informed and have given their written consent before studies ever begin. The U.S. Food and Drug Administration (FDA) monitors IRBs to protect and ensure the safety of participants in medical research. The purpose of IRB review is to ensure, both in advance and by periodic review, that appropriate steps are taken to protect the rights and welfare of human participants in research. To accomplish this purpose, IRBs use a group process to review research protocols and the related materials such as informed consent documents and investigator brochures. If an IRB determines that an investigation involves a significant risk device, it must notify the investigator and, if appropriate, the sponsor. The sponsor may not begin the investigation until approved by the FDA. Under FDA regulations, an IRB is an appropriately constituted group that has been formally designated to review and monitor biomedical research involving human subjects. In accordance with FDA regulations, an IRB has the authority to approve, require modifications in (to secure approval), or disapprove research. This group review serves an important role in the protection of the rights, safety, and welfare of human research subjects. An IRB must be composed of no less than five experts and lay people with varying backgrounds to ensure a complete and adequate This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/irb.htm, http://www.fda.gov/cder/about/smallbiz/humans. htm, and http://www.fda.gov/cdrh/devadvice/ide/ irb.shtml) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

INTEGRATED DATABASE

Concurrent with these methods for data integration was the development of principles of data warehousing. The classic approach to data warehousing involves three steps: data extraction, transformation, and loading. It is within the second step of transformation that data integration is performed. Here data is altered, if necessary, so that data values in the data warehouse are represented in a standard and consistent format. A simple example of this alteration is the transformation of laboratory values into standard international units. Often the term data warehouse is used interchangeably with integrated data. However, in a theoretical sense, they are not the same because data warehousing encompasses the entire process of designing, building, and accessing data in real time. Integrated data, however, may be a component of a data warehouse, but it can also exist as a separate entity and represents a harmonized view of data from multiple sources.

SUSAN J. KENNY Inspire Pharmaceuticals, Inc., Durham, North Carolina,

1 THEORY AND CONSTRUCTION OF INTEGRATED DATABASES Data integration is the problem of combining data that resides at different sources and providing the user with a unified view of these data (1). Integrated databases are widely used in many professions, including health care, epidemiology, postmarketing surveillance, and disease registries. The content of this article is restricted to the concepts, applications, and best practice as applied specifically to clinical trials. 1.1 The Purpose of Integrated Databases The primary purpose for creating an integrated database of clinical trial data is to have a database with a greater sample size so that potential differences among subpopulations of interest can be evaluated. These important differences may exist in the individual studies, but the smaller sample size may not yield enough statistical power to identify them adequately.

1.3 Constructing an Integrated Database The goal of an integrated database is to give the user the ability to interact with one large system of information that in actuality came from multiple sources. Thus, users must be provided with a homogeneous view of the data despite the reality that the data may have been represented originally in several forms. Before the creation of an integrated database, it must be established that the data to be integrated were collected in a similar manner so that the resulting integrated data accurately reflects the true associations between data elements. If studies used substantially different case report forms, had different inclusion/exclusion criteria, used different endpoint measurements, or had vastly different measures of statistical variability, then the use of an integrated database that is built from disparate data may lead to spurious or statistically biased conclusions. The first task in constructing an integrated database is to create a desired schema to describe the structure of the integrated database, the variables it will contain, and

1.2 The History of Integrated Databases The practice of combining heterogeneous data sources into a unified view is not new. With the introduction of computers into most business sectors in the 1960s, the use of electronic databases to store information became a standard practice. This practice led to an expansion of knowledge in the academic areas of database technology and computer science. Database technology was introduced in the late 1960s to support business applications. As the number of these applications and data repositories grew, the need for integrated data became more apparent. The first approach to integration was developed around 1980 (2), and over the past 20 years a rich history of research in the area of data integration and a rapid sophistication of methods has developed.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INTEGRATED DATABASE

how the values of these variables will be represented. A schema is akin to a blueprint that will guide the construction of the desired outcome. As a simple example, a schema to integrate the demographic data from three different clinical trials might specify that there will be one record per subject and that variables for age, age group category, sex, race, and survival status will be included. Associated with the database schema is a metadata dictionary to describe the attributes for each variable. The attributes would include the variable name, variable label, variable type (character or numeric), and the precision for numeric variables. In addition, a list of the acceptable values for each of the character variables and a range of acceptable values for numeric variables would be provided. The need for management of metadata is not limited to integrated databases, and various authors have presented approaches to sound practices of this important step (3,4). One of the most important steps in creating an integrated database is to specify clearly how variable values would be harmonized, or remapped, in the event that the originally collected values were not similar. This harmonization ranges from the simplistic, such as conversion of height to a standard unit, to the more difficult, such as the remapping of the recording of the states of disease progression. The recoding of adverse event data across several studies so that all verbatim terms are coded using the same version of the coding dictionary is another example of a type of harmonization that commonly occurs. It is important to document clearly the transformations because the integrity of the integrated data rests on the validation of this process. As new data is added to an existing integrated database, this documentation may need to be updated to reflect the remapping decision for any new and not previously seen data values. A consequence of harmonization is the loss of detail on an individual study basis. Depending on the extent of harmonization, it may not be possible to regenerate the original results from individual studies. Therefore, a trade-off occurs between standardization and the retention of important study details. Well-documented metadata should

support the efforts of the user of the integrated database to trace the path from the original data to the integrated, mapped data and clearly understand how important differences may have occurred. 1.4 The Benefit of Standards for Integration In an ideal setting, the knowledge of whether data from an individual clinical study eventually will be included in an integrated database would be known at the start of the study. However, this case is not practical because the decision to include data in an integrated database will likely depend on the success of the individual study and on the success of the entire product development. Therefore, the integration of data is most often done retrospectively, after the data has been collected and perhaps long after the study has been completed. This situation increases the likelihood that variables and values will need to be remapped to harmonize them with other data sources. In recent years, several initiatives to standardize clinical trial data have developed. The adoption of standards should serve to streamline any integration effort because it will reduce the variability in the individual study metadata. The Clinical Data Interchange Standards Consortium (CDISC) was founded in 1999 with the mission to ‘‘develop and support global, platform-independent data standards that enable information system interoperability to improve medical research and related areas of healthcare’’ (5). CDISC has developed standard metadata models for the representation of clinical trial data. The use of CDISC standards for each individual clinical trial at the beginning of the trial should result in a substantial reduction in the effort needed to integrate multiple studies into one integrated database. In a similar vein, Health Level Seven (HL7) is an accredited standards developing organization that operates in the healthcare arena with a mission to create standards for the exchange, management, and integration of electronic healthcare information. HL7 is working toward the creation of a standard vocabulary that will enable the exchange of clinical data and information so that a shared, well-defined, and unambiguous knowledge of the meaning of the data

INTEGRATED DATABASE

transferred is found (6). Both of these standards organizations have been referenced in the U.S. Food and Drug Administration’s (FDA’s) Critical Path Initiatives List (7,8) and undoubtedly will play an increased role in clinical trial data management in the future and result in the streamlining of integration efforts. Leading by example, the National Cancer Institute established the National Cancer Institute Center for Bioinformatics (NCICB) (9) in 2001 with the purpose to develop and provide interoperable biomedical informatics infrastructure tools and data that are needed by the biomedical communities for research, prevention, and care. Recognizing the problem of integrating data when multiple ways are available to describe similar or identical concepts, NCICB has embarked on developing a repository of common data elements, associated metadata, and a standard vocabulary. These efforts provide a robust example of how data integration can be streamlined and a solid vision for the future management of clinical trial data that can be applied to many therapeutic areas. 2 APPLICATION OF INTEGRATED DATABASES IN CLINICAL TRIAL RESEARCH Integrated databases have been used in clinical trial research for several years. In the classic sense, integrated databases are created for both regulatory purposes and postmarketing surveillance. With the rapid expansion of the field of bioinformatics, the use of integrated databases has expanded beyond this classic application. 2.1 Classic Applications The integration of data, especially the data related to patient safety, is required by most regulatory agencies when the drug is submitted for marketing approval. For submissions made to the U.S. Food and Drug Administration, the guidelines recommend that an application have sections entitled Integrated Summary of Efficacy and Integrated Summary of Safety (ISS) (10). In the ISS section, safety data from all available studies are integrated so that analyses of measures of

3

safety can be performed for the entire patient population exposed to the drug compound. Integrated databases, therefore, are often initially created as part of a new drug application. Using the integrated database of adverse events, sponsor companies can determine the most commonly occurring adverse events across the entire patient population. This information becomes part of the package insert and is used as a reference by informed patients, prescribing physicians, and marketing campaigns. In addition to defining the most common adverse events, the use of the integrated database to evaluate the serious adverse effects that would be too rare to be detected in a single study is of special interest. Often a sponsor may not be aware of potentially serious adverse effects or subpopulations that may have a differential response to treatment until the data is integrated and the sample size is large enough to detect these important differences. Using the integrated database, both sponsor companies and regulatory agencies can explore the safety profile across various subpopulations, such as age groups, racial or ethnicity groups, and gender. By employing data standards within a study project, data integration can be done as each clinical trial is completed rather than all at once just before submitting a new drug application. Sponsor companies profit from creating an integrated database as studies complete because it can reduce the time to submission, and it can highlight important differences between treatment groups or subpopulations earlier in the product development. In addition to creating integrated databases to fulfill regulatory requirements, many sponsors maintain an integrated database that contains results from late phase and postmarketing trials. The medical literature is replete with published examples of the use of integrated databases to monitor the safety and tolerability of a marketed drug. Many of these articles describe the use of integrated data specifically for postmarketing safety surveillance (11–16), and the sample size can be quite large when the drug compound under study is widely prescribed (17). The use of integrated databases

4

INTEGRATED DATABASE

is not restricted to late phase or postmarketing trials and can serve useful purposes in designing new trials and data collection instruments (18). 2.2 Recent Advances As the collection of data from clinical trials becomes less reliant on paper and increasingly collected via electronic means, the types of data that can be integrated has expanded, especially data from digital images. Constructing a database of images is inherently different than a database of textual values, and this difference impacts the design of the database and use of the data. New database strategies have been developed to address these issues (19,20). A recent study (21) has enumerated the issues associated with building an integrated database that contains the results of quantitative image analysis obtained from digital magnetic resonance images (MRIs). The authors present a workflow management system that uses a centralized database coupled with database applications that allow for seamless integration of the analysis of MRIs. This process could be generalized and applied to any clinical trial that uses image-based measurements. In an effort to improve drug safety programs, the FDA has recently embarked on several projects to discover better methods to predict cardiovascular risk of drugs. In two of these projects, the FDA has collaborated with nongovernment entities, Mortara Instruments, Inc. and Duke Clinical Research Institute, to design and build a repository to hold digital electrocardiograms. The use of this integrated warehouse will facilitate regulatory review and research and will aid in the development of tools to be used for the evaluation of cardiac safety of new drugs (22). The explosion of knowledge of the areas of genomics and proteomics has fostered the development of integrated databases that can be used to support clinical research. Matthew and colleagues (23) give a thorough review of the challenges that are being addressed in this new era of computational biology as applied to translational cancer research. By creating an integrated database

of gene expression and drug sensitivity profiles, Tsunoda and colleagues (24) could identify genes with expression patterns that showed significant correlation to patterns of drug responsiveness. Other authors have developed integrated databases of ‘‘omics’’ information that enables researchers working in the field of anticancer drug discovery to explore and analyze information in a systematic way (25). An example of the software infrastructure used to build biological data warehouse for the integration of bioinformatics data and for the retrieval of this information has been described and made freely available to the public (26). REFERENCES 1. M. Lenzerini, Data Integration: a theoretical perspective. Proceedings of the 21st ACM-SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems. 2002: 243–246. 2. A. Hurson and M. Bright, Multidatabase systems: An advanced concept in handling distributed data. Advances in Computers. 1991; 32: 149–200. 3. C. Brandt, R. Gadagkar, C. Rodriguez, and P. Nadkarni, Managing complex change in clinical study metadata. J. Am. Med. Inform. Assoc. 2004; 11(5): 380–391. 4. A. Vaduva and T. Vetterli, Meta data management for data warehousing: an overview. Int. J. Cooperative Inform. Syst. 2001; 10(3): 273–298. 5. Clinical Data Interchange Standards Consortium (CDISC). Available: www.cdisc.org. 6. Health Level Seven (HL7). Available: www.hl7.org. 7. U. S. Food and Drug Administration (FDA). Challenge and opportunity on the critical path to new medical products. March 2004. Available: http://www.fda.gov/ oc/initiatives/criticalpath/whitepaper.pdf. 8. U. S. Food and Drug Administration (FDA). Critical path opportunities list. March 2006. Available: http://www.fda.gov/oc/ initiatives/criticalpath/reports/opp list.pdf. 9. National Cancer Institute Center for Bioinformatics (NCICB). Available: http:// ncicb.nci.nih.gov/. 10. U. S. Food and Drug Administration (FDA). Guideline for the format and content of the clinical and statistical sections of an

INTEGRATED DATABASE application. July 1988. Available: http:// www.fda.gov/cder/guidance/statnda.pdf. 11. D. Payen, A. Sablotzk, P. Barie, G. Ramsay, S. Lowry, M. Williams, S. Sarwat, J. Northrup, P. Toland, F. V. McLBooth, International integrated database for the evaluation of severe sepsis and drotrecogin alfa (activated) therapy: analysis of efficacy and safety in a large surgical cohort. Surgery. 2006; 140(5): 726–739. 12. A. Gottlieb, C. Leonardi, B. Goffe, J. Ortonne, P. van der Kerhof, R. Zitnik, A. Nakanishi, and A. Jahreis, Etanercept monotherapy in patients with psoriasis: a summary of safety, based on an integrated multistudy database. J. Am. Acad. Dermatology. 2006; 54(3 Suppl 2): S92–100. 13. D. Hurley, C. Turner, I. Yalcin, L. Viktrup, and S. Baygani, Duloxetine for the treatment of stress urinary incontinence in women: an integrated analysis of safety. Eur. J. Obstet. Gynecol. Reprod. Biol. 2006; 125(1): 120–128. 14. R. Fleischmann, S. Baumgartner, M. Weisman, T. Liu, B. White, and P. Peloso, Long term safety of etanercept in elderly subjects with rheumatic diseases. Ann. Rheum. Dis. 2006; 65(3): 379–384. 15. C. Martin and C. Pollera, Gemcitabine: safety profile unaffected by starting dose. Int. J. Clin. Pharmacol. Res. 1996; 16(1): 9–18. 16. J. Rabinowitz, I. Katz, P. De Deyn, H. Brodaty, A. Greenspan, and M. Davidson, Behavioral and psychological symptoms in patients with dementia as a target for pharmacotherapy with risperidone. J. Clin. Psychiatry. 2004; 65(10): 1329–1334. 17. J. Shepherd, D. Vidt, E. Miller, S. Harris, and J. Blasetto,. Safety of rosuvastatin: Update of 16,876 rosuvastatin-treated patients in a multinational clinical trial program. Cardiology. 2007; 107(4): 433–443. 18. A. Farin and L. Marshall, Lessons from epidemiologic studies in clinical trials of traumatic brain injury. Acta. Neurochir. Suppl. 2004; 89: 101–107. 19. H. Tagare, C. Jaffe, and J. Duncan, Medical image databases: a content-based retrieval approach. J. Am. Med. Inform. Assoc. 1997; 4(3): 184–198. 20. C. Feng, D. Feng, and R. Fulton, Contentbased retrieval of dynamic PET functional images. IEEE Trans. Inf. Technol. Biomed. 2000; 4(2): 152–158. 21. L. Liu, D. Meir, M. Polgar-Turcsanyi, P. Karkocha, R. Bakshi, C. R. Guttman, Multiple

22.

23.

24.

25.

26.

5

sclerosis medical image analysis and information management. J. Neuroimaging. 2005; 15(4 Suppl): 103S–117S. U. S. Food and Drug Administration. The Future of Drug Safety – Promoting and Protecting the Health of the Public. January 2007. Available: http://www. fda.gov/oc/reports/iom013007.pdf. J. Matthew, B. Taylor, G. Bader, S. Pyarajan, M. Antoniotti, A. Chinnaiyan, C. Sander, J. Buarkoff, and B. Mishra, From bytes to bedside: data integration and computational biology for translational cancer research. PLOS Computational Biology. 2007; 3(2): 153–163. D. Tsunoda, O. Kitahara, R. Yanagawa, H. Zembutsu, T. Katagiri, K. Yamazaki, Y. Nakamura, and T. Yamori, An integrated database of chemosensitivity to 55 anticancer drugs and gene expression profiles of 39 human cancer cell lines. Cancer Res. 2002; 62(4): 1139–1147. F. Kalpakov, V. Poroikov, R. Sharipov, Y. Kondrakhin, A. Zakharov, A. Lagunin, L. Milanesi, and A. Kel, CYCLONET – an integrated database on cell cycle regulation and carcinogenesis. Nucleic Acids Res. 2007; 35(database issue): D550–556. S. Shah, Y. Huang, T. Xy, M. Yuen, J. Ling, and B. Ouellette, Atlas – a data warehouse for integrative bioinformatics. BMC Biominformatics. 2005; 21(6): 34.

CROSS-REFERENCES Clinical Data Management Integrated Summary of Safety Information Integrated Summary of Effectiveness Data Bioinformatics Data Mining of Health System Data

INTENTION-TO-TREAT ANALYSIS

that it is only possible to perform an analysis using a post-hoc selected subset. The essential concern with such subset selection, or equivalently post-hoc exclusions, is that the resulting subset may be susceptible to various forms of bias (3,5–7). A review is provided in Reference 8. Many who champion the efficacy subset analysis approach argue that statistical techniques may be applied to provide an unbiased analysis under certain assumptions (9). The essential statistical issue is the extent to which it can be assumed that missing data and omitted data do not introduce a bias under the missing information principle (10) (i.e., that missing/omitted data are ignorable).

JOHN M. LACHIN The George Washington University, Washington, DC

A clinical trial of a new therapy (agent, intervention, diagnostic procedure, etc.) has many objectives. One objective is to evaluate whether the therapy has the intended biological or physiologic effect, which is often termed pharmacologic efficacy in relation to a new pharmaceutical agent. Another is to evaluate the pragmatic use of the therapy in clinical practice, which is often termed simply effectiveness. The intention-to-treat principle refers to a strategy for the design, conduct, and analysis of a clinical trial aimed at assessing the latter, which is the pragmatic effectiveness of a therapy in clinical practice. An analysis for the assessment of pharmacologic efficacy generally excludes subjects who either did not comply with the assigned therapy or could not tolerate it because of adverse effects. Such analyses are often termed per-protocol, efficacy subset, or evaluable subset analyses and involve posthoc subset selection or post-hoc exclusions of randomized subjects. Conversely, for the assessment of effectiveness in an intent-totreat analysis, follow-up data for all randomized subjects should be obtained and included in the analysis. This design is the essence of the intention-to-treat principle. 1

1.2 Ignorable Missing Data Missing data refers to data that are hypothetically obtainable from a subject enrolled in a trial, but that are not obtained. The hypothetically obtainable data consists of every possible observation that could be obtained from a subject from the point of initial screening and randomization to the prespecified scheduled end of follow-up for that subject. In some trials, the prespecified end of study for a subject is a fixed period, such as 1 year of treatment and follow-up. In other trials, the prespecified end may depend on when the subject enters the trial, such as the case where patient entry is staggered over one period but there is a prespecified date on which all treatment and follow-up will end. If recruitment is conducted over a 3-year period and the total study duration is 5 years, then the first subject randomized hypothetically can contribute 5 years of data collection whereas the last can only contribute 2 years. Thus, for every clinical trial design, each randomized subject has an associated hypothetical complete set of data that could be collected, and the aggregate over all subjects is the set of hypothetically obtainable data. Data may be missing for many possible reasons or mechanisms, some of which may be ignorable, occur randomly, or occur completely by chance. Missing data that develop from an ignorable mechanism are called

MISSING INFORMATION

1.1 Background The intention-to-treat principle evolved from the evaluation by regulatory officials at the Food and Drug Administration (FDA) (1), as well as scientists from the National Institutes of Health (2) and academia (3), of clinical trials in which post-hoc subset selection criteria were applied (4). In some cases, the data are collected but are excluded at the time of analysis. More often, the protocol allows subjects to terminate follow-up, and thus allows subject data to be excluded (not collected), such

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INTENTION-TO-TREAT ANALYSIS

missing completely at random (MCAR) (10), in the sense that the unobserved value of the missing observation is statistically independent of other potentially observable information, most importantly including the treatment group assignment. In the context of survival analysis, the equivalent assumption is censoring at random. Few missing data mechanisms satisfy the MCAR criterion on face value. One is administratively missing data whereby data from a subject cannot be obtained because of administrative curtailment of follow-up. This situation usually applies in the setting of staggered patient entry with a fixed study end date, such as the last patient entered in the above example for whom at most 2 years of data could be collected before study end, and for whom the assessments during years 3–5 would be administratively missing. Missing data may occur for many other reasons. Some subjects may die while participating in the study. Others may be withdrawn from therapy because of poor compliance, lack of evidence of a therapeutic effect, or an adverse effect of therapy, and concurrently withdrawn from the study (i.e., with no additional follow-up). Others may be lost to follow-up (so-called dropouts) because of withdrawal of consent, moving away, incarceration, loss of interest, and so on. The fundamental issue is whether these mechanisms can be claimed to be ignorable. Of these, losses to follow-up are often considered to be missing completely at random, but missing data from such subjects would only be ignorable when effects of treatment, either positive or negative, played no role in the decision to curtail follow-up. This situation might be plausible if examination of the timing and characteristics of such losses are equivalent among the groups, and the characteristics of losses versus those who were not lost to follow-up are equivalent. If differences are detected, then it is possible or even likely that the missing data from such subjects is not ignorable. If baseline covariate differences are detected between those lost to follow-up versus those not, then a sensitivity analysis that compares groups adjusted for those covariates might provide a less-biased comparison of the treatment groups.

On-study deaths in some circumstances may also be claimed to be ignorable, as in a study of a disease or condition (e.g., topical skin therapy) that has a negligible risk of mortality. However, even in such cases, deaths may not be ignorable if the treatment itself may adversely affect vital organs, such as through hepatotoxicity (liver toxicity). The latter could be a hypothetical concern for any agent that is known to be metabolized and excreted by the liver. Drugs have been discovered with none of the other preclinical (i.e., animal) findings or clinical findings (e.g., elevated liver enzymes) that would trigger suspicion of hepatotoxicity, but in later trials or clinical practice are shown to pose a risk of possibly fatal hepatotoxicity in some patients. Clearl, subjects withdrawn from treatment and follow-up because of a specific adverse effect would only be ignorable when the adverse effect can be claimed to be statistically independent of the treatment assignment. This result can virtually never be proven. Other subjects withdrawn from treatment and follow-up because of insufficient therapeutic effect are clearly not ignorable. The fundamental issue is that in all of these cases, it cannot be proven that the missing data or the mechanism(s) for missing data are ignorable or missing completely at random. 1.3 Conditionally Ignorable Missing Data Many statistical methods require the assumption that missing data are missing completely at random to provide an unbiased analysis. However, other methods provide an unbiased analysis under the assumption that missing data are missing at random (MAR). This explanation is somewhat of a misnomer. Under MAR, it is assumed that the missing data are in fact nonignorable in the sense that the probability of being missing may depend on the unobserved value. However, MAR assumes that this dependence is reflected in other information that has been observed. Thus under MAR, it is assumed that the missing data are conditionally independent of the unobserved value, conditioning on the other information that has been observed including treatment assignment.

INTENTION-TO-TREAT ANALYSIS

Clearly, this is a big assumption. For example, a longitudinal analysis that uses a mixed model implicitly assumes that the data that are missing at a follow-up visit are a function of the baseline characteristics and other follow-up observations that preceded it. The treatment group comparison can then be claimed to be unbiased if this relationship applies (i.e., the structural and random model components are correctly specified), and the important covariates have been measured and observed. However, these assumptions cannot be verified.

In conclusion, although statistical methods can provide an unbiased analysis of virtually any data structure with missing data when certain assumptions apply, and although those assumptions, either MCAR or MAR conditionally, can be tested and rejected, those assumptions can never be proven to apply, and the resulting analysis can never be proven to be unbiased. Lachin (8) summarized the issue by stating that the only incontrovertibly unbiased study is one in which all randomized patients are evaluated and included in the analysis, assuming that other features of the study are also unbiased. This is the essence of the intent-to-treat philosophy. Any analysis which involves post hoc exclusions of information is potentially biased and potentially misleading.

1.4 Potential for Bias If the data that are missing develop from nonignorable mechanisms, then the missing data can introduce substantial bias in the results. Lachin (8) presents a model for the assessment of the possible bias for an analysis of the difference between two proportions and the pursuant inflation in the type I error probability α. As the proportion of subjects excluded from the analysis increases, the maximum possible bias and the resulting α increase, as expected. Furthermore, as the total sample size increases, the possible bias and α with a given fraction of missing data increases. Consider the simplest case in which all control group subjects are followed, but a fraction of the treated group has missing data, for example 20%. For a sample size of 200, a bias of 0.05 leads to an α = 0.14086 whereas for N = 800, the same bias leads to an α = 0.38964. Various analyses can be performed to assess whether the missing at random (conditionally) assumption applies, such as comparing the characteristics of those with missing versus observed data, in aggregate and within groups, and comparing the characteristics of those with missing data between groups. The null hypothesis of such tests is that the missing data are conditionally ignorable within the context of a particular model. The alternative hypothesis is that they are not ignorable, and the resulting analysis is biased. Thus, such tests can reject the null hypothesis of ignorable missing data in favor of the alternative of nonignorable missing data, but such tests cannot prove that missing data are ignorable.

3

2

THE INTENTION-TO-TREAT DESIGN

Thus, an intention-to-treat design makes every attempt to ensure complete follow-up and collection of outcome data for every participant from the time of randomization to the scheduled completion of study, regardless of other developments such as noncompliance or adverse effects of therapy. The International Conference on Harmonization (ICH) document Guidance on Statistical Principles for Clinical Trials (11), provides the following description of the intention-to-treat principle: The principle that asserts that the effect of a treatment policy can best be assessed by evaluating on the basis of the intention to treat a subject (i.e. the planned treatment regimen) rather than the actual treatment given. It has the consequence that subjects allocated to a treatment group should be followed up, assessed and analyzed as members of that group irrespective of their compliance with the planned course of treatment.

This guidance also states (Section 5.2.1): The intention-to-treat principle implies that the primary analysis should include all randomized subjects. Compliance with this principle would necessitate complete follow-up of all randomized subjects for study outcomes.

4

INTENTION-TO-TREAT ANALYSIS

The ICH the Guidance on General Considerations for Clinical Trials (12, Section 3.2.2) also states: The protocol should specify procedures for the follow-up of patients who stop treatment prematurely.

These principles are the essence of the intent-to-treat design. To conduct a study that provides high confidence that it is unbiased, the extent of missing data must be minimized. 2.1 Withdrawal from Treatment versus Withdrawal from Follow-up Every study protocol should include a provision for the investigator to withdraw a subject from the randomly assigned therapy because of possible adverse effects of therapy. However, the intent-to-treat design requires that such subjects should not also be withdrawn from follow-up. Thus, the protocol should distinguish between withdrawal from treatment versus withdrawal from follow-up. To the extent possible, every subject randomized should be followed as specified in the protocol. In fact, it would be advisable that the study protocol not include provision for the investigator to withdraw a subject from the study (i.e., follow-up). The only exceptions to this policy might be the death or incapacitation of a subject, or the subject’s withdrawal of patient consent. In some studies, time to a major clinical event is the primary outcome, such as a major cardiovascular adverse event or overall survival. In these cases, even if the subject withdraws consent to continue follow-up visits, it would also be important to ask the patient to consent to ascertainment of major events and/or vital status. Furthermore, in long-term studies, patients may withdraw consent, and then later be willing to resume follow-up and possibly the assigned treatment where not contraindicated. To allow patient participation to the extent possible, subjects should not be labeled as ‘‘dropouts’’ while the study is underway. Subjects who withdraw consent or who do not maintain contact may be termed inactive, temporarily, with the understanding that

any such subject may later become active. The designation of ‘‘dropout’’ or ‘‘lost to followup’’ should only be applied after the study has been completed. 2.2 Investigator and Subject Training/Education Unfortunately, many investigators have participated in clinical trials that were not designed in accordance with this principle. Even though the protocol may state that all patients should continue follow-up to the extent possible, investigators may fail to comply. In studies that have successfully implemented continued follow-up, extensive education of physicians, nurse investigators, and patients has been implemented. In the Diabetes Control and Complications Trial (1983–1993), the patients and investigators received intensive patient education on the components of the trial and the expectation that they would continue followup (13). Of the 1441 subjects randomized into the study, during the 10 years of study only 32 subjects were declared inactive at some point in time. Of these subjects, 7 later resumed treatment and follow-up. Among the 1330 surviving subjects at study end, only 8 subjects did not complete a study closeout assessment visit. During the trial, 155 of the 1441 patients deviated from the originally assigned treatment for some period (were noncompliant). Virtually all subjects continued to attend follow-up assessment visits and most resumed the assigned therapy later. The DCCT was unusual because two multifaceted insulin therapies were being compared in subjects with type 1 diabetes who must have insulin therapy to sustain life. Therefore, withdrawal from insulin therapy was not an option. In a study comparing a drug versus placebo, the issues are different. The Diabetes Prevention Program (14) included comparison of metformin versus placebo for preventing the development of overt diabetes among patients with impaired glucose tolerance. Metformin is an approved antihyperglycemic therapy for treatment of type 2 diabetes with known potential adverse effects that require discontinuation of treatment in about 4% of patients, principally

INTENTION-TO-TREAT ANALYSIS

because of gastrointestinal effects. The following is the text provided to the investigators to explain to the patient why the study desired to continue follow-up after therapy was withdrawn due to an adverse effect: When we designed the study we knew that a fraction of patients would not be able to tolerate metformin. You were told this when you agreed to participate in the study. However, we cannot tell beforehand which participants will be able to take metformin, and which will not. In order to answer the DPP study question as to whether any treatment will prevent diabetes, every participant randomized into the study is equally important. Thus, even though you will not be taking a metformin pill, it is just as important for us to know if and when you develop diabetes as it is for any other participant. That’s why it is just as important to the study that you attend your outcome assessment visits in the future as it was when you were taking your pills.

2.3 The Intent-to-Treat Analysis An intent-to-treat analysis refers to an analysis that includes all available data for all randomized subjects. However, for an intent-totreat analysis to comply with the intentionto-treat principle, all ‘‘available’’ data should represent a high fraction of all potentially observable data. Thus, an analysis of all randomized subjects, in which a high fraction have incomplete assessments and missing data, deviates from the intention-to-treat principle in its design and/or implementation, and thus it is possibly biased. 2.4 Intent-to-treat Subset Analysis In many studies, multiple analyses are conducted in different sets of subjects. The intention-to-treat ‘‘population’’ is often defined to include all subjects randomized who had at least one dose of the study medication. However, unless the protocol specifies systematic follow-up of all subjects, and a high fraction of the potentially obtainable data is actually obtained, then an analysis of the intention-to-treat population is simply another post-hoc selected subgroup analysis that is susceptible to bias because of nonignorable or informatively missing data.

5

2.5 LOCF Analysis In an attempt to reconstruct a complete data set from incomplete data, a simplistic approach that is now commonly employed is an analysis using the last observation carried forward (LOCF) for each subject with missing follow-up measurements. This method is popular because it makes it seem as though no substantial data are missing. However, the technique is statistically flawed (15,16). LOCF values would not be expected to provide the same level of information as values actually measured and would not be expected to follow the same distribution. Furthermore, such values will distort the variance/covariance structure of the data so that any confidence intervals or P-values are biased in favor of an optimistic assessment in that the sample size with LOCF values is artificially inflated and the variance of the measures is artificially deflated. The LOCF method has no formal statistical basis and has been soundly criticized by statisticians. 2.6 Structurally Missing Data In some studies, the primary outcome is the observation of a possibly right-censored event time and a secondary outcome is a mechanistic or other longitudinal measure. Often, the follow-up of a subject is terminated when the index event occurs, that causes all subsequent mechanistic measurements to be missing structurally. For example, in the Diabetes Prevention Program, the primary outcome was the time to the onset of type 2 diabetes, and measures of insulin resistance and insulin secretory capacity were obtained up to the time of diabetes or the end of study (or loss to follow-up). Thus, these mechanistic measures were missing beyond the time of diabetes, and it is not possible to conduct a straightforward intention-to-treat analysis to describe the long-term effect of each initial therapy on these mechanistic measures (e.g., the difference between groups at say 4 years in the total cohort of those entered into the study). 2.7 Worst Rank Analyses In some cases, it may be plausible to assume that subjects with missing data because of

6

INTENTION-TO-TREAT ANALYSIS

a prior index event are actually worse (or better) than all those who continue followup for a particular measure. For example, subjects who have died can be assumed to have a worse quality of life than any subject who survives. In this case, the subjects with structurally missing data because of such an index event can be assigned a ‘‘worst rank,’’ (i.e., a rank worse than that of any of the measures actually observed) (17). 3 EFFICIENCY OF THE INTENT-TO-TREAT ANALYSIS 3.1 Power The intent-to-treat design, which is necessary to conduct a true intent-to-treat analysis, requires the follow-up of all patients, which includes those who failed to comply with the therapy or those who were withdrawn from therapy by personal choice or by the investigator. Thus, the treatment effect observed may be diluted compared with a setting in which all subjects receive the therapy and are fully compliant. However, that comparison is specious. Virtually every agent or intervention will not be applied optimally in every subject in clinical practice. Therefore, an analysis aimed at the treatment effectiveness under optimal conditions is not relevant to clinical practice. Such an assessment, however, is of interest as a reflection of pharmacologic activity, which is the underlying mechanism by which a treatment is purported to have a clinical effect. This mechanism is the justification for the so-called ‘‘efficacy subset’’ or ‘‘perprotocol’’ analysis often conducted in pharmaceutical trials. The phrase ‘‘per-protocol’’ is used when the protocol itself specifies the post-hoc exclusions of patients and patient data based on compliance, adverse effects, or other factors that indicate suboptimal treatment. Such a subset analysis is highly susceptible to bias. Nevertheless, it is instructive to compare the power of an intention-to-treat design and analysis versus an efficacy subset analysis when it is assumed that the latter is unbiased. Using the test for proportions, Lachin (8) showed that there is a trade-off between the increasing sample size in the intentto-treat analysis versus the larger expected

treatment effect in the efficacy subset analysis. However, in some settings the intentto-treat design and analysis may be more powerful, especially when the treatment may have some long-term beneficial effects that persist beyond the period of therapy. For example, if the treatment arrests progression of the disease, then a subject treated for a period of time who is then withdrawn may still be expected to have a more favorable outcome long-term compared with a subject originally assigned to control. In this case, the intention-to-treat analysis can have substantially more power than the efficacy subset analysis, and it has the advantage that it is far less susceptible to bias. These issues were borne out in the analysis of the study of tacrine in the treatment of Alzheimer’s disease (18), in which 663 subjects were entered and 612 completed follow-up. However, only 253 continued on treatment. Thus, the intent-to-treat analysis of the 612 evaluated could be compared with the efficacy subset analysis of the 253 ‘‘ontherapy completers’’ as shown in Reference 8. For some outcomes, the intention-to-treat analysis produced results that were indeed significant whereas the efficacy subset analysis was not. 3.2 Sample Size In the planning of a study, it is common to adjust the estimate of the sample size for losses to follow-up. For a simple analysis of means or proportions, the sample size computed assuming complete follow-up is inflated to allow for losses. For example, to adjust for 20% losses to follow-up (subjects with missing outcome data), the sample size is inflated by the factor 1/0.8, or by 25%. Such an adjustment allows for the loss of information because of losses to followup or missing data. It does not adjust for the potential bias introduced by such losses if the mechanism for the missing data is informative.

4 COMPLIANCE ADJUSTED ANALYSES The efficacy subset analysis is a crude attempt to assess the treatment effect had

INTENTION-TO-TREAT ANALYSIS

all subjects remained fully compliant. However, if an intention-to-treat design is implemented, then noncompliance and the degree of compliance become outcome measures, which make it possible to conduct analyses that assess treatment group differences in the primary outcomes while taking into account the differences in compliance. Analyses could also be conducted to estimate the treatment group difference for any assumed degree of compliance in the treatment groups (19–21). However, these methods can only be applied when study outcomes are measured in all subjects, or a representative subset, which includes those who are noncompliant or who are withdrawn from therapy. These methods cannot be applied in cases where follow-up is terminated when a subject is withdrawn from therapy because of noncompliance or other factors. 5

CONCLUSION

The intention-to-treat principle encourages the collection of complete follow-up data to the extent possible under an intentionto-treat design and the inclusion of all data collected for each subject in an intentionto-treat analysis. An analysis of a so-called Intention-to-treat population of all subjects randomized, but without systematic followup, is simply another type of post-hoc subset analysis that is susceptible to bias. REFERENCES 1. R. Temple and G. W. Pledger, The FDA’s critique of the Anturane Reinfarction Trial. N. Engl. J. Med. 1980; 303: 1488–1492. 2. D. L. DeMets, L. M. Friedman, and C. D. Furberg, Counting events in clinical trials (letter to the editor). N. Engl. J. Med. 1980; 302: 924. 3. L. M. Friedman, C. D. Furberg, and D. L. DeMets, Fundamentals of Clinical Trials, 3rd ed. New York: Springer; 1998. 4. D. L. Sackett and M. Gent, Controversy in counting and attributing events in clinical trials. N. Engl. J. Med. 1979; 301: 1410–1412. 5. G. S. May, D. L. DeMets, L. M. Friedman, C. Furberg, E. Passamani, The randomized clinical trial: bias in analysis. Circulation 1981; 64: 669–673.

7

6. P. Armitage, Controversies and achievements in clinical trials. Control. Clin. Trials 1984; 5: 67–72. 7. R. Peto, M. C. Pike, P. Armitage, et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Br. J. Cancer 1976; 34: 585–612. 8. J. M. Lachin, Statistical Considerations in the Intent-to-treat Principle. Control. Clin. Trials 2000; 21: 167–189. 9. L. B. Sheiner and D. B. Rubin, Intention-totreat analysis and the goals of clinical trials. Clin. Pharmacol. Ther. 1995; 57: 6–15. 10. R. J. A. Little and D. B. Rubin, Statistical Analysis with Missing Data. New York: Wiley; 1987. 11. Food and Drug Administration, International Conference on Harmonization: Guidance on statistical principles for clinical trials. Federal Register September 16, 1998; 63: 49583–49598. 12. Food and Drug Administration, International Conference on Harmonization: Guidance on general considerations for clinical trials. Federal Register December 17, 1997; 62: 66113–66119. 13. Diabetes Control and Complications Trial Research Group, Implementation of a multicomponent process to obtain informed consent in the Diabetes Control and Complications Trial. Control. Clin. Trials 1989; 10: 83–96. 14. Diabetes Prevention Program Research Group, The Diabetes Prevention Program: design and methods for a clinical trial in the prevention of type 2 diabetes. Diabetes Care 1999; 22: 623–634. 15. G. Veberke, G. Molenberghs, L. Bijnens, and D. Shaw, Linear Mixed Models in Practice. New York: Springer, 1997. 16. F. Smith, Mixed-model analysis of incomplete longitudinal data from a high-dose trial of tacrine (cognex) in Alzheimer’s patient. J. Biopharm. Stat. 1996; 6: 59–67. 17. J. M. Lachin, Worst rank score analysis with informatively missing observations in clinical trials. Control. Clin. Trials 1999; 20: 408–422. 18. M. J. Knapp, D. S. Knopman, P. R. Solomon, et al., A 30-week randomized controlled trial of high-dose Tacrine in patients with Alzheimer’s disease. J. Am. Med. Assoc. 1994; 271: 985–991. 19. J. Rochon, Supplementing the intent-to-treat analysis: Accounting for covariates observed postrandomization in clinical trials. J. Am. Stat. Assoc. 1995; 90: 292–300.

8

INTENTION-TO-TREAT ANALYSIS

20. B. Efron and D. Feldman, Compliance as an explanatory variable in clinical trials. J. Am. Stat. Assoc. 1991; 86: 9–25. 21. J. W. Hogan and N. M. Laird, Intention-totreat analyses for incomplete repeated measures data. Biometrics 1996; 52: 1002–1017.

FURTHER READING Principles of Clinical Trial Design and Analysis: D. G. Altman, K. F. Schulz, D. Moher, M. Egger, D. Davidoff, D. Elbourne, P. C. Gøtzsche, and T. Lang, for the CONSORT Group, The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann. Intern. Med. 2001; 134: 663–694. P. Armitage, Controversies and achievements in clinical trials. Control. Clin. Trials 1984; 5: 67–72. D. L. DeMets, Statistical issues in interpreting clinical trials. J. Intern. Med. 2004; 255: 529–537 P. W. Lavori and R. Dawson, Designing for intent to treat. Drug Informat. J. 2001; 35: 1079–1086 G. S. May, D. L. DeMets, L. M. Friedman, C. Furberg, and E. Passamani, The randomized clinical trial: bias in analysis. Circulation 1981; 64: 669–673. R. Peto, M. C. Pike, P. Armitage, et al., Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Br. J. Cancer 1976; 34: 585–612. D. Schwartz and J. Lellouch, Explanatory and pragmatic attitudes in therapeutic trials. J. Chron. Dis. 967; 20: 637–648. Case Studies G. Chene, P. Moriat, C. Leport, R. Hafner, L. Dequae, I. Charreau, JP. Aboulker, B. Luft, J. Aubertin, J. L. Vilde, and R. Salamon, Intention-to-treat vs. on-treatment analyses from a study of pyrimethamine in the primary prophylaxis of toxoplasmosis in HIV-infected patients. ANRS 005/ACTG 154 Trial Group. Control. Clin. Trials 1998; 19: 233–248. The Coronary Drug Project Research Group, Influence of adherence to treatment and response of cholesterol on mortality in the Coronary Drug Project. N. Engl. J. Med. 1980; 303: 1038–1041. P. W. Lavori, Clinical trials in psychiatry: should protocol deviation censor patient data (with Discussion). Neuropsychopharm 1992; 6: 39–63.

P. Peduzzi, J. Wittes, K. Detre, and T. Holford, Analysis as-randomized and the problem of non-adherence: An example from the Veterans Affairs Randomized Trial of Coronary Artery Bypass Surgery. Stat. Med. 1993; 15: 1185–1195. C. Redmond, B. Fisher, H. S. Wieand, The methodologic dilemma in retrospectively correlating the amount of chemotherapy received in adjuvant therapy protocols with disease-free survival. Cancer Treat. Rep. 1983; 67: 519–526. Alternative Views J. H. Ellenberg, Intent-to-treat analysis versus astreated analysis. Drug Informat. J. 1996; 30: 535–544. M. Gent and D. L. Sackett, The qualification and disqualification of patients and events in longterm cardiovascular clinical trials. Thrombos. Haemostas. 1979; 41: 123–134. E. Goetghebeur and T. Loeys, Beyond Intention-totreat. Epidemiol. Rev. 2002; 24: 85–90. Y. J. Lee, J. H. Ellenberg, D. G. Hirtz, and K. B. Nelson, Analysis of clinical trials by treatment actually received: is it really an option? Stat. Med. 1991; 10: 1595-605. R. J. A. Little, Modeling the drop-out mechanism in repeated-measure studies. J. Am. Stat. Assoc. 1995; 90: 1112–1121. L. B. Sheiner, Is intent-to-treat analysis always (ever) enough? Br. J. Clin. Pharmacol. 2002; 54: 203–211.

CROSS-REFERENCES Adequate and Well-Controlled Trial Adherence Analysis Data Set Analysis Population Completer Analysis Casual Inference CONSORT Diabetes Control and Complications Trial Diabetes Prevention Program Effectiveness Evaluable Population Full Analysis Set Good Statistics Practice International Conference on Harmonization (ICH) Last Observation Carried Forward (LOCF) Lost-to-Follow-Up Missing Values

INTENTION-TO-TREAT ANALYSIS Per Protocol Set Analysis Protocol Deviations Protocol Deviators Protocol Violators Responder Analysis Subset Withdrawal from Study Withdrawal from Treatment

9

INTERACTION MODEL

1

GRAPHICAL INTERACTION MODELS

What is meant by association between two variables? The most general response to this question is indirect. Two variables are dissociated if they are conditionally independent given the rest of the variables in the multivariate framework in which the two variables are embedded. Association then simply means that the two variables are not dissociated. Association in this sense is, of course, not a very precise statement. It simply means that conditions exist under which the two variables are not independent. Analysis of association will typically have to go beyond the crude question of whether or not association is present, to find out what characterizes the conditional relationship—for instance, whether it exists only under certain conditions, whether it is homogeneous, or whether it is modified by outcomes on some or all the conditioning variables. Despite the inherent vagueness of statements in terms of unqualified association and dissociation, these statements nevertheless define elegant and useful models that may serve as the natural first step for analyses of association in multivariate frames of inference. These so-called graphical models are defined and described in the subsections that follow.

SVEND KREINER University of Copenhagen, Copenhagen, Denmark

Interaction models for categorical data are loglinear models describing association among categorical variables. They are called interaction models because of the analytic equivalence of loglinear Poisson regression models describing the dependence of a count variable on a set of categorical explanatory variables and loglinear models for contingency tables based on multinomial or product multinomial sampling. The term is, however, somewhat misleading, because the interpretation of parameters from the two types of models are very different. Association models would probably be a better name. Instead of simply referring the discussion of interaction and association models to the section on loglinear models, we will consider these models from the types of problems that one could address in connection with analysis of association. The first problem is a straightforward question of whether or not variables are associated. To answer this question, one must first define association and dissociation in multivariate frameworks and, secondly, define multivariate models in which these definitions are embedded. This eventually leads to a family of so-called graphical models that can be regarded as the basic type of interaction or association. The second problem concerns the properties of the identified associations. Are associations homogeneous or heterogeneous across levels of other variables? Can the strength of association be measured and in which way? To solve these problems, one must first decide upon a natural measure of association among categorical variables and, secondly, define a parametric structure for the interaction models that encapsulates this measure. Considerations along these lines eventually lead to the family of hierarchical loglinear models for nominal data and models simplifying the basic loglinear terms for ordered categorical data.

1.1 Definition A graphical model is defined by a set of assumptions concerning pairwise conditional independence given the rest of the variables of the model. Consider, for instance, a model containing six variables, A to F. The following set of assumptions concerning pairwise conditional independence defines four constraints for the joint distribution Pr(A, B, C, D, E, F). The family of probability distributions satisfying these constraints is a graphical model: A ⊥ C|BDEF ⇔ Pr(A, C|BDEF) = Pr(A|BDEF) Pr(C|BDEF), A ⊥ D|BCEF ⇔ Pr(A, D|BCEF) = Pr(A|BCEF) Pr(D|BCEF),

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INTERACTION MODEL

Graphical models for multidimensional tables were first discussed by Darroch et al. (5). Since then, the models have been extended both to continuous and mixed categorical and continuous data and to regression and block recursive models. Whittaker (9), Edwards (7), Cox & Wermuth (4), and Lauritzen (8) present different accounts of the theory of graphical models. The sections below summarize some of the main results from this theory. Figure 1. An interaction graph

1.2 The Separation Theorem B ⊥ E|ACDF ⇔ Pr(B, E|ACDF) = Pr(B|ACDF) Pr(E|ACDF), C ⊥ E|ABDF ⇔ Pr(C, E|ABDF) = Pr(C|ABDF) Pr(E|ABDF).

Interaction models defined by conditional independence constraints are called ‘‘graphical interaction models’’, because the structure of these models can be characterized by so-called interaction graphs, where variables are represented by nodes connected by undirected edges if and only if association is permitted between the variables. The graph shown in Figure 1 corresponds to the set of conditional independence constraints above, because there are no edges connecting A to C, A to D, B to E, and C to E. Interaction graphs are visual representations of complex probabilistic structures. They are, however, also mathematical models of these structures, in the sense that one can describe and analyze the interaction graphs by concepts and algorithms from mathematical graph theory and thereby infer properties of the probabilistic model. This connection between probability theory and mathematical graph theory is special to the graphical models. The key notion here is conditional independence, as discussed by Dawid (5). While the above definition requires that the set of conditioning variables always includes all the other variables of the model, the results described below imply that conditional independence may sometimes be obtained if one conditions with certain subsets of variables.

The first result connects the concept of graph separation to conditional independence. First, we present a definition: a subset of nodes in an undirected graph separate two specific nodes, A and B, if all paths connecting A and B intersect the subset. In Figure 1, (B, D, F) separate A and B, as does (B, E, F). E and C are separated by both (A, D, F) and (B, D, F). The connection between graph separation and conditional independence is given by the following result, sometimes referred to as the separation theorem. 1.2.1 Separation Theorem. If variables A and B are conditionally independent given the rest of the variables of a multivariate model, A and B will be conditionally independent given any subset of variables separating A and B in the interaction graph of the model. The four assumptions on pairwise conditional independence defining the model shown in Figure 1 generate six minimal separation hypotheses: A ⊥ C|BDF,

A ⊥ C|BEF,

A ⊥ D|BEF,

B ⊥ E|ADF,

C ⊥ E|ADF,

C ⊥ E|BDF.

1.3 Closure and Marginal Models It follows from the separation theorem that graphical models are closed under marginalization, in the sense that some of the independence assumptions defining the model transfer to marginal models. Collapsing, for instance, over variable C of the model shown in Figure 1 leads to a

INTERACTION MODEL

3

been deleted define a hierarchial loglinear model with parameters corresponding to each of the completely connected subsets of nodes in the graph. The interaction graph for the model shown in Figure 1 has four cliques, BCDF, ABF, AEF, and DEF, corresponding to a loglinear model defined by one four-factor interaction and three three-factor interactions. 1.5 Separation and Parametric Collapsibility

Figure 2. An interaction graph obtained by collapsing the model defined by Figure 1 over variable C

graphical model defined by conditional independence of A and D and B and E, respectively, because the marginal model contains separators for both AD and BE (Figure 2). 1.4 Loglinear Representation of Graphical Models for Categorical Data No assumptions have been made so far requiring variables to be categorical. If all variables are categorical, however, the results may be considerably strengthened both with respect to the type of model defined by the independence assumptions of graphical models and in terms of the available information on the marginal models. The first published results on graphical models (5) linked graphical models for categorical data to loglinear models: A graphical model for a multidimensional contingency table without structural zeros is loglinear with generators defined by the cliques of the interaction graph.

The result is an immediate result of the fact that any model for a multidimensional contingency table has a loglinear expansion. Starting with the saturated model, one removes all loglinear terms containing two variables assumed to be conditional independent. The loglinear terms remaining after all the terms relating to one or more of the independence assumptions of the model have

While conceptually very simple, graphical models are usually complex in terms of loglinear structure. The problems arising from the complicated parametric structure are, however, to some degree to be compensated for by the properties relating to collapsibility of the models. Parametric collapsibility refers to the situation in which model terms of a complete model are unchanged when the model is collapsed over one or more variables. Necessary conditions implying parametric collapsibility of loglinear models are described by Agresti [1, p. 151] in terms which translate into the language of graphical models: Suppose variables of a graphical model of a multidimensional contingency table are divided into three groups. If there are no edges connecting variables the first group with connected components of the subgraph of variables from the third group, then model terms among variables of the first group are unchanged when the model is collapsed over the third group of variables.

Parametric collapsibility is connected to separation in two different ways. First, parametric collapsibility gives a simple proof of the separation theorem, because a vanishing two-factor term in the complete model also vanishes in the collapsed model if the second group discussed above contains the separators for the two variables. Secondly, separation properties of the interaction graph may be used to identify marginal models permitting analysis of the relationship between two variables. If one first removes the edge between the two variables, A and B, and secondly identifies separators for A and B in the graph, then the model is seen to be parametric collapsible on to the model containing A

4

INTERACTION MODEL

and B and the separators with respect to all model terms relating to A and B. The results are illustrated in Figure 3, where the model shown in Figure 3(a) is collapsed on to marginal models for ABCD and CDEF. The separation theorem is illustrated in Figure 3(b). All terms relating to A and B vanish in the complete model. The model satisfies the condition for parametric collapsibility, implying that these parameters also vanish in the collapsed model. The second property for the association between E and F is illustrated in Figure 3(c). C and D separate E and F in the graph from which the EF edge has been removed. It follows, therefore, that E and F cannot be linked to one and the same connected component of the subgraph for the variables over which the table has been collapsed. The model is therefore parametric collapsible on to CDEF with respect to all terms pertaining to E and F. 1.6 Decomposition and Reducibility Parametric collapsibility defines situations in which inference on certain loglinear terms may be performed in marginal tables because these parameters are unchanged in the marginal tables. Estimates of, and test statistics for, these parameters calculated in the marginal tables will, however, in many cases differ from those obtained from the complete table. Conditions under which calculations give the same results may, however, also be stated in terms of the interaction graphs. An undirected graph is said to be reducible if it partitions into three sets of nodes—X, Y, and Z—if Y separates the nodes of X from those of Z and if the nodes of Y are completely connected. If the interaction graph meets the condition of reducibility, it is said to decompose into two components, X + Y and Y + Z. The situation is illustrated in Figure 4, which decomposes into two components, ABCD and CDEF. It is easily seen that reducibility above implies parametric collapsibility with respect to the parameters of X and Z, respectively. It can also be shown, however, that likelihoodbased estimates and test statistics obtained by analysis of the collapsed tables are exactly the same as those obtained from the complete table.

Figure 3. Collapsing the model given in (a) illustrates the separation theorem for A and B (b), and parametric collapsibility with respect to E and F (c)

1.7 Regression Models and Recursive Models So far, the discussion has focused on models for the joint distribution of variables. The models can, however, without any problems, be extended first to multidimensional regression models describing the conditional distribution of a vector of dependent variables given another vector of explanatory variables and, secondly, to block recursive systems of variables. In the first case, the model will be based on independence assumptions relating to either two dependent variables

INTERACTION MODEL

5

associations. For categorical data, the natural measures of association are measures based on the so-called cross product ratios (2). The question therefore reduces to a question of whether or not cross product ratios are constant across different levels of other variables, thus identifying loglinear models as the natural framework within which these problems should be studied.

3

Figure 4. An interaction graph of a reducible model

or one dependent and one independent variable. In the second case, recursive models have to be formulated as a product of separate regression models for each recursive block conditionally given variables in all prior blocks. To distinguish between symmetric and asymmetric relationships edges between variables in different recursive blocks, interaction graphs are replaced by arrows. 2 PARAMETRIC STRUCTURE: HOMOGENEOUS OR HETEROGENEOUS ASSOCIATION The limitations of graphical models for contingency tables lie in the way in which they deal with higher-order interactions. The definition of the graphical models implies that higher-order interactions may exist if more than two variables are completely connected. It is therefore obvious that an analysis of association by graphical models can never be anything but the first step of an analysis of association. The graphical model will be useful in identifying associated variables and marginal models where associations may be studied, but sooner or later one will have to address the question of whether or not these associations are homogeneous across levels defined by other variables and, if not, which variables modify the association. The answer to the question of homogeneity of associations depends on the type of measure that one uses to describe or measure

ORDINAL CATEGORICAL VARIABLES

In the not unusual case of association between ordinal categorical variables, the same types of argument apply against the hierarchical loglinear models as against the graphical models. Loglinear models are basically interaction models for nominal data; and, as such, they will give results that are too crude and too imprecise for ordinal categorical data. The question of whether or not the association between two variables is homogeneous across levels of conditioning variables can, for ordinal variables, be extended to a question of whether or not the association is homogeneous across the different levels of the associated variables. While not abandoning the basic loglinear association structure, the answer to this question depends on the further parameterization of the loglinear terms of the models. We refer to a recent discussion of these problems by Clogg & Shihadeh (3).

4

DISCUSSION

The viewpoint taken here on the formulation of interaction models for categorical data first defines the family of graphical models as the basic type of models for association and interaction structure. Loglinear models are, from this viewpoint, regarded as parametric graphical models, meeting certain assumptions on the nature of associations not directly captured by the basic graphical models. Finally, different types of models for ordinal categorical data represent yet further attempts to meet assumptions relating specifically to the ordinal nature of the variables.

6

INTERACTION MODEL

REFERENCES 1. Agresti, A. (1990). Categorical Data Analysis. Wiley, New York. 2. Altham, P. M. E. (1970). The measurement of association of rows and columns for an r × s contingency table, Journal of the Royal Statistical Society, Series B 32, 63–73. 3. Clogg, C. & Shihadeh, E. S. (1994). Statistical Models for Ordinal Variables. Sage, Thousand Oaks. 4. Cox, D. R. & Wermuth, N. (1996). Multivariate Dependencies. Models, Analysis and Interpretation. Chapman & Hall, London. 5. Darroch, J. N., Lauritzen, S. L. & Speed, T. P. (1980). Markov fields and log-linear models for contingency tables, Annals of Statistics 8, 522–539.

6. Dawid, A. P. (1979). Conditional independence in statistical theory, Journal of the Royal Statistical Society, Series B 41, 1–15. 7. Edwards, D. (1995). Introduction to Graphical Modelling. Springer-Verlag, New York. 8. Lauritzen, S. L. (1996). Graphical Models. Oxford University Press, Oxford. 9. Whittaker, J. (1990). Graphical Models in Applied Multivariate Statistics. Wiley, Chichester.

INTERIM ANALYSES

The alternative, and possibly the most familiar, purpose for interim analyses in a clinical trial is to allow interim assessment of treatment differences. Valid interim comparisons of the treatments and their use in deciding whether to stop a trial will be the main focus of the rest of this article. When using the term interim analysis hereafter, it is assumed that treatment comparisons are being conducted. The traditional approach to conducting phase III clinical trials has been to calculate a single fixed sample size in advance of the study, which depends on specified values of the significance level, power, and the treatment advantage to be detected. Data on all patients are then collected before any formal analyses are performed. Such a framework is logical when observations become available simultaneously, as in an agricultural field trial; but it may be less suitable for medical studies, in which patients are recruited over time and data are available sequentially. Here, results from patients who enter the trial early on are available for analysis while later patients are still being recruited. It is natural to be interested in such results. ‘‘However, the handling of treatment comparisons while a trial is still in progress poses some tricky problems in medical ethics, practical organisation and statistical analysis’’ (see Reference 3, Section 10.1), as discussed below. In methodological terms, the approach presented in this article is known as the frequentist approach and is the most widely used framework for clinical trials. An alternative school of thought, not discussed here but mentioned for completeness, is the Bayesian approach as described by, for example, Spiegelhalter et al. (4).

SUSAN TODD The University of Reading Medical and Pharmaceutical Statistics Research Unit Reading, Berkshire, United Kingdom

1

INTRODUCTION

The term ‘‘interim analysis’’ can, in its broadest sense, be used in relation to any evaluation of data undertaken during an ongoing trial. Whether examination of the data presents ethical and analytical challenges depends on the purpose of the analysis. Some routine monitoring of trial progress, usually blinded to treatment allocation, is often undertaken as part of a phase III trial. This monitoring can range from simple checking of protocol compliance and the accurate completion of record forms to monitoring adverse events in trials of serious conditions so that prompt action can be taken. Such monitoring may be undertaken in conjunction with a data and safety monitoring board (DSMB), established to review the information collected. As no direct comparison of the treatments in terms of their benefit is undertaken at such interim analyses, special methodology is not required. A second purpose of interim analyses is to undertake a sample size review, when the purpose of the interim analysis is to estimate one or more nuisance parameters (for example, σ 2 in the case of normally distributed data), and this information is used to determine the sample size required for the remainder of the trial. Sample size re-estimation based on the estimation of nuisance parameters, particularly on the variance of normally distributed observations, was proposed by Gould and Shih (1). A review of the methodology is given by Gould (2). A sample size review can be undertaken using data pooled over treatments in order to avoid any breaking of blindness. Simulations reported in the papers cited above show that there is negligible effect on the statistical properties of the overall test and so it is usually ignored.

2 OPPORTUNITIES AND DANGERS OF INTERIM ANALYSES The most compelling reason for monitoring trial data for treatment differences at interim analyses is that, ethically, it is desirable to terminate or modify a trial when evidence has emerged that one treatment is clearly

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

INTERIM ANALYSES

superior to the other, which is particularly important when life-threatening diseases are being studied. Alternatively, the data from a trial may support the conclusion that the experimental treatment and the control do not differ by some pre-determined clinically relevant magnitude, in which case it is likely to be desirable, both ethically and economically, to stop the study for futility and divert resources elsewhere. Finally, if information in a trial is accruing more slowly than expected, perhaps because the actual event or response rate observed in a trial is less than that anticipated when the trial was planned, then extension of enrollment may be appropriate, until a large enough sample has been recruited. Unfortunately, multiple analyses of accumulating data lead to problems in the interpretation of results (5). The main problem occurs when significance testing is undertaken at the various interim looks. Even if the treatments are actually equally effective, the more often one analyzes the accumulating data, the greater the chance of eventually and wrongly detecting a difference. Armitage et al. (6) were the first to numerically compute the extent to which the type I error probability (the probability of incorrectly declaring the experimental treatment as different from the control) is increased over its intended level if a standard hypothesis test is conducted at that level at each of a series of interim analyses. They studied the simplified setting of a comparison of normal data with known variance and set the significance level (or type I error probability) for each analysis to be 5%. If just one analysis is performed, then the error rate is 5% as planned. If one interim analysis and one final analysis are performed, the error rate rises to 8%. If four interim analyses and a final analysis are undertaken, this figure is 14%. This inflation continues if more interim analyses are performed. In order to make use of the advantages of monitoring the treatment difference, methodology is required to maintain the overall type I error rate at the planned level. The term ‘‘sequential trial’’ is most commonly used to describe a trial that uses this special methodology. Such trials implement pre-specified stopping rules to accurately maintain error rates.

A second problem associated with the conduct of interim analyses that involve treatment comparisons concerns the final analysis of the trial. When data are inspected at interim looks, the analysis appropriate for fixed sample size studies is no longer valid. Quantities such as P-values, point estimates, and confidence intervals are still well defined, but new methods of calculation are required. If a traditional analysis is performed at the end of a trial that stops because the experimental treatment is found better than control, the P-value will be too small (too significant), the point estimate too large, and the confidence interval too narrow. Again, special techniques are required. 3 THE DEVELOPMENT OF TECHNIQUES FOR CONDUCTING INTERIM ANALYSES It was in the context of quality control experiments that special methods for conducting interim analyses were first implemented. Manufactured items are often inspected one at a time, with a view to accepting or rejecting a batch in terms of its quality. Double sampling, or in the terms of this article, the use of a single interim analysis, was introduced to quality control by Dodge and Romig (7). The Second World War then provided the setting for the development of the sequential probability ratio test by Wald (8) in the United States and by Barnard (9) in the United Kingdom. In this procedure, an interim analysis is conducted after the inspection of every individual item. Sequential medical plans for comparing two sets of binary responses were introduced by Bross (10), and Kilpatrick and Oldham (11) applied the sequential t-test to a comparison of bronchial dilators. By 1960, enough accumulated theory and practice existed for the first edition of Armitage’s book on Sequential Medical Trials. During the 1970’s work in this area prescribed designs whereby traditional test statistics, such as the t-statistic or the chisquared statistic, were monitored after each patient’s response was obtained. Examples can be found in the book by Armitage (12). The principal limitation was, obviously, the need to perform an interim analysis so frequently, which was eventually overcome to

INTERIM ANALYSES

allow interim analyses to be performed after groups of patients. Work by Pocock (13) and O’Brien and Fleming (14) allowed inspections after the responses from predefined sets of patients. A more flexible approach leading on from these early repeated significance tests, referred to as the alpha-spending method, was proposed by Lan and DeMets (15) and extended by Kim and DeMets (16). An alternative flexible method, sometimes referred to as the boundaries approach, encompasses a collection of designs based on straight-line boundaries, which builds on work that has steadily accumulated since the 1940s. This approach is discussed by Whitehead (17), and the best known and most widely implemented design within this framework is the triangular test (18). 4 METHODOLOGY FOR INTERIM ANALYSES In his 1999 paper, Whitehead (19) lists the key ingredients required to conduct a trial incorporating interim analyses for assessing treatment differences: • A parameter that expresses the advan-

tage of the experimental treatment over control, which is an unknown population characteristic about which hypotheses will be posed and of which estimates will be sought. • A statistic that expresses the advantage of experimental over control apparent from the sample of data available at an interim analysis, and a second statistic that expresses the amount of information about the treatment difference contained in the sample. • A stopping rule that determines whether the current interim analysis should be the last, and, if so, whether it is to be concluded that the experimental is better than or worse than control or that no treatment difference has been established. • A method of analysis, valid for the specific design used, giving a P-value and point and interval estimates at the end of the trial.

3

The first two ingredients are common to both fixed sample size and sequential studies, but are worth revisiting for completeness. The second two are solutions to the particular problems of preserving error rates and obtaining a valid analysis on completion of a study in which interim analyses have been conducted. Any combination of choices for the four ingredients is possible, but, largely for historical reasons, particular combinations preferred by authors in the field have been extensively developed, incorporated into software, and used in practice. Each of the four ingredients will now be considered in turn. 4.1 The Treatment Effect Parameter As with a fixed sample size study, the first stage in designing a phase III clinical trial incorporating interim analyses is to establish a primary measure of efficacy. The authority of any clinical trial will be greatly enhanced if a single primary response is specified in the protocol and is subsequently found to show significant benefit of the experimental treatment. The choice of response should depend on such criteria as clinical relevance, familiarity to clinicians, and ease of obtaining accurate measurements. An appropriate choice for the associated parameter measuring treatment difference can then be made, which should depend on such criteria as interpretability, for example, whether a measurement based on a difference or a ratio is more familiar, and precision of the resulting analysis. If the primary response is a continuous measure such as the reduction in blood pressure after 1 month of antihypertensive medication, then the difference in true (unknown) means is of interest. If a binary variable is being considered, such as the occurrence (or not) of deep vein thrombosis following hip replacement, the log-odds ratio may be the parameter of interest. Finally, suppose that in a clinical trial the appropriate response is identified as survival time following treatment for cancer, then a suitable parameter of interest might be the log-hazard ratio. 4.2 Test Statistics for Use at Interim Analyses At each interim analysis, a sequential test monitors a statistic that summarizes the

4

INTERIM ANALYSES

current difference between the experimental treatment and control, in terms of the chosen parameter of interest. If the value of this statistic lies outside some specified critical values, the trial is stopped and appropriate conclusions can be drawn. The timing of the interim looks can be defined directly in terms of number of patients, or more flexibly in terms of information. It should be noted that the test statistic measuring treatment difference may increase or decrease between looks, but the statistic measuring information will always increase. Statisticians have developed flexible ways of conducting sequential trials allowing for the number and the timing of interim inspections. Whitehead (17) describes monitoring a statistic measuring treatment difference known in technical terms as the efficient score and schedules the interim looks in terms of a second statistic approximately proportional to study sample size known as observed Fisher’s information. Jennison and Turnbull (20) employ a direct estimate of the treatment difference itself as the test statistic of interest and record inspections in terms of a function of its standard error. Other choices are also possible. 4.3 Stopping Rules at Interim Analyses As highlighted above, a sequential test compares the test statistic measuring treatment difference with appropriate critical values at each interim analysis. These critical values form a stopping rule or boundary for the trial. At any interim analysis in the trial, if the boundary is crossed, the study is stopped and an appropriate conclusion drawn. If the statistic stays within the boundary, then not enough evidence exists to come to a conclusion at present and a further interim look should be taken. It is possible to look after every patient or to have just one or two interim analyses. When interims are performed after groups of patients, it may be referred to as a ‘‘group sequential trial.’’ A design incorporating inspections after every patient may be referred to as a ‘‘fully sequential test.’’ The advantage of looking after every patient is that a trial can be stopped as soon as an additional patient response results in the boundary being crossed. In

contrast, performing just one or two looks reduces the potential for stopping, and hence delays it. However, the logistics of performing interim analyses after groups of subjects are far easier to manage. In practice, planning for between four and eight interim analyses appears sensible. Once it had been established that a problem existed with inflating the type I error when using traditional tests and the usual fixed sample size critical values, designs were suggested that adjusted for this problem. It is the details of the derivation of the stopping rule that introduces much of the variety of sequential methodology. In any trial, the important issues to focus on are the desirable reasons for stopping or continuing a study at an interim analysis. Reasons for stopping may include evidence that the experimental treatment is obviously better than the control, evidence that the experimental treatment is already obviously worse than the control, or alternatively it may be established that little chance exists of showing that the experimental treatment is better than the control. Reasons for continuing may include belief that a moderate advantage of the experimental treatment is likely and it is desired to estimate this magnitude carefully or, alternatively, evidence may exist that the event rate is low and more patients are needed to achieve power. Criteria such as these will determine the type of stopping rule that is appropriate for the study under consideration. Stopping rules are now available for testing superiority, non-inferiority, equivalence, and even safety aspects of clinical trials. Some designs even aim to deal with both efficacy and safety aspects in a combined stopping rule, but these designs are more complex. As an example, consider a clinical trial conducted by the Medical Research Council Renal Cancer Collaborators between 1992 and 1997 (21). Patients with metastatic renal carcinoma were randomly assigned to treatment with either the hormone therapy, oral medroxyprogesterone acetate (MPA), or the biological therapy, interferon-α. The primary endpoint in the study was survival time and the treatment difference was measured by the log-hazard ratio. It was decided that if a difference in 2-year survival from 20%

INTERIM ANALYSES

on MPA to 32% on interferon-α (log-hazard ratio −0.342) was present, then a significant treatment difference at the two-sided 5% significance level should be detected with 90% power. The use of interferon-α was experimental and this treatment is known to be both costly and toxic. Consequently, its benefits over MPA needed to be substantial to justify its wider use. A stopping rule was required to satisfy the following two requirements: early stopping if data showed a clear advantage of interferon-α over MPA and early stopping if data showed no worthwhile advantage of interferon-α, which suggested use of an asymmetric stopping rule. Such a rule handles both of these aspects. The design chosen was the triangular test (17), similar in appearance to the stopping rule in Fig. 1. Interim analyses were planned every 6 months from the start of the trial. The precise form of the stopping rule was defined, as is the sample size in a fixed sample size trial, by consideration of the significance level, power, and desired treatment advantage, with reference to the primary endpoint. 4.4 Analysis following a Sequential Trial Once a sequential trial has stopped, a final analysis should be performed. The interim analyses determine only whether stopping should take place; they do not provide a complete interpretation of the data. An appropriate final analysis must take account of the fact that a sequential design was used. Unfortunately, many trials that have been terminated at an interim analysis are finally reported with analyses that take no statistical account of the inspections made (22). In a sequential trial, the meaning and interpretation of data summaries such as significance levels, point estimates, and confidence intervals remain as for fixed sample size trials. However, various alternative valid methods of calculation have been proposed. These methods can sometimes lead to slightly different results when applied to the same set of data. The user of a computer package may accept the convention of the package and use the resulting analysis without being concerned about the details of calculation. Those who wish to develop a deeper understanding

5

of statistical analysis following a sequential trial are referred to Chapter 5 of Whitehead (17) and Chapter 8 of Jennison and Turnbull (23). The generally accepted method is based on orderings methodology, whereby potential outcomes in a trial are ordered by degree of support for the alternative hypothesis. The original, and most successful, form of ordering was introduced by Fairbanks and Madsen (24) and explored further by Tsiatis et al. (25). 5 AN EXAMPLE: STATISTICS FOR LAMIVUDINE The effectiveness of lamivudine for preventing disease progression among patients with chronic hepatitis B and advanced fibrosis or cirrhosis is unknown and prompted the conduct of a large survival study (26). An efficient trial methodology for reaching a reliable conclusion with as few subjects as possible was required. The annual rate of disease progression was assumed to be 20% for placebo and a reduction of one-third (to 13.3%) for the lamivudine group was taken as a clinically relevant treatment effect. It was desired to detect this with power 0.9. A significance level of 0.05 was specified. When the objectives of the trial were considered in detail, interim analyses using an appropriate stopping rule were planned. The methodology followed was the boundaries approach, as discussed by Whitehead (17), and the triangular test was selected as an appropriate design. Clinically compensated chronic hepatitis B patients with histologically confirmed fibrosis or cirrhosis were randomized 2:1 to receive lamivudine (100 mg/day) or placebo for up to 5 years. Overall, 651 patients were randomized at 41 sites in nine Asia-Pacific countries, 436 to lamivudine and 215 to placebo. These people were then followed up for evidence of disease progression. An independent data and safety monitoring board was established to study the progress of the trial at interim analyses. At the first interim analysis, 36 patients had experienced disease progression. The first point plotted on Fig. 1 (x) represents those data. The statistic Z signifies the advantage seen so far on lamivudine and is the efficient score for the log-hazard ratio.

6

INTERIM ANALYSES

Z 25 20 15 10 5 0 −5

10

20

30

40

50

60

70

80

−10

90 V

−15 Figure 1. Sequential plot for the trial of lamivudine. Z: Efficient score statistic measuring the advantage of lamivudine over placebo. V: Statistic measuring the information on which the comparison is based.

Here, Z is calculated adjusting for 5 covariates, center, gender, baseline fibrosis staging score, baseline Childs-Pugh score, and baseline ALT values. An unadjusted Z would be the usual log-rank statistic for survival data. The statistic V measures the information on which the comparison is based, which is the variance of Z and is approximately one-quarter the number of events. The inner dotted boundaries, known as the Christmas tree correction for discrete looks, form the stopping boundary: reach this boundary and the trial is complete. Crossing the upper boundary results in a positive trial conclusion. At the second interim analysis, a total of 67 patients had experienced disease progression. The upper boundary was reached and the trial was stopped. When the results from data that had accumulated between the second interim analysis and confirmation of stopping were added, the final ‘‘overunning’’ point was added to the plot. A final overrunning analysis of the data was conducted using the computer package PEST. The P-value obtained was 0.0008, a highly significant result in favor of lamivudine. By using a series of interim looks, the design allowed a strong positive conclusion to be drawn after only 71 events had been observed.

6

INTERIM ANALYSES IN PRACTICE

Interim analyses are frequently being implemented in clinical trials. Early examples can

be found in the proceedings of two workshops, one on practical issues in data monitoring sponsored by the U.S. National Institutes of Health (NIH) held in 1992 (27) and the other on early stopping rules in cancer clinical trials held at Cambridge University in 1993 (28). The medical literature also demonstrates the use of interim analyses. Examples of such studies include trials of lithium gamolenate in pancreatic cancer (29), of viagra in erectile disfunction (30), and of implanted defibrilators in coronary heart disease (31), as well as the example detailed in Section 5 above and many others. Two books dealing exclusively with the implementation of sequential methods in clinical trials are those by Whitehead (17) and Jennison and Turnbull (23). In addition, three major commercial software packages are currently available. The package PEST (32) is based on the boundaries approach. The package EaSt (33) implements the alpha-spending boundaries of Wang and Tsiatis (34) and Pampallona and Tsiatis (35). An addition to the package S-Plus is the S + SeqTrial module (36). If a trial is to be a comparison of two treatments in respect to a single primary endpoint, with the objective of discovering whether one treatment is superior, non-inferior, or equivalent to the other, then it is extremely likely that a suitable sequential method exists, which means that infeasibility and unfamiliarity are no longer valid reasons for avoiding interim analyses and stopping rules in such trials, if ethical or economic purposes would be served by them.

INTERIM ANALYSES

When planning to include interim analyses in any clinical trial, the implications of introducing a stopping rule need to be thought out carefully in advance of the study. In addition, all parties involved in the trial should be consulted on the choice of clinically relevant difference, specification of an appropriate power requirement, and the selection of a suitable stopping rule. As part of the protocol for the study, the operation of any sequential procedure should be clearly described in the statistical section. Decision making as part of a sequential trial is both important and time sensitive. A decision taken to stop a study not only affects the current trial, but also is likely to affect future trials planned in the same therapeutic area. However, continuing a trial too long puts participants at unnecessary risk and delays the dissemination of important information. It is essential to make important ethical and scientific decisions with confidence. Wondering whether the data supporting interim analyses are accurate and up-to-date is unsettling and makes the decision process harder. It is, therefore, necessary for the statistician performing the interim analyses to have timely and accurate data. Unfortunately, a trade-off exists—it takes time to ensure accuracy. Potential problems can be alleviated if data for interim analyses are reported separately from the other trial data, as part of a fast-track system. Less data means that they can be validated more quickly. If timeliness and accuracy are not in balance, not only may real-time decisions be made on old data, but more seriously, differential reporting may lead to inappropriate study conclusions. If a DSMB is appointed, one of their roles should be to scrutinize any proposed sequential stopping rule before the start of the study and to review the protocol in collaboration with the trial Steering Committee. The procedure for undertaking the interim analyses should also be finalized in advance of the trial start-up. The DSMB would then review results of the interim analyses as they are reported. Membership of the DSMB and its relationship with other parties in a clinical trial has been considered in Reference 27 and by Ellenberg et al. (37). It is important that the interim results of an ongoing trial are not

7

circulated widely as it may have an undesirable effect on the future progress of the trial. Investigators’ attitudes will clearly be affected by whether a treatment looks good or bad as the trial progresses. It is usual for the DSMB to be supplied with full information and, ideally, the only other individual to have knowledge of the treatment comparison would be the statistician who performs the actual analyses. 7

CONCLUSIONS

Use of interim analyses in phase III clinical trials is not new, but it is probably the more recent theoretical developments, together with availability of software, which have precipitated their wider use. The methodology is flexible as it enables choice of a stopping rule from a number of alternatives, allowing the trial design to meet the study objectives. One important point is that a stopping rule should not govern the trial completely. If external circumstances change the appropriateness of the trial or assumptions made when choosing the design are suspected to be false, it can and should be overridden, with the reasons for doing so carefully documented. Methodology for conducting a phase III clinical trial sequentially has been extensively developed, evaluated, and documented. Error rates can be accurately preserved and valid inferences drawn. It is important that this fact is recognized and that individuals contemplating the use of interim analyses conduct them correctly. Regulatory authorities, such as the Food and Drug Administration (FDA), do not look favorably on evidence from trials incorporating unplanned looks at data. In the United States, the FDA (38) published regulations for NDAs, which included the requirement that the analysis of a phase III trial ‘‘assess . . . the effects of any interim analyses performed.’’ The FDA guidelines were updated by publication of ‘‘ICH Topic E9: Notes for Guidance on Statistical Principles for Clinical Trials’’ (39). Section 3 of this document discusses group sequential designs and Section 4 covers trial conduct including trial monitoring, interim analysis, early stopping, sample size adjustment, and the role of an independent DSMB. With such

8

INTERIM ANALYSES

acknowledgement from regulatory authorities, interim analyses are likely to become even more commonplace. REFERENCES 1. A. L. Gould and W. J. Shih, Sample size re-estimation without unblinding for normally distributed data with unknown variance. Commun. Stat. – Theory Meth. 1992; 21: 2833–3853. 2. A. L. Gould, Planning and revising the sample size for a trial. Stat. Med. 1995; 14: 1039–1051. 3. S. J. Pocock, Clinical Trials: A Practical Approach. New York: Wiley, 1983. 4. D. J. Spiegelhalter, K. R. Abrams, and J. P. Myles, Bayesian Approaches to Clinical Trials and Health-Care Evaluation. Chichester: Wiley, 2004. 5. K. McPherson, Statistics: the problem of examining accumulating data more than once. N. Engl. J. Med. 1974; 28: 501–502. 6. P. Armitage, C. K. McPherson, and B. C. Rowe, Repeated significance tests on accumulating data. J. Royal Stat. Soc. Series A 1969; 132: 235–244. 7. H. F. Dodge and H. C. Romig, A method of sampling inspection. Bell Syst. Tech. J. 1929; 8: 613–631. 8. A. Wald, Sequential Analysis. New York: Wiley, 1947. 9. G. A. Barnard, Sequential test in industrial statistics. J. Royal Stat. Soc. 1946;(Suppl 8): S1–S26. 10. I. Bross, Sequential medical plans. Biometrics 1952; 8: 188–205. 11. G. S. Kilpatrick and P. D. Oldham, Calcium chloride and adrenaline as bronchial dilators compared by sequential analysis. Brit. Med. J. 1954; ii: 1388–1391. 12. P. Armitage, Sequential Medical Trials, 2nd ed. Oxford: Blackwell, 1975. 13. S. J. Pocock, Group sequential methods in the design and analysis of clinical trials. Biometrika 1977; 64: 191–199. 14. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics 1979; 35: 549–556. 15. K. K. G. Lan and D. L. DeMets, Discrete sequential boundaries for clinical trials. Biometrika 1983; 70: 659–663. 16. K. Kim and D. L. DeMets, Design and analysis of group sequential tests based on the type I

error spending rate function. Biometrika 1987; 74: 149–154. 17. J. Whitehead, The Design and Analysis of Sequential Clinical Trials, rev. 2nd ed. Chichester: Wiley, 1997. 18. J. Whitehead, Use of the triangular test in sequential clinical trials. In: J. Crowley (ed.), Handbook of Statistics in Clinical Oncology. New York: Dekker, 2001, pp. 211–228. 19. J. Whitehead, A unified theory for sequential clinical trials. Stat. Med. 1999; 18: 2271–2286 20. C. Jennison and B. W. Turnbull, Group sequential analysis incorporating covariate information. J. Amer. Stat. Assoc. 1997; 92: 1330–1341. 21. Medical Research Council Renal Cancer Collaborators, Interferon-α and survival in metastatic renal carcinoma: early results of a randomised controlled trial. Lancet 1999; 353: 14–17. 22. K. M. Facey and J. A. Lewis, The management of interim analyses in drug development. Stat. Med. 1998; 17: 1801–1809. 23. C. Jennison and B. W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. Boca Raton, FL: Chapman and Hall/CRC, 2000. 24. K. Fairbanks and R. Madsen, P values for tests using a repeated significance test design. Biometrika 1982; 69: 69–74. 25. A. A. Tsiatis, G. L. Rosner, and C. R. Mehta, Exact confidence intervals following a group sequential test. Biometrics 1984; 40: 797–803. 26. Y. F. Liaw et al. on behalf of the CALM study group, Effects of lamivudine on disease progression and development of hepatocellular carcinoma in advanced chronic hepatitis B: a prospective placebo-controlled clinical trial. N. Engl. J. Med. 2004; 351: 1521–1531. 27. S. Ellenberg, N. Geller, R. Simon, and S. Yusuf (eds.), Proceedings of ‘Practical issues in data monitoring of clinical trials’. Stat. Med. 1993; 12: 415–616. 28. R. L. Souhami and J. Whitehead (eds.), Workshop on early stopping rules in cancer clinical trials. Stat. Med. 1994; 13: 1289–1499. 29. C. D. Johnson et al., Randomized, dose-finding phase III study of lithium gamolenate in patients with advanced pancreatic adenocarcinoma. Brit. J. Surg. 2001; 88: 662–668. 30. F. A. Derry et al., Efficacy and safety of oral sildenafil (viagra) in men with erectile dysfunction caused by spinal cord injury. Neurology 1998; 51: 1629–1633.

INTERIM ANALYSES 31. A. J. Moss et al., Improved survival with implanted defibrillator in patients with coronary disease at high risk of ventricular arrhythmia. N. Engl. J. Med. 1996; 335: 1933–1940. 32. MPS Research Unit, PEST 4: Operating Manual. Reading, UK: The University of Reading, 2000. 33. Cytel Software Corporation, EaSt 2000: A Software Package for the Design and Interim Monitoring of Group-Sequential Clinical Trials. Cambridge, MA: Cytel Software Corporation, 2000. 34. S. K. Wang and A. A. Tsiatis, Approximately optimal one-parameter boundaries for group sequential trials. Biometrics 1987; 43: 193–199. 35. S. Pampallona and A. A. Tsiatis, Group sequential designs for one-sided and twosided hypothesis testing with provision for early stopping in favor of the null hypothesis. J. Stat. Plan. Inference 1994; 42: 19–35.

9

36. MathSoft Inc., S-Plus 2000. Seattle, WA: MathSoft Inc., 2000. 37. S. Ellenberg, T. R. Fleming, and D. L. DeMets, Data Monitoring Committees in Clinical Trials: A Practical Perspective. Chichester: Wiley, 2002. 38. US Food and Drug Administration Guideline for the Format and Content of the Clinical and Statistical Sections of an Application. Rockville, MD: FDA. http://www.fda.gov/cder/guidance/statnda.pdf 1988. 39. International Conference on Harmonisation Statistical Principles for Clinical Trials, Guideline E9. http://www.ich. org/LOB/media/MEDIA485.pdf 1998.

INTERIM CLINICAL TRIAL/STUDY REPORT An Interim Clinical Trial/Study Report is a report of intermediate results and their evaluation based on analyses performed during the course of a trial.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

INTERNATIONAL CONFERENCE ON HARMONISATION OF TECHNICAL REQUIREMENTS FOR REGISTRATION OF PHARMACEUTICALS FOR HUMAN USE (ICH) The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) is a cooperative effort between the drug regulatory authorities and innovative pharmaceutical company professional organizations of the European Union, Japan, and the United States to reduce the need to duplicate the testing conducted during the research and development of new medicines. Through harmonization of the technical guidelines and requirements under which drugs for human use are approved within the participating nations, ICH members seek more economical use of human, animal, and material resources and the elimination of delay in availability of new drugs, while maintaining the quality, safety, and effectiveness of regulated medicines.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

International Studies of Infarct Survival (ISIS) The ISIS began in 1981 as a collaborative worldwide effort to evaluate the effects of several widely available and practical treatments for acute myocardial infarction (MI). The ISIS Collaborative Group randomized more than 134 000 patients into four large simple trials assessing the independent and synergistic effects of beta-blockers, thrombolytics, aspirin, heparin, converting enzyme inhibitors, oral nitrates, and magnesium in the treatment of evolving myocardial acute infarction (Table 1). More than 20 countries participated in these trials, which were coordinated worldwide by investigators in Oxford, England.

ISIS-1: Atenolol in Acute MI [1] Beta-blocking agents reduce the heart rate and blood pressure, as well as their product, inhibit the effects of catecholamines, and increase thresholds for ventricular fibrillation. Thus, it is not surprising that beta-blockers were among the first agents to be evaluated in randomized trials of evolving acute MI. Even by 1981, the available trials of beta-blocking agents for acute infarction were too small to demonstrate a significant benefit. However, based on an overview of the available evidence (see Meta-analysis of Clinical Trials), it was judged that the prevention of even one death per 200 patients treated with beta-blockers (see Number Needed to Treat (NNT)) would represent a worthwhile addition to usual care. Unfortunately, detecting such an effect would require the randomization of over 15 000 patients. It was toward this end that the First International Study of Infarct Survival (ISIS-1) trial was formed. Table 1 Trial

In a collaborative effort involving 245 coronary care units in 11 countries, the ISIS-1 trial randomized 16 027 patients with suspected acute MI to a regimen of intravenous atenolol versus no betablocker therapy. Patients assigned to active treatment received an immediate intravenous injection of 5–10 mg atenolol, followed by 100 mg/day orally for seven days. Similar agents were avoided in those assigned at random to no beta-blocker therapy unless it was believed to be medically indicated. As in the subsequent ISIS collaborations, all other treatment decisions were at the discretion of the responsible physician. During the seven-day treatment period in which atenolol was given, vascular mortality was significantly lower in the treated group (3.89% vs. 4.57%, P < 0.04), representing a 15% mortality reduction. Almost all of the apparent benefit was observed in days 0 to 1 during which time there were 121 deaths in the atenolol group as compared with 171 deaths in the control group. The early mortality benefit attributable to atenolol was maintained at 14 days and at the end of one year follow-up (10.7% atenolol vs. 12.0% control). Treatment did not appear to decrease infarct size substantially, although the ability of a large and simple trial such as ISIS-1 to assess such a reduction was limited. Despite its large size, the 95% confidence limits of the risk reductions associated with atenolol in ISIS-1 were wide and included relative risk reductions between 1% and 25%. However, an overview that included ISIS1 and 27 smaller completed trials of beta-blockade suggested a similar sized mortality reduction (14%). When a combined endpoint of mortality, nonfatal cardiac arrest and nonfatal reinfarction was considered from all available trials, the 10%–15% reduction persisted with far narrower confidence limits. Taken together, these data suggest that early treatment of 200 acute MI patients with beta-blocker therapy

The International Studies of Infarct Survival (ISIS) Year completed

ISIS-1 ISIS-2

1985 1988

ISIS-3

1991

ISIS-4

1993

Agents studied Atenolol vs. control Streptokinase vs. placebo Aspirin vs. placebo Streptokinase vs. tPA vs. APSAC Aspirin + SC heparin vs. aspirin Captopril vs. placebo oral mononitrate vs. placebo Magnesium vs. control

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

Patients randomized 16 027 17 187 41 299 58 050

2

International Studies of Infarct Survival (ISIS)

would lead to avoidance of one reinfarction, one cardiac arrest, and one death during the initial seven-day period. Unfortunately, beta-blocker use in the setting of acute MI remains suboptimal with utilization rates ranging between 30% in the US to less than 5% in the UK. This underutilization appears related in part to poor physician education. In the GUSTO1 trial, beta-blockers were encouraged by the study protocol and almost 50% of all patients received the drugs without any apparent increase in adverse effects.

ISIS-2: Streptokinase and Aspirin in Acute MI [2] As with beta-blockers, data from randomized trials of thrombolytic therapy completed prior to 1985 did not yield truly reliable results. Indeed, the largest of the early studies enrolled 750 patients, a totally inadequate sample size to detect the most plausible 20%–25% reduction in mortality. Given this situation, the Second International Study of lnfarct Survival (ISIS-2) was designed to test directly in a single randomized, double-blind, placebo-controlled trial (see Blinding or Masking) the risks and benefits of streptokinase and aspirin in acute MI. To accomplish this goal, the ISIS-2 collaborative group randomized 17 187 patients presenting within 24 hours of symptom onset using a 2 × 2 factorial design to one of four treatment groups: 1.5 million units of intravenous streptokinase over 60 minutes; 162.5 mg/day of oral aspirin for 30 days; both active treatments; or neither. In brief, the primary endpoint (see Outcome Measures in Clinical Trials) of the trial, total vascular mortality, was reduced 25% by streptokinase alone (95% Cl, −32 to −18, P < 0.0001) and 23% by aspirin alone (95% CI, −30% to −15%, P < 0.00001). Patients allocated to both agents had a 42% reduction in vascular mortality (95% Cl, −50 to −34, P < 0.00001), indicating that the effects of streptokinase and aspirin are largely additive. When treatment was initiated within six hours of the onset of symptoms, the reduction in total vascular mortality was 30% for streptokinase, 23% for aspirin, and 53% for both active agents. For aspirin, the mortality benefit was similar when the drug was started 0–4 hours (25%), 5–12 hours (21%), or 13–24 hours (21%) after the onset of

clinical symptoms. Aspirin use also resulted in highly significant reductions for nonfatal reinfarction (49%) and nonfatal stroke (46%). As regards side-effects, for bleeds requiring transfusion, there was no significant difference between the aspirin and placebo groups (0.4% vs. 0.4%), although there was a small absolute increase of minor bleeds among those allocated to aspirin (0.6%, P < 0.01). For cerebral hemorrhage, there was no difference between the aspirin and placebo groups. For streptokinase, those randomized within four hours of pain onset experienced the greatest mortality reduction, although statistically significant benefits were present for patients randomized throughout the 24 hour period. As expected, there was an excess of confirmed cerebral hemorrhage with streptokinase (7 events vs. 0; 2P < 0.02), all of which occurred within one day of randomization. Reinfarction was slightly more common among those assigned streptokinase alone, but this difference was not statistically significant. Furthermore, aspirin abolished the excess reinfarction attributable to streptokinase. In addition to demonstrating the independent as well as synergistic effects of streptokinase and aspirin, ISIS-2 also supplied important information concerning which patients to treat. Because the ISIS-2 entry criteria were broad, the trial included the elderly, patients with left bundle branch block, and those with inferior as well as anterior infarctions. In each of these subgroups, clear mortality reductions were demonstrated. Thus, in addition to changing radically the premise that thrombolysis should be avoided in patients already on aspirin, the ISIS-2 trial was largely responsible for widening the eligibility criteria for patients who would benefit from thrombolytic therapy.

ISIS-3: Streptokinase vs. APSAC vs. tPA and Subcutaneous Heparin vs. No Heparin in Acute MI [3] While ISIS-2 (streptokinase), the first Gruppo Italiano per lo Studio della Sopravvivenza nell’Infarto miocardico (GISSI-1, streptokinase), the APSAC Intervention Mortality Study (AIMS, anisoylated plasminogen streptokinase activator complex [APSAC]), Anglo–Scandinavian Study of Early Thrombolysis (ASSET, tissue plasminogen activator [tPA]) and ISIS-2 all documented clear mortality benefits for

International Studies of Infarct Survival (ISIS) thrombolysis, they did not provide information that allowed for directly comparing these agents. It was also unclear whether patients given aspirin would further benefit from the addition of heparin. These questions were the focus of the Third International Study of Infarct Survival (ISIS-3). In brief, the ISIS-3 collaborative group randomized 41 299 patients to streptokinase, APSAC, and tPA. Patients presenting within 24 hours of the onset of evolving acute MI and with no clear contraindication to thrombolysis were assigned randomly to IV streptokinase (1.5 MU over one hour), IV tPA (duteplase, 0.50 million U/kg over four hours), or IV APSAC (30 U over three minutes). All patients received daily aspirin (162.5 mg), with the first dose crushed or chewed in order to achieve a rapid clinical antithrombotic effect. In addition, half were randomly assigned to receive subcutaneous heparin (12 500 IU twice daily for seven days), beginning four hours after randomization. ISIS-3 demonstrated no differences in mortality between the three thrombolytic agents. Specifically, among the 13 780 patients randomized to streptokinase, there were 1455 deaths (10.5%) within the initial 35-day follow-up period as compared with 1448 deaths (10.6%) among the 13 773 patients randomized to APSAC and 1418 deaths (10.3%) among the 13 746 randomized to tPA. Long-term survival was also virtually identical for the three agents at both three and six months. With regard to in-hospital clinical events, cardiac rupture, cardiogenic shock, heart failure requiring treatment, and ventricular fibrillation were similar for the three agents. For nonfatal reinfarction, there was a reduction with tPA, while streptokinase and APSAC allocated patients had higher rates of allergy and hypotension requiring treatment. Streptokinase produced fewer noncerebral bleeds than either APSAC or tPA. While there were no major differences between thrombolytic agents in terms of lives saved or serious in-hospital clinical events, significant differences were found in ISIS-3 for rates of total stroke and cerebral hemorrhage. Specifically, there were 141 total strokes in the streptokinase group as compared with 172 and 188 in the APSAC and tPA groups, respectively. For cerebral hemorrhage there were 32 events (two per 1000) in the streptokinase group as compared with 75 (five per 1000) in the APSAC group and 89 (seven per 1000) in the tPA group. While

3

the absolute rates for cerebral hemorrhage for all three agents was low, this apparent advantage for streptokinase was highly statistically significant (P < 0.0001 for streptokinase vs. APSAC, P < 0.00001 for streptokinase vs. tPA). With regard to the addition of delayed subcutaneous heparin to thrombolytics there was no reduction in the prespecified endpoint of 35 day mortality. During the scheduled seven day period of heparin use, there were slightly fewer deaths in the aspirin plus heparin group compared with the aspirin group alone, a difference of borderline significance. There was, however, a small but significant excess of strokes deemed definite or probable cerebral hemorrhages among those allocated aspirin plus heparin (0.56% vs. 0.40%, P < 0.05). In contrast, reinfarction was more common among those randomized to aspirin alone as compared with those receiving aspirin plus subcutaneous heparin.

ISIS-4: Angiotensin Converting Enzyme Inhibition, Nitrate Therapy, and Magnesium in Acute MI [4] In 1991 the ISIS collaboration chose to investigate several other promising but unproven approaches to the treatment of acute MI. Specifically, the Fourth International Study of Infarct Survival (ISIS-4) sought to examine treatment strategies that would benefit both high- and low-risk patients presenting with acute MI, not simply those who are eligible for thrombolysis. To attain this goal, the ISIS collaborative group chose to study three promising agents: a twice daily dose of the angiotensin converting enzyme (ACE) inhibitor captopril for 30 days, a once daily dose of controlled release mononitrate for 30 days, and a 24-hour infusion of intravenous magnesium. As was true in each of the preceding ISIS trials, the available data were far too limited to allow reliable clinical recommendations concerning these therapies. For example, while ACE inhibiting agents had been shown to be successful in reducing mortality in patients with congestive heart failure and in patients a week or two past acute infarction, it was unclear whether these agents provided a net benefit for all patients in the setting of evolving acute MI. Similarly, while nitrates were often used in evolving MI because of their ability to reduce myocardial

4

International Studies of Infarct Survival (ISIS)

afterload and potentially limit infarct size, barely 3000 patients had received intravenous nitroglycerin in randomized trials and even fewer patients had been studied on oral nitrate preparations. Finally, because of its effects on calcium regulation, arrhythmia thresholds, and tissue preservation, magnesium therapy had often been considered as an adjunctive therapy for acute infarction even though no data from a randomized trial of even modest size had been available. Based on statistical overviews, the ISIS investigators estimated that each of these therapies had the potential to reduce mortality in acute infarction by as much as 15%–20%. However, because many patients presenting with acute infarction were treated with thrombolytic therapy and aspirin, the estimated mortality rates at one month were estimated to be as low as 7%–8%. Thus, to assess reliably whether these potentially important clinical effects were real required the randomization of a very large number of patients, perhaps as many as 60 000. To achieve this goal, a 2 × 2 × 2 factorial design was employed in which patients were randomized first to captopril or captopril placebo, then to mononitrate or mononitrate placebo, and then to magnesium or magnesium control. Thus, it was possible in the trial for any given patient to receive all three active agents, no active agents, or any combination.

Captopril Use of the ACE inhibitor captopril was associated with a significant 7% decrease in five-week mortality (2088 [7.19%] deaths among patients assigned to captopril vs. 2231 [7.69%] deaths among those assigned to placebo), which corresponds to an absolute difference of 4.9 ± 2.2 fewer deaths per 1000 patients treated for one month. The absolute benefits appeared to be larger (possibly as high as 10 fewer deaths per 1000) in some higher-risk groups, such as those presenting with heart failure or a history of MI. The survival advantage appeared to be maintained at 12 months. In terms of side-effects, captopril produced no excess of deaths on days 0–1, even among patients with low blood pressure at entry. It was associated with an increase of 52 patients per 1000 in hypotension considered severe enough to require termination of study treatment, of five per 1000 in reported cardiogenic shock, and of five per 1000 in some degree of renal dysfunction.

Mononitrate Use of mononitrate was not associated with any significant improvements in outcomes. There was no significant reduction in overall five-week mortality, nor were there reductions in any subgroup examined (including those not receiving short-term nonstudy intravenous or oral nitrates at entry). Continued follow-up did not indicate any later survival advantage. Somewhat fewer deaths on days 0–1 were reported among individuals allocated to active treatment, which is reassuring about the safety of using nitrates early in evolving acute MI. The only significant side-effect of the mononitrate regimen was an increase in hypotension of 15 per 1000 patients.

Magnesium As with mononitrate, use of magnesium was not associated with any significant improvements in outcomes, either in the entire group or any subgroups examined (including those treated early or late after symptom onset or in the presence or absence of fibrinolytic or antiplatelet therapies, or those at high risk of death). Further follow-up did not indicate any later survival advantage. In contrast to some previous small trials, there was a significant excess of heart failure with magnesium of 12 patients per 1000, as well as an increase of cardiogenic shock of five patients per 1000 during or just after the infusion period. Magnesium did not appear to have a net adverse effect on mortality on days 0–1. In terms of side-effects, magnesium was associated with an increase of 11 patients per 1000 in hypotension considered severe enough to require termination of the study treatment, of three patients per 1000 in bradycardia, and of three patients per 1000 in a cutaneous flushing or burning sensation. Because of its size, ISIS-4 provided reliable evidence about the effects of adding each of these three treatments to established treatments for acute MI. Collectively, GISSI-3, several smaller studies, and ISIS-4 have demonstrated that, for a wide range of patients without clear contraindications, ACE inhibitor therapy begun early in evolving acute MI prevents about five deaths per 1000 in the first month, with somewhat greater benefits in higher-risk patients. The benefit from one month of ACE inhibitor therapy persists for at least the first year. Oral nitrate

International Studies of Infarct Survival (ISIS) therapy, while safe, does not appear to produce a clear reduction in one-month mortality. Finally, intravenous magnesium was ineffective at reducing one-month mortality.

Conclusion Because of their simplicity, large size, and strict use of mortality as the primary endpoint, the ISIS trials have played a critical substantive role in establishing rational treatment plans for patients with acute MI. Methodologically, they have clearly demonstrated the utility of large simple randomized trials. Three principles guided the design and conduct of the ISIS trials. The first was the belief that a substantial public health benefit would result from the identification of effective, widely practical treatment regimens that could be employed in almost all medical settings, as opposed to those that can be administered only at specialized tertiary care facilities. For this reason, the ISIS investigations focused on strategies to decrease mortality which, in and of themselves, did not require cardiac catheterization or other invasive procedures for either diagnostic or therapeutic purposes. The second principle was that the benefits of truly effective therapies would be applicable to a wide spectrum of patients with diverse clinical presentations. Thus, the entry criteria for the ISIS trials were intentionally broad and designed to mimic the reality all health care providers encounter when deciding whether or not to initiate a given treatment plan. This is one reason that the ISIS trials focused on evolving acute MI in the view of the responsible physician. The third and perhaps most important principle was that most new therapies confer small to moderate benefits, on the order of 10%–30%. While such benefits on mortality are clinically very meaningful, these effects can be detected reliably only by randomized trials involving some tens of thousands of patients. Thus, the ISIS protocols were

5

streamlined to maximize randomization and minimize interference with the responsible physician’s choice of nonprotocol therapies and interventions. Nonetheless, by selectively collecting the most important entry and follow-up variables that relate directly to the efficacy or adverse effects of the treatment in question, the ISIS trials yielded reliable data for providing a rational basis for patient care. By limiting paperwork and not mandating protocol-driven interventions, the ISIS approach proved to be remarkably cost-effective. Indeed, the large ISIS trials were conducted at a small fraction of the usual cost of other smaller trials which, because of their inadequate sample sizes, failed to demonstrate either statistically significant effects or informative null results.

References [1]

[2]

[3]

[4]

ISIS-1 (First International Study of lnfarct Survival) Collaborative Group (1986). Randomised trial of intravenous atenolol among 16027 cases of suspected acute myocardial infarction: ISIS-1, Lancet 2, 57–65. ISIS-2 (Second International Study of lnfarct Survival) Collaborative Group (1998). Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17187 cases of suspected acute myocardial infarction: ISIS-2, Lancet 2, 349–360. ISIS-3 (Third International Study of lnfarct Survival) Collaborative Group (1992). ISIS-3: Randomised comparison of streptokinase vs. tissue plasminogen activator vs. anistreplase and of aspirin plus heparin vs. aspirin alone among 41299 cases of suspected acute myocardial infarction: ISIS-3. Lancet 339, 75–70. ISIS-4 (Fourth International Study of Infarct Survival) Collaborative Group (1995). ISIS-4: a randomised factorial trial assessing early oral captopril, oral mononitrate, and intravenous magnesium sulphate in 58 050 patients with suspected acute myocardial infarction: ISIS4. Lancet 345, 669–685.

CHARLES H. HENNEKENS & P.J. SKERRETT

Xij = ξ i +  ij where in the first treatment group, T1, the mean of ξ i were µ1 and in the other treatment group, T2, µ2 . The assumptions underlying the two-sample t-test hold. With N subjects randomly assigned to T1 and T2, the power to detect a deviation from the null hypothesis of equivalence of T1 and T2 using outcome measure O is the effect size:

INTER-RATER RELIABILITY HELENA CHMURA KRAEMER Stanford University, Palo Alto, California

1

DEFINITION

The classic model that underlies reliability stipulates that a rating of unit i, i = 1,2, . . . can be expressed as Xij = ξ i +  ij , where ξ i is that unit’s ‘‘true value’’ (i.e., that free of rater error), and εij is error made by the jth independent rater sampled from the population of raters (1). The inter-rater reliability coefficient is defined as:

δ(ρ) =

(µ1 − µ2 ) √ (µ1 − µ2 ) √ = ρ = ρδ (σx2 )1/2 σξ

(1)

Because the sample size necessary to achieve specified power for a two-sample t-test is inversely related to the square of the effect size, the sample size necessary to achieve given power is inversely related to the reliability coefficient. Thus, in a situation in which one would need 100 subjects per group with a perfectly reliability measure, one would need 200 subjects if the measure had a reliability of .5, and 500 subjects if the measure had a reliability of .2. Similarly, suppose two variables X and Y were to be correlated. Then, the product moment correlation coefficient between X and Y can be shown to be:

ρ = σ 2 ξ/σ 2 X where σ 2 ξ is the variance of the ξ i in the population of interest, and σ 2 X that of a single observation per subject. Thus, in a sense, reliability relates to signal-to-noise ratio where σ 2 ξ relates to ‘‘signal’’ and σ 2 X combines ‘‘signal’’ and ‘‘noise.’’ According to this definition, the reliability coefficient is zero if and only if subjects in the population are homogeneous in whatever X measures. This situation should almost never pertain when considering measures for use in randomized clinical trials. Consequently, testing the null hypothesis that ρ = 0 is virtually never of interest, although admittedly it is often observed in the research literature. Instead, the tasks of greatest interest to clinical research are (1) obtaining a confidence interval for ρ, (2) judging the adequate of ρ, and (3) considering how to improve ρ.

Corr(X, Y) = Corr(ξ , ν)(ρX ρY )1/2 Where ξ and ν are the ‘‘true values’’ of X and Y, and ρ X and ρ Y are their reliabilities, the attenuation of correlation is always associated with unreliability of measurement. In this case, the sample size needed to achieve a certain power in testing whether the observed correlation deviates from zero is proportional to the product of the two reliability coefficients. However, if the only effect of unreliability of measurement were to decrease the power of statistical tests, researchers could compensate for unreliability by increasing sample size. Increasing the sample size is not always feasible, and if feasible, it is not easy. Increasing sample size increases the time and cost of performing clinical trials, increasing the difficulty of obtaining adequate funding, and delaying the process of settling important clinical questions. Yet, all these processes

2 THE IMPORTANCE OF RELIABILITY IN CLINICAL TRIALS Concern about inter-rater reliability in randomized clinical trials stems from the fact that a lack of reliability results in a lack of precision in estimation of population parameters as well as a loss of power in testing hypotheses. For example, suppose the response to treatment in a clinical trial were given by

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INTER-RATER RELIABILITY

could be managed. However, the effect of unreliability of measurement also leads to attenuation of effect sizes. Thus, it is possible to increase sample size enough to declare the treatment difference statistically significant, only to have the effect size indicate that the treatment difference is unlikely to be of clinical significance.

randomly allocated to positions in the analysis to assure that any disagreement between raters observing one subject is attributed to error, and raters must be ‘‘blinded’’ to each other’s ratings to ensure independence of errors. It is the design of the reliability study that assures that what is estimated adequately indicates reliability.

3 HOW LARGE A RELIABILITY COEFFICIENT IS LARGE ENOUGH?

5 ESTIMATE OF THE RELIABILITY COEFFICIENT—PARAMETRIC

No standards guarantee that inter-rater reliability is ‘‘large enough,’’ although ‘‘rules of thumb’’ have been suggested. For example, ρ > .8 might be considered ‘‘almost perfect,’’ .6 to .8 ‘‘substantial, .4 to .6 ‘‘moderate,’’ .2 to .4 ‘‘low,’’ and under .2 ‘‘unacceptable’’ (2). In fact, relatively few clinical measurements have reliability above .8, and many with reliability under .4 (3,4). Moreover, many medical measures, even those in common use, have never been assessed for reliability.

When it is assumed that ξ i and  ij are independent of each other, the intraclass correlation coefficient (q.v.) is generally used to estimate ρ. In practice, the easiest approach is to use a two-way ANOVA (N subjects by M raters). Then

4 DESIGN AND ANALYSIS OF RELIABILITY STUDIES

1 + (M − 1)r 1 + (M − 1)ρ ≈ F(M−1),(M−1)(N−1) 1−r 1−ρ

The design to assess the inter-rater reliability of a measurement requires sampling N subjects from the population of interest, then sampling M independent (‘‘blinded’’) raters of each subject at the same time (to avoid additional variance caused by inconsistency of the subject’s responses over time) from the population of ratings to which the results are meant to apply. It is important to realize that the reliability of any measure may vary from one clinical population to another. For example, a diagnostic procedure may be reliable in a clinical population but unreliable in a community population. At the same time, a diagnostic procedure may be reliable when done by well-trained physicians in a clinical population but unreliable when done by technicians (or vice versa) in the same population. Consequently, having a representative sample of patients from the relevant population and a representative sample of raters from that relevant population is crucial. Because of the practical difficulty of having multiple simultaneous raters of a subject at one time, most often M equals 2. Raters are

Easier to use in applications is an extension of Fisher’s z-transformation. Here for M ≥ 2,

r=

Fs − 1 Fs + M − 1

where FS is the F-statistic to test for the effect of subjects (5). Under these assumptions:


zM (r1 ) = .5 ln

1 + (M − 1)r1 1 − r1



is approximately normally distributed with mean zM (ρ) and variance 1/(h − 1) where: h=

2(M − 1)(N − 1) M

In particular, when M = 2, this means that
.5 ln

1+r 1−r



is approximately normally distributed with mean 
1+ρ .5 ln 1−ρ and variance 1/(N − 2). If a product moment correlation coefficient between the two ratings exists, then the variance of the z-transformation would have been 1/(N − 3), which

INTER-RATER RELIABILITY

indicates the minor loss of power associated with using an intraclass correlation rather than a product moment correlation coefficient applied to data generated in a reliability design. With the above information, confidence intervals for the intraclass correlation coefficient can be computed, presented, and evaluated for adequacy. The intra-class correlation coefficient has generated some confusion. Several different forms of the intraclass do not, in general, lead to the same conclusion (6–9). However, in a reliability study, when the multiple raters per subject are randomly assigned to the m positions, all various forms do estimate the same population parameter, although not necessarily equally efficiently. The problem related to choice of intra-class correlation coefficient therefore need not confuse reliability estimation. 6 ESTIMATION OF THE RELIABILITY COEFFICIENT—NONPARAMETRIC The above distribution theory for the intraclass correlation coefficient is robust to deviations from the normality assumptions, but if there is serious deviation for ordinal X, a nonparametric alternative is related to Kendall’s Coefficient of Concordance, W (10). If the N ratings in the jth position (j = 1, 2, . . . , M) are rank-ordered (tied ranks averaged), application of the above ANOVA leads to a nonparametric inter-rater reliability coefficient rKI = (MW − 1)/(M − 1). The distribution of rKI is well approximated by that of r when some nonlinear transformation of X ij exists that satisfies the normality assumptions (11). Otherwise, bootstrap methods (12–14) might be used to obtain confidence intervals for the reliability coefficient. 7 ESTIMATION OF THE RELIABILITY COEFFICIENT—BINARY On example of what would be considered a very serious deviation from the assumptions that underlie the parametric intraclass correlation coefficient occurs if X ij is binary (0 or 1), for then the within subject variance

3

usually depends on the true value. In that situation, the intraclass kappa coefficient estimates the inter-rater reliability coefficient (15). To compute the intraclass kappa coefficient, the above ANOVA may be applied to the 0/1 data (16). It is the distribution of the resulting reliability coefficient (the intraclass kappa) that is affected, not its computation. For M > 2, bootstrap methods are recommended to obtain confidence intervals, for the distribution depends on unknown higher order moments of ξ i = Ej (X ji ). For M = 2, the asymptotic distribution of the intraclass kappa is known (16), but unless sample size is very large, bootstrap methods are still preferred.

8 ESTIMATION OF THE RELIABILITY COEFFICIENT—CATEGORICAL Finally, suppose that X ij represents more than two nonordered categories (ordered categories are ordinal data for which rKI is more suited.). An intraclass kappa appropriate for this situation estimates 

Pk (1 − Pk )κk /



Pk (1 − Pk )

where Pk is the probability of falling into category k, k = 1, 2, . . . , K, and κ k is the intraclass kappa coefficient for the binary variable in which each subject is classified either in category k or not in category k (17). However, as can be observed from this formula, it is possible that the multicategory intraclass kappa is near zero when some individual categories are nearly perfectly reliable, or conversely, that the multicategory intraclass kappa is high one when some rarer categories have near zero reliability. For this reason, for multicategory X ij , it is recommended that the intraclass kappa be computed and evaluated for each of the categories separately (18). Then, if some individual categories are unreliable and other acceptably reliable, attention can be focused on improving the reliability of the flawed categories, either by redefinition, perhaps by combining categories that are ill distinguished from each other, or by retraining raters.

4

INTER-RATER RELIABILITY

9 STRATEGIES TO INCREASE RELIABILITIY (SPEARMAN-BROWN PROJECTION) What if the reliability coefficient estimated by the above methods is not high enough to satisfy the standards set by the designers of the randomized clinical trial? Then, several strategies might be considered to improve the inter-rater reliability of a measure prior to its use in a RCT: better training of raters and clarification of the measurement system. Alternatively, one can always propose to average multiple independent ratings for each subject in the RCT (common practice with assay procedures in which three independent assay results are often averaged per tissue sample.). Under the classic model, the reliability of the average of m ratings per subject, ρ m , is given by the Spearman-Brown projection formula: ρm =

mρ (m − 1)ρ + 1

Thus, for example, to raise the reliability of a measure from ρ to ρ m , one would need the average of m independent raters for each subject, where: m=

10

ρm (1 − ρ) (1 − ρm )ρ

OTHER TYPES OF RELIABILITIES

Inter-observer reliability is one of several types of reliabilities that might be of interest, depending on which sources of error are of interest in determining the accuracy of a measurement. Two others of considerable importance are test-retest reliability and intra-rater reliability. For test-retest reliability, the multiple ratings per subjects are sampled over a span of time short enough that the subject characteristic of interest is unlikely to change but long enough that the subject inconsistency in expressing that characteristic can be included in the error (e.g., over several weeks for the diagnosis of a long-lasting condition). Once again, to ensure the independence of errors within each subject, the ratings are

preferably made by different raters. Generally, because the error included in testretest reliability combines both rater error and error caused by subject inconsistency, test-retest reliability is generally lower than is inter-observer reliability that includes only rater errors. Because inconsistency of subject’s expression of the characteristic of interest is often the major source of error, and thus of attenuation of power in statistical hypothesis testing, or attenuation of effect size in estimation, test-retest reliability provides greater assurance of accuracy of measurement than does inter-observer reliability. For intra-rater reliability, the multiple ratings per subject are obtained by multiple independent measurements made by the same rater on the same observation. For example, one might sample N cancer patients, obtain a tumor tissue sample from each, divide that sample into M subsamples, each mounted separately for pathological examination, without any subject labels. Then, one rater would examine all unlabeled MN tissue samples in random order and classify each into the type or stage of cancer. The question is to what extent the rater classifies the M tissue samples from each patient the same. The error of interest now develops only from the inconsistencies within a rater, which is one component of the inconsistencies from one rater to another that is reflected in inter-observer unreliability. Consequently, one would expect that intra-observer reliability would be higher than would be interobserver reliability, which, in turn, would be higher than test-retest reliability. The discrepancies between these reliability coefficients provide clues as to the sources of error of measurement: from inconsistencies within each rater, from inconsistencies between one rater and another, and from inconsistencies in the patient’s expression of the characteristic.

REFERENCES 1. F. M. Lord and M. R. Novick, Statistical Theories of Mental Test Scores. Reading, MA: Addison-Weslely Publishing Company, Inc., 1968.

INTER-RATER RELIABILITY 2. J. R. Landis and G. G. Koch, The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174.

CROSS-REFERENCES

3. L. M. Koran, The reliability of clinical methods, data and judgments, part 1. N. Engl. J. Med. 1975; 293: 642–646.

Boostrapping Categorical Variables Correlation Intraclass Correlation Coefficient

4. L. M. Koran, The reliability of clinical methods, data and judgments, part 2. N. Engl. J. Med. 1975; 293: 695–701. 5. E. A. Haggard, Intraclass Correlation and the Analysis of Variance. New York: Dryden Press, 1958. 6. J.J. Bartko, The intraclass correlation coefficient as a measure of reliability. Psychol. Reports 1966; 19: 311. 7. J. J. Bartko, Corrective note to: The Intraclass Correlation Coefficient as a Measure of 1974; 1974: 34. 8. J. J. Bartko, On various intraclass correlation reliability coefficients. Psychol. Reports 1976; 83: 762–765. 9. P. E. Shrout and J. L. Fleiss, Intraclass correlations: uses in assessing rater reliability. Psychol. Bull. 1979; 86: 420–428. 10. M. Kendall and J. D. Gibbons, Rank Correlation Methods, 5th ed. New York: Oxford University Press, 1990. 11. H. C. Kraemer, The small sample non-null properties of Kendall’s Coefficient of Concordance for normal populations. J. Am. Stat. Assoc. 1976; 71: 608–613. 12. C. E. Lunneforg, Estimating the correlation coefficient: the bootstrap. Psychol. Bull. 1985; 98: 209–215. 13. J. L. Rasmussen, Estimating correlation coefficients: bootstrap and parametric. Psychol. Bull. 1987; 101: 136–139. 14. M. J. Strube, Bootstrap Type I error rates for the correlation coefficient: an examination of alternate procedures. Psychol. Bull. 1988; 104: 290–292. 15. H. C. Kraemer, V. S. Periyakoil, and A. Noda tutorial in biostatistics: kappa coefficients in medical research. Stats. Med. 2002; 21: 2109–2129. 16. J. L. Fleiss, Statistical Methods For Rates and Proportions. New York: John Wiley & Sons, 1981. 17. H. C. Kraemer, Ramifications of a population model for k as a coefficient of reliability. Psychometrika 1979; 44: 461–472. 18. H. C. Kraemer, Measurement of reliability for categorical data in medical research. Stat. Methods Med. Res. 1992; 1: 183–199.

Analysis of Variance (ANOVA)

Kappa Statistic Reliability Analysis Repeated Measurements Type II Error (False Negative)

5

INTERVAL CENSORED

are still in use. For instance, a potentially biased analysis would result from using the Kaplan–Meier estimator or the Cox regression model after transforming the interval censored observations into the right censoring situation. Replacing intervals with their middle or maximal point, however, approximates the true results only in exceptional cases, for instance, when the observed intervals are generally small and when the accuracy needed in the specific problem is low.

THOMAS A GERDS and CAROLINA MEIER-HIRMER Institute for Medical Biometry and Medical Informatics, University Hospital Freiburg Center for Data Analysis and Modeling Freiburg, Germany

1

CENSORING

In this article, the response is always the time of an event, the occurrence of which becomes known at examination times. Some special cases shall be distinguished. One speaks of left censoring, if the event of interest occurred before the examination time and of right censoring if the event did not occur until the examination time. The situation with only one examination for each patient is called ‘‘case 1’’ interval censoring, and the resulting observations are often called current status data. Left and right censoring can be generalized to ‘‘case k’’ interval censoring for situations in which the information from exactly k examinations is available for each patient (k is a positive integer). Since in the clinical practice the number of examination times is typically different among patients, most frequently one has to deal with ‘‘mixed case’’ interval censoring. This term refers to situations in which some observations are exact event times, some are right or left censored, and others are really censored to intervals. It is important to emphasize that the name ‘‘interval censoring’’ is often used to generally describe data consisting of such a mixture. Using artificial data, it is demonstrated in Fig. 1 and Table 1 how interval censored observations can be obtained from the longitudinal data of the examination process. Note that although patients 2 and 4 have the same event time, the observed intervals differ considerably. A note of caution: Complicated censoring schemes arise in medical practice as well as in other fields (see the statistical literature). But the connection between the theoretical results and the applications is not yet well developed in all cases nor easily available. Moreover, computer programs are not generally available. As a result, ad hoc methods that may cause biased conclusions

2

CLASSIFICATION AND EXAMPLES

In this section, the types of interval censoring are classified and illustrated more formally. For the special cases of left and right censoring, see the article on survival analysis. ‘‘Case 1’’ interval censoring: The observations are also called current status data. Here the information of one examination is available for each patient, and it is known if the event occurred before or after the date of examination. As a result, the observations are either left censored or right censored. Crosssectional studies in clinical or epidemiological projects often result in current status data. Another example are tumor incidence data in animal experiments where independent individuals are exposed to carcinogens. Typically, a valid histological diagnosis of the onset of a tumor is only possible after death. Thus, the day of death is the only examination time revealing whether a tumor has grown. The limitation of detecting a tumor only if it exceeds a critical size is a technical reason for censoring. For tumor incident experiments, the censoring occurs because of the inability to measure the event of interest exactly (occult tumors). For cross-sectional studies, the censoring is introduced by the study design. ‘‘Case k’’ interval censoring: For each patient, the results from k examinations are available. As the same number of examinations are almost never available for all patients, ‘‘case k’’

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

INTERVAL CENSORED

Examination process I

II

III

IV V VI

Right

1

Interval 2

Left 3

Interval 4

Exact Time

5

0

12

30

42

54

Figure 1. Interval-censored observations (thick lines) corresponding to the artificial data in Table (1). Filled dots in the intervals represent the (unobservable) true event times. The respective type of censoring is marked on the left axis of the diagram. Although observations 2 and 4 have the same true event time, the observed intervals differ considerably.

Table 1. Illustration of How to Obtain Interval-Censored Observations from Hypothetical Examination Processes Examination Process Time

0

12

30

42

48

Event

NO

NO

NO

NO

NO

Time

0

12

42

–

–

Event

NO

NO

YES

–

–

Time

0

–

–

–

–

Event

YES

–

–

–

–

Time

0

12

30

54

–

Event

NO

NO

NO

YES

–

Time

0

30

–

–

–

Event

NO

YES

–

–

–

Patient 1

Patient 2

Patient 3

Patient 4

Patient 5

interval censoring (for k greater than one) occurs sparsely in the medical praxis. An example is the supervision of children learning to speak where learned words are collected on a fixed number of questionnaires for each child. However, if single questionnaires are missing for some children, and hence the number of available examinations differs, the data have to be considered as ‘‘mixed case’’ interval censored. ‘‘Mixed case’’ interval censoring: For each patient, the event status is known at a differing number of examination

True Event Time

Censored Observation

60

[48, ∞)

36

[12, 42]

− 10

( − ∞, 0]

36

[30, 54]

30

30

times. Therefore, ‘‘mixed case’’ interval censored data typically consist of a mixture of interval-censored and rightcensored observations. Sometimes they include left-censored observations and even exact time points, if the event happens to occur at the day of examination. An example for mixed case interval censoring is given by breast cancer studies where the event of interest is the first occurrence of breast retraction. Shortly after therapy, which can be a combination of radiotherapy and chemotherapy, the time interval between two adjacent examinations is

INTERVAL CENSORED

typically small but lengthens as the recovery progresses. The exact time of retraction is only known to fall into the interval between two visits or after the last examination time. Double interval censoring: If the variable of interest is the duration between two events, then both the start point and the endpoint of the duration can be interval censored. A well-known example is the incubation time between HIV-infection and AIDS diagnosis: The infection time is interval censored between the last negative and the first positive antibody test, and the time to AIDS diagnosis is right censored, when AIDS is not diagnosed within the study time. Double censoring: Double censoring refers to situations in which the observations are either exact event times, left censored, or right censored. Double censoring is therefore a special case of mixed case interval censoring. Interval censored covariates: In the framework of multistate models, the influence of an intermediate event on the main outcome variable is sometimes of interest. For instance, if the endpoint of interest is death, then the recurrence of a tumor can be an influential intermediate event. In this example, the occurrence of the intermediate event is an interval-censored covariate. Situations in which the time of the intermediate event is interval censored occur frequently in such frameworks, in particular for illnessdeath models.

3 STUDY DESIGN AND STATISTICAL MODELING Considering interval-censored data resulting from periodic follow-up in a clinical trial or longitudinal study, the information is generally the greater the smaller the intervals are between adjacent examinations. The length of the observed intervals evidently influences the power of statistical methods, and it is important for the significance of statistical

3

analysis to gather as much information as possible. However, acquisition on a dense time schedule or even continuous in time can be prohibited for various reasons. Financial costs or stress of patients are typical factors that limit the accuracy of measurement. The statistical model building for censored data proceeds in two steps: In a first step, the survival model is specified, that is, a family of probability distributions that includes the underlying survival function. The second step deals with the dependence of the event time on the censoring scheme. Sometimes it is important to specify a probabilistic model also for the distribution of the censoring process. For convenience, it is often assumed that the censoring is independent of the event time. Survival models can be parametric, semiparametric, or nonparametric. Besides the differences of the statistical methods used for the three model classes, one must be aware of the well-known tradeoff between bias and variance, which may differ among the three approaches: On the one hand, if the parametric model assumptions are violated, the statistical inference can be misleading due to biased estimates. On the other hand, with semiparametric or nonparametric models, the power of the statistical methods is typically low for small or moderate sample sizes that are often present in clinical studies. In the presence of covariates, a suitable survival regression model has to be specified. For instance, the proportional hazards model, the proportional odds model, and the accelerated failure time model are frequently used regression models that have extensions to interval-censored data structures. The task of modeling the censoring mechanism has a similar impact: Strong assumptions on the distribution of the censoring mechanism can result in biased inference, whereas allowing general censoring schemes may lead to low statistical power of the statistical procedures. As noted, most statistical procedures for interval-censored data assume that the examination times are independent of the event time. This assumption is satisfied for externally scheduled examination times. However, there are situations in which the examination process is not independent of the

4

INTERVAL CENSORED

event time. The random censorship assumption is often violated when the data arise from a serial screening and the timing of screening depends on the patient’s health status. Or, for time-to-infection data, if the infection can be suspected after certain undertakings or accidents, cool-headed patients would likely urge a laboratory testing. Then, the infection time and the examination time are not independent. 4

STATISTICAL ANALYSIS

Several characteristics of survival distributions are of interest in clinical trials: the survival probability function, the difference or ratio of the survival probabilities in two (treatment) groups, and the influence of covariates on the survival probabilities, to name the most important ones. Under the burden of complicated censoring schemes, for each of these cases, a valid estimation method is needed. The methods for interval-censored data are nonstandard and need advanced techniques. Consistent estimators may be available only under restrictive assumptions on the censoring mechanism, and the distributional properties of estimates, confidence intervals, or valid testing procedures are only approximately constructed. To this end, one should note that the development of statistical methods and their distributional properties for interval-censored data are at present not fully developed and research is an ongoing process. In particular, examples with appropriate statistical treatment of dependent interval censoring are only very recent; see References 1 and 2. There is also ongoing work in mathematical statistics. The mathematically interested reader is referred to References 3 and 4 and the references given therein. In the remaining section, some established statistical methods are named for which it is assumed that the censoring is independent of the event time. The inference in parametric survival models is relatively straightforward. As a consequence of the independence assumption, likelihood based methods are applicable; see Reference 5. In particular, the maximum likelihood estimator has the usual properties, as are consistency and the usual convergence

rate n1/2 , where n is the sample size. Software for the parametric inference should be available for most standard statistic programs. For the nonparametric estimation of the survival probability function, the so-called nonparametric maximum likelihood estimator (NPMLE) can be used. The estimator is related to the familiar Kaplan–Meier estimator, which is the nonparametric maximum likelihood estimator for right-censored data. However, the Kaplan–Meier estimator cannot be applied directly and only in exceptional cases to interval-censored data; see the note of caution at the end of Section (1). NPMLE for interval-censored data is not uniquely defined: Any function that jumps the appropriate amount in the so-called equivalence sets represents a NPMLE; see Reference 6. Briefly, the equivalence sets are found by ordering all unique left-hand limits and all unique right-hand limits of all observed intervals in a dataset; see Reference 7 for details. Outside the equivalence sets, the nonparametric estimator defines constant survival probability and the graph is horizontal in these parts. Although the graph of NPMLE is formally undefined in the equivalence sets, some authors visualize NPMLE as if it was a step function, some interpolate between the horizontal parts, and others leave the graph indefinite outside the horizontal parts. Technical differences occur with the computation of NPMLE for the different types of censoring: For ‘‘case 1’’ interval-censored data, NPMLE is given by an explicit formula, (8). For the other types of interval censoring, NPMLE has to be computed recursively; see References 6, and (8–10). For instance, the self-consistency equations developed by Turnbull (9) yield an algorithm that is a special case of the EM-algorithm. Turnbull’s algorithm is implemented in some major statistical software packages (SAS, Splus). The more recently suggested algorithms achieve improvement concerning stability and computational efficiency (8, 11, 12). Unlike the Kaplan–Meier estimator, the NPMLE for interval-censored data converges at a rate slower than n1/2 , where n is the sample size. In particular, the distribution of this survival function estimator cannot be approximated by a Gaussian distribution. However, at least for ‘‘case 1’’ and ‘‘case 2’’

INTERVAL CENSORED

5

Semiparametric models are prominent for the analysis of regression problems in survival analysis. The frequently used regression models that have extensions for intervalcensored data structures are the proportional hazards model (14, 15), the proportional odds model (16), and the accelerated failure time model (3). These model classes have semiparametric and parametric subclasses. The main difference is that the estimators of the survival curve in the semiparametric models perform as NPMLE; i.e., the convergence rate is slow, and the distribution is not

interval censoring, the asymptotic distribution of NPMLE has been derived in Reference 8. In some cases, the bootstrap provides an alternative method for approximating the distribution of NPMLE (3). By using such tools, confidence intervals for the survival probability at a fixed time can be constructed. Nonparametric tests for the two group comparison have been proposed in References 12 and 13.

Table 2. Breast Cancer Retraction Data in Two Treatment Arms. The Time of First Occurrence of Breast Retraction Lies Between the Left and the Right Endpoint of the Intervals; Right-Censored Observations have ∞ as the Right Endpoint Treatment 1 (n = 46)

[0, 5], [0, 7], [0, 8], [4, 11], [5, 11], [5, 12], [6, 10], [7, 14], [7, 16], [11, 15], [11, 18], [17, 25], [17, 25], [18, 26], [19, 35], [25, 37], [26, 40], [27, 34], [36, 44], [36, 48], [37, 44], [15, ∞), [17, ∞), [18, ∞), [22, ∞), [24, ∞), [24, ∞), [32, ∞), [33, ∞), [34, ∞), [36, ∞), [36, ∞), [37, ∞), [37, ∞), [37, ∞), [38, ∞), [40, ∞), [45, ∞), [46, ∞), [46, ∞), [46, ∞), [46, ∞), [46, ∞), [46, ∞), [46, ∞), [46, ∞)

Treatment 2 (n = 48)

[0, 5], [0, 22], [4, 8], [4, 9], [5, 8], [8, 12], [8, 21], [10, 17], [10, 35], [11, 13] [11, 17], [11, 20], [12, 20], [13, 39], [14, 17], [14, 19], [15, 22], [16, 20] [16, 24], [16, 24], [16, 60], [17, 23], [17, 26], [17, 27], [18, 24], [18, 25] [19, 32], [22, 32], [24, 30], [24, 31], [30, 34], [30, 36], [33, 40], [35, 39] [44, 48], [11, ∞), [11, ∞), [13, ∞), [13, ∞), [13, ∞), [21, ∞), [23, ∞) [31, ∞), [32, ∞), [34, ∞), [34, ∞), [35, ∞), [48, ∞)

Patients

Treatment 1

0

5

10

15

20

25

30

35

40

45

50

55

60

∞

40

45

50

55

60

∞

Months

Patients

Treatment 2

0

5

10

15

20

25

30

35

Months Figure 2. Graphical representation of the breast deterioration data in two treatment groups. Right-censored observations are shown at the end of each treatment group.

6

INTERVAL CENSORED

approximately Gaussian. In case of a parametric survival model, the usual properties retain for the maximum likelihood estimators of the covariate effects. The hypothesis of zero covariate effects can then be tested, e.g., under the proportional hazard assumption (14); see also References 17 and 18 for related test statistics. 5

It was suggested that additional chemotherapy leads to earlier breast retraction. The data consist of a mixture of interval-censored and right-censored observations. A complete listing taken from Reference 7 is presented in Table 2. A special diagram has been introduced in Reference 12 for the graphical representation of interval censored data; see Fig. 2. In each treatment group, the interval-censored part of the data is sorted by the length of the observed interval and the right-censored part by the time of the last examination. Figure 3 compares the parametric estimate of the survival curve in the Weibull survival model to NPMLE. In addition, the Kaplan–Meier estimator was computed by treating the center of the 56 closed intervals, where the right endpoint is not ∞, as if these observations were exact. The graph of the Kaplan–Meier estimator, which is only an ad hoc method in this situation, is also displayed in Fig. 3. All estimation methods show late differences among the survival probabilities in the treatment groups. The graphs of the competing methods are quite close for treatment 1, but they differ for treatment 2. To test the hypothesis of no treatment effect, the nonparametric tests proposed in References 12 and 13 are compared with the

WORKED EXAMPLE

1.0

In this section, an illustration of the mainstream statistical methods for interval censoring is given. The data are taken from the overview article (7), where the reader also finds comprehensive statistical analysis and a comparison of statistical methods. The source of the data is clinical studies on the cosmetic effect of different treatments of breast cancer (19, 20). Women with early breast cancer received a breastconserving excision followed by either radiotherapy alone or a combination of radiotherapy and chemotherapy. The event of interest was the first occurrence of breast retraction. The time interval between two adjacent examinations was in the mean 4–6 months, stretching wider as the recovery progresses. In what follows, treatment 1 corresponds to radiotherapy alone and treatment 2 to a combination of radiotherapy and chemotherapy.

0.0

0.2

Survival Probability 0.4 0.6 0.8

NPMLE Weibull Kaplan−Meier

0

10

20

30

40

50

60

Months Figure 3. Comparison of estimated survival curves in the treatment groups of the breast deterioration data. The respective upper curves belong to treatment 1 for all methods of estimation.

INTERVAL CENSORED Table 3. Test Results for the Treatment Effect in the Breast Deterioration Data Test

P-value

Finkelstein (14) ¨ Dumbgen et al. (12) Sun (13) Parametric

0.004 0.0028 0.0043 0.0012

11.

12.

13.

test under the proportional hazard assumption given in Reference 14, and with the test in a fully parametric survival model (Weibull family). The resulting P-values presented in Table 3 are different. However, in this example, all methods show significant differences among the treatment arms. REFERENCES 1. R. A. Betensky, On nonidentifiability and noninformative censoring for current status data. Biometrika 2000; 218–221. 2. D. M. Finkelstein, W. B. Goggins, and D. A. Schoenfeld, Analysis of failure time data with dependent interval censoring. Biometrics 2002; 298–304. 3. J. Huang and J. A. Wellner. Interval censored survival data: A review of recent progress. Proc. First Seattle Symposium in Biostatistics: Survival Analysis, 1997. 4. J. Sun, Encyclopedia of Biostatistics: Interval Censoring, P. Armitage and T. Colton (eds.), New York: Wiley, 2002: 2090–2095. 5. J. P. Klein and M. L. Moeschberger, Survival Analysis—Techniques for Censored and Truncated Data. Statistics in Biology an Health. New York: Springer, 1997. 6. R. Peto, Experimental survival curves for interval-censored data. Appl. Stat. 1973; 22: 86–91. 7. J. C. Lindsey and L. M. Ryan, Tutorial in biostatistics—methods for interval-censored data. Stat. Med. 1998; 17: 219–238. 8. P. Groeneboom and J. A. Wellner, Information Bounds and Nonparametric Maximum Likelihood Estimation, vol. 19 of DMV-Seminar. ¨ New York: Birkhauser, 1992. 9. B. W. Turnbull, The empirical distribution function with arbitrarily grouped, censored and truncated data. J. R. Stat. Soc. Series B 1976; 38: 290–295. 10. A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood from incomplete data via

14.

15.

16.

7

the EM algorithm (with discussion). J. R. Stat. Soc. Series B 1977; 39: 1–38. G. Jongbloed, The iterative convex minorant algorithm for nonparametric estimation. J. Comput. Graphical Stat. 1998; 7: 310–321. ¨ L. Dumbgen, S. Freitag, and G. Jongbloed, Estimating a unimodal distribution from interval-censored data. Technical report. University of Bern, 2003. J. Sun, A nonparametric test for intervalcensored failure time data with application to AIDS studies. Stat. Med. 1996; 15: 1387–1395. D. M. Finkelstein. A proportional hazards model for interval-censored failure time data. Biometrics 1986; 42: 845–854. J. Huang, Efficient estimation for the proportional hazards model with interval censoring. Ann. Stat. 1996; 24: 540–568. A. J. Rossini and A. A. Tsiatis, A semiparametric proportional odds regression model for the analysis of current status data. J. Amer. Stat. Assoc. 1996; 91: 713–721.

17. M. P. Fay. Rank invariant tests for interval censored data under the grouped continous model. Biometrics 1996; 52: 811–822. 18. G. R. Petroni and R. A. Wolfe. A two-sample test for stochastic ordering with intervalcensored data. Biometrics 1994; 50: 77–87. 19. G. F. Baedle, S. Come, C. Henderson, B. Silver, and S. A. H. Hellman, The effect of adjuvant chemotherapy on the cosmetic results after primary radiation treatment for early stage breast cancer. Int. J. Radiation Oncol. Biol. Phys. 1984; 10: 2131–2137. 20. G. F. Baedle, J. R. Harris, B. Silver, L. Botnick, and S. A. H. Hellman, Cosmetic results following primary radiation therapy for early beast cancer. Cancer 1984; 54: 2911–2918.

INTRACLASS CORRELATION COEFFICIENT

to different dietary interventions. This form of randomization at a higher level than the individual is known as a cluster-randomized (or group randomized) design. Although randomization occurs at this higher level, observations are usually still made at the level of the individual. Cluster trials can adopt a completely randomized design (in which clusters are randomized to each intervention without restriction), a stratified design (in which similar clusters are grouped into strata and randomization to the interventions takes place within each stratum), or a paired design (in which clusters are paired and one cluster from each pair is randomized to each intervention). Adoption of a cluster-randomized design, however, has implications for the design, conduct, and analysis of such studies. A fundamental assumption of an individually randomized trial is that the outcome for an individual patient is completely unrelated to that for any other patient (i.e., they are ‘‘independent’’). This assumption no longer holds when cluster randomization is adopted, because patients within any one cluster are more likely to respond in a similar manner. For example, the management of patients in a single hospital is more likely to be consistent than management across several hospitals. This correlation among individuals within clusters has to be taken into account when planning and analyzing the cluster trial design. The statistical measure of this correlation is known as the intraclass or intracluster correlation coefficient (ICC). In this article, the measurement and impact of the ICC is discussed. The most common trial design—the completely randomized design—is assumed throughout.

MARION K. CAMPBELL University of Aberdeen, Aberdeen, UK

JEREMY M. GRIMSHAW Ottawa Health Research Institute, Ottawa, Canada

GILDA PIAGGIO World Health Organisation, Geneva, Switzerland

1

INTRODUCTION

The randomized controlled trial is the design of choice for evaluating healthcare interventions (1,2). Most commonly, randomization takes place at the level of the patient, in which individual patients are allocated to the different arms of the trial. Randomization by individual minimizes bias in study results and provides maximal efficiency regarding the precision of estimates and power of statistical tests. In some situations, however, allocation at an individual patient basis is problematic, for example, if there is a risk of contamination of interventions across trial groups (i.e., the unintentional transfer of elements of one intervention to participants allocated to another arm of the trial) (3). For example, if some patients in an inpatient ward were allocated randomly to receive charts to record symptoms to trigger particular actions and some patients were not, then it would difficult to prevent those patients given the charts from talking to those patients not assigned charts. This latter group might then begin to note their symptoms, even without charts, thereby potentially distorting the results of the trial. In such situations, randomizing at some ‘‘higher’’ level may be desirable to obtain a more reliable estimate of the treatment effect (4). In this case, randomization by hospital ward would have been more appropriate. Similarly, when trials that evaluate dietary interventions are being designed, families are often randomized as an entire unit to avoid the possibility of different family members being assigned

2 THE INTRACLUSTER (OR INTRACLASS) CORRELATION COEFFICIENT Individuals within a cluster are usually more similar than individuals from different clusters. Because individuals within a cluster receive the same intervention and those who receive different interventions belong to different clusters, randomizing clusters implies

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INTRACLASS CORRELATION COEFFICIENT

a loss in efficiency compared with randomizing individuals. The variability between units that receive the same intervention, which is the basis for estimating the uncontrolled variation, is thus greater than that between individuals who receive the same intervention had the individuals been randomized. The similarity between individuals within clusters is measured by the intracluster correlation coefficient (which is commonly referred to by the Greek letter rho, ρ). 2.1 Definition of the ICC The ICC can be defined in general as the correlation between pairs of subjects within clusters, and thus it can vary between −1/(j−1) and 1, where j is the number of clusters (5). However, the most commonly used definition of the ICC, which is appropriate within the cluster trial context, is based on the analysis of variance and specifically on the relationship of the between to within-cluster variance. Using this definition, the ICC is calculated as the proportion of the total variation in the outcome that can be attributed to the difference between clusters: ρ=

σb2 σb2

+ σw2

where σb2 is the between-cluster variance component, and σw2 is the within-cluster variance component (6). In this context, therefore, the ICC takes a value of between 0 and 1. 2.2 Accounting for the ICC in Sample-Size Calculations The presence of clustering within trial data has a direct impact on sample-size requirements for a trial, primarily because standard sample-size calculations assume that data within the trial are independent (which is not the case in a cluster trial). The size of the ICC has a direct impact on the efficiency of the sample size within the clustered design. For example, for a completely randomized cluster trial design, to achieve the equivalent power of a patient-randomized trial, standard sample-size estimates require to be inflated by a factor: 1 + (n − 1)ρ

to accommodate for the clustering effect, where n is the average cluster size, and ρ is the estimated ICC (assuming the clusters are of a similar size) (7). (It is important to note that cluster size can be defined as either the number of available individuals with the cluster or the number of individuals within the cluster being selected to take part in the trial.) This inflation factor is commonly known as the ‘‘design effect’’ or the ‘‘variance inflation factor’’ (8). The impact of randomization by cluster on the required sample size can be substantial, and as both the ICC and the cluster size influence the calculation, the design effect can be substantial even when the ICC seems to be numerically small. Table 1 shows the example of a trial designed to detect a change from 40% to 60% in a dichotomous outcome, with 80% power and a 5% significance level. For an individually randomized trial, a sample size of 194 patients would be required to detect this level of difference. If, however, cluster randomization was required because of the potential for contamination, samplesize estimates would have to be inflated to account for the ICC and the cluster size. Table 1 shows that, even at relatively modest levels of ICC (say ρ = 0.01), a doubling of the original estimate would not be unusual. Sample-size calculators have been developed to aid in sample size calculations for trials, which also allow trade-offs between cluster sizes and number of clusters to be assessed (9,10). 2.3 Methods for Estimating the ICC The aim of this section is to provide a brief overview for the reader of the most common approaches to estimating the ICC, providing references for additional detail and methods. Some of these estimation procedures can return ICC values of less than zero, but in such cases the ICC estimate should be truncated at zero. For the estimation of an ICC from continuous data, the most common method adopted is the analysis of variance (ANOVA) approach (6,11,12). This method draws on the fact that, as alluded to above, an ICC can be described as the relationship between the two components of variation in outcome in

INTRACLASS CORRELATION COEFFICIENT

3

Table 1. Impact of the ICC on Sample Size Requirements Example:

To detect a change from 40% to 60%, with 80% power and a 5% significance level, assuming different levels of ICC and cluster size (under individual randomization a total of 194 patients would be required).

Cluster size

ICC

10 Number of clusters (total number of individuals) required

20 Number of clusters (total number of individuals) required

50 Number of clusters (total number of individuals) required

100 Number of clusters (total number of individuals) required

0.01 0.05 0.10 0.15

22 (220) 30 (300) 38 (380) 46 (460)

12 (240) 20 (400) 30 (600) 38 (760)

6 (300) 14 (700) 24 (1200) 34 (1700)

4 (400) 12 (1200) 22 (2200) 32 (3200)

a cluster trial— the variation among individuals within a cluster and the variation between clusters. These two sources of variation can be identified separately using a standard ANOVA approach—by adopting a one-way ANOVA, with the cluster-level variable as a random factor (13). For the ANOVA approach, assuming a two-level model (i.e., cluster and individual) the appropriate model can expressed as: yij = µ + βj + eij where yij is the response of the ith individual in the jth cluster, µ is the overall mean response, β j is the random effect of the jth cluster, and eij is the random error. It is assumed that the β j are independently and identically distributed with zero mean and constant variance σb2 , and that the eij are similarly identically and independently distributed with zero mean and constant variance σw2 . The following variance component estimates are then identified from the ANOVA table: (MSB − MSW) σˆ b2 = n0 and σˆ w2 = MSW where MSB is the between-cluster mean square, MSW is the within-cluster mean square, and n0 is the ‘‘average’’ cluster size.

The ‘‘average’’ cluster size is calculated using:
 n2j 1 n0 = N− J−1 N where J is the number of clusters, N is the total number of individuals, and nj is the number of individuals in the jth cluster. This version of the ‘‘average’’ cluster size is used over the arithmetic average, as the arithmetic mean estimate can lead to underestimation of the between-cluster variance component when the cluster sizes vary. The ICC can then be estimated as: ρˆ =

σˆ b2 σˆ b2

+ σˆ w2

Most random effects modeling packages also produce estimates for these betweencluster σˆ b2 and within-cluster σˆ w2 components of variation directly as part of their standard output. These estimates can then be used directly in the formula to estimate the ICC. Random effects modeling also allows estimation of the ICC when covariates need to be accounted for in the model. Donner and Koval (11) note that using this derivation of the ICC requires no assumption that the dependent data be normally distributed. Donner and Koval (6) also describe an alternative approach to calculating the ICC from continuous data that can be calculated by assuming that the observations yij , are distributed about the same mean µ and variance σ 2 in such a way that two observations

4

INTRACLASS CORRELATION COEFFICIENT

within the same cluster have a common correlation ρ. This result adopts a maximum likelihood estimator (MLE) approach. Donner and Koval note that in samples of equal size n, this expression of the ICC is equivalent to the Pearson product-moment correlation as computed over all pairs of observations that can be constructed within clusters. In this article, Donner and Koval compared the performance of ANOVA and maximum likelihood estimates using Monte Carlo simulation. The results of the simulation suggested that for low to moderate ICCs (< 0.5), the ANOVA method performed well and provided accurate estimates of the ICC, especially when single-member clusters were excluded from the analysis. However, if the ICC was large (> 0.5), or completely unknown, then the maximum likelihood approach tended to produce more accurate estimates. For the estimation of an ICC from binary data, the ANOVA approach is again most commonly used, as it has also been shown not to require any strict distributional assumptions (14,11,12). Although the ANOVA estimate of the ICC may be used with dichotomous data, however, standard methods to calculate confidence intervals around this estimate do not hold in the dichotomous case. Moore and Tsiatis (15) and Feng and Grizzle (16) describe an alternative approach for use in the binary case—the moment estimator for dichotomous data. The moment estimator is based on an extension of the generalized Pearson statistic to estimate overdispersion. Ridout et al. (17) reviewed the performance of several methods for calculating ICCs from binary data and showed that the ANOVA method performed well. Feng and Grizzle (16) also showed that the ANOVA method and the moment estimator method were similar, but they noted that bias increases as the estimate of ρ increases and as the prevalence nears zero or one (estimates were lower than they should be).

3

THE MAGNITUDE OF THE ICC

As has been described above, robust estimates of ICCs are required for informed sample size calculations to be made.

3.1 What is the Magnitude of the ICC? Estimates on ICC size are available from several settings and for several outcomes. The ICC estimate varies depending on the type of cluster under examination (e.g., community, hospital, family, and so on). As the definition of the ICC suggests, ICCs are likely to be high if the variation between clusters is high and the variation within clusters is low. Gulliford et al. (18) published a range of ICCs suitable for use in community intervention studies based on a systematic analysis of the 1994 English Health Survey. This research showed that for community intervention trials where communities were at a regional or district level ICCs were typically lower than 0.01. However, as the definition of ‘‘community’’ was varied, so the typical range of ICC values varied. Where communities were defined at postcode/zipcode level, ICCs were generally lower than 0.05. When community was defined at a household level, however, ICCs ranged up to 0.3. This finding highlights well the contribution of the between-to-within cluster variation to the estimate of the ICC. In small units, such as households, within-cluster variation tends to be small so the between-cluster variation dominates, which leads to a higher estimate of ICC. For larger clusters, however, withincluster variation is relatively large; hence, the effect of the between-cluster variation is lessened, which leads to a smaller ICC estimate. Todd et al. (19) also showed how different design features could affect the size of the ICC in community-intervention trials. They showed that effective matching of clusters resulted in a marked lowering of the ICC. As the community trials referred to by Todd et al. (19) all involved large cluster sizes, however, reported ICCs, were generally lower than 0.01 irrespective of the chosen design. Individual community-intervention studies have also published several ICCs. For example, Hannan et al. (20) published ICCs from the Minnesota Health Program—these ranged typically from 0.002 to 0.012. A range of ICCs from other settings has also been identified. Murray and Blitstein (21) highlighted more than 20 papers that had reported ICCs from a range of settings,

INTRACLASS CORRELATION COEFFICIENT

which included workplace and school-based studies. Smeeth and Ng (22) provided a comprehensive analysis of ICCs for cluster trials that involved older people in primary care and observed ICCs that ranged from very small (<0.01) to 0.05. Parker et al. (23), Adams et al. (24), and Cosby et al. (25) also reported ICCs from primary care settings and observed that although most ICCs were very small some were higher (up to 0.2). Campbell et al. (26) provided estimates of the magnitude of ICCs in implementation research settings, and again they showed evidence of higher ICCs (up to 0.3) for process outcomes. Although several sources are available for ICC estimates, they are often based on a relatively small number of observations, and hence, confidence intervals around these estimates are often wide. 3.2 What Affects the Magnitude of the ICC? Campbell et al. (27) sought to identify potential determinants of the magnitude of ICCs in cluster trials. They examined a range of hypotheses (that had been generated through a survey of statisticians involved in the design of cluster trials and known experts in the field) across a range of datasets that involved 220 ICC estimates. The authors concluded that many factors affected the size of the ICC. In particular, the results suggested that if an ICC relates to a ‘‘process’’ rather than an ‘‘outcome’’ variable, then the ICC would be higher (the multivariate model suggested that, on average, the ICC would be 0.05 higher for a process variable). This result is likely observed because patient biological and compliance variability modifies the effect of interventions. Thus, although the process may be consistent within clusters, the clinical outcome within cluster is likely to be more variable. For example, although clinicians and organizations may practice according to similar guidelines (the process measure), patients are likely to be much more variable in how they comply and respond to that treatment guidance (the outcome measure). Similarly, the data showed that if an ICC represents data from secondary rather than community-based care, then the size of the ICC increased (on average 0.01 higher for a secondary care ICC compared with

5

a community-based ICC). Although numerically small, such differences can have a substantial effect on sample size, especially when the average cluster size is large. The influence of the other factors examined was less clear cut. For ICCs related to dichotomous variables, ICCs were shown to be higher for variables with mid-range prevalences compared with those for low- or high-prevalence variables. These differences are likely to be caused partly by known biases which can occur in the estimation of ICCs when the prevalence nears zero or one. Feng and Grizzle (16) showed that bias increases in the estimate of the ICC as the prevalence nears zero or one (ICCs were underestimated). This was particularly the case when ICCs are estimated from datasets with small numbers of clusters and small cluster sizes. This feature may also be observed because when prevalence is low, there is less scope for a high ICC unless several clusters have a prevalence that is relatively high compared with the others. Earlier research (13,28–30) has also shown that the larger the cluster size, the lower the observed ICC. It has been noted, however, that whereas the magnitude of the ICC tends to decline with cluster size, it does so at a relatively slow rate (8). 4

REPORTING THE ICC

Although the reporting of ICCs would seem to be increasing, there is little consistency in the information presented to describe the ICC. Empirically informed guidance on the appropriate reporting of the trial ICC estimate was published by Campbell et al. (31). This research suggested that three dimensions should be routinely reported to aid the interpretation of the ICC: • a description of the dataset (including

characteristics of the outcome and the intervention)—including information on the demographic distribution within and between clusters, a description of the outcome, and a structured description of the intervention. • information on how the ICC was calculated, which includes what statistical

6

INTRACLASS CORRELATION COEFFICIENT

method had been used to calculate the ICC, the software program used to calculate the ICC, what data were used to calculate the ICC (e.g., whether from control data and/or intervention data, preintervention, and/or post-intervention data, etc.), and whether any adjustment had been made for covariates • information on the accuracy of the ICC (including confidence intervals where possible) These recommendations were adopted in the CONSORT statement for the reporting of cluster randomized trials (32). 5 ACCOUNTING FOR THE ICC IN ANALYSIS The correlation between individuals should be accounted for in the analysis of cluster trials. If this correlation is ignored, then the analysis will yield P-values that are artificially low and confidence intervals that will be over-narrow, which increases the chances of identifying spuriously significant findings and misleading conclusions (8,33). Three main approaches can be used for the analysis of data that are generated from a clustered design—analysis at the cluster level, adjustment of standard tests to accommodate for the clustering in the data, or more advanced analysis using the data from both the cluster and the individual level (13,34). The traditional approach to the analysis of cluster-randomized trials was the cluster level approach, which consists of calculating a summary measure for each cluster, such as a cluster mean or proportion, and conducting a standard analysis with the cluster as the unit of analysis. This approach is appropriate when the inference is intended to be at the cluster level. When the inference is intended to be at the individual level, the traditional approach is no longer appropriate. Given more recent advances in statistical computing, the approach adopted in recent years in this situation has been to use the data available at both the individual and cluster level. These techniques, such as random effects (or multilevel modeling) and generalized estimating equations, incorporate the hierarchical

nature of the data directly into the analysis structure. They also allow the incorporation of individual and cluster-level characteristics and can handle more complex correlation structures. They do, however, require a larger number of clusters to be available for analysis; for example, at least 20 clusters per group are recommended for the use of generalized estimating equations (33). The most appropriate analysis option depends on several factors, which include factors such as the research question, including the level at which inference will be made, whether individual or cluster, the study design, the type of outcome (whether binary or continuous), and the number and size of clusters available. A worked example of the three approaches is presented by Mollison et al. (34). 6 SUMMARY In this article, we have shown that the ICC is of fundamental importance to the planning and execution of a cluster-randomized trial. It is the formal statistical measure of the impact of clustering and can have a substantial effect on sample-size requirements (even when the ICC estimate is numerically small). Different types of outcomes are likely to have different magnitudes of ICC, and it is important that this factor is considered when planning and analyzing any cluster-randomized trial. In this article, we concentrated our discussion on completely randomized cluster designs as they are the most commonly adopted designs in practice. In a completely randomized cluster design, the calculation and interpretation of the ICC is at its most straightforward— which is appropriate for an introductory article such as this. Variations to the calculation of the ICC and the adjustment for sample size calculations are available for more complex designs such as the stratified randomized cluster trial designs and the pair-matched cluster trial design. Readers should refer to more specialized texts (8,33) for discussion of these cases. REFERENCES 1. S. J. Pocock, Clinical Trials: A Practical Approach. Chichester, UK: Wiley, 1983.

INTRACLASS CORRELATION COEFFICIENT 2. B. Sibbald and M. Roland, Understanding controlled trials: why are randomized controlled trials important? BMJ 1998; 316: 201. 3. M. R. Keogh-Brown, M. O. Bachmann, L. Shepstone, C. Hewitt, A. Howe, C. R. Ramsay, F. Song, J. N. V. Miles, D. J. Torgerson, S. Miles, D. Elbourne, I. Harvey, and M. J. Campbell, Contamination in trials of educational interventions. Health Technol. Assess. 2007; 11:43. 4. P. M. Fayers, M. S. Jordhoy, and S. Kaasa, Cluster-randomized trials. Palliative Med. 2002; 26: 69–70. 5. R. A. Fisher, Statistical Methods for Research Workers, 12th ed. Edinburgh, UK: Oliver and Boyd, 1954. 6. A. Donner and J. J. Koval, The estimation of intraclass correlation in the analysis of family data. Biometrics 1980; 36: 19–25. 7. A. Donner, N. Birkett, and C. Buck, Randomization by cluster. Sample size requirements and analysis. Am. J. Epidemiol. 1981; 114: 906–914. 8. A. Donner and N. Klar, Design and Analysis of Cluster Randomization Trials in Health Research, 1st ed. London: Arnold, 2000. 9. A. P. Y. Pinol and G. Piaggio, ACLUSTER Version 2.0. Geneva, Switzerland: World Health Organisation, 2000. Available: www.updatesoftware.com/publications/acluster. 10. M. K. Campbell, S. Thomson, C.R. Ramsay, G.S. MacLennan, and J. M. Grimshaw, Sample size calculator for cluster randomized trials. Comp. Biol. Med. 2004; 34: 113–125. 11. A. Donner and J. J. Koval, Design considerations in the estimation of intraclass correlation. Ann. Hum. Genet. 1982; 46: 271–277. 12. D. M. Murray, B. L. Rooney, P. J. Hannan, A. V. Peterson, D. V. Ary, A. Biglan, G. J. Botvin, R. I. Evans, B. R. Flay, R. Futterman, J. G. Getz, P. M. Marek, M. Orlandi, M. A. Pentz, C. L. Perry, and S. P. Schinke, Intraclass correlation among common measures of adolescent smoking: estimates, correlates, and applications in smoking prevention studies. Am. J. Epidemiol. 1994; 140: 1038–1050. 13. O. C. Ukoumunne, M. C. Gulliford, S. Chinn, J. A. C. Sterne, and P. G. J. Burney, Methods for evaluating area-wide and organization based interventions in health and health care: a systematic review. Health Technol. Assess. 2000; 3: 5. 14. A. Donner and A. Donald, Analysis of data arising from a stratified design with the cluster as unit of randomization. Stat. Med. 1987; 6: 43–52.

7

15. D. F. Moore and A. Tsiatis, Robust estimation of the variance in moment methods for extra-binomial and extra-poisson variation. Biometrics 1991; 47: 383–401. 16. Z. Feng and J. E. Grizzle, Correlated binomial variates: properties of estimator of intraclass correlation and its effect on sample size calculation. Stat. Med. 1992; 11: 1607–1614. 17. M. S. Ridout, C. G. B. Demetrio, and D. Firth, Estimating intraclass correlation for binary data. Biometrics 1999; 55: 137–148. 18. M. Gulliford, O. Ukuomunne, and S. Chinn, Components of variance and intraclass correlations for the design of community based surveys and intervention studies: data from the health survey for England 1994. Am. J. Epidemiol. 1999; 149: 876–883. 19. J. Todd, L. Carpenter, X. Li, J. Nakiyingi, R. Gray, and R. Hayes, The effects of alternative study designs on the power of community randomized trials: evidence from three studies of human immunodeficiency virus prevention in East Africa. Internat. J. Epidemiol. 2003; 32: 755–762. 20. P. J. Hannan, D. M. Murray, D. R. Jacobs, and P. G. McGovern, Parameters to aid in the design and analysis of community trials: intraclass correlations from the Minnesota Heart Health Program. Epidemiology 1994; 5: 88–95. 21. D. M. Murray and J. L. Blitstein Methods to reduce the impact of intraclass correlation in group-randomized trials. Eval. Rev. 2003; 27: 79–103. 22. L. Smeeth and E. S. Ng, Intraclass correlation coefficients for cluster randomized trials in primary care: data from the MRC trial of the assessment and management of older people in the community. Control. Clin. Trials 2002; 23: 409–421. 23. D. R. Parker, E. Evangelou, and C. B. Eaton, Intraclass correlation coefficients for cluster randomized trials in primary care: the cholesterol education and research trial (CEART). Contemp. Clin. Trials 2005; 26: 260–267. 24. G. Adams, M. C. Gulliford, O. C. Ukoumunne, S. Eldridge, S. Chinn, and M. J. Campbell, Patterns of intra-cluster correlation from primary care research to inform study design and analysis. J. Clin. Epidemiol. 2004; 57: 785–794. 25. R. H. Cosby, M. Howard, J. Kacorowski, A. R. Willan, and J. W. Sellors, Randomizing patients by family practice: sample size estimation, intracluster correlation and data analysis. Family Practice 2003; 20: 77–82.

8

INTRACLASS CORRELATION COEFFICIENT

26. M. K. Campbell, J. M. Mollison, and J. M. Grimshaw, Cluster trials in implementation research: estimation of intracluster correlation coefficients and sample size. Stat. Med. 2000; 20: 391–399. 27. M. K. Campbell, P. M. Fayers, and J. M. Grimshaw, Determinants of the intracluster correlation coefficient in cluster randomized trials. Clinical Trials 2005; 2: 99–107. 28. A. Donner, An empirical study of cluster randomization. Internat. J. Epidemiol. 1982; 11: 283–286. 29. W. W. Hauck, C. L. Gilliss, A. Donner, and S. Gortner, Randomization by cluster. Nursing Res. 1991; 40: 356–358. 30. O. Siddiqui, D. Hedeker, B. R. Flay, and F. B. Hu, Intraclass correlation estimates in a school-based smoking prevention study. Am. J. Epidemiol. 1996; 144: 425–433. 31. M. K. Campbell, J. M. Grimshaw, and D. R. Elbourne, Intracluster correlation coefficients in cluster randomized trials: empirical insights into how should they be reported. BMC Med. Res. Methodol. 2004; 4: 9. 32. M. K. Campbell, D. R. Elbourne, and D. G. Altman, for the CONSORT group, The CONSORT statement: extension to cluster randomized trials. BMJ 2004; 328: 702–708. 33. D. M. Murray, The Design and Analysis of Group Randomized Trials. Oxford, UK: Oxford University Press, 1998. 34. J. Mollison, J. A. Simpson, M. K. Campbell, and J. M. Grimshaw, Comparison of analytic methods for cluster randomized trials: an example from a primary care setting. J. Epidemiol. Biostat. 2000; 5: 339–348.

FURTHER READING S. M. Eldridge, D. Ashby, G. S. Feder, A. R. Rudnicka, and O. C. Ukoumunne, Lessons for cluster randomized trials in the twenty-first century: a systematic review of trials in primary care. Clinical Trials 2004; 1: 80–90. S. Killip, Z. Mahfoud, and K. Pearce, What is an intracluster correlation coefficient? Crucial concepts for primary care researchers. Ann. Family Med. 2004; 2: 204–208.

CROSS-REFERENCES Cluster randomization Group randomized trials

INTRARATER RELIABILITY

of the between-subject variation (BSV) to the total variation [i.e., the sum of the BSV and the within-subject variation (WSV)], and it is the statistical measure most researchers adopted for quantifying the intrarater reliability of continuous data. Note that ICC reaches its maximum value of 1 when WSV (i.e., the average variation for a subject) reaches its lower bound of 0, a situation indicating that any variation in the data is because the subjects are different and not because the rater is being inconsistent. Used previously as a measure of reliability by Ebel (3) and Barko (4), the ICC has proved to be a valid measure of raters’ self-consistency. Shrout and Fleiss (5) discuss various forms of the ICC as a measure of inter-rater reliability, which quantifies the extent of agreement between raters as opposed to intrarater reliability used to measure self-consistency. The selection of a particular version of the ShroutFleiss ICCs is dictated by the design adopted for the intrarater reliability study. Lachin (6) also discusses various techniques for evaluating the quality of clinical trial data, which includes the ICC among others.

KILEM L. GWET STATAXIS Consulting, Gaithersburg, Maryland

The notion of intrarater reliability will be of interest to researchers concerned about the reproducibility of clinical measurements. A rater in this context refers to any datagenerating system, which includes individuals and laboratories; intrarater reliability is a metric for rater’s self-consistency in the scoring of subjects. The importance of data reproducibility stems from the need for scientific inquiries to be based on solid evidence. Reproducible clinical measurements are recognized as representing a well-defined characteristic of interest. Reproducibility is a source of concern caused by the extensive manipulation of medical equipment in test laboratories and the complexity of the judgmental processes involved in clinical data gathering. Grundy (1) stresses the importance of choosing a good laboratory when measuring cholesterol levels to ensure their validity and reliability. This article discusses some basic methodological aspects related to intrarater reliability estimation. For continuous data, the intraclass correlation (ICC) is the measure of choice and will be discussed in the section entitled ‘‘Intrarater reliability for continuous scores.’’ For nominal data, the kappa coefficient of Cohen (2) and its many variants are the preferred statistics, and they are discussed in the section entitled ‘‘nominal scale score data.’’ The last section is devoted to some extensions of kappa-like statistics aimed at intrarater reliability coefficients for ordinal and interval data.

1.1 Defining Intrarater Reliability Ratings in a typical intrarater reliability study that involves m subjects and n replicates per subject are conveniently organized as shown in Table 1. The entry yij represents the ith replicate score that the rater assigned to subject j. This table may be transposed if the number of subjects is very large. The relationship between the ICC and the analysis of variance (ANOVA) techniques motivated the proposed disposition of rows and columns of Table 1. The WSV is the average of the m subjectlevel variances s2j calculated over the n replicates. More formally, the WSV is defined as follows:

1 INTRARATER RELIABILITY FOR CONTINUOUS SCORES

1
2 Sj , m m

WSV =

A continuous clinical measurement, such as blood pressure, will be considered reproducible if repeated measures taken by the same rater under the same conditions show a rater variation that is negligible compared with the subject variation. ICC is the ratio

j=1

where S2j =

n 1
(yij − y·j )2 n−1 i=1

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

(1)

2

INTRARATER RELIABILITY

Table 1. Scores Assigned to m Subjects with n Replicates per Subject Subject Observation

1

1 2 .. . i . .. n

y11 y21 .. . yil . .. yn1

2

···

j

···

m

y12 y22 .. . yi2 . .. yn2

··· ··· .. . ··· . .. ···

ylj y2j .. . yij . .. ynj

··· ··· .. . ··· .. . ···

y1 m y2 m .. . yim . .. ynm

and y·j is the average of the n replicate observations related to subject j. The BSV is obtained by taking the variance of the m subject-level mean scores trimmed of the fraction of the WSV it contains. It is formally defined as follows: BSV = BSVo − m 

where BSVo =

Examination Survey (7). For the sake of illustration, I assume that the data was collected on two occasions (times 1 and 2) by the same laboratory. Although Equations (1)–(3) can be used to obtain the ICC from Table 2 data, the more convenient approach for computing the ICC will generally be to use an ANOVA procedure either from Microsoft Excel (Microsoft Corporation, Redmond, WA), or from a standard statistical package such as SPSS (SPSS Inc., Chicago, IL) or SAS (SAS Institute Inc., Cary, NC). The ANOVA analysis will produce two mean squares (MS) known as the Mean Square for Treatments (MST ), which is also referred to as the Mean Square for the model, and the Mean Square for Error (MSE ). The ICC can be expressed as a function of the two mean squares as follows:

WSV , n

(y·j − y·· )2

j=1

(2)

m−1

and y·· is the overall mean of all mn observations. Using Equations (1) and (2), we define the intrarater reliability coefficient γˆ (read gamma hat with the hat indicating an estimation of the ‘‘true’’ parameter γ to be defined later) as follows: γˆ =

BSV BSV + WSV

γˆ =

(3)

MST − MSE MST + (n − 1)MSE

(4)

Using MS Excel’s Analysis ToolPak and the cholesterol data, I created the output shown in Table 3 known as the ANOVA table, in which the column labeled ‘‘MS’’ contains the two Mean Squares (MST = 1323.56, and MSE = 18.25) needed to compute the ICC. Therefore, the intrarater reliability associated with

To illustrate the calculation of the ICC, let us consider the cholesterol level data of Table 2. Table 2 data represent cholesterol levels taken from 10 individuals who participated in the 2005 National Health and Nutrition

Table 2. Total Cholesterol Measures (in mg/dL) taken on 10 subjects with 2 Replicates per Subject Subjects Time

1

1 2

152 155

Source: Reference 7.

2

3

4

5

6

7

8

9

10

202 210

160 156

186 200

207 214

205 209

160 163

188 189

147 146

151 153

INTRARATER RELIABILITY

3

Table 3. ANOVA Table Created with MS Excel from Table 2 Cholesterol Data ANOVA Source of Variation Between Groups Within Groups Total

SS

df

MS

F

P-value

F crit

11,912.05 182.5 12,094.55

9 10 19

1323.56 18.25

72.52

6.4E-08

3.02

Table 2 data is computed as follows: γˆ =

1, 323.56 − 18.25 = 0.973 1, 323.56 + (2 − 1) × 18.25

Consequently, the subject factor accounts for 97.3% of the total variation observed in the cholesterol data of Table 2, which is an indication of high intrarater reliability. This estimation gives us a sense of the reproducibility of cholesterol levels. However, it also raises questions about its accuracy and the steps that could be taken to improve it. These issues can only be addressed within a well-defined framework for statistical inference. 1.2 Statistical Inference The primary objective of this section is to present a framework for statistical inference that will help answer the following fundamental questions about the intrarater reliability estimate: • Is the obtained ICC sufficiently accu-

rate? • Can the obtained ICC be considered

valid? • Have we used a sufficiently large num-

ber of replicates? • Have we used a sufficiently large number of subjects? • Can the data be collected by multiple raters? These questions can be addressed only if this theoretical framework provides the following two key components: 1. The definition of a population parameter γ (i.e., gamma without the hat) that represents the ‘‘true’’ unknown intrarater reliability being measured

2. Methods for evaluating the precision of proposed statistics γˆ with respect to the parameter of interest γ Let us consider the simple scenario in which yij results from the additive effect of a common score µ, the subject effect tj , and an error εij committed on the ith replicate score of subject j. This relation is mathematically expressed as follows: yij = µ + tj + ij , i = 1, ·n and j = 1, · · · m (5) This example is a single-factor ANOVA model that goes along with the following assumptions: • The m subjects that participate in the

intrarater reliability study form a random sample selected from a larger population of subjects of interest. Moreover, tj is a normally distributed random variable with mean 0 and variance σt2 > 0. • The error ε ij is a normally distributed random variable with mean 0, and variance σˆ2 > 0, and it is independent of tj . A small value for σ2 will lead to a small variation between replicate scores, which in turn should lead to a high intrarater reliability. Therefore, the theoretical parameter that represents the intrarater reliability γ is defined as follows: γ =

σt2 σt2 + σ2

(6)

which is one of the parameters studied by McGraw and Wong (8). It follows from Equation (5) that the denominator of γ is the total variation in the scores, and Equation (6) is the most popular form of ICC found in the

4

INTRARATER RELIABILITY

statistical literature and represents the portion of total variance that is accounted for by the between-subject variance. The statistic γˆ of Equation (3) is a sample-based approximation of γ widely accepted in the statistical literature. We will now present a method for evaluating the precision of the statistic γˆ . How close is γˆ to γ ? To answer this question we will construct a 95% confidence interval around γ ; that is a range of values expected to contain the unknown γ with 95% certainty. Constructing a 95% confidence interval for γ requires the calculation of the 2.5th and 97.5th percentiles of the F distribution with m−1 and m(n−1) degrees of freedom. These two percentiles are denoted by F0.975,m−1,m(n−1) and F0.025,m−1,m(n−1) , respectively, where 0.975 = 1–(1–0.95)/2 and 0.025 = (1–0.95)/2. Although textbooks’ statistical tables provide these percentiles, they are also readily obtained from MS Excel as follows: ‘‘ =FINV(0.025,m−1,m*(n−1))’’ gives the 97.5th percentile, whereas ‘‘FINV(0.975,m−1,m*(n−1))’’ for the 2.5th percentile. Let F be defined as follows: Fo =

BSVo MST = WSV/n MSE

(7)

The 95% confidence interval for γ is obtained by noting that: P(L95 ≤ γ ≤ U95 ) = 0.95 L95 =

F0 /F0.025,m−1,m(n−1) − 1 and (m − 1) + F/F0.025,m−1,m(n−1)

U95 =

F0 /F0.975,m−1,m(n−1) − 1 (m − 1) + F/F0.975,m−1,m(n−1)

(8)

The 95% confidence interval width is given by: (9) W95 = U95 − L95 The design of an intrarater reliability study must aim at minimizing the magnitude of the confidence interval width. The optimal values for the number m of subjects and the number n of replicates per subject are those that minimize W 95 of Equation (9).

1.3 Optimizing the Design of the Intrarater Reliability Study An intrarater reliability study is well designed if the number of observations is determined to minimize the confidence interval length and the experimental error is kept small. This section addresses the following two questions: 1. What is the optimal number m of subjects, and number n of replicates per subject? 2. Can the intrarater reliability study involve two raters or more? 1.3.1 Sample Size Determination. Finding the optimal number of subjects and replicates per subject based the confidence interval length is the approach Giraudeau and Mary (9) used to propose guidelines for planning a reproducibility study. Let ω95 be the expected width of the 95% confidence interval associated with γ . Note that σ2 Fo /(σ2 + mσt2 ) = Fo /(1 + mγ /(1 − γ )) where Fo is defined by Equation (7) and follows the F distribution with m−1 and m(n−1) degrees of freedom. Because W 95 defined by Equation (9) is a function of Fo , its expected value ω95 is a function of γ . The relationship between ω95 and γ is depicted in Figs. 1, 2, and 3 for various values of m and n. The ω95 values are calculated using a Monte-Carlo simulation approach because of the difficulty to derive a mathematical expression of the probability distribution of W95 . For values of γ that vary from 0 to 1 by step of 0.05, and for various combinations of (m,n) we simulated 10,000 observations from the F distribution with m−1 and m(n−1) degrees of freedom, and calculated 10,000 confidence intervals using Equation (8). The mean length of the 10,000 intervals was used as an estimate for ω95 . Each of the three figures contains two plots, and each plot shows how different values of m and n affect the relationship between γ and ω95 for a fixed total number of observations mn. For the two plots of Fig. 1, the total sample sizes mn are 20 and 40. For Fig. 2, the total sample sizes are 60 and 80, whereas Fig. 3’s plots are based on the sample sizes of 100 and 120. All three figures tend to indicate

Expected Interval Width

INTRARATER RELIABILITY

1. .75 .50

Expected Interval Width

0

.25

.50 .75 Intra Rater Reliability

1.

.75

.50 (m,n) = (20,2) (m,n) = (10,4) (m,n) = (8,5)

.25

0

Expected Interval Width

(m,n) = (10,2) (m,n) = (5,4)

.25 0

Figure 1. Expected width of the 95% confidence interval as a function of γ for m and n values that correspond to mn = 20 and mn = 40.

5

0

.25

.50 .75 Intra Rater Reliability

1.0

0.6 0.5 0.4 (m,n) = (30,2) (m,n) = (20,3) (m,n) = (15,4) (m,n) = (12,5)

0.3 0.2 0.1 0 0

.25

.50

.75

1.0

.75

1.0

Figure 2. Expected width of the 95% confidence interval as a function of γ for m and n values that correspond to mn = 60 and mn = 80.

Expected Interval Width

Intra Rater Reliability 0.6 0.5 0.4 0.3

(m,n) = (40,2) (m,n) = (20,4) (m,n) = (16,5) (m,n) = (10,8)

0.2 0.1 0

0

.25

.50 Intra Rater Reliability

INTRARATER RELIABILITY

Expected Interval Width

6

0.6 0.5 0.4 0.3

0.1

Expected Interval Width

0

Figure 3. Expected width of the 95% confidence interval as a function of γ for m and n values that correspond to mn = 100 and mn = 120.

(m,n) = (50,2) (m,n) = (25,4) (m,n) = (20,5) (m,n) = (10,10)

0.2

0

.25

.50 .75 Intra Rater Reliability

1.0

0.5 0.4 0.3 (m,n) = (60,2) (m,n) = (40,3) (m,n) = (30,4) (m,n) = (24,5)

0.2 0.1 0

that for high intrarater reliability coefficients (i.e., greater than 0.5), and a fixed total number of observations mn, using 2, 3, or at most 4 replicates per subjects is sufficient to obtain the most efficient intrarater reliability coefficient. Having more than four replicates is likely to lead to a loss of precision. For smaller γ values, the recommendation is to use four or five replicates per subject. One would also note that if the ‘‘true’’ value of the intrarater reliability is smaller than 0.80, then its estimation will generally not be very precise. 1.3.2 Blocking the Rater Factor. If two raters or more are used in a completely randomized intrarater reliability experiment, the resulting coefficient will be inaccurate. In a completely randomized design, subjects and replicates are assigned randomly to different raters. Consequently, the rater effect will increase the experimental error, which thereby decreases the magnitude of the intrarater reliability coefficient. This problem can be resolved by designing the experiment so that the rater effect can be measured and separated from the experimental error. A design that permits this method requires each rater to rate all subjects and provide the same number of replicates per

0

.25

.50 .75 Intra Rater Reliability

1.0

subject. Under this design referred to as Randomized Block Design (RBD), the data is gathered by block (i.e., by rater in this case) with random rating within the block, and it would be organized as shown in Table 4, where γ is the number of raters. In Table 4, yijk represents the kth replicate observation on subject i provided by rater j. The intrarater reliability coefficient γˆ under an RBD design is still defined by Equation (3) with the exception that the within-subject variation (WSV) and the between-subject variation (BSV) are defined as follows: 1

2 Sij , and mr m

WSV =

r

i=1 j=1

BSV = BSVo − where S2ij = BSVo =

1 n−1

1 m−1

WSV nr n
(yijk − yij· )2 and k=1

m
(yi·· − y··· )2 ;

(10)

i=1

yij is the average of all n scores rater j assigned to subject i, yi·· is the average of

INTRARATER RELIABILITY

7

Table 4. Intrarater Reliability Data on m Subjects with r Raters and n Replicates per Subject and per Rater Subjects 1 2 .. . i . .. m

Rater 1

···

Rater j

···

Rater r

y111 , . . . y11n y211 , . . . y21n .. . yill , . . . yiln . .. ymll , . . . ymln

... ... .. . ... . .. ...

y1j1 , . . . y1jn y2j1 , . . . y2jn .. . yijl , . . . yijn . .. ymjl , . . . ymjn

... ... .. . ... . .. ...

y1r1 , . . . y1rn y2rl , . . . y2rn .. . yirl , . . . yirn . .. ymrl , . . . ymrn

nr scores assigned to subject i, and y··· the overall mean scores. Note that the WSV is obtained by averaging the mr sample variances calculated at the cell level. Equation (10) offers the advantage of removing any influence of inter-rater variation when calculating the intrarater reliability. The number of replicates in an RBD design may vary by rater and by subject. We assumed it to be fixed in this section for the sake of simplicity. Although a single rater is sufficient to carry out an intrarater reliability experiment, the use of multiple raters may be recommended for burden reduction or for convenience. The techniques and the inferential framework discussed in this section work well for continuous data, such as the cholesterol level, but they are not suitable for nominal data. In the next section, I present some techniques specifically developed for nominal data. 2

NOMINAL SCALE SCORE DATA

Although the ICC is effective for quantifying the reproducibility of continuous data, nominal data raise new statistical problems that warrant the use of alternative methods. Rating subjects on a nominal scale amounts to classifying them into one of q possible response categories. The discrete nature of that data has the following two implications: 1. The notion of reproducibility is exact. A response category is reproduced when the initial and the replicate categories are identical, and unlike continuous data, nominal data are not subject to random measurement errors.

2. A rater may classify a subject on two occasions into the exact same category by pure chance with a probability that is non-negligible. Table 5 shows the distribution of 100 individuals with identified pulmonary abnormalities examined by a medical student on two occasions. On both occasions, the medical student found the same 74 individuals with pulmonary abnormalities and the same 15 individuals without any abnormalities. However, the student disagreed with himself on 11 individuals. These data, which are a modification of the results reported by Mulrow et al. (10), shows how analysts may organize intrarater reliability data, and it is used later in this section for illustration purposes. For intrarater reliability experiments based on two replicates per subject, analysts may organize the observations as shown in Table 6, where m is the number of subjects rated, and mkl the number of subjects classified in category k on the first occasion and in category l on the second occasion. If the experiment uses three replicates per subject or more, then a more convenient way to organize the data is shown in Table 7 where n is the number of replicates per subject, and nik the number of times subject i is classified into category k. 2.1 Intrarater Reliability: Single Rater and Two Replications When ratings come from a simple experiment based on a single rater, two replicates per subject, and two categories such as described in Table 5, the kappa coefficient of Cohen (2) or an alternative kappa-like statistic may

8

INTRARATER RELIABILITY

Table 5. Distribution of 100 Subjects with Respect to the Presence of Pulmonary Abnormalities Observed on two Occasions by a Medical Student First

Second Observation

Observation

Present

Present Absent Total

Absent

Total

1 15 16

75 25 100

74 10 84

be used to estimate the intrarater reliability coefficient. The medical student who generated Table 5 data could have obtained some of the 89 matches by pure chance because of the small number of response categories limited to two. Consequently, 89% will overestimate student’s self-consistency. Cohen’s (2) solution to this problem was a chance-corrected agreement measure γˆκ , which is known in the literature as kappa, and it is defined as follows: pa − pe γˆκ = (11) 1 − pe where for Table 5 data pa = (74 + 15)/100 = 0.89 is the overall agreement probability, and pe = 0.75 × 0.84 + 0.21 × 0.16 = 0.6636 is the chance-agreement probability. Consequently, the kappa coefficient that measures the medical student intrarater reliability is γˆ = 0.673. According to the Landis and Koch (11) benchmark, a kappa value of this magnitude is deemed substantial. In a more general setting with m subjects, two replicates per subject, and an arbitrary number q of response categories (see Table 6), the kappa coefficient of Cohen (2) is still defined by Equation (11), except the overall

agreement probability pa and the chanceagreement probability pe that are respectively defined as follows: pa =

q q

pkk , and pe = pk+ p+k k=1

(12)

k=1

where pkk = mkk /m, P+k = m+k /m, and Pk+ = mk+ /m The overall agreement probability is the proportion of subjects classified into the exact same category on both occasions (i.e., the diagonal of Table 6). The kappa coefficient will at times yield unduly low values when the ratings suggest high reproducibility. Cicchetti and Feinstein (12), as well as Feinstein and Cicchetti (13) have studied these unexpected results known in the literature as the kappa paradoxes. Several alternative more paradox-resistant coefficients are discussed by Brennan and Prediger (14). A Brennan-Prediger alternative denoted by γˆGI , which is often referred to as the G-Index (GI) and should be considered by practitioners, is defined as follows: γˆGI =

pa − 1/q . 1 − 1/q

(13)

Applied to Table 5 data, the BrennanPrediger coefficient becomes γˆGI = (0.89 −

Table 6. Distribution of m Subjects by Response Category and Replication Number. First-Replication Category 1 .. . k . .. q Total

Second-Replication Response Category 1

···

k

···

q

Total

m11 .. . mk1 . .. mq1 m+1

... .. . ... . .. ... ...

m1k .. . mkk . .. mqk m+k

... .. . ... .. . ... ...

m1q .. . mkq . .. mqq m+k

m+ .. . mk+ . .. mq+ m

INTRARATER RELIABILITY

0.5)/(1 − 0.5) = 0.78, which is slightly higher than the kappa. Aickin (15) presents an interesting discussion about kappa-like Intrarater reliability coefficients and suggests the use of his α coefficient. The α coefficient is based on a sound theory and uses the maximum likelihood estimation of some of its components obtained with a computationally intensive iterative algorithm. Gwet (16) proposed the AC1 statistic as a simple way to resolve the kappa paradoxes. The AC1 coefficient is defined as follows: γˆAC1 =

pa − p(1) e

reliability coefficient. The techniques discussed in this section generalize those of the previous section, and they are suitable for analyzing Table 7 data that involve three replicates or more per subject. All kappa-like statistics presented in the previous section can still be used with Table 7 data. However, the overall agreement probability pa is defined as the probability that two replicates random by chosen from the n replicates associated with a randomly selected subject, are identical. More formally pa is defined as follows:   q m 1

nik (nik − 1) (16) pa = m n(n − 1)

(14)

1 − p(1) e

where pa is defined by Equation (12), and the chance-agreement probability is as follows:

i=1

k=1

pk = (pk+ + p+k )/2

k=1

Concerning the calculation of chance-agreement probability, several versions have been proposed in the literature, most of which are discussed by Conger (17) in the context of inter-rater reliability, rather than in the context of intrarater reliability. Fleiss (18) suggested that chance-agreement probability be estimated as follows:

1
= pk (1 − pk ), where q−1 q

p(1) e

9

(15)

For Table 5 data, p1 = (0.75 + 0.84)/2 = 0/795, p2 = (0.25 + 0.16)/2 = 0.205, and p2 = 1−p1 . Consequently, Gwet’s chanceagreement probability is P(1) e = 2 × 0.795 × 0.205 = 0.32595. The AC1 statistic is then given by γˆAC1 = (0.89 − 0.32595)/(1 − 0.32595) ∼ = 0.84, which is more in line with the observed extent of reproducibility. Gwet (16) extensively discusses the statistical properties of the AC1 statistic as well as the origins of the kappa paradoxes.

p(F) e =

q


p2k where pk =

k=1

m 1
nik m n

(17)

i=1

Note that pk represents the relative number of times that a subject is classified into category k. Fleiss’ generalized kappa is then given by: (F) γˆF = (pa − p(F) e )/(1 − pe ).

2.2 Intrarater Reliability: Single Rater and Multiple Replications

Conger (17) criticized Fleiss’ generalized kappa statistic for not reducing to Cohen’s kappa when the number of replicates is

Using more than two replicates per subject can improve the precision of an intrarater

Table 7. Frequency Distribution of mn Observations by Subject and Response Category. Response Category Subject

1

···

k

···

q

Total

1 .. . i . .. m Total

n11 .. . ni1 . .. nn1 n+1

... .. . ... . .. ... ...

n1k .. . nik . .. nnk n+k

... .. . ... .. . ... ...

n1q .. . niq . .. nnq n+q

n .. . n . .. n mn

10

INTRARATER RELIABILITY

limited to two and proposed the following chance-agreement probability: p(C) e =

q


p2k −

k=1

q
s2k /n

occur in practice, and it should be dealt with using special methods that eliminate the impact of inter-rater variation. The general approach consists of averaging various probabilities calculated independently for each rater as previously discussed.

(18)

k=1

where s2k is the sample variance

2.3 Statistical Inference

1
s2k = (θjk − θ ·k )2 n−1 n

(19)

For an intrarater reliability coefficient to be useful, it must be computed with an acceptable level of precision; this notion can be defined and measured only within a formal framework for statistical inference. This section gives an overview of the main inferential approaches proposed in the literature and provides references for more inquiries. Several authors have proposed frameworks for statistical inference based on various theoretical models. Kraemer et al. (19) review many models that have been proposed in the literature. Kraemer (20) proposed a model under which the kappa coefficient can be interpreted as an intraclass correlation. Donner and Eliasziw (21), Donner and Klar (22), and Aickin (15) have proposed different models that may be useful in different contexts. This model-based approach poses two important problems for practitioners. The first problem stems from the difficulty of knowing which model is most appropriate for a particular situation. The second problem is the dependency of inferential procedures on the validity of the hypothesized model. Fortunately, a different approach to inference based on finite population sampling and widely used in the social sciences can resolve both problems.

j=1

where θ jk = mjk /m is the percent of subjects classified into category k on the jth occasion, and θ ·k is the average of these n values. To compute the variances s2k , k = 1, · · · , q, it could be useful to organize ratings as in Table 8. Both the Fleiss and Conger versions of kappa are vulnerable to the paradox problem previously discussed, and they yield reasonable intrarater reliability coefficients only when pk , the propensity for classification in category k, remain fairly constant from category to category. The generalized version of the AC1 statistic of Gwet (16) is a more paradox-resistant alternative to kappa, and it is based on Equation (14) with the exception that the chance-agreement probability is defined as follows: 1
pk (1 − pk ) q−1 q

p(1) e =

(20)

k=1

where pk is defined as in Equation (17). The situation where intrarater reliability data are collected by multiple raters may

Table 8. Frequency Distribution of mn Observations by Replicate Number and Response Category. Response Category Replication

1

···

k

···

q

Total

1 .. . j . .. n Total

m11 .. . mj1 . .. mn1 m+1

... .. . ... . .. ... ...

m1k .. . mjk . .. mnk m+k

... .. . ... .. . ... ...

m1q .. . miq . .. mnq m+q

m .. . m . .. m mn

INTRARATER RELIABILITY

The randomization approach or designbased inference is a statistical inference framework in which the underlying random process is created by the random selection of m subjects out of a predefined finite population of M subjects of interest. This approach is described in textbooks such as Kish (23) and Cochran (24), and it has been used extensively in the context of inter-rater reliability assessment by Gwet (16,25). The variances of many intrarater reliability coefficients presented in the second section can be found in Gwet (16,25). The first two sections present various approaches for evaluating the reproducibility of continuous and nominal data. These approaches are not recommended for ordinal or interval data, although ordinal clinical measurements such as the presence (no, possible, probable, definite) of a health condition as read on a radiograph, are commonplace. The objective of the next section is to present a generalization of kappa susceptible for use with ordinal and interval data. 3

ORDINAL AND INTERVAL SCORE DATA

Berry and Mielke (26) as well as Janson and Olsson (27,28) have generalized the kappa statistic to handle ordinal and interval data. In addition to being applicable to ordinal and interval data, these extensions can analyze multivariate data of subjects rated on multiple characteristics. Although Berry and Mielke (26) deserve credit for introducing the notions of vector score and Euclidean distance behind these extensions, Janson and Olsson (27) improved and expanded them substantially. Let us consider a simple intrarater reliability study in which a rater must rate all five subjects (m = 5) on two occasions (n = 2) on a three-level nominal scale (q = 3). If the rater classifies subject 1 into category 2 on the first occasion, then the corresponding score can be represented as a vector a11 = (0,1,0), with the second position of digit ‘‘1’’ indicating the category number where the subject is classified. The vector score associated with the classification of subject 1 into category 3 on the second occasion is a12 = (0,0,1). The squared Euclidean distance between a11 and a12 is

11

obtained by summing all three squared differences between the elements of both vectors and is given by: d2 (a11 , a12 ) = (0 − 0)2 + (0 − 1)2 + (1 − 0)2 = 2 Following Janon and Olsson (26), Cohen’s kappa coefficient can re-expressed as follows: 1
2 d ( ai1 , ai2 ) 5 5

γˆJO = 1 −

i=1 5 5

1

2 d ( ai1 , aj2 ) 52

(21)

i=1 j=1

The kappa coefficient as written in Equation (21) depends solely on several distance functions. Its generalization relies on the distance function’s ability to handle ordinal and interval data. If the scoring is carried out on a three-level ordinal scale, then each score will represent a single rank instead of threedimensional vector of 0s and 1s. If the categories in Table 6 are ordinal, then Equation (21) can be adapted to that data and yield the following more efficient kappa coefficient: q  q 

γˆJO = 1 −

pkl (k − l)2

k=1i=1 q q  

(22)

pk+ p+l (k − l)2

k=1l=1

To illustrate the use of kappa with ordinal scores, let us consider Table 9 data, which are a modification of the initial chest radiograph data that Albaum et al. (29) analyzed. A radiologist has examined 100 initial chest radiographs on two occasions to determine the presence of a radiographic pulmonary infiltrate. The four levels of the measurement scale for this experiment are ‘‘No,’’ ‘‘Possible,’’ ‘‘Probable,’’ and ‘‘Definite.’’ Because classifications of radiographs into the ‘‘Probable’’ and ‘‘Definite’’ categories agree more often than those in the ‘‘No’’ and ‘‘Definite’’ categories, the use of classic kappa of Cohen (2) will dramatically underestimate the intrarater reliability. Cohen’s kappa for Table 9 is given by γˆk = (0.57 − 0.3151)/(1 − 0.3151) ≈ 0.37. The generalized version of kappa based on Equation

12

INTRARATER RELIABILITY

Table 9. Distribution of 100 Subjects by Presence of Radiographic Pulmonary inFiltrate and Assessment Time Radiographic Assessment

Radiographic Assessment in Time 2

in Time 1

No

Possible

Probable

Definite

TOTAL

No Possible Probable Definite TOTAL

6 2 2 1 11

7 7 4 4 22

2 6 7 7 22

1 2 5 37 45

16 17 18 49 100

(22) yields an intrarater reliability coefficient of γˆJO = 1 − 0.89/2.41 = 0.63. This generalized version of kappa yields an intrarater reliability coefficient substantially higher and accounts for partial agreement in a more effective way. 4

CONCLUDING REMARKS

This article introduces the notion of intrarater reliability for continuous, nominal, ordinal, as well as interval data. Although the intraclass correlation coefficient is the measure of choice for continuous data, kappa and kappa-like measures defined by Equations (11), (13), (14), and (18)–(20) are generally recommended for nominal data. The extension of kappa to ordinal data is more efficient than classic kappa when the data is ordinal, and it is an important addition to the kappa literature. The literature on inter-rater reliability is far more extensive than that on intrarater reliability, particularly for discrete data, which is explained partially by the tendency researchers have to underestimate the importance of data reproducibility. Although many techniques were developed to measure interrater reliability, very few specifically address the problem of intrarater reliability. In this article, we have adapted some inter-rater reliability estimation procedures so they can be used for computing intrarater reliability coefficients. Unlike inter-rater reliability experiments that involve multiple raters, multiple subjects, and a single replicate per subject, intrarater reliability experiments typically involve a single rater and several replicates per subject. Consequently interrater reliability methods have been modified

by considering the replicates as ratings from different independent raters. Several authors, such as Fleiss and Cohen (30), Kraemer (20), and others, have attempted to interpret kappa as well as other kappa-like reliability measures as a form of intraclass correlation under certain conditions. The main justification for this effort stems from the need to link kappa to a population parameter and to create a framework for statistical inference. So far, no clear-cut theory can establish such a link in a broad setting. The connection of kappa to the intraclass correlation is unnecessary to have a good statistical inference framework. A satisfactory solution to this problem is the use of the finite population inference framework discussed in Gwet (16, 25). REFERENCES 1. S. M. Grundy, Second report of the expert panel on detection, evaluation, and treatment of high blood cholesterol in adults (Adult Treatment Panel II). 1993; National Institutes of Health, NIH Publication No. 93-3095. 2. J. Cohen, A coefficient of agreement for nominal scales. Educat. Psychol. Measurem. 1960; 20: 37–46. 3. R. L. Ebel, Estimation of the reliability of ratings. Psychometrika 1951; 16: 407–424. 4. J. J. Barko, The intraclass correlation coefficient as a measure of reliability. Psychol. Reports 1966; 19: 3–11. 5. P. E. Shrout and J. L. Fleiss, Intraclass correlations: uses in assessing rater reliability. Psychol. Bull. 1979; 86: 420–428. 6. J. M. Lachin, The role of measurement reliability in clinical trials. Clin. Trials 2004; 1: 553–566. 7. National Health and Nutrition Examination Survey. 2005.

INTRARATER RELIABILITY 8. K. O. McGraw and S. P. Wong, Forming inferences about some intraclass correlation coefficients. Psychol. Methods 1996; 1: 30–46. 9. B. Giraudeau and J. Y. Mary, Planning a reproducibility study: how many subjects and how many replicates per subject for an expected width of the 95 per cent confidence interval of the intraclass correlation coefficient. Stats. Med. 2001; 20: 3205–3214. 10. C. D. Mulrow, B. L. Dolmatch, E. R. Delong, J. R. Feussner, M. C. Benyunes, J. L. Dietz, S. K. Lucas, E. D. Pisano, L. P. Svetkey, B. D. Volpp, R. E. Ware, and F. A. Neelon, Observer variability in the pulmonary examination. J. Gen. Intern Med. 2007; 1: 364–367. 11. J. R. Landis and G. G. Koch, The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174. 12. D. V. Cicchetti and A. R. Feinstein, High agreement but low kappa: II. Resolving the paradoxes. J. Clin. Epidemiol. 1990; 43: 551–558. 13. A. R. Feinstein and D. V. Cicchetti, High agreement but low kappa: I. The problems of two paradoxes. J. Clin. Epidemiol. 1990; 43: 543–549. 14. Brennan, RL, and Prediger, DJ. Coefficient kappa: some uses, misuses, and alternatives. Educat. Psychol Measurem. 1981; 41: 687–699. 15. M. Aickin, Maximum likelihood estimation of agreement in the constant predictive probability model, and its relation to Cohen’s kappa. Biometrics 1990; 46: 293–302. 16. K. L. Gwet, Computing inter-rater reliability and its variance in the presence of high agreement. Br. J. Mathemat. Stat. Psychol. 2008. In press. 17. A. J. Conger, Integration and generalization of kappas for multiple raters. Psychol. Bull. 1980; 88: 322–328. 18. J. L. Fleiss, Measuring nominal scale agreement among many raters. Psychol. Bull. 1971; 76: 378–382. 19. H. C. Kraemer, V. S. Periyakoil, and A. Noda, Kappa coefficients in medical research. Stats. Med. 2002; 21: 2109–2129.

13

20. H. C. Kraemer, Ramifications of a population model for κ as a coefficient of reliability. Psychometrika 1979; 44: 461–472. 21. A. Donner, M. A. Eliasziw, A hierarchical approach to inferences concerning interobserver agreement for multinomial data. Stats. Med. 1997; 16: 1097–1106. 22. Donner, A, Klar, N. The statistical analysis of kappa statistics in multiple samples. Journal of Clinical Epidemiology. 1996; 49(9): 1053–1058. 23. K. Kish, Survey Sampling. New York: Wiley, 1965. 24. W. G. Cochran, Sampling Techniques, 3rd ed. New York: Wiley, 1977. 25. K. L. Gwet, Variance estimation of nominalscale inter-rater reliability with random selection of raters. Psychometrika. 2008. In press. 26. K. J. Berry and P. W. Mielke Jr., A generalization of Cohen’s Kappa agreement measure to interval measurement and multiple raters. Educat. Psychol. Measurem. 1988; 48: 921–933. 27. H. Janson and U. Olsson, A measure of agreement for interval or nominal multivariate observations. Educat. Psychol. Measurem. 2001; 61: 277–289. 28. H. Janson and U. Olsson, A measure of agreement for interval or nominal multivariate observations by different sets of judges. Educat. Psychol. Measurem. 2004; 64: 62–70. 29. M. N. Albaum, L. C. Hill, M. Murphy, Y. H. Li, C. R. Fuhrman, C. A. Britton, W. N. Kapoor, and M. J. Fine, PORT Investigators. Interobserver reliability of the chest radiograph in community-acquired pneumonia. CHEST 1996; 110: 343–350. 30. J. L. Fleiss and J. Cohen, The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educat. Psychol. Measurem. 1973; 33: 613–619.

FURTHER READING D. C. Montgomery, Design and Analysis of Experiments. New York: John Wiley & Sons, 2004. D. T. Haley, Using a New Inter-rater Reliability Statistic, 2007. Available: http://computingreports.open.ac.uk/index.php/2007/200716. ANOVA Using MS Excel. Available: http:// higheredbcs.wiley.com/legacy/college/mann/ 0471755303/excel manual/ch12.pdf.

14

INTRARATER RELIABILITY

CROSS-REFERENCES Inter-Rater Reliability Intraclass Correlation Coefficient Kappa Statistic Weighted Kappa Analysis of Variance (ANOVA)

INVESTIGATIONAL DEVICE EXEMPTION (IDE)

apply to devices in commercial distribution. Sponsors do not need to submit a PMA or Premarket Notification 510(k), register their establishment, or list the device while the device is under investigation. Sponsors of IDE’s are also exempt from the Quality System (QS) Regulation except for the requirements for design control.

An Investigational Device Exemption (IDE) refers to the regulations under 21 CFR (Code of Federal Regulations) 812. An approved IDE means that the Institutional Review Board (IRB) [and the Food and Drug Administration (FDA) for significant risk devices] has approved the sponsor’s study application and that all requirements under 21 CFR 812 are met. An IDE allows the investigational device to be used in a clinical study to collect safety and effectiveness data required to support a Premarket Approval (PMA) application or a Premarket Notification [510(k)] submission to FDA. Clinical studies are most often conducted to support a PMA. Only a small percentage of 510(k)’s require clinical data to support the application. Investigational use also includes clinical evaluation of certain modifications or new intended uses of legally marketed devices. All clinical evaluations of investigational devices, unless exempt, must have an approved IDE before the study is initiated. Clinical evaluation of devices that have not been cleared for marketing requires: • an IDE approved by an IRB. If the study

• • • •

involves a significant risk device, the IDE must also be approved by the FDA; informed consent from all patients; labeling for investigational use only; monitoring of the study and; required records and reports.

An approved IDE permits a device to be shipped lawfully for the purpose of conducting investigations of the device without complying with other requirements of the Food, Drug, and Cosmetic Act (Act) that would This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cdrh/devadvice/ide/index. shtml) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

INVESTIGATIONAL NEW DRUG APPLICATION PROCESS (IND)

an investigation and under whose immediate direction the investigational drug is administered or dispensed. A physician might submit a research IND to propose studying an unapproved drug or an approved product for a new indication or in a new patient population. 2. Emergency Use IND allows the FDA to authorize use of an experimental drug in an emergency situation that does not allow time for submission of an IND in accordance with 21 Code of Federal Regulations (CFR) Section 312.23 or Section 312.24. It is also used for patients who do not meet the criteria of an existing study protocol or when an approved study protocol does not exist. 3. Treatment IND is submitted for experimental drugs that show promise in clinical testing for serious or immediately life-threatening conditions while the final clinical work is conducted and the FDA review takes place.

In the United States, current federal law requires that a drug be the subject of an approved marketing application before it is transported or distributed across state lines. Because a sponsor will probably want to ship the investigational drug to clinical investigators in many states, it must seek an exemption from that legal requirement. The Investigational New Drug (IND) process (Fig. 1) is the means through which the sponsor technically obtains this exemption from the U.S. Food and Drug Administration (FDA). During a new drug’s early preclinical development, the sponsor’s primary goal is to determine whether the product is reasonably safe for initial use in humans and if the compound exhibits pharmacologic activity that justifies commercial development. When a product is identified as a viable candidate for further development, the sponsor then focuses on collecting the data and information necessary to establish that the product will not expose humans to unreasonable risks when used in limited, early-stage clinical studies. The FDA’s role in the development of a new drug begins after the drug’s sponsor (usually the manufacturer or potential marketer) has screened the new molecule for pharmacologic activity and acute toxicity potential in animals, and wants to test its diagnostic or therapeutic potential in humans. At that point, the molecule changes in legal status under the Federal Food, Drug, and Cosmetic Act and becomes a new drug subject to specific requirements of the drug regulatory system. There are three IND types:

There are two IND categories: 1. Commercial 2. Research (noncommercial) The IND application must contain information in three broad areas: 1. Animal Pharmacology and Toxicology Studies. Preclinical data to permit an assessment as to whether the product is reasonably safe for initial testing in humans. Also included are any previous experience with the drug in humans (often foreign use). 2. Manufacturing Information. Information pertaining to the composition, manufacturer, stability, and controls used for manufacturing the drug substance and the drug product. This information is assessed to ensure that the company can adequately produce and supply consistent batches of the drug.

1. Investigator IND is submitted by a physician, who both initiates and conducts This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/regulatory/applications/ ind page 1.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

INVESTIGATIONAL NEW DRUG APPLICATION PROCESS (IND)

Applicant (Drug Sponsor)

IND

Review by CDER

Medical

Pharmacology/ Toxicology

Chemistry

Sponsor Submits New Data

Safety Review

Safety Acceptable for Study to Proceed ?

Statistical

No

No

Clinical Hold Decision Yes

Yes Complete Reviews

Reviews Complete and Acceptable ?

Notify Sponsor

No

Sponsor Notified of Deficiencies

Yes No Deficiencies

Study Ongoing*

* While sponsor answers any deficiencies Figure 1. Flowchart of the Investigational New Drug process.

3. Clinical Protocols and Investigator Information. Detailed protocols for proposed clinical studies to assess whether the initial-phase trials will expose study participants to unnecessary risks. Also, information must be provided on the qualifications of clinical investigators— professionals (generally physicians) who

oversee the administration of the experimental agent—to assess whether they are qualified to fulfill their clinical trial duties. Finally, commitments must be made to obtain informed consent from the research participants, to obtain review of the study by an institutional review board, and to adhere to the investigational new drug regulations.

INVESTIGATIONAL NEW DRUG APPLICATION PROCESS (IND)

Once the IND has been submitted, the sponsor must wait 30 calendar days before initiating any clinical trials. During this time, the FDA has an opportunity to review the IND for safety to ensure that research participants will not be subjected to unreasonable risk.

3

INVESTIGATIONAL PRODUCT

The investigator should ensure that the investigational product(s) are used only in accordance with the approved protocol. The investigator, or a person designated by the investigator/institution, should explain the correct use of the investigational product(s) to each subject and should check, at intervals appropriate for the trial, that each subject is following the instructions properly. The sponsor is responsible for supplying the investigator(s)/institution(s) with the investigational product(s). The sponsor should not supply an investigator/institution with the investigational product(s) until the sponsor obtains all required documentation [e.g., approval/favorable opinion from IRB (Institutional Review Board)/IEC (Independent Ethics Committee) and regulatory authority(ies)]. The sponsor should ensure that written procedures include instructions that the investigator/institution should follow for the handling and storage of investigational product(s) for the trial and documentation thereof. The procedures should address adequate and safe receipt, handling, storage, dispensing, retrieval of unused product from subjects, and return of unused investigational product(s) to the sponsor [or alternative disposition if authorized by the sponsor and in compliance with the applicable regulatory requirement(s)]. The sponsor should

An Investigational Product is a pharmaceutical form of an active ingredient or placebo being tested or used as a reference in a clinical trial, including a product with a marketing authorization when used or assembled (formulated or packaged) in a way different from the approved form, when used for an unapproved indication, or when used to gain further information about an approved use. Responsibility for investigational product(s) accountability at the trial site(s) rests with the investigator/institution. Where allowed/required, the investigator/ institution may/should assign some or all of the duties of the investigator/institution for investigational product(s) accountability at the trial site(s) to an appropriate pharmacist or another appropriate individual who is under the supervision of the investigator/institution. The investigator/institution and/or a pharmacist or other appropriate individual, who is designated by the investigator/institution, should maintain records of the delivery of the product to the trial site, the inventory at the site, the use by each subject, and the return to the sponsor or alternative disposition of unused product(s). These records should include dates, quantities, batch/serial numbers, expiration dates (if applicable), and the unique code numbers assigned to the investigational product(s) and trial subjects. Investigators should maintain records that document adequately that the subjects were provided the doses specified by the protocol and should reconcile all investigational product(s) received from the sponsor. The investigational product(s) should be stored as specified by the sponsor and in accordance with applicable regulatory requirement(s).

• Ensure timely delivery of investigational

product(s) to the investigator(s), • Maintain records that document ship-

ment, receipt, disposition, return, and destruction of the investigational product(s), • Maintain a system for retrieving investigational products and documenting this retrieval (e.g., for deficient product recall, reclaim after trial completion, and expired product reclaim), and • Maintain a system for the disposition of unused investigational product(s) and for the documentation of this disposition.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

INVESTIGATIONAL PRODUCT

The sponsor should • Take steps to ensure that the investi-

gational product(s) are stable over the period of use, and • Maintain sufficient quantities of the investigational product(s) used in the trials to reconfirm specifications, should this become necessary, and to maintain records of batch sample analyses and characteristics. To the extent that stability permits, samples should be retained either until the analyses of the trial data are complete or as required by the applicable regulatory requirement(s), whichever represents the longer retention period.

INVESTIGATOR

The investigator(s) should be qualified by education, training, and experience to assume responsibility for the proper conduct of the trial; should meet all the qualifications specified by the applicable regulatory requirement(s); and should provide evidence of such qualifications through up-to-date curriculum vitae and/or other relevant documentation requested by the sponsor, the IRB/IEC, and/or the regulatory authority(ies). The investigator should be thoroughly familiar with the appropriate use of the investigational product(s) as described in the protocol, in the current Investigator’s Brochure, in the product information, and in other information sources provided by the sponsor. The investigator should be aware of, and should comply with, GCP and the applicable regulatory requirements. The investigator/institution should permit monitoring and auditing by the sponsor and inspection by the appropriate regulatory authority(ies). The investigator should maintain a list of appropriately qualified persons to whom the investigator has delegated significant trial-related duties. The investigator should be able to demonstrate (e.g., based on retrospective data) a potential for recruiting the required number of suitable subjects within the agreed recruitment period. The investigator should have sufficient time to properly conduct and complete the trial within the agreed trial period. The investigator should have available an adequate number of qualified staff and adequate facilities for the foreseen duration of the trial to conduct the trial properly and safely. The investigator should ensure that all persons assisting with the trial are adequately informed about the protocol, the investigational product(s), and their trialrelated duties and functions. During and following the participation of a subject in a trial, the investigator/institution should ensure that adequate medical care is provided to the subject for any adverse events, including clinically significant laboratory values, related to the trial. The investigator/institution should inform a subject when medical care is needed for intercurrent illness(es) of which the investigator becomes

An Investigator is a person responsible for conducting the clinical trial at a trial site. If a trial is conducted by a team of individuals at a trial site, the investigator is the responsible leader of the team and may be called the principal investigator. The sponsor is responsible for selecting the investigator(s)/institution(s). If a coordinating committee and/or coordinating investigator(s) are to be used in multicenter trials, their organization and/or selection are the responsibility of the sponsor. Before entering an agreement with an investigator/institution to conduct a trial, the sponsor should provide the investigator(s)/institution(s) with the protocol and an up-to-date Investigator’s Brochure and should provide sufficient time for the investigator/institution to review the protocol and the information provided. The sponsor should obtain the agreement of the investigator/institution • To conduct the trial in compliance with

Good Clinical Practice (GCP), with the applicable regulatory requirement(s), and with the protocol agreed to by the sponsor and given approval/favorable opinion by the IRB (Institutional Review Board)/IEC (Independent Ethics Committee), • To comply with procedures for data recording/reporting, • To permit monitoring, auditing, and inspection, and • To retain the essential documents that should be in the investigator/institution files until the sponsor informs the investigator/institution that these documents no longer are needed. The sponsor and the investigator/institution should sign the protocol or an alternative document to confirm this agreement. This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

INVESTIGATOR

aware. It is recommended that the investigator inform the primary physician of the subject about the participation of the subject in the trial if the subject has a primary physician and if the subject agrees to the primary physician being informed. Although a subject is not obliged to give his/her reason(s) for withdrawing prematurely from a trial, the investigator should make a reasonable effort to ascertain the reason(s), while fully respecting the rights of the subject.

INVESTIGATOR/INSTITUTION

market segment with a 38% share of total clinical trial grants. Nearly two thirds of the 9,000 investigative sites that conduct clinical trials in the United States annually fall into the part-time segment. Select sites in this segment are conducting as many as 25 or 30 clinical trials each year whereas other sites may conduct less than one trial annually. As industry sponsors continue to favor conducting their clinical trials in actualuse, community-based settings, and given the relatively low clinical trial operating costs within part-time sites, this segment is well positioned to continue to grow the faster than its peers (5). As Figure 1 shows, AMCs receive nearly all clinical research grants from government agencies and from foundations, and approximately 36% of all clinical trial grants from private sector pharmaceutical, biotechnology, and medical device companies (1, 6). The dedicated investigative site segment captures 26% of the market for clinical trial grants from industry. Dedicated sites derive nearly 100% of their annual revenue from clinical trial grants. This segment has undergone profound structural changes over the past decade and now includes a wide variety of private entities that include site management organizations, stand-alone dedicated research facilities, and managed site networks. These various entities will be discussed shortly (1).

KENNETH A. GETZ Tufts Center for the Study of Drug Development Boston, Massachusetts

Overall spending on clinical research grants to investigative sites exceeded $20 billion in 2007. Government agencies and foundations contribute $9.9 billion—approximately 50%—to the total amount spent on clinical research grants. However, these sponsors support a broad variety of activities that include patient-oriented research, epidemiologic and behavioral studies, outcomes research, and health services research (1). In 2007, government agencies and foundations spent approximately $1.9 billion on clinical trials of new medical interventions. The lion’s share of spending on clinical trials came from private sector sources that include pharmaceutical, biotechnology, and medical device manufacturers. Industry spent more than $10 billion on clinical trial grants to investigative sites in 2007 (2). Given the high failure rates of new molecular entities—which include both traditional chemical entities and biologics—under clinical investigation, the U.S. Department of Commerce estimates that industry sponsors spend more than $1 billion in capitalized costs to bring a single new drug through the 17-year R&D cycle and into the market place (3, 4). This figure includes the sponsor’s costs to develop drugs that fail to demonstrate efficacy and present a poor safety profile. 1

2

SEGMENT PROFILE: AMCs

Until the early 1990s, AMCs were the primary and the predominant home of industrysponsored clinical trials. As these programs became larger, more complex, and costly, industry sponsors grew tired of the inherent inefficiencies in working with academia that include protracted contractual and budget negotiations, bureaucratic and slow-moving institutional review boards (IRBs) and higher relative cost associated with poorer performance. (7) A select group of institutions— which include the University of Kansas Medical Center, Columbia-Presbyterian Medical Center, Johns Hopkins University School

LANDSCAPE SIZE AND STRUCTURE

Today, the market for clinical research grants is composed of three primary segments: academic medical centers (AMCs), part-time investigative sites, and dedicated investigative sites. The part-time segment consists of physicians in private, community settings that derive most (85% or more) of their annual revenue from clinical practice. In 2007, this highly fragmented and transient group has surpassed AMCs as the largest

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

INVESTIGATOR/INSTITUTION

Privatization of US Clinical Investigators

70% 51%

41%

36%

Figure 1. Privatization of US clinical investigators.

of Medicine, University of Rochester Medical School, New York University Medical Center, and George Washington Medical Center—centralized their administrative clinical research functions to compete with community-based investigative sites. Institutional culture, philosophy, and politics made it difficult to implement and to establish these offices. Central clinical trial offices serve faculty and study personnel by freeing up investigator and staff time; improving business development and marketing; and accelerating contract, budget, and IRB approval processes. Specific initiatives by central offices included the following: (1) building databases of a given institutions clinical research capabilities and patient populations to respond quickly to sponsor inquiries, (2) working with the IRB to expedite the approval process, (3) obtaining and negotiating new contracts on behalf of faculty, (4) training study coordinators and investigators, (5) providing patient recruitment support, and (6) tracking study performance. Today, most (70%) of the nearly 130 AMCs have established central offices to assist in supporting clinical research activities at their institutions (6). Still, AMCs continue to lose market share of clinical trial grants from industry. This loss is caused by several factors: many academic-based investigative sites

report that they receive poor returns on infrastructure investment, minimal institutional support, and limited incentive given renewed emphasis on the more prestigious governmentfunded clinical research programs as NIH funding surged (1). During the most recent 5-year period between 2002 and 2007, for example, total NIH funding grew nearly 4% annually to $28.8 billion FY2007. Moving forward, however, NIH funding growth is expected to level off and the NIH is beginning to phase out its long-established General Clinical Research Center (GCRC) program to make room for the new Clinical and Translational Science Awards (CTSA) model. The result will be an intensification of competition specifically for NIH clinical research grants (2). Many AMCs are turning their sights again toward industry-sponsored clinical trials as a vital source for much-needed financial resources in the near future. Fortunately, many clinical research professionals within pharmaceutical and biotechnology companies remain interested in forging, and in building, stronger clinical research relationships with academia. Whereas many AMCs are considering whether to make a more concerted push to expand their involvement in industry-funded clinical research activity, others have been implementing new strategies and tactics designed to improve their ability to compete

INVESTIGATOR/INSTITUTION

for industry study grants that include the implementation of services to expedite study start-up (e.g., contract and budget approvals, IRB review and approval), to provide higher quality training of study investigators and staff, and to accelerate the collection and management of clinical research data. Rising financial pressures will prompt many AMCs to expand their involvement in industry-funded clinical trials over the next several years. The NIH Roadmap for Medical Research and its push to establish Clinical and Translational Science Awards will serve as a catalyst in driving up AMC’s commitment to industry-sponsored clinical trials (6). By 2012, the NIH’s CTSA initiative will provide approximately $500 million annually to 60 AMCs to encourage more effective interaction and communication between clinical research and medical practice. 3 SEGMENT PROFILE: PART-TIME INVESTIGATIVE SITES Part-time investigative sites are a vital component of the study conduct services engine, yet this group of investigative sites is the most difficult to define given its highly fragmented nature. Investigative sites in this segment are very transient with extremely variable clinical research infrastructure and experience. Top sites in this segment conduct as many as 25 or 30 clinical trials each year, whereas bottom sites may do less than one annually. Although 40% of this segment has full-time coordinators, 31% of the segment does not have a single research nurse on staff. Turnover is the highest among investigators in this segment. One of every three part-time investigators who file the FDA-required 1572 form discontinues their involvement once the trial has ended (8). Sponsors have long favored placing a significant portion of their phase II and III clinical studies in actual-use settings. With rapidly expanding pipelines of promising new chemical entities, part-time sites can expect to see increasing numbers of annual clinical trial opportunities in the future. Additionally, industry anticipates that the demand for clinical research centers will exceed supply within the next several years. Given their

3

more flexible involvement in conducting clinical trials, part-time investigative sites play a valuable role in providing much needed capacity. With typically low overhead and minimal fixed costs, part-time sites can play a broader role as a variable study conduct resource that can transition flexibly into and out of clinical projects on demand. In essence, part-time sites may find larger opportunities to act as contract study conduct partners with dedicated clinical research centers. The dramatic growth in post-approval research spending represents another important trend that will increase the demand for part-time investigators. Post-approval studies are now conducted by sponsor companies for a wide variety of purposes: to extend knowledge about efficacy, safety and outcomes within actual use settings, to position newly launched drugs against standard and competing therapies, to prime the market and stimulate drug use, to observe prescribing behaviors and patient experiences, and to extend drug life cycles by targeting new populations and new markets. The FDA is also increasing its pressure on companies to continue the testing of new therapies after they are launched. Typically, phase IV research programs are conducted in community-based, clinical practice settings. Part-time sites report modest growth in year-over-year revenue and relatively high profitability, which is largely caused by the flexible and transient way that part-time sites approach conducting clinical trials. Although many very experienced part-time investigators with established clinical trial operations exist, as a group this segment is composed largely of investigative sites with wide variability in operating structure, experience, and performance. Annual revenue reported by part-time investigative sites ranges from $25,000 annually to as much as $1,000,000– $2,000,000 (1). Most (83%) part-time investigative sites report that they primarily focus on one or two specialty research areas. Few part-time sites consider themselves multispecialty research centers. Nearly half of all part-time investigative sites operate out of a group or network of

4

INVESTIGATOR/INSTITUTION

practicing physicians. Many sites in this segment report that several physicians within their provider network have been involved in conducting at least one clinical trial during the past five years. On average, part-time sites have one study coordinator on staff. The average age of a principal investigator in a part-time investigative site setting is 50 years. Most investigators are getting involved in industry-sponsored clinical research at later stages in their professional careers. These individuals are more established and more financially secure. Less than 10% of part-time principal investigators who conduct industry-sponsored clinical trials are under the age of 40. An estimated 45% of the more than 40,000 active clinical investigators— some 16,000 to 18,000 people—in any given year are conducting their clinical trials on a part-time basis in independent research centers (1). Operating profits among part-time sites vary widely—and it is a function of research infrastructure. Some of the largest part-time sites report operating profits as low as 5–7% of net revenue. As expected, part-time sites that conducted only a few clinical trials in 2006 report operating profits as high as 60–70% of grant revenue. Almost 20 cents of every clinical grant award dollar is profit for the part-time site. Minimal infrastructure and integrated and shared resources help part-time sites maintain higher relative operating margins. In addition, these sites are well positioned to handle the typical cash flow problems that result from late sponsor and CRO (contract research organization) payments. Part-time sites have a steady clinical practice revenue stream. But it is certainly not the economics alone that attracts practicing physicians to conduct clinical trials. These physicians are drawn to the science of new drug development, and they want to offer their patients access to medicines on the frontiers of medical knowledge. Both small and large part-time research centers have unique needs that must be addressed if this segment is to meet increasing demand and performance standards. To reach a comparable level of performance across this segment, experienced and new

entrant part-time sites will need to improve their abilities to start studies and enroll patients, to comply with evolving federal guidelines and regulations, to use emerging electronic solutions technologies more effectively, and to improve their operating efficiencies. At this time, part-time sites largely learn to conduct clinical research through actual experience. A more systematic and uniform approach, which is integrated into health professional training and supported by research sponsors, will likely be necessary. Ultimately, this study conduct services segment holds great promise in meeting growing demand and in offering much needed capacity. 4 SEGMENT PROFILE: DEDICATED INVESTIGATIVE SITES An estimated 500 dedicated investigative sites operate in the United States currently. Approximately two thirds of this total is stand-alone research centers. Close to 35% of all dedicated sites is part of a network (1). Dedicated sites are typically multispecialty and are relatively sophisticated in terms of their approach to managing their clinical research activity— from financial management to staffing and recordkeeping. They conduct an average of 30 clinical research studies per year. Similar to parttime community-based investigative sites, dedicated sites interact with centralized IRBs for human subject protection oversight. Most dedicated sites—nearly 90%— report that they are profitable with net operating margins of 11–15%. Dedicated investigative sites do not have a clinical practice with which to draw patients for their clinical trials. Instead, dedicated sites rely heavily on advertising and promotion to attract study volunteers. Approximately 40% of dedicated sites report that they have a full or part-time patient recruitment specialist on staff (1). In the mid-1990s, a group of dedicated sites formed to meet growing sponsor demands for clinical trial cost and time efficiency and improved data. The conceptual promise offered by these new Site Management Organizations (SMOs) was compelling:

INVESTIGATOR/INSTITUTION • Centralized clinical research operations • Standardized contracts and operating

procedures • Trained and accredited staff • New technologies to manage informa-

tion and to track performance management of patient recruitment and retention Systematic management of clinical data Streamlined regulatory and legal review and approval processes Reduced fixed costs to offer more competitive pricing Applied business and management principles

• Systematic • • • •

But since their introduction 15 years ago, most SMOs have struggled to deliver on these conceptual promises, and they have been through a wide variety of incarnations. The first SMOs emerged under a cloud of scandal when Future HealthCare, which was an early entrant, was indicted for manipulating its financial records to influence the capital markets. The mid-1990s saw a wave of new entrants— owned-site and affiliation model SMOs offering single- and multispecialty expertise—including Affiliated Research Centers (ARC), Clinical Studies Limited (CSL), Collaborative Clinical Research, Hill Top Research, Health Advance Institute (HAI), InSite Clinical Trials, Protocare, Integrated Neuroscience Consortium (INC), Rheumatology Research International (RRI), and several hybrid CRO-SMOs such as Clinicor, MDS Harris, and Scirex In the late 1990s, SMOs entered their most active period of venture capital fund raising and aggressively pursued expansion and diversification strategies. Collaborative Clinical raised $42 million in a 1996 initial public offering. ARC, INC, InSite, and HAI each closed rounds of venture capital financing. Phymatrix, which is a $200 million physician’s practice management group, acquired CSL for $85 million. Having raised more than $14 million in 1999, Radiant Research purchased peer SMO Hill Top Research. And nTouch Research (formerly Novum) raised $8 million in venture capital funding to double the size of its investigative site network through the acquisition of HAI.

5

By the early 2000s, many SMOs had exited the market whereas others diversified their services even more, venturing into traditional CRO and patient recruitment offerings. To name but a few, after a public offering, Collaborative Clinical Research renamed itself Datatrak and announced that it would be exiting the SMO business to become a provider of electronic clinical trial technology solutions. Integrated Neuroscience Consortium, MDS, and RRI focused attention on offering CRO services; The Essential Group (formerly ARC) abandoned its SMO services to focus on offering contract patient recruitment services; in 2005 nTouch Research was acquired by CRO Accelovance; InSync Research shut down its operations in 2000 after selling four of its seven sites to Radiant Research; Clincare, which had hoped to expand its regional network of eight owned sites, also exited the business in 2000. In late 2001, ICSL sold its SMO assets to Comprehensive Neuroscience. And in 2003, Radiant Research completed the acquisition of Protocare—another top five SMO—thereby expanding Radiant’s network to nearly 60 sites. Radiant Research has clearly demonstrated that it is an outlier—it is growing and operating profitably among traditional SMOs that have either exited or struggled to survive in this market. Today, Radiant owns and operates more than 30 dedicated sites, and it has 550 employees including 225 study coordinators. With the exception of Radiant Research, industry observers have concluded that traditional SMOs failed to demonstrate and deliver the value of centralized investigative site operating controls. SMO management structures were cumbersome and challenged the organization’s ability to operate profitably for a sustainable period of time. Traditional SMOs struggled to compete for sufficient levels of new business, to manage positive cash flows, and they failed to achieve revenue and earnings growth that would satisfy their investors. SMO insiders express the failures of traditional SMOs differently: Many believe that research sponsors failed to use the SMO properly by neglecting to empower the SMO to manage an entire study.

6

INVESTIGATOR/INSTITUTION

The past several years has observed strong growth in new structures among managed site networks. The Tufts Center for the Study of Drug Development has been tracking this development. These new players tend to operate regionally, and they have extremely lean centralized operations. They are composed of small networks of sites—on average five—that are connected loosely through minimal standardized management services provided across the network. Basic management services include contract, budget, and regulatory assistance. Although many decentralized site networks have established exclusive arrangements with their investigators, they encourage autonomy and direct interaction with study sponsors. As a result, they address sponsor’s ‘‘investigator-centric’’ preferences while offering minimal—although essential—operating support. Examples of decentralized site networks include Pivotal Research, Benchmark Research, ResearchAcrossAmerica, and RxTrials. Decentralized site networks are building momentum. They generated $215 million in study grant revenue in 2006, and they are growing 9.3% annually. They seem well positioned to capture growing market share over the next several years while containing operating costs (1).

5 THE LANDSCAPE MOVES OVERSEAS Recently, a dramatic shift has occurred in the use of ex–U.S.-based investigative sites. As Figure 2 indicates, major pharmaceutical and biotechnology companies have been placing an increasing proportion of their clinical trials at offshore locations throughout Central and Eastern Europe, Latin America, and parts of Asia. Several sponsors report that they now routinely place most of their clinical trials among ex–U.S.-based investigative sites. This shift has major implications not only for U.S.-based investigative sites, but also for sponsors and for CROs who are looking to optimize their relationships with an increasingly global network of investigators. It is widely accepted that conducting trials overseas is far less expensive and a healthy supply of Western-trained physicians are located there, who are eager to serve as investigators and to study coordinators. Drug shipment issues have largely been overcome and the adoption of e-clinical technology solutions helped to address operating support issues that used to bottleneck overseas projects. Most importantly, the abundance of treatment-na¨ıve patients abroad has translated into speedier clinical trial recruitment and improved retention rates. In the developing world, vast numbers of patients suffer

An Increasingly Global Mix of FDA- Regulated Investigators

Figure 2. An increasingly global mix of FDA-regulated clinical investigators.

INVESTIGATOR/INSTITUTION

from diseases that range from the familiar to the exotic. These patients are often eager to enroll in clinical trials at the request of their physicians and to gain access to free medical care, tests, and complimentary therapies that they could otherwise not afford. In an analysis of 1572 filings with the FDA, the Tufts Center for the Study of Drug Development has found that most investigative sites still hail from the United States, but the geographic mix is changing rapidly. A decade ago, less than 1 in 10 FDA-regulated investigators was based outside the U.S.; in 2006, 40% of FDA-regulated investigators was ex-U.S. The Tufts Center also found that, whereas the overall number of principal investigators within the United States has been growing by 5.5% annually since 2002, the number of FDA-regulated principal investigators based outside the US has been growing by more than 15% annually during that same time period (8). Major pharmaceutical companies have already tipped the balance in favor of ex-U.S. investigative sites, or they are planning to do so within the next couple of years. Wyeth, for example, reports that it does more than half of its clinical trials among ex–U.S.-based investigative sites. In 2007, nearly 60% of all investigative sites recruited by Novartis were ex-U.S. Mid-sized P&G Pharmaceuticals, which now conducts more than one third of its trials among ex-U.S. sites, says that it is routinely looking to place trials overseas, often as replacements for traditionally U.S.-based investigators. And GlaxoSmithKline said that it conducted 29% of its clinical trials abroad in 2006andhopes to increase that figure to 50% of its clinical trials by the end of 2006. Merck is conducting significantly more studies internationally than in the past. More than 40% of its investigative sites are now based outside the U.S. in regions that include Central and Eastern Europe, Latin America, Australia, New Zealand, and, increasingly, the Asia-Pacific (8). As clinical trial volume and scope continues to increase, an increasingly global community of investigative sites seems ready to meet drug development capacity needs. For U.S.-based investigative sites, this trend signals that competition for a smaller number

7

of trials will intensify domestically. Research sponsors have already been turning to more sophisticated criteria (e.g., historical performance metrics and details on access to specific patient populations) to justify the higher relative costs of U.S.-based clinical investigators. Competition among ex-U.S. investigators is also intensifying. As these sites take on more operating infrastructure, they will need to sustain a larger stream of clinical trial activity. Clinical research managers who represent different global regions within biopharmaceutical companies must compete with their own colleagues to gain a larger share of their organization’s finite clinical trial work. At this time, Eastern Europe and Latin America tend to be the favored regions. Growing use of ex-U.S. based investigative sites is also expected to increase outsourcing to contract service providers well positioned within markets abroad and offering significantly lower operating expenses. Given the added complexity of simultaneously conducting clinical trials internationally, several sponsors report a heavier reliance on CROs to meet more aggressive development timelines. Global Phase I clinical trials activity is one area that is beginning to migrate back to the United States and Canada largely due to a weaker US dollar combined with tighter global regulatory requirements. During the next several years, Phase I investigative sites can expect intensifying competition and consolidation among CRO-owned and community-based investigator-owned facilities. The shift toward increasing use of investigative sites outside the U.S., particularly in Phase II and III clinical research studies, will likely attract public scrutiny. With mounting political pressure, the FDA is expected to weigh-in with new restrictions and reforms. Media and policy-makers have already identified this trend as a potentially explosive issue. This issue will fuel the fire of an already damaged and eroded relationship with the public. Despite data suggesting that regardless of where clinical trials are conducted, the U.S. market typically gains early access to new medical innovations, and the media tends to depict the growing prevalence of ex–U.S.-based investigative sites as

8

INVESTIGATOR/INSTITUTION

profit-motivated and exploitive of vulnerable global markets. It is critical for research sponsors to educate the public and policymakers proactively about the full rationale behind broad-based global development programs. New drug approval in the United States requires that sponsors submit an application that contains substantial evidence of effectiveness based on the results of adequate and well-controlled studies. A few cases in which a new drug has been approved for the U.S. market are based on data from well-controlled studies conducted solely by investigative sites abroad. With regard to the exact mix of U.S.-based and ex–U.S.-based investigative sites, the FDA remains elusive. No minimum number of U.S. patients is required in a New Drug Application. Sponsors may design and conduct their studies in any manner provided that they demonstrate, via prospectively planned analyses, that the ex-U.S. patients are similar to U.S. patients, both in terms of pretreatment disease characteristics and treatment outcomes. Now and in the near future, sponsors and the FDA will continue to evaluate their studies for their applicability to patient populations and medical care settings in the United States. Select diseases and their respective patient affinity groups may vie to keep some clinical trials within the United States given wide variations in diet, lifestyle, and health care consumption. Among these diseases are experimental treatments for age-related illnesses (such as Alzheimer’s, Parkinson’s, and ALS) gastro-intestinal and endocrine disorders. The investigative site landscape is poised to continue to evolve substantially. Research sponsors increasingly seek more efficient and productive collaborations with study conduct providers. As the cost and the duration of clinical research studies continue to increase, ample opportunities exist for investigative sites to improve. Streamlined operating processes, improved financial controls, increased adoption of integrated clinical research data collection and management technologies, the implementation of more effective patient recruitment and retention practices are but a few of the key areas in which successful investigative sites will excel and differentiate.

REFERENCES 1. J. DiMasi et al., The price of innovation: new estimates of drug development Costs. J. Health Econ. 2003; 22: 151–185 2. J. DiMasi, Risk in new drug development: approval success rates for investigational drugs. Clin. Pharmacol. Ther. 2001; 69: 297–307. 3. K. Getz, The Market for Site Management Organizations and Clinical Trial Conduct Services. Dorland Medical and Healthcare Marketplace. 2003; 18: 216–218 4. K. Getz, Number of active investigators in FDA-regulated clinical trials drop. The Impact Report, The Tufts Center for the Study of Drug Development at Tufts University. 2005; 7(3). 5. K. Getz, The Evolving SMO in the United States. New York: Marcel Dekker, 2004. 6. H. Moses et al., Financial anatomy of biomedical research. JAMA 2005; 1333–1342. 7. R. Rettig, The industrialization of clinical research. Health Aff. 2000; 129–146. 8. N. Sung et al., Central challenges facing the national clinical research enterprise. JAMA 2003; 289: 1278–1287.

INVESTIGATOR’S BROCHURE

of importance to the investigator. If a marketed product is being studied for a new use (i.e., a new indication), then an IB specific to that new use should be prepared. The IB should be reviewed at least annually and revised as necessary in compliance with the written procedures of the sponsor. More frequent revision may be appropriate depending on the stage of development and the generation of relevant new information. However, in accordance with Good Clinical Practice (GCP), relevant new information may be so important that it should be communicated to the investigators and possibly to the Institutional Review Boards (IRBs)/Independent Ethics Committees (IECs) and/or regulatory authorities before it is included in a revised IB. Generally, the sponsor is responsible for ensuring that an up-to-date IB is made available to the investigator(s), and the investigators are responsible for providing the up-to-date IB to the responsible IRBs/IECs. In the case of an investigator-sponsored trial, the sponsor-investigator should determine whether a brochure is available from the commercial manufacturer. If the investigational product is provided by the sponsorinvestigator, then he or she should provide the necessary information to the trial personnel. In cases where preparation of a formal IB is impractical, the sponsor-investigator should provide, as a substitute, an expanded background information section in the trial protocol that contains the minimum current information described in this guideline.

The Investigator’s Brochure (IB) is a compilation of the clinical and nonclinical data on the investigational product(s) that are relevant to the study of the product(s) in human subjects. Its purpose is to provide the investigators and others involved in the trial with the information to facilitate their understanding of the rationale for, and their compliance with, many key features of the protocol, such as the dose, dose frequency/interval, methods of administration, and safety monitoring procedures. The IB also provides insight to support the clinical management of the study subjects during the course of the clinical trial. The information should be presented in a concise, simple, objective, balanced, and nonpromotional form that enables a clinician or potential investigator to understand it and make his/her own unbiased risk–benefit assessment of the appropriateness of the proposed trial. For this reason, a medically qualified person generally should participate in the editing of an IB, but the contents of the IB should be approved by the disciplines that generated the described data. This guideline delineates the minimum information that should be included in an IB and provides suggestions for its layout. It is expected that the type and extent of information available will vary with the stage of development of the investigational product. If the investigational product is marketed and its pharmacology is widely understood by medical practitioners, then an extensive IB may not be necessary. Where permitted by regulatory authorities, a basic product information brochure, package leaflet, or labeling may be an appropriate alternative, provided that it includes current, comprehensive, and detailed information on all aspects of the investigational product that might be This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

KAPLAN–MEIER PLOT

In its original form, the K–M estimator is applicable to right censored data, meaning that, for some patients, it is only known that their true survival time exceeds a certain censoring time. Right censoring is exclusively considered here, and extensions of the K–M estimator to other types of censoring are merely mentioned at the end of this article. Different types of censoring are described elsewhere in the encyclopedia. Nearly all textbooks on survival analysis discuss the K–M estimator in detail. The classic book by Kalbfleisch and Prentice (2), a standard reference on survival analysis for many years, increased the popularity of the K–M estimator substantially. The more practically oriented reader may refer to the textbooks by Marubini and Valsecchi (3) or Klein and Moeschberger (4). Rigorous derivations of the statistical properties of the Kaplan–Meier estimator are provided, for example, in the book by Andersen et al. (5). The practical analysis can be performed using any sound statistical software package. For instance, SAS provides the procedure PROC LIFETEST, SPSS the procedure KM, R/S-Plus the procedure SURVFIT, and STATA the procedure STS GRAPH. Next, the K–M estimator is described including an estimation of its variance and the calculation of confidence intervals. The application of the plot is illustrated using data from a particular clinical study. Then, the estimation of the median survival time and of survival probabilities of specific time points is discussed. Furthermore, practical notes are provided on the use of the K–M estimator for the analysis of clinical survival data and its interpretation. Finally, a few additional topics are considered.

¨ NORBERT HOLLANDER

University Hospital of Freiburg Institute of Medical Biometry and Medical Informatics Freiburg, Germany

1

INTRODUCTION

The Kaplan–Meier plot or Kaplan–Meier curve is the standard method to describe and compare graphically the overall survival of groups of patients. This method is applied to survival time, which is defined as time from treatment start or date of randomization to death or last date of follow up, and, more general, to any type of time-to-event data, such as time-to-disease progression, time-to-detoriation, or time-to-first toxicity. Sometimes, the time-to-event is also called failure time. In the sequel, the term ‘‘survival time’’ is used as a synonym for any time-to-event. In addition to its broad applicability, the Kaplan–Meier plot has further advantages: It displays without bias nearly all information concerning survival time obtained in the dataset. Furthermore, with the Kaplan–Meier plot, the outcome of a trial can easily be explained to clinicians. Therefore, the Kaplan–Meier plot is omnipresent in clinical trials literature. No trial investigating survival time should be published without showing the curve. The Kaplan–Meier plot is the graphical presentation of the nonparametric Kaplan–Meier estimator (K–M estimator) of the survival curve. The K–M estimator generalizes the empirical distribution function of a sample in the presence of censoring. It’s name is derived from the seminal paper of E. L. Kaplan and P. Meier entitled ‘‘Nonparametric estimation from incomplete observations’’ in the Journal of the American Statistical Association (1). As a result of the importance of the Kaplan–Meier plot, this paper has become one of the most cited statistical papers. Sometimes, the K–M estimator is also denoted as product-limit estimator, which was the name originally used by Kaplan and Meier.

2

ESTIMATION OF SURVIVAL FUNCTION

Consider the situation in which the survival time is studied for a homogenous population. The survival function S(t) = Pr(T > t) is the basic quantity to describe time-to-event data, where S(t) is defined as the probability of an individual surviving time t. Censoring is

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

KAPLAN–MEIER PLOT

usually assumed to be independent from the occurrence of the event (e.g., death), which assumes that the additional knowledge of censoring before any time t does not alter the risk of failure at t. Let n denote the sample size of a study (= number of patients) and assume that deaths are observed at k different time points with k ≤ n. To calculate the K–M estimator, the data has to be arranged as in Table 1: The observed death times are ordered by ascending values t1 < t2 < . . .< tk . The symbol dj denotes the number of deaths at time point tj (dj ≥1) and rj is the number of patients ‘‘at risk’’ just before time point tj (j = 1, . . ., k). The latter contains all patients without an observed event just before time point tj . ‘‘At risk’’ just before tj are, therefore, all patients dying at tj or thereafter and those censored at tj or thereafter. Censored observations are given in Table 1 indirectly only by the number of patients ‘‘at risk,’’ between tj−1 and tj , rj−1 − rj − dj−1 censored survival times exist. The K–M estimator for the survival distribution function S(t) is then obtained by ˆ = S(t)


 tj ≤t

1−

dj rj

 j = 1, . . . k

where rj is the number of patients at risk (i.e., aliveand not censored) just before time tj and denotes the product over all observed tj ≤t

death times tj less than or equal to t. The variance of the K–M estimator is commonly estimated by the so-called Greenwood formula: ˆ ˆ 2 Var(S(t)) = S(t)

 tj ≤t

dj rj (rj − dj )

Table 1. Arrangement of Time-to-Event Data Time points

Number of patients ‘‘at risk’’

Number of deaths

t1 t2 . .. tk

r1 r2 . .. rk

d1 d2 . .. dk

where
tj ≤t denotes the sum over all observed death times tj less than or equal to t. This formula allows the calculation of the standard ˆ at each time point tj . deviation of S(t) In large samples, the K–M estimator, evaluated at a given time t, is approximately normally distributed so that a standard 100(1 − α)% pointwise confidence interval for S(t) can be calculated by  ˆ ˆ ± z1−α/2 Var(S(t)) S(t)

(1)

where z1−α/2 denotes the 1 − α/2 quantile of the standard normal distribution. The confidence interval, which is routinely calculated by several statistical packages, is appropriate for studies with n = 200 or more patients with k > 100 observed deaths (6). However, for small sample sizes (e.g., n = 50), Equation (1) serves as a rough approximation only and may be improved (see Practical Note 7). If the upper confidence bound is larger than one, it is generally set equal to one and the lower confidence bound is set equal to zero if it is negative. 2.1 An Example The computation of the K–M estimator is illustrated with the data on remission durations of leukemia patients. This dataset is taken from a controlled clinical trial published by Freireich et al. (7) and has been used previously by several authors (e.g., References 3 and 4) for illustration. In the clinical trial, it was investigated whether patients who had achieved complete remission after some induction treatment could benefit from further treatment. Here, only the patients treated with 6mercaptopurine (6-MP) chemotherapy are considered. The time-to-event of interest is the time from complete remission to relapse. For the 21 patients of the 6-MP group, the observed remission durations or rightcensored times are recorded in weeks. These times were, rearranged in ascending order: 6,6,6,6*,7,9*,10,10*,11,11*,13,16,17*,19*, 20*,22, 23,25*, 32*,34* and 35*, where * indicates a censored observation. Table 2 illustrates the single steps of the computation, and Fig. 1 displays the resulting Kaplan–Meier plot including the standard

0.50

0.75

1.00

3

0.25

symbol 'l' denotes a censored observation 95 % confidence Interval for the median survival time tnz

0.00

Survival Probability (= Probability of staying in remission)

KAPLAN–MEIER PLOT

12 weeks

0

5

~ t0.5 = 23 weeks

15 20 25 10 Time since complete remission (weeks)

30

∞

35

Figure 1. Kaplan–Meter plot with standard pointwise 95% confidence interval for the leukemla data

pointwise 95% confidence intervals derived from Equation (1). Estimated survival probˆ abilities S(t), describing, in this example, the probability of staying in complete remission, are obtained for each time point t since complete remission. At time point t = 0, all patients are in complete remission and the estimated survival probability is, therefore, ˆ S(0) = 1. After 12 weeks, for example, the estimated probability of being still in ˆ remission is S(12) = 0.753, with a 95% confidence interval of [0.564; 0.942]. 2.2 Practical Notes 1. The K–M estimator of the survival curve is a step function with jumps at the observed uncensored survival times. The size of these jumps depends not only on the number of events observed at each event time tj , but also on the pattern of the censored observations before tj . If and only if all observed times are uncensored, the K–M estimator is equal to one minus the empirical distribution function and jumps by the amount 1/n if n is the sample size. Note that, in the latter special case, the K–M

plot and the respective software are still applicable. 2. If both an event and a censored observation are observed at exactly the same time point, the event is assumed to have occurred first. This assumption is crucial for the mathematical analysis but convenient from the practical point of view. 3. To calculate the K–M estimator, the time is partitioned into a sequence of time intervals, where the number and lengths of intervals are determined by the observed uncensored survival times in the data. In the example provided, the intervals are 0 < t ≤ 6 weeks, 6 < t ≤ 7 weeks, 7 < t ≤ 10 weeks, and so on. A similar approach for estimating survival curves is the so-called life-table estimator, also known as actuarial method. With this approach, the time is partitioned into a data-independent fixed sequence of intervals that are almost always of equal lengths. For instance, in lifetable analysis for human populations, the length of each interval is usually

4

KAPLAN–MEIER PLOT

1 year. Once the set of intervals has been chosen, the construction of the estimator follows the basic idea used for the Kaplan–Meier estimator. More details on various types of life-table estimators can be found in the book by Lee (8). In clinical trials in which individual data on survival time are available, the K–M estimator should be preferred because it is more precise. 4. The K–M estimator is a product of conditional probabilities. For each time interval (see Practical Note 3), the probability of being a survivor at the end of the interval is estimated on the condition that the subject was a survivor at the beginning of the interval. This conditional probability is denoted as ‘‘factor contributed to the K–M estimator’’ ˆ in Table 2. The survival probability S(t) at some time point t is calculated as the product of the conditional probabilities of surviving each time interval up to time point t, which is done recursively as described in the second footnote of Table 2. 5. The K–M estimator is well defined for all time points less than the largest observed survival or censoring time tlast . If tlast corresponds to an event

(i.e., an uncensored observation), then the estimated survival curve is zero beyond this point. If tlast corresponds to a censored observation, as in the above example, where tlast = 35 weeks, the K–M estimator should be considered as undefined beyond tlast , because when this last survivor would have died if the survivor had not been censored is not known. Therefore, a correct Kaplan–Meier plot should end at this last observed time point and the curve should not be prolonged to the right (right-censored!). 6. To improve the interpretability of results, the K–M plot should contain information on the censoring distribution, which is especially relevant if many patients are censored after the last observed death, thereafter the estimated survival probability remains constant until the last censored observation. In Fig. 1, censored observations are marked by a symbol; 4 patients are censored after the last observed death with censoring times of 25*, 32*, 34*, and 35* weeks. Considering the K–M plot without this information, the authors only would have known that the last observed censored time is 35*

Table 2. Illustration of the Computation of the Kaplan-Meier Estimator with Standard Pointwise 95% Confidence Bounds for the Leukemia Data of Freireich et al. (7)

j 1 2 3 4 5 6 7

Number of Factor KM-estimator Ordered patients Number of contributed for all patients to the t in the distinct at risk§ failure just before dying at KM-estimator interval time tj at t [tj , tj+1 ) times time tj j   dj ˆ # tj rj dj 1− S(t) rj 6 7 10 13 16 22 23

21 17 15 12 11 7 6

3 1 1 1 1 1 1

18/21 16/17 14/15 11/12 10/11 6/7 5/6

18/12 = 0.857 0.857 × 16/17 = 0.807 0.807 × 14/15 = 0.753 0.753 × 11/12 = 0.690 0.690 × 10/11 = 0.627 0.627 × 6/7 = 0.538 0.538 × 5/6 = 0.448

Standard deviation based on Greenwood variance estimator  ˆ Var(S(t))

Standard pointwise 95% confidence interval for S(t)

0.0764 0.0869 0.0963 0.1068 0.1141 0.1282 0.1346

[0.707; 1.000] [0.636; 0.977] [0.564; 0.942] [0.481; 0.900] [0.404; 0.851] [0.286; 0.789] [0.184; 0.712]

§ Note that not only d but also the censored observations 6*,9*,10*,11*,17*,19*,20*,25*,32*,34* and 35* change the set j of patients at risk # S ˆ (0) = 1, for t the K-M estimator ˆ (t) is calculated recursively: at time point t = 0 all patients are in remission leading to S 1     ˆ ˆ is S(t 1 ) = S(0) ·

d 1− 1 r1

d2 ˆ ) = S(t ˆ , S(t 2 1 ) · 1− r 2

etc.

KAPLAN–MEIER PLOT

weeks and that the remaining 3 censored observations (this number would usually be unknown too) were somewhere between 23 weeks and 35 weeks. Thus, without any information on the censoring distribution, the K–M plot should be interpreted until the last observed death only. For large samples, information concerning the censoring distribution could be given by adding the number of patients ‘‘at risk’’ for equidistant time points to the time axis. 7. For small samples, better pointwise confidence intervals can be constructed ˆ using transformations of S(t) such as ˆ log(− log(S(t))) (6). Based on this transformation, the resulting 100(1 − α)% confidence interval is not symmetric about the K–M estimator of the survival function. In the example provided, one would obtain a 95% confidence interval for the probability of being still in remission after 12 weeks, S(12), of [0.503; 0.923] in contrast to that reported above, which was [0.564; 0.942] around 0.753. 8. It should be noted that alternatives exist to the Greenwood formula for variance estimation (see, for example, Reference 9). Using these alternatives may effect pointwise confidence intervals!

2.3 Median Survival Time For the presentation of survival data with censoring, the entire estimated survival ˆ curve, S(·), together with standard errors or confidence intervals, is usually the best choice. Nevertheless, summary statistics such as location estimates are sometimes also useful. Beside the presentation of survival probabilities for a specific time point t, the median, or 0.5 quantile, is a useful estimator of location. The median survival time t˜0.5 is determined by the relation S(t˜0.5 ) = 0.5. Notice that the K–M estimator is a step function and, hence, does not necessarily attain the value 0.5. Therefore, t˜0.5 is usually defined ˆ as the smallest time t, for which S(t) ≤ ˆ 0.5 (i.e., the time t where S(t) jumps from

5

a value greater than 0.5 to a value less or equal to 0.5). A commonly used method to calculate confidence intervals for the median survival time t˜0.5 has been proposed by Brookmeyer and Crowley (10), which is based on a modification of the construction of pointwise confidence intervals for S(t) described above. Based on the standard confidence interval in Equation (1), a 100(1 − α)% confidence interval for t˜0.5 is the set of all time points t that satisfy the following condition: ˆ − 0.5 S(t) ≤ z1−α/2 −z1−α ≤   S(t) ˆ Var The resulting confidence interval for t˜0.5 can easily be obtained from the pointwise confidence intervals for S(t): The lower confidence bound is the smallest value for which the lower confidence bound of S(t) is less than or equal to 0.5, and the upper confidence bound for t˜0.5 is the smallest t-value for which the upper confidence bound of S(t) is less than or equal to 0.5. Therefore, one can easily determine the corresponding confidence interval from the graph of the Kaplan–Meier Plot and its pointwise 95% confidence band (see Fig. 1). The median survival time (i.e., the median duration from complete remission to relapse) for leukaemia patients tretated with 6-MP is 23 weeks. The corresponding lower confidence bound is 13 weeks. As the upper confidence bound for S(t) does not fall below the value 0.5, one may set the upper confidence bound for t˜0.5 to infinity (= ∞). 2.4 More Practical Notes 9. If the estimated survival probability is larger than 0.5 at any time point t (which may occur when considering patients with a good prognosis), the median survival time t˜0.5 cannot be calculated. 10. Instead of the median t˜0.5 , the K–M estimator can also be used to provide estimates of other quantiles of the survival time distribution. Recall that the p-quantile t˜p is estimated by ˆ the smallest time t at which S(t) is less than or equal to 1-p. Analogous to t˜0.5 , the corresponding confidence

6

KAPLAN–MEIER PLOT

interval for t˜p can be obtained from the pointwise confidence interval of S(t). 11. Using a transformation to calculate pointwise confidence intervals for S(t) (see Practical Note 7), the confidence interval for t˜p should be based on the same transformation and, therefore, may differ from the approach described above. In the example provided, the 95% confidence interval for t˜0.5 does not change. 12. Use of the median survival time has obvious advantages as compared with the mean, which is highly sensitive to the right tail of the survival distribution, where estimation tends to be imprecise because of censoring. The ∞ mean survival time µ = 0 S(t) dt can naturally be estimated by substituting ˆ the K–M estimator S(t) for S(t). This estimator is appropriate only when tlast corresponds to a death (see Practical Notes 2) and censoring is light. Consequently, it should be used with caution.

3

ADDITIONAL TOPICS 1. The pointwise confidence intervals described above are valid only at a fixed single time point t0 (in the example, t0 = 12 weeks was considered). Although these intervals are plotted by many statistical software packages in addition to the K–M estimator as a function of time (see Fig. 1), the curves cannot be interpreted as a confidence band with, for example, 95% confidence that the entire survival function lies within the band. Proper confidence bands can be √ derived from the weak convergence of n(Sˆ − S)/S to a mean zero Gaussian martingale. Two important types of such confidence bands are the bounds of Hall and Wellner (11) and the equal precision bands (12). 2. Besides right-censoring other kinds of data incompleteness, such as left truncation, may be present in survival analysis. In epidemiologic applications, for

example, individuals are often not followed from time zero (e.g., the birth, if age is the relevant time scale), but only from a later entry time (conditional on survival until this entry time). For such left-truncated data, the usual K–M estimator may be calculated with a modified risk set, but it is of little practical use, because the estimates ˆ S(t) have large sampling errors. More useful is the estimation of the conditional survival distribution function (see, for example, Reference 4). 3. Left-censoring occurs when some individuals have experienced the event of interest before the start of the observation period. For samples that include both left-censoring and right–censoring, a modified K–M estimator of the survival function has been suggested by Turnbull (13). 4. Besides the event of interest (e.g., relapse) and right-censored observations, other events (e.g., non-diseaserelated deaths) may occur. Thus, for patients dying before relapse, the event of interest, namely relapse, could not be observed. Therefore, it is said that the two events, death and relapse, compete with each other. In such socalled competing risk situations, the occurrence of the competing risk (here the non-disease-related death) is often inadequately treated as right-censored observation, and the survival function for the event of interest is estimated by the K–M estimator. However, as illustrated, for example, by Schwarzer et al. (14), the use of the K–M estimator is not appropriate in competing risk situations: Calculating the K–M estimator for both Sˆ (time to the event of first interest) and Sˆ (time to non-diseaserelated death) where the corresponding competing risk (death in the first case, event of interest in the second case) is treated as right-censored observation the sum of both survival probabilities for a fixed time point t0 may be larger than 1. However, as the two events are mutually exclusive, the sum must not exceed 1. For competing risks, the estimation of survival functions based on

KAPLAN–MEIER PLOT

cumulative incidences is appropriate, however (14, 15).

REFERENCES 1. E. L. Kaplan and P. Meier, Non-parametric estimation from incomplete observations. J. Amer. Stat. Assoc. 1958; 53: 457–481. 2. J. Kalbfleisch and R. Prentice, The Statistical Analysis of Failure Time Data. New York: Wiley, 1980. 3. E. Marubini and M. G. Valsecchi, Analysing Survival Data from Clinical Trias and Observational Studies. Chichester: Wiley, 1995. 4. J. P. Klein and M. L. Moeschberger, Survival Analysis: Techniques for Censored and Truncated Data. New York: Springer, 1997. 5. P. K. Andersen, O. Borgan, R. D. Gill, and N. Keiding, Statistical Methods Based on Counting Processes. New York: Springer, 1992. 6. Ø. Borgan and K. Liestøl, A note on confidence intervals and bands for the survival function based on transformations. Scand. J. Stat. 1990; 17: 35–41. 7. E. J. Freireich et al., The effect of 6-mercaptopurine on the duration of steroid-induced remissions in acute leukaemia: a model for evaluation of other potentially useful therapy. Blood 1963; 21: 699–716. 8. E. T. Lee, Statistical Methods for Survival Data Analysis. New York: Wiley, 1992. 9. O. O. Aalen and S. Johansen, An empirical transition matrix for nonhomogeneous Markov chains based on censored observations. Scand. J. Stat. 1978; 5: 141–150. 10. R. Brookmeyer and J. J. Crowley, A confidence interval for the median survival time. Biometrics 1982; 38: 29–41. 11. W. Hall and J. A. Wellner, Confidence bands for a survival curve from censored data. Biometrika 1980; 67: 133–143. 12. V. N. Nair, Confidence bands for survival functions with censored data: a comparative study. Technometrics 1984; 14: 265–275. 13. B. W. Turnbull, Nonparametric estimation of the survivorship function with doubly censored data. J. Amer. Stat. Assoc. 1974; 69: 169–173. 14. G. Schwarzer, M. Schumacher, T. B. Maurer, and P. E. Ochsner, Statistical analysis of failure times in total joint replacement. J. Clin. Epidemiol. 2001; 54: 997–1003.

7

15. T. A. Gooley, W. Leisenring, J. Crowley, and B. E. Storer, Estimation of failure probabilities in the presence of competing risks: new representations of old estimators. Stat. Med. 1999; 18: 695–706.

KAPPA

Table 1.

RICHARD J. COOK University of Waterloo, Waterloo, Ontario, Canada

In medical research it is frequently of interest to examine the extent to which results of a classification procedure concur in successive applications. For example, two psychiatrists may separately examine each member of a group of patients and categorize each one as psychotic, neurotic, suffering from a personality disorder, or healthy. Given the resulting data, questions may then be posed regarding the diagnoses of the two psychiatrists and their relationship to one another. The psychiatrists would typically be said to exhibit a high degree of agreement if a high percentage of their diagnoses concurred, and poor agreement if they often made different diagnoses. In general, this latter outcome could arise if the categories were ill-defined, the criteria for assessment were different for the two psychiatrists, or their ability to examine these criteria differed sufficiently, possibly as a result of different training or experience. Poor empirical agreement might therefore lead to a review of the category definitions and diagnostic criteria, or possibly retraining with a view to improving agreement and hence consistency of diagnoses and treatment. In another context, one might have data from successive applications of a test for dysplasia or cancer from cervical smears. If the test indicates normal, mild, moderate, or severe dysplasia, or cancer, and the test is applied at two time points in close proximity, ideally the results would be the same. Variation in the method and location of sampling as well as variation in laboratory procedures may, however, lead to different outcomes. In this context, one would say that there is empirical evidence that the test is reliable if the majority of the subjects are classified in the same way for both applications of the test. Unreliable tests would result from the sources of variation mentioned earlier. Again, empirical evidence of an unreliable test may lead to refinements of the testing procedure.

T2 = 1

T2 = 2

Total

T1 = 1 T1 = 2

x11 x21

x12 x22

x1· x2·

Total

x·1

x·2

x·· = n

1 THE KAPPA INDEX OF RELIABILITY FOR A BINARY TEST For convenience, consider a diagnostic testing procedure generating a binary response variable T indicating the presence (T = 1) or absence (T = 2) of a particular condition. Suppose this test is applied twice in succession to each subject in a sample of size n. Let T k denote the outcome for the kth application with the resulting data summarized in the two-by-two table (Table 1). where xij denotes the frequency at which
T 1 = i and T 2 = j, xi· = 2j=1 xij , and x·j =
2 i=1 xij , i = 1, 2, j = 1, 2. Assuming that test results on different subjects are independent, conditioning on n leads to a multinomial distribution for the outcome of a particular table with  f (x; p) =

x11

 2  2 xij n pij , x12 x21 x22 i=1 j=1

x = (x11 , x12 , x21 , x22 )
, p = (p11 , p12 , p21 , and p22 = 1 − p11 p22 )
,

− p12 − p21 . Let pi· = 2j=1 pij and p·j = 2i=1 pij . Knowledge of p would correspond to a complete understanding of the reliability of the test. Since knowledge of p is generally unattainable and estimation of p does not constitute a sufficient data reduction, indices of reliability/agreement typically focus on estimating one-dimensional functions
of p. A natural choice is p0 = 2i=1 pii , the probability of
raw agreement, which is estimated as pˆ 0 = 2i=1 xii /n. If p0 = 1, then the test is completely reliable since the probability of observing discordant test results is zero. Similarly, if pˆ 0 is close to unity, then it suggests that the outcomes of the two

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

KAPPA

applications concurred for the vast majority of the subjects. However, several authors have expressed reluctance to base inferences regarding reliability on the observed level of raw agreement (see (3) and references cited therein). The purported limitations of pˆ 0 as a measure of reliability stem from the fact that p0 reflects both ‘‘chance’’ agreement and agreement over and above that which would be expected by chance. The agreement expected by chance, which we denote by pe , is computed on the basis of the marginal distribution, defined by p1· and p·1 , and under the assumption that the outcomes of the two tests are independent conditional on the
true status. Specifically, pe = 2i=1 pi· p·i is
estimated by pˆ e = 2i=1 x1· x·1 /n2 . To address concerns regarding the impact of nonnegligible chance agreement, Cohen (3) defined the index kappa which takes the form κ=

p0 − pe , 1 − pe

and indicated that it can be interpreted as reflecting ‘‘the proportion of agreement after chance agreement is removed from consideration’’. This can be seen by noting that p0 − pe is the difference in the proportion of raw agreement and the agreement expected by chance, this being the agreement arising due to factors not driven by chance. If p0 − pe > 0, then there is agreement arising from nonchance factors; if p0 − pe = 0, then there is no additional agreement over that which one would expect based on chance; and if p0 − pe < 0, then there is less agreement than one would expect by chance. Furthermore, 1 − pe is interpreted by Cohen (3) as the proportion ‘‘of the units for which the hypothesis of no association would predict disagreement between the judges’’. Alternatively, this can be thought of as the maximum possible agreement beyond that expected by chance. An estimate of κ, denoted κ, ˆ is referred to as the kappa statistic and may be obtained by replacing p0 and pe with their corresponding point estimates, giving κˆ =

pˆ 0 − pˆ e . 1 − pˆ e

(1)

Table 2. T 2 = 1 T 2 = 2 T 2 = 3 . . . T 2 = R Total T1 = 1 T1 = 2 T1 = 3 .. . T1 = R

x11 x21 x31 .. . xR1

x12 x22 x32 .. . xR2

x13 x23 x33 .. . xR3

Total

x·1

x·2

x·3

... ...

x1R x2R x3R .. . xRR

x1· x2· x3· .. . xR·

...

x·R

x·· = n

... ... ...

2 THE KAPPA INDEX OF RELIABILITY FOR MULTIPLE CATEGORIES When the classification procedure of interest has multiple nominal categories, assessment of agreement becomes somewhat more involved. Consider a diagnostic test with R possible outcomes and let T k denote the outcome of the kth application of the test, k = 1, 2. Then T k takes values on {1, 2, 3, . . ., R} and interest lies in assessing the extent to which these outcomes agree for k = 1 and k = 2. An R × R contingency table may then be constructed (see Table 2), where again xij denotes the frequency with which the first application of the test led to outcome i and the second led to outcome j, i = 1, 2, . . ., R, j = 1, 2, . . . , R. A category-specific measure of agreement may be of interest to examine the extent to which the two applications tend to lead to consistent conclusions with respect to outcome r, say. In this problem there is an implicit assumption that the particular nature of any disagreements are not of interest. One can then collapse the R × R table to a 2 × 2 table constructed by crossclassifying subjects with binary indicators such that T k = 1 if outcome r was selected at the kth application, T k = 2 otherwise, k = 1, 2. A category-specific kappa statistic can then be constructed in the fashion indicated earlier. This can be repeated for each of the R categories giving R such statistics. In addition to these category-specific measures, however, an overall summary index of agreement is often of interest. The kappa statistic in (1) is immediately generalized for the R × R (R > 2) table as follows. Let pij denote the probability of T 1

KAPPA

= i and T 2 = j, one of the R2 multino
mial probabilities, pi· = R j=1 pij , and p·j =
R p , i = 1, 2, . . . , R, j = 1, 2, . . . , R. Then, ij i=1 as before, pˆ ij = xij /n, pˆ i· = xi· /n, pˆ ·j = x·j /n,

pˆ 0 = R ˆ ii , pˆ e = R ˆ i· pˆ ·i , and the overi=1 p i=1 p all kappa statistic takes the same form as in (1). This overall kappa statistic can equivalently be written as a weighted average of category-specific kappa statistics (6). The kappa statistic has several properties that are widely considered to be attractive for measures of agreement. First, when the level of observed agreement, reflected by pˆ 0 , is equal to the level of agreement expected by chance (pˆ e ), κˆ = 0. Secondly, κˆ takes on its maximum value of 1 if and only if there is perfect agreement (i.e. pˆ 0 = 1 arising from a diagonal table). Thirdly, the kappa statistic is never less than −1. The latter two features require further elaboration, however, as the actual upper and lower limits on κˆ are functions of the marginal frequencies. In particular, κˆ takes on the value 1 only when the marginal frequencies are exactly equal and all off-diagonal cells are zero. Values less than 1 occur when the marginal frequencies are the same but there are different category assignments in the table or, more generally, when the marginal frequencies differ (when the marginal frequencies differ there are necessarily nonzero diagonal cells and hence some disagreements). It is natural then to expect the kappa statistic for such a table to be less than unity. Cohen (3) shows that the maximum possible value of κˆ takes the form x·· κˆ M =

R 

min(xi· , x·i ) −

i=1

R  i=1

x2·· −

R 

If the marginal frequencies for the two tests are uncorrelated (as measured by the product–moment correlation of the margins (3)), then the lower bound for κˆ is κˆ L = −(R − 1)−1 . When the marginal frequencies are negatively correlated, κˆ L > − (R − 1)−1 . However, when the marginal frequencies are positively correlated, κˆ L < −(R − 1)−1 . It is only as the number of categories reduces to two, the correlation of the marginal frequencies approaches 1, and the variances of the marginal frequencies increase, that κˆ L approaches −1 (3). Having computed a kappa statistic for a given contingency table it is natural to want to characterize the level of agreement in descriptive terms. Landis & Koch (11) provide ranges that suggest, beyond what one would expect by chance, 0.75 < κˆ typically represents excellent agreement, 0.40 < κˆ < 0.75 fair to good agreement, and κˆ < 0.40 poor agreement. While there is some appeal to this convenient framework for the interpretation of κ, ˆ caution is warranted. Frequently, it will be of interest to construct confidence intervals for the index kappa. Fleiss et al. (8) derive an approximate large sample estimate for the variance of κ, ˆ v ar(κ), ˆ as



R 

pˆ ii [1 − (pˆ i· + pˆ ·i )(1 − κ)] ˆ 2

i=1

+ (1 − κ) ˆ 2

xi· x·i

 i

,

pˆ ij (pˆ ·i + pˆ j· )2

j=i

(2)

xi· x·i

3

− [κˆ − pˆ e (1 − κ)] ˆ 2

 [x..(1 − pˆ e )2 ], (3)

i=1

and argues that this is intuitively reasonable since differences in the marginal frequencies necessarily lead to a reduction in the level of agreement and hence κ. ˆ Cohen then suggests that if one is interested in assessing the proportion of the agreement permitted by the margins (correcting for chance), then one computes κ/ ˆ κˆ M . We return to the topic of marginal frequencies and their influence on the properties of κ later in the article.

and Fleiss (6) recommends carrying out tests (see Hypothesis Testing) and constructing confidence intervals by assuming approximate normality of (κˆ − κ)/[ var(κ)] ˆ 1/2 and proceeding in the standard fashion. For tests regarding the null hypothesis H0 : κ = 0, an alternate variance estimate may be derived from (3) by substituting 0 for κ, ˆ and pˆ i· pˆ ·j for

4

KAPPA

pˆ ij , giving ˆ v ar0 (κ)  R   pˆ i· pˆ ·i [1 − (pˆ i· + pˆ ·i )]2 + pˆ i· pˆ ·j = k=1

 × (pˆ ·i + pˆ j· )2 − p2e

i=j

[x·· (1 − pˆ e )2 ],

(4)

with tests carried out as described above. 3

THE WEIGHTED KAPPA INDEX

The discussion thus far has focused on situations in which the test serves as a nominal classification procedure (e.g. as in the psychiatric diagnosis example at the beginning of the article). In such settings, since there is no natural ordering to the outcomes, any disagreements are often considered to be equally serious and the methods previously described are directly applicable. In some circumstances with nominal scales, however, certain types of disagreements are more serious then others and it is desirable to take this into account. Furthermore, when the outcome is ordinal (as in the cervical cancer screening example), it is often of interest to adopt a measure of agreement that treats disagreements in adjacent categories as less serious than disagreements in more disparate categories. For the test based on cervical smears designed to classify the condition of the cervix as healthy, mildly, moderately, or severely dysplastic, or cancerous, if on one occasion the test suggested mild dysplasia and on another moderate, this type of disagreement would be considered less serious than if a cervix previously diagnosed as cancerous was subsequently classified as mildly dysplastic. In general, the seriousness reflects clinical implications for treatment and the consequences of wrong decisions. Weighted versions of the kappa statistic were derived by Cohen (4) to take into account the additional structure arising from ordinal measures or from nominal scales in which certain types of disagreement are of more importance than others. In particular, the objective of adopting a weighted kappa

statistic is to allow ‘‘different kinds of disagreement’’ to be differentially weighted in the construction of the overall index. We begin by assigning a weight to each of the R2 cells; let wij denote the weight for cell (i, j). These weights may be determined quite arbitrarily but it is natural to restrict 0 ≤ wij ≤ 1, set wii to unity to give exact agreement maximum weight, and set 0 ≤ wij < 1 for i = j, so that all disagreements are given less weight than exact agreement. The selection of the weights plays a key role in the interpretation of the weighted kappa statistic and also impacts the corresponding variance estimates, prompting Cohen (4) to suggest these be specified prior to the collection of the data. Perhaps the two most common sets of weights are the quadratic weights, with wij = 1 − (i − j)2 /(R − 1)2 , and the so-called Cicchetti weights, with wij = 1−|i − j|/(R − 1) (1,2). The quadratic weights tend to weight disagreements just off the main diagonal more highly than Cicchetti weights, and the relative weighting of disagreements farther from the main diagonal is also higher with the quadratic weights. Clearly, these two weighting schemes share the minimal requirements cited above. The weighted kappa statistic then takes the form κˆ (w) =

pˆ (w) ˆ (w) e 0 −p 1 − pˆ (w) e

,

(5)



R = R ˆ ij and pˆ (w) where pˆ (w) e = i=1 j=1 wij p
R
R
R 0 wij pˆ i· pˆ ·j . If wi· = j=1 pˆ ·j wij and i=1
j=1 w·j = R p i=1 ˆ i· wij , then the large-sample variance of κˆ (w) is estimated by v ar(κˆ (w) )  R  R  = pˆ ij [wij − (wi· + w·j )(1 − κˆ (w) )]2 i=1 j=1

 − [κˆ (w) − pˆ (w) ˆ (w) )]2 e (1 − κ 2 [x2·· (1 − pˆ (w) e ) ]

(6)

and, as before, tests and confidence intervals may be carried out and derived in the standard fashion assuming asymptotic normality var(κˆ (w) )]1/2 . As of the quantity (κˆ (w) − κ (w) )/[

KAPPA

in the unweighted case, a variance estimate appropriate for testing H0 :κ (w) = 0 may be derived by substituting pˆ i· pˆ ·j for pˆ ij , and 0 for κˆ (w) in (6). We note in passing that the weighted kappa with quadratic weights has been shown to bear connections to the intraclass correlation coefficient. Suppose that with an ordinal outcome the categories are assigned the integers 1 through R from the ‘‘lowest’’ to ‘‘highest’’ categories, respectively, and assignment to these categories is taken to correspond to a realization of the appropriate integer value. Fleiss & Cohen (7) show that the intraclass correlation coefficient computed by treating these integer responses as coming from a Gaussian general linear model for a two-way analysis of variance, is asymptotically equivalent to the weighted kappa statistic with quadratic weights.

4 THE KAPPA INDEX FOR MULTIPLE OBSERVERS Thus far we have restricted consideration to the case of two applications of the classification procedure (e.g. two successive applications of a diagnostic test, two physicians carrying out successive diagnoses, etc.). In many situations, however, there are multiple (>2) applications and interest lies in measuring agreement on the basis of several applications. Fleiss (5) considered the particular problem in which a group of subjects was examined and classified by a fixed number of observers, but where it was not necessarily the same set of observers carrying out the assessments for each patient. Moreover, Fleiss (5) assumed that it was not possible to identify which observers were involved in examining the patients. For this problem, we require some new notation. Let M denote the number of subjects, N denote the number of observers per subject, and R denote the number of categories as before. Therefore, NM classifications are to be made. Let nij denote the number of times the ith subject was assigned to the jth category. A measure of overall raw agreement for the assignments on the ith

5

subject is given by R  nij (nij − 1)

qˆ i =

j=1

N(N − 1)

,

which can be interpreted as follows. N  With  N posobservers per subjects there are 2   nij sible pairs of assignments. There are 2 which agree on category   j and hence a total
R nij number of pairs of assignments j=1 2 which concur altogether for the ith subject. Thus, (7) simply represents the proportion of all paired assignments on the ith subject for which there was agreement on the category. The overall measure of raw observed agreement over all subjects is then given by
ˆ i , which equals qˆ 0 = M −1 M i=1 q M  R 

qˆ 0 =

n2ij

i=1 j=1

MN(N − 1)

−

1 . N−1

(8)

As before, however, some agreement would be expected among the observers simply by chance and the kappa statistic in this setting corrects for this. The expected level of agreement is computed by noting that M  nij

pˆ j =

i=1

MN

is the sample proportion of all assignments
R made to category j, with ˆ j = 1. So if j=1 p pairs of observers were simply assigning subjects to categories at random and independently one can estimate that they would be expected to agree according to pˆ e =

R 

pˆ 2j ,

(9)

j=1

then the kappa statistic is computed by correcting for chance in the usual way as κˆ =

qˆ 0 − pˆ e . 1 − pˆ e

(10)

6

KAPPA

The sample variance for (10) is derived by Fleiss et al. (9) to be v ar(κ) ˆ



2 R R    pj (1 − pj ) − pj (1 − pj ) = 2 j=1

j=1

 (1 − 2pj )



 MN(N − 1)

R 

2 pj (1 − pj )

j=1

(11) and is typically used for tests or interval estimation in the standard fashion. When the same set of raters assesses all subjects and individual raters scores are known, it is not possible to use the results of Fleiss (5) without ignoring the rater-specific assignments. For this context, Schouten (13) proposed the use of indices based on weighted sums of pairwise measures of observed and expected levels of agreement. In particular, for a given pair of raters and a given pair of categories, observed and expected measures of agreement may be computed as earlier. Then, for each pair of raters, a measure of overall observed agreement may be obtained by taking a weighted average of such measures over all pairwise combinations of categories. Given a corresponding measure of expected agreement, an overall kappa statistic can be computed in the usual fashion. Schouten (13) then described how to obtain kappa statistics reflecting agreement over all observers, agreement between a particular observer and the remaining observers, and agreement within and between subgroups of observers. 5

GENERAL REMARKS

MaClure & Willett (12) provide a comprehensive review and effectively highlight a number of limitations of the kappa statistics. In particular, they stress that for ordinal data derived from categorizing underlying continuous responses, the kappa statistic depends heavily on the often arbitrary category definitions, raising questions about interpretability. They also suggest that the

use of weights, while attractive in allowing for varying degrees of disagreement, introduces another component of subjectivity into the computation of kappa statistics. Perhaps the issue of greatest debate is the so-called prevalence, or base-rate, problem of kappa statistics. Several other authors have examined critically the properties and interpretation of kappa statistics (10,14,15), and the debate of the merits and demerits continues unabated. Despite the apparent limitations, the kappa statistic enjoys widespread use in the medical literature and has been the focus of considerable statistical research. REFERENCES 1. Cicchetti, D. V. (1972). A new measure of agreement between rank ordered variables, Proceedings of the American Psychological Association 7, 17–18. 2. Cicchetti, D. V. & Allison T. (1973). Assessing the reliability of scoring EEG sleep records: an improved method, Proceedings and Journal of the Electro-physiological Technologists’ Association 20, 92–102. 3. Cohen, J. (1960). A coefficient of agreement for nominal scales, Educational and Psychological Measurement 20, 37–46. 4. Cohen, J. (1968). Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit, Psychological Bulletin 70, 213–220. 5. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters, Psychological Bulletin 76, 378–382. 6. Fleiss, J. L. (1981). Statistical Methods for Rates and Proportions, 2nd Ed. Wiley, New York. 7. Fleiss, J. L. & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability, Educational and Psychological Measurement 33, 613–619. 8. Fleiss, J. L., Cohen, J. & Everitt, B. S. (1969). Large sample standard errors of kappa and weighted kappa, Psychological Bulletin 72, 323–327. 9. Fleiss, J. L., Nee, J. C. M. & Landis, J. R. (1979). Large sample variance of kappa in the case of different sets of raters, Psychological Bulletin 86, 974–977. 10. Kraemer, H. C. & Bloch, D. A. (1988). Kappa coefficients in epidemiology: an appraisal of a

KAPPA reappraisal, Journal of Clinical Epidemiology 41, 959–968. 11. Landis, J. R. & Koch, G. G. (1977). The measurement of observer agreement for categorical data, Biometrics 33, 159–174. 12. MaClure, M. & Willett, W. C. (1987). Misinterpretation and misuse of the kappa statistic, American Journal of Epidemiology 126, 161–169.

7

13. Schouten, H. J. A. (1982). Measuring pairwise interobserver agreement when all subjects are judged by the same observers, Statistica Neerlandica 36, 45–61. 14. Thompson, W. D. & Walter S. D. (1988). A reappraisal of the kappa coefficient, Journal of Clinical Epidemiology 41, 949–958. 15. Thompson, W. D. & Walter S. D. (1988). Kappa and the concept of independent errors, Journal of Clinical Epidemiology 41, 969–970.

KEFAUVER–HARRIS DRUG AMENDMENTS

advertising from the Federal Trade Commission to the FDA.

In 1962, news reports about how Food and Drug Administration (FDA) Medical Officer Frances O. Kelsey, M.D., Ph.D., had kept the drug thalidomide off the U.S. market aroused public interest in drug regulation. Thalidomide had been marketed as a sleeping pill by the German firm Chemie Grunenthal, and it was associated with the birth of thousands of malformed babies in Western Europe. ‘‘In the years before 1962, Senator Estes Kefauver had held hearings on drug costs, the sorry state of science supporting drug effectiveness, and the fantastic claims made in labeling and advertising,’’ Temple says. ‘‘Well-known clinical pharmacologists explained the difference between well-controlled studies and the typical drug study. With the [Food, Drug and Cosmetic] FD&C Act ‘in play’ because of thalidomide, Congress had the opportunity to make major changes.’’ In October 1962, Congress passed the Kefauver–Harris Drug Amendments to the Federal FD&C Act. Before marketing a drug, firms now had to not only prove safety, but also they had to provide substantial evidence of effectiveness for the product’s intended use. Temple says, ‘‘That evidence had to consist of adequate and well-controlled studies, a revolutionary requirement.’’ ‘‘Also critically, the 1962 amendments required that the FDA specifically approve the marketing application before the drug could be marketed, another major change.’’ The Kefauver–Harris Drug Amendments also asked the Secretary to establish rules of investigation of new drugs, which include a requirement for the informed consent of study subjects. The amendments also formalized good manufacturing practices, required that adverse events be reported, and transferred the regulation of prescription drug This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/fdac/features/2006/106 cder.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

LANS-DEMETS: ALPHA SPENDING FUNCTION

Assume that the procedure is a one-sided test but that the process can be easily generalized to introduce two one-sided symmetric or asymmetric boundaries. In general, a test statistic Z(j), j= 1, 2, 3, . . . ,J is computed at each successive interim analysis. In the large sample case and under the null hypothesis, the Z(j)s are standard N(0,1). At each analysis, the test statistic is compared with a critical value Zc (j). The trial would continue as long as the test statistic does not exceed the critical value. That is, continue the trial as long as

DAVID L. DEMETS University of Wisconsin-Madison Madison, Wisconsin

K. K. GORDON LAN Johnson & Johnson Raritan, New Jersey

The randomized control clinical trial (RCT) is the standard method for the definitive evaluation of the benefits and risks of drugs, biologics, devices, procedures, diagnostic tests, and any intervention strategy. Good statistical principles are critical in the design and analysis of these RCTs (1,2). RCTs also depend on interim analysis of accumulating data to monitor for early evidence of benefit, harm, or futility. This interim analysis principle was established early in the history of RCTs (3) and was implemented in early trials such as the Coronary Drug Project (4,5). Evaluation of the interim analysis may require the advice of an independent data monitoring committee (DMC) (6,7), including certain trials under regulatory review (8,9). However, although ethically and scientifically compelling, interim repeated analysis of accumulating data has the statistical consequence of increased false positive claims unless special steps are taken. The issue of sequential analysis has a long tradition (10,11) and has received special attention for clinical trials (12,13). In particular, increasing the frequency of interim analysis can substantially increase the Type I error rate if the same criteria are used for each interim analysis (13). This increase was demonstrated in the Coronary Drug Project, which used sequential analysis for monitoring several treatment arms compared with a placebo (4). Most of the classical sequential methods assumed continuous analysis of accumulating data, a practice not realistic for most RCTs. Rather than continuous monitoring, most clinical trials review accumulating data periodically after additional data has been collected.

Z(j) < Zc (j) for j = 1, 2, 3, . . . ., J-1 Otherwise, the trial might be considered for termination. We would fail to reject the null hypothesis if Z(j) < Zc (j) for all j (j = 1,2, . . . ,J). We would reject the null hypothesis if at any interim analysis, Z(j) ≥ Zc (j) for j = 1, 2, . . . , J Peto et al. (14) recommended using a very conservative critical value for each interim analysis, say a standardized value of Zc (j) = 3.0 for all j (j = 1,2, . . . ,J), such the impact on the overall Type I error would be minimal. In 1977, Pocock (15) published a paper based on the earlier work of Armitage and colleagues (13) that formally introduced the idea of a group sequential approach. This modification developed a more conservative critical value than the na¨ıve (e.g., 1.96 for one-sided Type I error of 0.025) to be used at each analysis such that the overall Type I error was controlled. For example, if a total of 5 interim analyses were to be conducted with an intended Type I error of 0.025, then Zc (j) = 2.413 would be used at each interim analysis (j = 1,2, . . . ,5). Note that this final critical value is much larger than the standard critical value. In 1979, O’Brien and Fleming (16) introduced an alternative group sequential boundary for evaluating interim analysis. In this approach, the critical values change with each interim analysis, starting with a very conservative (i.e., large) value and shrinking to a final value close to the nominal critical value at the scheduled com-

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

LANS-DEMETS: ALPHA SPENDING FUNCTION

pletion. The exact√ form for each critical value is Zc (j) = ZOBF (J) (J/j). In this case, for the same 5 interim analyses and an overall Type I error of 0.025, the ZOBF (5) value is 2.04, which √ makes the 5 critical values 2.05 (5/j) for j = 1,2, . . . ,5 or (4.56, 3.23, 2.63, 2.28, and 2.04). Both of these latter models assume an equal increment in information between analyses and that the number of interim analyses J is fixed in advance. These three group sequential boundaries have been widely used, and examples are shown in Fig. 1. In fact, the OBF group sequential method was used in the Betablocker Heart Attack Trial (BHAT) (17), which terminated early because of an overwhelming treatment benefit for mortality. In 1987, Wang and Tsiatis generalized the idea of Pocock and O’Brien-Fleming and introduced a family of group sequential boundaries. For given α, J and a shape parameter φ, a constant C will be chosen so that Z(j) ≥ C(J/j)φ for some j = 1,2, . . . ,J is equal to α. The choice of φ = 0.5 yields the OBF boundary, and φ = 0 yields the Pocock boundary. 1 ALPHA SPENDING FUNCTION MOTIVATION The BHAT trial was an important factor in the motivation for the alpha spending function approach to group sequential monitoring. BHAT was a cardiovascular trial that evaluated a betablocker class drug to reduce mortality following a heart attack (17). An independent DMC reviewed the data periodically, using the OBF group sequential boundaries as a guide. A beneficial mortality trend emerged early in the trial and continued to enlarge with subsequent evaluations. At the sixth of a planned seven interim analyses, the logrank test statistic crossed the OBF boundary. After careful examination of all aspects, the DMC recommended that the trial be terminated, approximately 1 year earlier than planned. However, although the OBF sequential boundaries were used, the assumptions of these models were not strictly met. The increment in the number of deaths between DMC meetings was not equal. Furthermore, additional interim analyses were contemplated although not done. This experience suggested the need for more flexible

sequential methods for evaluating interim results. Neither the number nor the timing of interim analyses can be guaranteed in advance. A DMC may need to add additional interim analyses as trends that suggest benefit or harm emerge. As described by Ellenberg, Fleming, and DeMets (7), many factors must be considered before recommendations for early termination are made, and an additional interim analysis may be necessary to confirm or more fully evaluate these issues. Thus, the need for a flexible group sequential method seemed compelling. 2 THE ALPHA SPENDING FUNCTION The initial (or classical) group sequential boundaries are formed by choosing boundary values such that the sum of the probability of exceeding those critical values during the interim analyses is exactly the specified alpha level set in the trial design, assuming the null hypothesis of no intervention effect. That is, the total available alpha is allocated or ‘‘spent’’ over the prespecified times of interim analyses. The alpha spending function proposed by Lan and DeMets (18) allocated the alpha level over the interim analyses by a continuous monotone function, α(t), where t is the information fraction, 0 ≤ t ≤ 1. Here t could be the fraction of target patients recruited (n/N) or the fraction of targeted deaths observed (d/D) at the time of the interim analysis. In general, if the total information for the trial design is I, then at the j-th analysis, the information fraction tj = Ij /I. The total expected information I should have been determined by the trial design if properly done. The function α(t) is defined such that α(0) = 0 and α(1) = α. Boundary values Zc (j), which correspond to the α-spending function α(t), can be determined successfully so that under the null P0 {Z(1) ≥ Zc (1), or Z(2) ≥ Zc (2), or . . . , or Z(j) ≥ Zc (j)} = α(tj )

(1)

where {Z(1), . . . ,Z(j),} represent the test statistics from the interim analyses 1, . . . , . The specification of α(t) will create a boundary of critical values for interim test statistics,

LANS-DEMETS: ALPHA SPENDING FUNCTION

3

Group Sequential Boundaries 6 5

Alpha Spending OBF Type OBF

Z Score

4 3 2 Figure 1. Upper boundary values corresponding to the α 1 (t*) spending function for α – 0.05 at information fraction t* = 0.25, 0.50, 0.75, and 1.0 and for a truncated version at a critical value of 3.0.

1 0

and we can specify functions that approximate O’Brien–Fleming or Pocock boundaries as follows: √ α1 (t) = 2 − 2 − ( Zα/2 ´ / t) O’Brien–Fleming Type α2 (t) = α1n (1 + (e − 1)t) Pocock Type where  denotes the standard normal cumulative distribution function. The shape of the alpha spending function is shown in Fig. 2 for both of these boundaries. Two other families of spending functions proposed (19,20) are α(θ , t) = α tθ for θ > 0 α(γ , t) = α[(1 − e−γ t )/(1 − e−γ )], for γ = 0 The increment α(tj ) – α(tj−1 ) represents the additional amount of alpha or Type I error probability that can be used at the jth analysis. In general, to solve for the boundary values Zc (j), we need to obtain the multivariate distribution of Z(1), Z(2), . . . ,Z(J). In the cases to be discussed, the distribution is asymptotically multivariate normal with covariance structure  = (σ jk ) where σjk = cov (Z(j), Z(k)) √ √ = (tj /tk ) = (iJ /iK ) for j ≤ k where ij and ik are the amount of information available at the j-th and k-th data

0

0.2

0.4 0.6 Information Fraction

0.8

1.0

monitoring, respectively. Note that at the jth data monitoring, ij and ik are observable and σ jk is known even if I (total information) is unknown. However, if I is not known during interim analysis, we must estimate I by Iˆ and tj by tj = xj /Iˆ so that we can estimate α(tj ) by α(tˆ). If these increments have an independent distributional structure, which is often the case, then derivation of the values of the Zc (j) from the chosen form of α(t) is relatively straightforward using Equation (1) and the methods of Armitage et al. (21,22). If the sequentially computed statistics do not have an independent increment structure, then the derivation of the Zc (j) involves a more complicated numerical integration and sometimes is estimated by simulation. However, as discussed later, for the most frequently used test statistics, the independent increment structure holds. This formulation of the alpha spending function provides two key flexible features. Neither the timing nor the total number of interim analyses have to be fixed in advance. The critical boundary value at the j-th analysis only depends on the information fraction tj and the previous j-1 information fractions, t1 ,t2 , . . . ,tj−1 , and the specific spending function being used. However, once an alpha spending function has been chosen before the initiation of the trial, that spending function must be used for the duration of the trial. A DMC can change the frequency of the interim analyses as trends emerge without appreciably affecting the overall α level (23,24). Thus,

4

LANS-DEMETS: ALPHA SPENDING FUNCTION

Spending Functions .05 .04

Alpha

α2 (t∗) .03 α3 (t∗)

.02

α1 (t∗)

.01 0 0

0.2

0.4 0.6 Information Fraction

0.8

it is difficult to abuse the flexibility of this approach. The timing and spacing of interim analyses using the alpha spending function approach have been examined (19,25–27). For most trials, two early analyses with less than 50% of the information fraction are adequate. An early analysis say at 10% is often useful to make sure that all of the operational and monitoring procedures are in order. In rare cases, such early interim reviews can identify unexpected harm such as in the Cardiac Arrhythmia Suppression Trial (28) that terminated early for increased mortality at 10% of the information fraction using an alpha spending function. A second early analysis at 40 or 50% information fraction can also identify strong convincing trends of benefit as in two trials that evaluated beta blocker drugs in chronic heart failure (29,30). Both trials terminated early at approximately 50% of the information fraction with mortality benefits. Computation of the alpha spending function can be facilitated by available software on the web (www.biostat.wisc.edu/landemets)

1.0

Figure 2. Comparison of spending functions α 1 (t*), α 2 (t*), and α 3 (t*) at information fractions t* = 0.2, 0.4, 0.6, 0.8, and 1.0.

or by commercial software packages (www. cytel.com/Products/East/default.asp). 3 APPLICATION OF THE ALPHA SPENDING FUNCTION Initial development of group sequential boundaries was for comparison of proportions or means (15,16,26). In these cases, the increments in information are represented by additional groups of subjects and their responses to the intervention. For comparing means or proportions, the information fraction t can be estimated by the n/N, the observed sample size divided by the expected sample size. However, later work expanded the use to other common statistical procedures. Tsiatis and colleagues (31,32) demonstrated that sequential logrank test statistics and the general class of rank statistics used in censored survival data had the independent increment structure that made the application to group sequential boundary straightforward. Later, Kim and Tsiatis (33)

Table 1. Comparison of boundaries using spending functions with Pocock (P) and O’Brien–Fleming (OBF) methods (α = 0.05, t* = 0.2, 0.4, 0.6, 0.8, and 1.0) t*

α 1 (t*)

OBF

α 2 (t*)

P

0.2 0.4 0.6 0.8 1.0

4.90 3.35 2.68 2.29 2.03

4.56 3.23 2.63 2.28 2.04

2.44 2.43 2.41 2.40 2.36

2.41 2.41 2.41 2.41 2.41

LANS-DEMETS: ALPHA SPENDING FUNCTION

demonstrated that the alpha spending function approach for sequential logrank tests was also appropriate. In this case, the information fraction is approximated by d/D, the number of observed events or deaths divided by the expected or design for number of events or deaths (34). Application of the alpha spending function for logrank tests has been used in several clinical trials (e.g., 28–30). Group sequential procedures including the alpha spending function have also been applied to longitudinal studies using a linear random effects model (35,36). Longitudinal studies have also been evaluated using generalized estimating equations (37). In a typical longitudinal clinical trial, subjects are added over time, and more observations are gathered for each subject during the course of the trial. One statistic commonly used is to evaluate the rate of change by essentially computing the slope of the observations for each subject and then taking a weighted average of these slopes over the subjects in each intervention arm. The sequential test statistics for comparison of slopes using the alpha spending function must take into account their distribution. If the information fraction is defined in terms of the Fisher information (i.e., inverse of the variance for the slopes), then the increments in the test statistic are independent, and the alpha function can be applied directly (38). The total expected information may not be known exactly, but it often can be estimated. Wu and Lan (36) provide other approaches to estimate the information fraction in this setting. Scharfstein and Tsiatis (39) demonstrated that any class of test statistics that satisfies specific likelihood function criteria will have this property and thus can be used directly in this group sequential setting. 4 CONFIDENCE INTERVALS AND ESTIMATION Confidence intervals for an unknown parameter θ following early stopping can be computed by using the same ordering of the sample space described by Tsiatis et al. (32) and by using a process developed by Kim and DeMets (25,40) for the alpha spending function procedures. The method can be briefly

5

summarized as follows: A 1–γ lower confidence limit is the smallest value of θ for which an event at least as extreme as the one observed has a probability of at least γ . A similar statement can be made for the upper limit. For example, if the first time the Z-value exists the boundary at tj with the observed Z*(j) ≥ Zc (j), then the upper θ U and lower θ L confidence limits are θ U = sup {θ : Pθ {Z(1) ≥ Zc (1), or · · · , or Z( j − 1) > Zc ( j − 1), or Z( j) ≥ Z∗ (j)} ≤ 1 − γ }} and θ L = sup {θ : Pθ {Z(1) ≥ Zc (1), or · · · , or Z( j − 1) ≥ Zc ( j − 1), or Z( j) ≥ Z∗ (j)} ≥ 1 − γ }} Confidence intervals obtained by this process will have coverage closer to 1 – γ than na¨ıve ˆ confidence intervals using θˆ ± Zγ /2 SE(θ). As an alternative to computing confidence intervals after early termination, Jennison and Turnbull (41) have allocated the calculation of repeated confidence intervals. This calculation is achieved by inverting a sequential test to obtain the appropriate coefficient Z∗α/2 in the general form for the confidence interval, θˆ ± Z∗α/2 SE(θˆ ). This inversion can be achieved when the sequential test is based on an alpha spending function. If we compute the interim analyses at the tj , obtaining corresponding critical values Zc (j), then the repeated confidence intervals are of the form θˆk ± Zc ( j)SE(θˆj ) where θˆj is the estimate for the parameter θ at the j-th analysis. Methodology has also been developed to obtain adjusted estimates for the intervention effect (42–47). Clinical trials that terminate early are prone to exaggerate the magnitude of the intervention effect. These methods shrink the observed estimate closer to the null. The size of the adjustments may depend on the specific sequential boundary employed. Conservative boundaries such as that proposed by Peto or O’Brien and Fleming

6

LANS-DEMETS: ALPHA SPENDING FUNCTION

generally require less adjustment, and the na¨ıve point estimate and confidence intervals may be quite adequate. Another issue is the relevance of the estimate to clinical practice. The population sample studied is usually not a representative sample of current and future practice. Subjects were those who passed all of the inclusion and exclusion criteria and volunteered to participate. Early subjects may differ with later subjects as experience is gained with the intervention. Thus, the intervention effect estimate may represent populations like the one studied, the only solid inference, but not as relevant to how the intervention will be used. Thus, complex adjustments may not be as useful. 5

TRIAL DESIGN

If any trial is planning to have interim analyses for monitoring for benefit or harm, then that plan must be taken into account in the design. The reason is that group sequential methods will impact the final critical value, and thus power, depending on which boundary is used. For the alpha spending function approach, the specific alpha function must be chosen in advance. In addition, for planning purposes, the anticipated number of interim analyses must be estimated. This number does not have to be adhered to in the application, but it is necessary for the design. Variance with this number in the application will not practically affect the power of the trial. Thus, the design strategy for the alpha spending function is similar to that strategy described by Pocock for the initial group sequential methods (15). The key factor when the sample size is computed is to take into consideration the critical value at the last analysis when the information fraction is 1.0. One simple approach is to use this new critical value in the standard sample size formula. This estimate will reasonably approximate a more exact approach described below. To illustrate, consider a trial that is comparing failure rates of successive groups of subjects. Here, H0 : pC − pT = 0 HA : pC − pT = δ = 0

where pC and pT denote the unknown response rates in the control and newtreatment groups, respectively. We would estimate the unknown parameter by pˆ C and pˆ T , the observed event rates in our trial. For a reasonably large sample size, we often use the following test statistics, Z=


pˆ C − pˆ T ˆ − p)(1/m ˆ p(1 C + 1/mT )

to compare event rates where pˆ is the combined event rate across treatment groups. For sufficiently large n where n = mC = mT , this statistics has an approximate standard normal distribution with mean and unit variance under the null hypothesis H0 : = 0. In this case, assuming equal sample size (n) per group in each arm,
n(pC − pT )/ 2p(1 − p)
√ = nδ/ 2p(1 − p)

=

√

where p = (pC + pT )/2 It follows that n=

2 2 p(1 − p) δ2

To design our studies, we evaluate the previous equation for n, the sample size per treatment per sequential group. Because the plan is to have J groups each of size 2n, the total sample size 2N equals 2nJ. Now, to obtain the sample size in the context of the alpha spending function, we proceed as follows: 1. For planning purposes, estimate the number of planned interim analyses J at equally spaced increments of information (i.e., 2n subjects). It is also possible to specify unequal increments, but equal space is sufficient for design purposes. 2. Obtain the boundary values of the K interim analyses under the null hypothesis H0 to achieve a prespecified overall alpha level, α, for a specific spending function α(t). 3. For the boundary obtained, obtain the value of to achieve a desired power (1–β).

LANS-DEMETS: ALPHA SPENDING FUNCTION

4. Determine the value of n that determines the total sample size 2N = 2nJ. 5. Having computed these design parameters, one may conduct the trial with interim analysis to be done based on the information fraction tj approximated by tj = Number of subjects observed/2N at the jth analysis (38). The number of actual interim analyses may not be equal to J, but the alpha level and the power will be affected only slightly (26). As a specific example, consider using an O’Brien–Fleming-type alpha spending function α 1 (t) with a one-sided 0.025 alpha level and 0.90 power at equally spaced increments at t = 0.2, 0.4, 0.6, 0.8 and 1.0. Using previous publications (16) or available computer software, we obtain boundary values 4.56, 3.23, 2.63, 2.28, and 2.04. Using these boundary values and available software, we find that = 1.28 provides the desired power of 0.90. If we specify pC = 0.6, pT = 0.4 (p = 0.5) under the alternative hypothesis, then we can obtain a sample size as follows. For = 1.28, n=

2(1.28)2 (0.5)(0.5) = 20.5, (0.2)2

and we have a total sample size of 2(21)5 = 210 subjects. We can then proceed to conduct interim analysis times at information fraction tj equal to the observed number of subjects divided by 210. Similar formulations can be developed for the comparison of means, repeated measures, and survival analysis (48). However, for most applications, the standard sample size formulas with the new alpha spending function final critical value will be a very good approximation. 6

CONCLUSIONS

The alpha spending function approach for group sequential interim analysis has provided the necessary flexibility that allows

7

data monitoring committees to fulfill their task. DMCs can adjust their analysis as data accumulates and trends emerge. As long as the alpha spending function is specified in advance, there is little room for abuse. Many trials sponsored by industry and government have successfully used this approach. Although the decision to terminate any trial early, for benefit or harm, is a very complex decision process, the alpha spending function can be an important factor in that process. REFERENCES 1. L. Friedman, C. Furberg, and D. DeMets, Fundamentals of Clinical Trials. Littleton, MA: John Wright – PSG Inc., 1981. 2. S. J. Pocock, Clinical Trials: A Practical Approach, New York: Wiley, 1983. 3. Heart Special Project Committee. Organization, review and administration of cooperative studies (Greenberg Report): a report from the Heart Special Project Committee to the National Advisory Council, May 1967. Control. Clin. Trials 1988; 9:137–48. 4. P. L. Canner, Monitoring treatment differences in long–term clinical trials. Biometrics 1977; 33:603–615. 5. Coronary Drug Project Research Group. Practical aspects of decision making in clinical trials: The Coronary Drug Project as a case study. Control. Clin. Trials 1982; 9:137–148. 6. D. L. DeMets, L. Friedman, C. D. Furberg, Data Monitoring in Clinical Trials: A Case Studies Approach. 2005. Springer Science + Business Media, New York, NY. 7. S. Ellenberg, T. Fleming and D. DeMets, Data Monitoring Committees in Clinical Trials: A Practical Perspective. West Sussex, UK: John Wiley & Sons, Ltd., 2002. 8. ICH Expert Working Group: International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline. Statistical principles for clinical trials. Stats. Med. 1999; 18:1905–1942. 9. U.S. Department of Health and Human Services. Food and Drug Administration. Docket No. 01D–0489. Guidance for Clinical Trial Sponsors on the Establishment and Operations of Clinical Trial Data Monitoring Committees. Federal Register 66:58151–58153, 2001. Available:

8

LANS-DEMETS: ALPHA SPENDING FUNCTION http://www.fda.gov/OHRMS/DOCKETS/98fr/ 112001b.pdf.

10. F. J. Anscombe, Sequential medical trials. Journal of the American Statistical Association 1963; 58:365–383. 11. I. Bross, Sequential medical plans. Biometrics 1952; 8:188–205. 12. P. Armitage, Sequential Medical Trials, 2nd ed. New York: John Wiley and Sons, 1975. 13. P. Armitage, C. K. McPherson, and B. C. Rowe, Repeated significance tests on accumulating data. J. Royal Stat. Soc. Series A 1969; 132:235–244. 14. R. Peto, M. C. Pike, P. Armitage, et al., Design and analysis of randomized clinical trials requiring prolonged observations of each patient. 1. Introduction and design. Br. J. Cancer 1976; 34:585–612. 15. S. J. Pocock: Group sequential methods in the design and analysis of clinical trials. Biometrika 1977; 64:191–199. 16. P. C. O’Brien and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics 1979; 35:549–556. 17. Beta–Blocker Heart Attack Trial Research Group. A randomized trial of propranolol in patients with acute myocardial infarction. I Mortality results. J. Amer. Med. Assoc. 1982; 247:1707–1714. 18. K. K. G. Lan and D. L. DeMets, Discrete sequential boundaries for clinical trials. Biometrika 1983; 70:659–663. 19. K. Kim and D. L. DeMets, Design and analysis of group sequential tests based on the type I error spending rate function. Biometrika 1987; 74:149–154.

45(3):1017–1020. 25. K. Kim and D. L. DeMets, Confidence intervals following group sequential tests in clinical trials. Biometrics 1987; 4:857–864. 26. K. Kim and D. L. DeMets, Sample size determination for group sequential clinical trials with immediate response. Stats. Med. 1992; 11:1391–1399. 27. Z. Li and N. L. Geller, On the choice of times for date analysis in group sequential trials. Biometrics 1991; 47:745–750. 28. Cardiac Arrhythmia Suppression Trial (CAST) Investigators. Preliminary report: Effect of endainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N. Engl. J. Med. 1989; 321:406–412. 29. MERIT–HF Study Group. Effect of metoprolol CR/XL in chronic heart failure: Metoprolol CR/XL randomised intervention trial in congestive heart failure. Lancet 1999; 353:2001–2007. 30. M. Packer, A. J. S. Coats, M. B. Fowler, H. A. Katus, H. Krum, P. Mohacsi, J. L. Rouleau, M. Tendera, A. Castaigne, C. Staiger, et al. for the Carvedilol Prospective Randomized Cumulative Survival (COPERNICUS) Study Group. Effect of Carvedilol on survival in severe chronic heart failure. New Engl. J. Med. 2001; 334:1651–1658. 31. A. A. Tsiatis, Repeated significance testing for a general class of statistics used in censored survival analysis. J. Am. Stat. Assoc. 1982; 77:855–861. 32. A. A. Tsiatis, G. L. Rosner, and C. R. Mehta, Exact confidence intervals following a group sequential test. Biometrics 1984; 40:797–803.

20. I. K. Hwang, W. J. Shih, and J. S. DeCani, Group sequential designs using a family of type I error probability spending function. Stats. Med. 1990; 9:1439–45.

33. K. Kim and A. A. Tsiatis, Study duration for clinical trials with survival response and early stopping rule. Biometrics 1990; 46:81–92.

21. K. K. G. Lan and D. L. DeMets, Group sequential procedures: Calendar versus information time. Stats. Med. 1989; 8:1191–1198.

34. K. K. G. Lan and J. Lachin, Implementation of group sequential logrank tests in a maximum duration trial. Biometrics 1990; 46:759–770.

22. D. M. Reboussin, D. L. DeMets, K. M. Kim, and K. K. G. Lan, Computations for group sequential boundaries using the Lan–DeMets spending function method. Control. Clin. Trials 2000; 21:190–207.

35. J. W. Lee and D. L. DeMets, Sequential comparison of changes with repeated measurement data. J. Am. Stat. Assoc. 1991; 86:757–762.

23. M. A. Proschan, D. A. Follman, and M. A. Waclawiw. Effects of assumption violations on type I error rate in group sequential monitoring. Biometrics 1992; 48:1131–1143. 24. K. K. G. Lan and D. L. DeMets, Changing frequency of interim analyses in sequential monitoring. Biometrics 1989 Sept;

36. M. C. Wu and K. K. G. Lan, Sequential monitoring for comparison of changes in a response variable in clinical trials. Biometrics 1992; 48:765–779. 37. S. J. Gange and D. L. DeMets, Sequential monitoring of clinical trials with correlated categorical responses. Biometrika 1996; 83:157–167.

LANS-DEMETS: ALPHA SPENDING FUNCTION 38. K. K. G. Lan, D. M. Reboussin, D. L. DeMets: Information and information fractions for design and sequential monitoring of clinical trials. Communicat. Stat.– Theory Methods 1994; 23:403–420. 39. D. O. Scharfstein, A. A. Tsiatis, and J. M. Robins, Semiparametric efficiency and its implication on the design and analysis of group–sequential studies. J. Am. Stat. Assoc. 1997; 92:1342–1350. 40. K. Kim, Point estimation following group sequential tests. Biometrics 1989; 45:613–617. 41. C. Jennison and B. W. Turnbull: Interim analyses: The repeated confidence interval approach. J. Royal Stat. Soc., Series B 1989; 51:305–361. 42. S. S. Emerson and T. R. Fleming, Parameter estimation following group sequential hypothesis testing. Biometrika 1990; 77:875–892. 43. M. D. Hughes and S. J. Pocock, Stopping rules and estimation problems in clinical trials. Stats. Med. 1981; 7:1231–1241. 44. Z. Li and D. L. DeMets, On the bias of estimation of a Brownian motion drift following group sequential tests. Stat. Sinica 1999; 9:923–937. 45. J. C. Pinheiro and D. L. DeMets: Estimating and reducing bias in group sequential designs with Gaussian independent structure. Biometrika 1997; 84:831–843. 46. D. Siegmund, Estimation following sequential tests. Biometrika 1978; 65:341–349. 47. J. Whitehead, On the bias of maximum likelihood estimation following a sequential test. Biometrika 1986; 73:573–581. 48. D. L. DeMets and K. K. G. Lan, The alpha spending function approach to interim data analyses. In: P. Thall (ed.), Recent Advances in Clinical Trial Design and Analysis. Dordrecht, The Netherlands: Kluver Academic Publishers, 1995, pp. 1–27.

FURTHER READING M. N. Chang and P. C. O’Brien, Confidence intervals following group sequential tests. Control. Clin. Trials 1986; 7:18–26. T. Cook and D. L. DeMets, Statistical Methods in Clinical Trials. Boca Raton, FL: CRC Press/Taylor & Francis Co., 2007. D. L. DeMets, Data monitoring and sequential analysis – An academic perspective. J. Acq. Immune Def. Syn. 1990; 3(Suppl 2): S124–S133.

9

D. L. DeMets, R. Hardy, L. M. Friedman, and K. K. G. Lan, Statistical aspects of early termination in the Beta–Blocker Heart Attack Trial. Control. Clin. Trials 1984; 5:362–372. D. L. DeMets and K. K. G. Lan, Interim analysis: the alpha spending function approach. Stats. Med. 1994; 13:1341–1352. D. L. DeMets, Stopping guidelines vs. stopping rules: A practitioner’s point of view. Communicat. Stats.–Theory Methods 1984; 13:2395–2417. T. R. Fleming and D. L. DeMets, Monitoring of clinical trials: issues and recommendations. Control. Clin. Trials 1993; 14:183–197. J. L. Haybittle, Repeated assessment of results in clinical trials of cancer treatment. Brit. J. Radiol. 1971; 44:793–797. K. K. G. Lan, W. F. Rosenberger, and J. M. Lachin: sequential monitoring of survival data with the Wilcoxon statistic. Biometrics 1995; 51:1175–1183. J. W. Lee, Group sequential testing in clinical trials with multivariate observations: a review. Stats. Med. 1994; 13:101–111. C. L. Meinert, Clinical Trials: Design, Conduct, and Analysis. New York: Oxford University Press, 1986. S. Piantadosi. Clinical Trials: A Methodologic Perspective, 2nd ed. Hoboken, NJ: John Wiley and Sons, Inc., 2005. S. J. Pocock. Statistical and ethical issues in monitoring clinical trials. Stats. Med. 1993; 12:1459–1469. S. J. Pocock. When to stop a clinical trial. Br. Med. J. 1992; 305:235–240. E. Slud and L. J. Wei, Two–sample repeated significance tests based on the modified Wilcoxon statistic. J. Am. Stat. Assoc. 1982; 77:862–868. A. Wald, Sequential Analysis. New York: John Wiley and Sons, 1947.

LARGE, SIMPLE TRIALS

impact of daily aspirin reduces the risk of myocardial infarction. When carefully evaluated with precise estimates of effect and applied judiciously for maximum impact, relatively inexpensive interventions can produce disproportionate health benefits. Rare effects, especially those that are differential by treatment group, are difficult to detect or to distinguish from background rates in conventional studies. The demonstrated efficacy of a new drug or other medical product is established at a time when the limited clinical experience leaves the safety profile not yet fully defined. A large, simple trial with sufficient power to detect orders of magnitude differences in rare event rates can be very important (7). Serious adverse events that are not of concern at the start of the study may be explored using large health-care databases. Mitchell and Lesko (8) suggested that such was the case for the use of pediatric ibuprofen as an antipyretic. If such a difference is estimated reliably, health policy and practices can be developed to manage the risk while preserving the benefit. The need for larger trials to detect adverse event rates as low as 1 in 1000 or less has been emphasized for the full evaluation of new vaccines (9–11), highlighting the limitations of information derived from smaller trials with the ability to detect rates of 1 in 100 or higher. Similarly, large, simple trials have been used to address primary prevention questions (12). Large, simple trials have posed practical treatment questions relevant to a wide range of patients. If personalized effects applicable only to a single individual, characterized by gene expression analysis, are considered as one end of a spectrum, then large, simple trial issues would be at the opposite end of that spectrum. Cardiovascular disease research has been an active area for the application of large, simple clinical trials (13–15).

MARY A. FOULKES Center for Biologics Evaluation and Research U.S. Food and Drug Administration Rockville, Maryland

1

LARGE, SIMPLE TRIALS

Since the launch of the first randomized clinical trials in the middle of the twentieth century, immediate, major, and ubiquitous impacts on clinical practice and public health policy have been the result of very large randomized trials. For example, within hours of the results announced from the first oral polio vaccine field trial, the clarity of those results led to the regulatory approval of the vaccine and then to the rapid immunization of millions. That field trial was large (almost half a million children), simple (capture of incident cases of paralytic polio through public health records), randomized, and double blind (1, 2). The conclusions of that trial were unambiguous, leading to a seismic revolution in the fight against polio. This article describes all the characteristics that differentiate large, simple trials, sometimes called ‘‘megatrials’’ (3), from other types of trial, and we address important clinical questions that can have substantive impact: sizing the trial appropriate to the question, reliance on the bias control of randomization, minimizing the data collection, and other trial processes. 2 SMALL BUT CLINICALLY IMPORTANT OBJECTIVE The clinical effect size of interest dictates the number of subjects needed (or events observed) in any randomized, controlled trial. To identify a relatively small or moderate but clinically meaningful effect, the number of subjects needed (or number of events observed) in the trial may be very large (4–6). There are numerous instances where clinical effects that are limited can be used to substantial benefit from a population perspective. For example, in cardiovascular disease, the demonstration of a small but meaningful

2.1 Minimal Detectable Difference The difference to be estimated between the treatment groups needs to be clinically important when balanced against potential side effects, and practical enough to alter clinical practice. These differences could have major

Wiley Encyclopedia of Clinical Trials, Copyright  2007, John Wiley & Sons, Inc.

1

2

LARGE, SIMPLE TRIALS

public health impact when the interventions might ultimately be available to millions of patients. For example, the VITATOPS trial (16) aims to investigate whether a 15% relative reduction in the rate of stroke, myocardial infarction, or death from any vascular cause in patients with recent transient ischemic attack or stroke can result from a multivitamin intervention. The VITATOPS trial is expected to enroll approximately 8,000 patients, with a trial completion in 2010. It is important for the progress of medicine and public health that small to moderate (but real) benefits be reliably distinguished from no benefit, and this distinction requires large trials. 2.2 Sample Size Based on the minimal detectable difference, the sample size should be sufficient to reliably estimate the difference between treatment groups. When these differences are assumed to be relatively small, the resulting sample size estimates are large, in the tens of thousands of patients. The large sample size is necessary ‘‘not only to be correct but also to be convincing,’’ argues Peto (17). The sample size estimation process should focus on minimizing the risk of a false-negative result, a type II error. A trial of sufficient size will also limit the width of the confidence interval for the effect estimate, reducing uncertainty in applying the trial results. 2.3 Cost and Time Annual health-care expenditures, both direct and indirect, can be substantially impacted by small but important health-care advances that influence large numbers of people, as with cardiovascular disease, diabetes, or many other diseases. Effective preventive measures can also have major health-care cost benefits. The resource costs of a large trial, including time spent on protocol mandated follow-up procedures and data collection, can be minimized. If trial designs are stripped to the essentials, once the intervention is delivered, the time and cost of the trial are a function of the primary outcome measure. For a large, simple trial to be something other than a conventional trial

multiplied by orders of magnitude in sample size, cost, time, and effort, every action and every requirement of the trial should be questioned and only maintained as part of the trial if the contribution to the end result is substantial. This implies judicious inclusion, if at all, of extraneous substudies, ancillary studies, and other nonessential additions. Another costly aspect of trials that could be avoided in large, simple trials is the overhead associated with reliance on tertiary care institutions or clinical research institutions as the clinical sites for a trial. These may provide ideal sites for early clinical studies of complex interventions, or for studies requiring extensive clinical work-up to establish eligibility, but the usual health-care settings are the sites for large, simple trials. These usual health-care settings are also the ultimate site of application of large, simple trial results. 3 ELIGIBILITY Simple eligibility will allow easy, rapid identification and enrollment of eligible patients. The eligibility screen should be easily interpretable not only to those enrolling patients into the trial, but ultimately to those who may use the results if applicable. To be most widely interpretable, such eligibility requirements should not rely on the highest end ‘‘high-tech’’ diagnostics, which might not be widely available or may be completely unavailable outside the trial. 3.1 Simple, Broad In contrast to many clinical trials with very specific, thoroughly documented eligibility criteria, a large, simple trial engulfs a broad swath of patients with the general indication (18). Patients may enter from many different sites, from many different treating clinicians and practices, and from many different health-care systems, so the eligibility criteria need to be clear, interpretable, and consistent. Minimal testing should be required to determine eligibility. The exclusions should be a function of contraindications to any of the interventions. Patients thought to be at high risk of adverse events or with an independent life-threatening disease are generally

LARGE, SIMPLE TRIALS

excluded. In the Fourth International Study of Infarct Survival (ISIS-4) trial, for example, all patients presenting within 24 hours of a suspected acute myocardial infarction (with no clear indications for or contraindications to any one of the trial treatments) were eligible (19). 3.2 Multicenter and Multinational To rapidly enroll large numbers of patients, these trials are necessarily multicenter trials. They are often conducted in many countries simultaneously. This lends to the generalizability of results, but it also requires that definitions, eligibility screening, intervention delivery, and data capture be coordinated and harmonized multinationally. The uncertainty principle (20) as it applies to substantial uncertainty about which intervention is more beneficial can easily vary across countries, across health-care systems, and over time. Although internal validity would not be impacted, variation across countries and over time may impact the enrollment and the ultimate generalizability of results. One example of accommodation of such variations across countries was in the Options in Management with Antiretrovirals (OPTIMA) trial (21), a factorial design in which some countries enrolled to both factors and some to only one factor. 4

RANDOMIZED ASSIGNMENT

The strength of a large, simple trial over any similar-sized comparison of differently treated groups of individuals is the a priori assignment by chance to treatment group. Randomization ensures that the treatment groups are similar in both measured and unmeasured factors such that the treatment can be assumed to be the primary difference between the groups. Randomization addresses selection bias in the assignment of patients to groups. The strength of randomization has been widely articulated (22) and is a major strength in inferences from largescale randomized evidence and large, simple trials in particular. Randomization also provides a method of controlling confounding, of both known and unknown confounders. With or without stratification, randomization

3

is relied on to control confounding where the simple character of the trial would be compromised by other methods such as eligibility restriction. 4.1 Simple Process The actual randomization process should be as simple as possible. This process has benefited substantially from global telecommunications, networks, and interactive websites. It is possible for a single, central organization to manage and execute the randomization process. The randomization process is often the only point of capture for baseline information, so only the most important and clearly defined baseline factors (e.g., gender) should be recorded at this point. 4.2 Blinding Many clinical trials are blinded, but the incorporation of blinding into the conduct of a large, simple trial must be carefully weighed. To effect blinding, the interventions must be sufficiently similar that the large, simple nature of the trial is not compromised. If a blinded trial might require an extra intravenous line, as might have been the case had the Global Utilization of Streptokinase and t-PA for Occulted Arteries (GUSTO) trial (23) been blinded, the large size and the simple conduct might have been impossible. The Francis Field Trial (1), on the other hand, was blinded and involved delivery of identical placebo or polio vaccine. The large, simple nature of that trial was not impinged by blinding. Blinding of the outcome evaluation or adjudication, however, does not depend on blinding the patients to their randomized treatment groups. The integrity of trial results can be strengthened by evaluation by blinded reviewers. When the primary outcome is allcause mortality, blinded reviewers and adjudication are unnecessary. 4.3 Intervention Administered/Compliance The typical interventions chosen for evaluation in a large, simple trial are practical to deliver, such as the oral polio vaccine, and relatively short-term, such as a few hours of intravenous thrombolytic agent infusion.

4

LARGE, SIMPLE TRIALS

Longer term or self-administered interventions bring up issues of compliance. As with all trials, achieving and maintaining high compliance is important; the less compliance, the more the intervention effects begin to coincide. The usual methods of maintaining and improving compliance—minimizing the length and complexity of intervention, selecting expected compliant patients, maintaining frequent follow-up contact, and designing simple measures of compliance—should be applied to the extent possible while maintaining the large, simple character of the trial (24). 4.4 Patient Management All other aspects of patient management, other than the randomized comparison, should mimic local clinical practice and should be at the discretion of the treating clinician. Although this may introduce wide variation in the ancillary treatments, the results over a very large group of patients and clinicians will reflect the heterogeneity in practice. There should be no additional record keeping, laboratory tests, or other investigations and no extra follow-up visits. 5

OUTCOME MEASURES

The primary trial endpoint or outcome measure in a large, simple trial should be one that is directly relevant to the subject, that is easily and objectively measured with minimal resources. The outcome measure for a given trial substantially affects how large and simple the trial can feasibly be. It also affects the reliability of the effect estimates and the relevancy of the trial results. 5.1 Objectively Determined Survival, or mortality, is often the primary outcome measure in large, simple trials. It has many advantages: it is an easy to ascertain, objectively determined, binary, absorbing state. With the availability of healthcare databases and other national databases, information on survival status, including date, location, and cause of death, can often be captured without active follow-up efforts. Outcomes other than mortality can also be objectively determined.

5.2 Minimal Missing The amount and detail of the data collected, particularly after baseline, should be limited. The clear necessity of each data item will help to ensure that the participating patients, clinicians, and others will consistently provide the limited data requested, thus minimizing the amount of essential data that is missing. 5.3 Minimal Follow-up Depending on the primary endpoint of the trial, frequent, lengthy, labor-intensive, and high-tech direct medical assessments are not necessary in clinical trials. If the relevant primary endpoint can be adequately captured by indirect contact, clinical follow-up efforts can be minimized. Mail and telephone follow-up contacts have been employed, and e-mail could similarly be used. Lubsen and Tijssen (25) have argued that simple protocols with minimal follow-up contact are only appropriate when the mechanism of action is well understood and when the sideeffect profile is well understood. In general, minimal follow-up contact does not provide information on intermediate outcomes and does not contribute to greater understanding the mechanism of action. 5.4 Limited and Automated Data Capture The availability of health-care databases and other sources of automated data further simplify the capture of outcome data. The use of automated means to transmit data as effortlessly as possible also contributes to the completeness of the resulting data. The Francis Field Trial (1) used the existing system of public health surveillance and the required reporting of incident cases. National health systems can be used to provide hospitalization or mortality information. In the United Kingdom, this is facilitated when trial participants are ‘‘flagged’’ in the National Health Service (NHS) central register. Horsburgh (26) has described the infrastructure requirements for a public health surveillance system to capture a common disease outcome (e.g., tuberculosis). The practice of trying to answer as many questions as possible within a single trial is the antithesis of the simplicity required for

LARGE, SIMPLE TRIALS

a large, simple trial. A trial that attempts to address quality of life, economic issues, and other questions simultaneously is not feasible on a large-sample scale. The capture of data items extraneous to the primary central question must be avoided (27). The design and conduct of a large, simple trial must be diligently and vigilantly preserved; otherwise, the trial will become too complex to be either large or simple. Maintaining the simplicity of the follow-up and data capture procedure, without a routine comprehensive assessment of adverse events, and relying on randomization to control biases, the differences between treatment groups in rates of unexpected and rare adverse events can be evaluated by using large health-care databases, such those used in pharmcoepidemiology. These health-care databases could include the U.S. Centers for Medicare and Medicaid Services (CMS) Database, the U.K. and Italian General Practice Research Databases, the German MediPlus Database, various health maintenance organization databases, and many more. Serious adverse events, hospitalizations, and deaths can be captured by these sources rather than by sacrificing the simplicity of the large simple trial. 5.5 Subgroups The expectation in a large, simple trial is that the primary effect may vary by subgroups but is not likely to be reversed (28). The direction and consistency of effect across subgroups is one rationale for avoiding the collection of data to identify subgroups. Gray et al. (28) concluded that ‘‘most studies could be much larger, and of so much greater scientific value, if they had wider entry criteria and collected ten times less data.’’ Trial designers for many years have been warning that ‘‘marked heterogeneity of the treatment comparison in different strata can arise by chance more easily than would intuitively be expected’’ (29). The pitfalls of subgroup analyses are widely known (30); the ISIS-2 trial report provides a classic example, an analysis of effects across astrological subgroups (31). Qualitative rather than quantitative differences, though unexpected and uncommon, should be of concern.

5

Large, simple trials can include planned, small substudies, for example, at only certain sites, to address more specific hypotheses and requiring more complex follow-up information. These substudies would have no impact on the overall simplicity of the trial for the majority of patients. 5.6 Impact on Practice An additional outcome measure from a large, simple trial is the timely and pervasive impact on clinical practice. When a clear and convincing result to important questions shows a real effect that benefits the patient, that result should be widely reported and disseminated. The external influences on clinical decisions (e.g., financial) often respond rapidly to these results, accelerating the uptake of the results. The Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto Miocaridico 1 (GISSI-1) trial result, for example, became common (near universal) practice in Italian coronary care units within 1 year of reporting (32). Contrast this rapid pace of uptake with control of hypertension: despite numerous trials showing the positive effects of blood pressure control, according to a recent survey of the U.S. population, control of hypertension is stagnant at 34% (33). Because most individuals with elevated blood pressure are not being adequately treated, the demonstration within randomized controlled trials of a treatment effect is only a piece of a larger, more complex cluster of conditions. Even trials that have begun to enroll can benefit from further simplification, as seen in a trial of trastuzumab (Herceptin) (34). The simplification in this case involved broadening the eligibility criteria to more closely reflect the actual target patient population, streamlining study procedures (reducing frequencies and durations of procedures), and reconsidering and eliminating the placebo (and any need for blinding). As the result of these simplifications, the enrollment rate accelerated, risks of infections and other complications of multiple infusions dropped, and the resources to enroll and follow each patient were cut. These trial design and conduct aspects can be simplified in many trials, making a large trial feasible.

6

6

LARGE, SIMPLE TRIALS

CONCLUSIONS

By design, bias and confounding are minimized in randomized trials, and substantial power to detect moderate-sized effects is afforded by large sample sizes. These attributes lead to relevant, usable information that is generalizable to a broad population. Large, simple trials are not appropriate to every clinical question, but they can certainly contribute to clinical practice and public health policy. Practice and policy decisions to be addressed in the future will include decision-making based on knowledge of small but important clinical effects. Large, simple trials have a place in the full range of clinical study designs. REFERENCES 1. T. Francis Jr., Evaluation of the 1954 Field Trial of Poliomyelitis Vaccine: Final Report. 1957. Ann Arbor, MI: Edwards Brothers/National Foundation for Infantile Paralysis; School of Public Health, University of Michigan, 1957. 2. T. Francis, Jr., An evaluation of the 1954 Poliomyelitis Vaccine Trials—summary report. Am J Public Health. 1955; 45: 1–63. 3. D. Heng, Megatrials of drug treatment: strengths and limitations. Ann Acad Med Singap. 2000; 29: 606–609. 4. S. Yusuf, R. Collins, and R. Peto. Why do we need some large, simple randomized trials? Stat Med. 1984; 3: 409–420. 5. C. Baigent, The need for large-scale randomized evidence. Br J Clin Pharmacol. 1997; 43: 349–353. 6. R. M. Califf and D. L. DeMets, Principles from clinical trials relevant to clinical practice: part I. Circulation. 2002; 106: 1015–1021. 7. R. Temple, Meta-analysis and epidemiologic studies in drug development and postmarketing surveillance. JAMA. 1999; 281: 841–844. 8. A. A. Mitchell and S. M. Lesko, When a randomized controlled trial is needed to assess drug safety. The case of paediatric ibuprofen. Drug Saf. 1995; 13: 15–24. 9. B. L. Strom, How the US drug safety system should be changed. JAMA. 2006; 295: 2072–2075. 10. S. S. Ellenberg, M. A. Foulkes, K. Midthun, and K. L. Goldenthal, Evaluating the safety of

new vaccines: summary of a workshop. Am J Public Health. 2005; 95: 800–807. 11. J. Clemens, R. Brenner, M. Rao, N. Tafari, and C. Lowe, Evaluation of new vaccines for developing countries. Efficacy or effectiveness? JAMA. 1996; 275: 390–397. 12. J. E. Buring and C. H. Hennekens, The contributions of large, simple trials to prevention research. Prev Med. 1994; 23: 595–598. 13. C. D. Furberg, S. Yusuf, and T. J. Thom, Potential for altering the natural-history of congestive heart-failure—need for large clinical trials. Am J Cardiol. 1985; 55: A45–A47. 14. S. Yusuf and R. Garg, Design, results, and interpretation of randomized, controlled trials in congestive-heart-failure and leftventricular dysfunction. Circulation. 1993; 87: 115–121. 15. M. D. Flather, M. E. Farkouh, and S. Yusuf. Large simple trials in cardiovascular disease: their impact on medical practice. In: R. M. Califf, B. Mark, and G. S. Wagner (eds.), Acute Coronary Care. St. Louis, MO: Mosby, 1995, pp. 131–144. 16. VITATOPS Trial Study Group. The VITATOPS (Vitamins to Prevent Stroke) Trial: rationale and design of an international, large, simple, randomized trial of homocysteine-lowering multivitamin therapy in patients with recent transient ischaemic attack or stroke. Cerebrovasc Dis. 2002; 13: 120–126. 17. R. Peto, Clinical trial methodology. Biomedicine. 1978; 28(Special): 24–36. 18. S. Yusuf, P. Held, K. K. Teo, et al. Selection of patients for randomized controlled trials: implications of wide or narrow eligibility criteria. Stat Med. 1990; 9: 73–86. 19. ISIS-4 (Fourth International Study of Infarct Survival) Collaborative Group. A randomised factorial trial assessing early oral captopril, oral mononitrate, and intravenous magnesium sulphate in 58,050 patients with suspected acute myocardial infarction. Lancet. 1995; 345: 669–685. 20. R. Peto and C. Baigent, Trials: the next 50 years. Large scale randomized evidence of moderate benefits. BMJ. 1998; 317: 1170–1171. 21. T. C. Kyriakides, A. Babiker, and J. Singer, et al. An open-label randomized clinical trial of novel therapeutic strategies for HIV-infected patients in whom antiretroviral therapy has failed: rationale and design of OPTIMA trial. Control Clin Trials. 2003; 24: 481–500.

LARGE, SIMPLE TRIALS 22. R. Peto, R. Collins, and R. Gray, Largescale randomized evidence—large, simple trials and overviews of trials. J Clin Epidemiol. 1995; 48: 23–40. 23. Global Utilization of Streptokinase and t-PA for Occulted Arteries (GUSTO) Investigators. An international randomized trial comparing four thrombolytic strategies for acute myocardial infarction. N Engl J Med. 1993; 329: 673–682. 24. C. Hennekens and J. E. Buring, Need for large sample sizes in randomized trials. Pediatrics. 1987; 79: 569–571. 25. J. Lubsen and J. G. P. Tijssen, Large trials with simple protocols: indications and contraindications. Control Clin Trials. 1989; 10: 151S–160S. 26. C. R. Horsburgh, A large, simple trial of a tuberculosis vaccine. Clin Infect Dis. 2000; 30 (Suppl 3): S213–216. 27. S. Yusuf, Randomised controlled trials in cardiovascular medicine: past achievements, future challenges. BMJ. 1999; 319: 564–568. 28. R. Gray, M. Clarke, R. Collins, and R. Peto, Making randomized trials larger: a simple solution? Eur J Surg Oncol. 1995; 21: 137–139. 29. R. Peto, M. C. Pike, P. Armitage, N. E. Breslow, D. R. Cox, et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. Analysis and examples. Br J Cancer. 1977; 35: 1–39. 30. S. Yusuf, J. Wittes, J. Probstfield, and H. A. Tyroler, Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. JAMA. 1991; 266: 93–98. 31. ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet. 1988; 2: 349–360. 32. G. Tognoni, M. G. Franzosi, S. Garattini, and A. Maggioni, The case of GISSI in changing the attitudes and practice of Italian cardiologists. Stat Med. 1990; 9: 17–27. 33. A. V. Chobanian G. L. Bakris, H. R. Black, W. C. Cushman, L. A. Green, et al., for the National Heart, Lung, and Blood Institute Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure; National High Blood Pressure Education Program Coordinating Committee. The seventh report of the Joint National Committee on Prevention, Detection, Evaluation,

7

and Treatment of High Blood Pressure: the JNC 7 report. JAMA. 2003; 289: 2560–2572. 34. T. R. Fleming, Issues in the design of clinical trials: insights from the transuzumab (Herceptin) experience. Semin Oncol. 1999; 26: 102–107.

FURTHER READING Selected Examples of Large, Simple Trials ALLHAT Collaborative Research Group. Major cardiovascular events in hypertensive patients randomized to doxazosin vs chlorthalidone: the antihypertensive and lipid-lowering treatment to prevent heart attack trial (ALLHAT). JAMA. 2000; 283: 1967–1975. M. L. Barreto, L. C. Rodrizues, S. S. Cunha, S. Pereira, et al. Design of the Brazilian BCGREVAC trial against tuberculosis: a large, simple randomized community trial to evaluate the impact on tuberculosis of BCG revaccination at school age. Control Clin Trials. 2002; 23: 540–553. S. Black, H. Shinefield, M. A. Fireman, et al. Efficacy, safety and immunogenicity of the heptavalent pneumococcal conjugate vaccine in children. Pediatr Infect Dis J. 2000; 19: 187–195. Chinese Acute Stroke Trial (CAST) Collaborative Group. CAST: a randomised placebo-controlled trial of early aspirin use in 20,000 patients with acute ischaemic stroke. Lancet. 1997: 349: 1641–1649. R. Collins, R. Peto, and J. Armitage. The MRC/BHF Heart Protection Study: preliminary results. Int J Clin Pract. 2002; 56: 53–56. CRASH Trial Pilot Study Collaborative Group. The MRC CRASH Trial: study design, baseline data, and outcome in 1000 randomised patients in the pilot phase. Emerg Med J. 2002; 19: 510–514. Digitalis Investigation Group (DIG). The effect of digoxin on mortality and morbidity in patients with heart failure. N Engl J Med. 1997; 336: 525–533. Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto miocaridico (GISSI). Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Lancet. 1986; 1: 397–402. P. Edwards, M. Aragno, L. Balica, and R. Cottingham, Final results of the MRC CRASH, a randomized placebo-controlled trial of intravenous corticosteroid in adults with head injury—outcomes at 6 months. Lancet. 2005; 365: 1957–1959.

8

LARGE, SIMPLE TRIALS

Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto Miocaridico (GISSI). GISSI-2: a factorial randomized trial of alteplase versus streptokinase and heparin versus no heparin among 12,490 patients with acute myocardial infarction. Lancet. 1990; 336: 65–71. HOPE Study Investigators. The HOPE (Heart Outcomes Prevention Evaluation) Study: the design of a large, simple randomized trial of an angiotensin-converting enzyme inhibitor (ramipril) and vitamin E in patients at high risk of cardiovascular events. Cardiovasc Med. 1996; 12: 127–137. International Stroke Trial Collaborative Group. International Stroke Trial (IST): a randomised trial of aspirin, subcutaneous heparin, both or neither among 19,435 patients with acute ischaemic stroke. Lancet. 1997; 349: 1569–1581. ISIS-1 (First International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous atenolol among 16,027 cases of suspected acute myocardial infarction: ISIS-1. Lancet. 1986; 2: 57–66. ISIS-3 (Third International Study of Infarct Survival) Collaborative Group. ISIS-3: a randomised trial of streptokinase vs tissue plasminogen activator vs anistreplase and of aspirin plus heparin vs aspirin alone among 41,299 cases of suspected acute myocardial infarction. Lancet. 1992; 339: 753–770.

S. M. Lesko and A. A. Mitchell, An assessment of the safety of pediatric ibuprofen. A practitionerbased randomized clinical trial. JAMA. 1995; 273: 929–933. MAGIC Steering Committee. Rationale and design of the magnesium in coronaries (MAGIC) study: a clinical trial to reevaluate the efficacy or early administration of magnesium in acute myocardial infarction. Am Heart J. 2000; 139: 10–14. SOLVD Investigators. Studies of left ventricular dysfunction (SOLVD)—rationale, design and methods: two trials that evaluate the effect of enalapril in patients with reduced ejection fraction. Am J Cardiol. 1990; 66: 315–322. E. Van Ganse, J. K. Jones, N. Moore, J. M. Le Parc, R. Wall, and H. Schneid, A large simple clinical trial prototype for assessment of OTC drug effects using patient-reported data. Pharmacoepidemiol Drug Saf. 2005; 14: 249–255.

CROSS-REFERENCES Effect size Eligibility Expanded safety trials Hypothesis Sample size

LINEAR MODEL

variation, and goodness of fit. Understanding the basic structure of a (General) Linear Model is extremely helpful in the practical application of statistical models and the interpretation of their results. This article presents the Linear Model in its general form, discusses its three major subtypes, namely Linear Regression, Analysis of Variance (ANOVA), and Analysis of Covariance (ANCOVA), and explains how extensions are related to Linear Model and how they are interrelated between themselves. Therefore, the aim is to provide a comprehensive overview. For specific modeling approaches, refer to the respective articles on Linear Regression, ANOVA, and Generalized Linear Models.

DR. IRIS BURKHOLDER DR. LUTZ EDLER German Cancer Research Center Department of Biostatistics Heidelberg, Germany

1

INTRODUCTION

Linear Models for simple and multiple regression, Analysis of Variance (ANOVA), and Analysis of Covariance (ANCOVA) have broad and often direct application in clinical trials. Various extensions and generalizations of the Linear Model have been developed more recently often motivated by medical research problems. Most prominent are the loglinear models for the evaluation of count data; the logistic regression model for the evaluation of the influence of clinical factors on a dichotomous outcome as, for example, cancer incidence; and, last but not least, the Cox regression model for the analysis of survival data. The Linear Model, also called General Linear Model (GLM) when extended to incorporate classes of predictive factors, is built on the assumption of response data following a Gaussian (Normal) distribution. In contrast, extensions of the GLM no longer require this assumption, but are either based on other distributions or on allowing more general classes of distributions. The Generalized Linear Model—not to be confused with the General Linear Model—became an extremely valuable and versatile modeling approach for more general types of data (distributions from the exponential family) including the General Linear Model as a special case. Further extensions have been models for repeated measurements and the General Estimation Equation (GEE) approach. It is important to note that central features of the Linear Model were retained in those extensions. Features of similarity and analogy include such important components as model building, variable selection, maximum likelihood technique, residual analysis, explained

2

LINEAR MODEL

The Linear Model represents the relationship between a continuous response variable y and one or more predictor variables X in the form: y = Xβ + ε

(1)

where • y is a vector of observations of the

response variable • X is the design matrix determined by

the predictors • β is a vector of unknown parameters to

be estimated • ε is a vector of random disturbances,

independent of each other and usually having a normal distribution with mean zero and variance σ 2 . Remarks: 1. As an example, consider as response variable y the blood pressure (mmHg) of a patient and as predictor variables x1 = gender, x2 = age, x3 = smoking status, and x4 = treatment status (beta-blocker therapy). The variables gender, smoking status, and treatment status are measured on a qualitative

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

LINEAR MODEL

scale, whereas age is a quantitative variable. 2. It is very common to specify by subscripts the dimension of the quantities of model Equation (1)

where • yi denotes the ith observed response • β 0 is a constant intercept • β j , j = 1, . . . , p are the unknown regres-

sion parameters

ynx1 = Xnxp βpx1 + εnx1

• xij , j = 1, . . . , p are the values of the p

indicating that y is a column vector of length n of n single observations (y1 , . . ., yn ) and X is a matrix (Xij )i=1,...,n j=1,...,p with n columns and p rows. Notice, n is the sample size of the respective clinical study. 3. Model Equation (1) assumed ε ∼ N(0, σ 2 ) as normal distribution with mean zero and the same variance σ 2 for each observation. More general models exist that allow different variance σi2 for the individual observations (heteroscedasticity). Then, one writes ε = εnx1 ∼ N(0, ) where 0 = (0, . . . , 0) is a vector of length n with all components equal to zero and where  is a diagonal matrix with σ12 , . . . , σn2 in the diagonal and all other elements equal to zero ( = diag(σ12 , . . . , σn2 )). Three subtypes of classic Linear Models can be distinguished by the composition of the design matrix X (see Fig. 1). The Linear Model leads to – Regression analysis, if X contains only quantitative values (covariates), – Analysis of variance, if X contains only categorical variables, – Analysis of covariance, if X contains quantitative as well as categorical variables.

explanatory variables • ε i is the ith error term.

Remark: 1. If all p explanatory variables are zero (i.e., if they add no information) Equation (2) reduces to the constant regression y = β0 + ε. In simple regression yi = β0 + β1 x1 + ε, β 0 is exactly the intercept on the y-axis and β 1 the slope of the regression line. 2. Details about estimation of the regression parameters and interpretation of the parameters can be found in the article on Linear Regression Analysis. 2.1.1 Historical Example of Simple Linear Regression. The British scientist Sir Francis Galton (1) was studying the inheritance of physical characteristics. In particular, he wondered if he could predict a boy’s adult height based on the height of his father. He plotted the heights of fathers and the heights of their sons for a number of father-son pairs and then tried to fit a straight line through the data. Let • y be the height of the first, fully grown,

son (continuous response) and • x be the height of the father (the contin-

uous predictor). 2.1 Simple and Multiple Regression Linear regression is a technique for modeling a continuous response or outcome variable as a linear combination of one (simple regression) or several potential (multiple regression) continuous explanatory factors. The general model equation for p explanatory variables is given by yi = β0 + β1 xi1 + . . . + βp xip + εi , i = 1, . . . , n

(2)

One can say that, in mathematical terms, Galton wanted to determine the intercept constant β 0 and the regression parameter β 1 . He observed that when choosing a group of parents of a given height, the mean height of their children was closer to the mean height of the population than is the given height. In other words, tall parents tend to be taller than their children and short parents tend to be shorter. Galton termed this phenomenon ‘‘regression towards mediocrity’’

LINEAR MODEL

meaning ‘‘going back towards the average.’’ The mean height of the children was closer to the mean height of all children than the mean height of their parents was to the mean height of all parents. In mathematical terms, unless X and Y are exactly linearly related for a given value of X, the predicted value of Y is always fewer standard deviations from its mean than is X from its mean. Regression toward the mean appears unless X and Y are perfectly correlated, so it always occurs in practice. 2.2 Analysis of Variance (ANOVA) The Analysis of variance is used to uncover the main and interaction effects of categorical independent variables (factors) X on the dependent variable y. A main effect is the direct effect of an independent variable on the dependent variable. An interaction effect is the joint effect of two or more independent variables on the dependent variable. Depending on the number of factors used in an ANOVA model, one distinguishes one-way ANOVA and multi-way (p-way) ANOVA models. The one-way ANOVA investigates the effects of one categorical factor on the dependent variable. If that factor has k different values (features), one can test for differences of the dependent variable between the groups defined by those different features. A dichotomous categorical variable can be included as predictor variable in the ANOVA model by creating a dummy variable that is 0 if the characteristic is absent and 1 if it is present. For example, a dummy variable representing the sex of patients in a clinical trial can take the value 0 for females and 1

3

for males. The coefficient then represents the difference in mean between male and female. Variables with more than two categories cannot simply be inserted as one factor in the ANOVA, unless it can be assumed that the categories are ordered in the same way as their codes, and that adjoining categories are in some sense the same distance apart (i.e., the variable possesses a metric scale). As this assumption is very strong, a set of dummy variables is created instead to represent the predictive factor. The number of dummy variables is the number of categories of that factor minus one. For example, in patients with nonsmall-cell lung cancer, three different histologic tumor types are usually distinguished: adenocarcinoma, squamous cell carcinoma, and large cell carcinoma. The variable histologic tumor type could be included in an ANOVA model by creating two dummy variables: histo1 = 1 if patient has adenocarcinoma, 0 otherwise, and histo2 = 1 if patient has squamous cell carcinoma, 0 otherwise.

Obviously, the patient has large cell carcinoma if histo1 = 0 and histo2 = 0. Therefore, the two dummy variables histo1 and histo2 characterize each patient of that study. The mathematical model that describes the relationship between the response and a categorical factor for the one-way ANOVA is

Linear modeling

Simple and multiple regression

Analysis of Variance ANOVA

Analysis of Covariance ANCOVA

only quantitative covariates

only categorical variables

quantitative covariates and categorical variables

Figure 1. Differences between the three subtypes of Linear Models: regression, analysis of variance, and analysis of covariance

4

LINEAR MODEL

given by

where yij = µ + τi + εij

(3)

• yij represents the kth observation on the

where • yij represents the jth observation on the

ith level of the categorical factor • µ is the common effect for the whole population • τ i denotes the effect of the ith level • ε ij is the random error present in the jth observation on the ith level and the errors are assumed to be normally and independently distributed with mean zero and variance σ 2 . 2.2.1 Example: One-way ANOVA. In four groups of patients, measures of gut permeability are obtained (2) and the differences between the groups should be investigated. In the one-way ANOVA model, let • yij the continuous response variable indi-

cating the result of the permeability test for the jth individual in the ith patient group (i = 1, . . . 4) • τ i denotes the effect of the ith patient group (i = 1, . . . 4). The basic idea of a one-way ANOVA consists in considering the overall variability in the data and partitioning the variability into two parts: between-group and within-group variability. If between-group variability is much larger than within-group variability, it suggests that differences between the groups are real, not just random noise. Two-way ANOVA analyzes one dependent variable in terms of the categories (groups) formed by two independent factors, which generalises to p-way ANOVA, which deals p independent factors. It should be noted that as the number of independent factors increases, the number of potential interactions proliferates. Two independents have one single first-order interaction only. The two-way ANOVA model that describes the relationship between the response, two categorical factors, and their interaction is given by yijk = µ + τi + δj + (τ δ)ij + εijk

(4)

• • • •

•

ith level of the first categorical factor and on the jth level of the second categorical factor µ is the common effect for the whole population τ i denotes the effect of the ith level of the first categorical factor δ j denotes the effect of the jth level of the second categorical factor (τ δ)ij represents the interaction between the ith level of the first categorical factor and the jth level of the second categorical factor εijk is the random error present in the kth observation on the ith level of the first categorical factor and on the jth level of the second categorical factor and the errors are assumed to be normally and independently distributed with mean zero and variance σ 2 .

2.2.2 Example: Interaction in Two-way ANOVA. Consider an experiment to determine the effect of exercise and diet on cholesterol levels. Both factors are categorical variables with two levels. For each factor, a dummy variable is created; for example, the dummy variable exercise is 0 if no exercise is done and is 1 if the patient does exercise, and the dummy variable is 0 if patient is on normal diet and is 1 if the patient is on low-fat diet. Besides the two main effects of the two factors of interest, the design matrix X contains a column representing the interaction between exercise and diet. In this example, the interaction column could be formed by combining the two singles columns of the main effects. The model equation is, therefore, given by • y the continuous response describing the

cholesterol level • µ the common effect for the whole popu-

lation • τ 1 the effect of no exercise and τ 2 the

effect of exercise • δ 1 the effect of normal diet and δ 2 the

effect of low-fat diet

LINEAR MODEL • (τ δ) the interaction effects: (τ δ)11 no

exercise—normal diet, (τ δ)12 no exercise—low-fat diet, (τ δ)21 exercise—normal diet and (τ δ)22 exercise—low-fat diet. Three independent factors give rise to three first-order interactions and one secondorder interaction; four independent factors have six first-order interactions, three second-order interactions, and one thirdorder interaction. As the number of interactions increases, it may become extremely difficult to interpret the model outcome. More information about one-way and twoway ANOVA and hypothesis testing could be found in the article on Analysis of Variance ANOVA. 2.3 Analysis of Covariance (ANCOVA) The analysis of covariance (ANCOVA) is a technique that combines elements of regression and variance analysis. It involves fitting a model where some elements are effects corresponding to levels of factors and interactions, in the manner of analysis of variance, and some others are regression-style coefficients. ANCOVA compares regression within several groups. Of main interest is the explanation of the relationship between the response and the quantitative variable within each group. The general model equation is given by y = Xα + Zβ + ε

(5)

where • y is the vector of observations of the

5

2.3.1 Example. The relationship between salt intake and blood pressure may be investigated for male and female patients. ANCOVA can be used to analyze whether the relationship between salt intake and blood pressure holds for both sexes. The model equation is given by • the continuous response y describing the

blood pressure • the design matrix X containing the categorical covariate sex, and • the matrix Z containing the continuous covariate of daily salt intake. ANCOVA allows one to test more precisely whether the treatment (group) means are equal. It can also be used to study the linear relationship between the dependent and the quantitative variable within each group. 3 GENERALIZATIONS OF THE LINEAR MODEL

Generalized Linear Models are an extension of the linear modeling that allows models to be fit to data that follow probability distributions other than the normal distribution, such as the Poisson, Binomial, Multinomial, and so on. Three extensions that have broad range of application (see Fig. 2) are represented here: the loglinear model for Poisson-distributed data, the logistic regression analysis applicable for binomial-distributed data, and the Cox regression for survival data. For a complete discussion, consult the article on Generalized Linear Models.

response variable • X is the matrix of dummy (0,1) variables • α is the parameter vector of the general

mean and the effects corresponding to levels of factors and their interactions • Z contains the values of the covariates (‘‘regression part’’) • β is the parameter vector for the regression-style coefficients of the covariates • ε is the error term and it is assumed that the errors are identical independent normal N(0,σ 2 ) distributed.

3.1 Loglinear Models Loglinear analysis is an extension of the two-way contingency table where the conditional relationship between two categorical variables is analyzed by taking the natural logarithm of the cell frequencies within a contingency table assuming Poissondistributed data. No distinction is made between dependent and independent variables and, therefore, loglinear models can only analyze the association between variables. Loglinear modeling is an analogue to

6

LINEAR MODEL

Generalized Linear Models

Loglinear Model

Logistic Regression

Cox Regression

Poisson distributed data

Binomial distributed data

Survival data

Figure 2. Extensions of the classical Linear Model (Generalized Linear Models)

multiple regression for categorical variables, and it can be applied to involve not only two but also three or more variables corresponding to a multiway contingency analysis. The loglinear model equation for a 2 × 2 × 2 contingency table is given by B C AB ln(Fijk ) = µ + λA i + λj + λk + λij BC ABC + λAC ik + λjk + λijk

(6)

where • ln(F ijk ) is the natural logarithm of the

expected cell frequency of the cases for cell ijk in the 2 × 2 × 2 contingency table where i indexes the first variable A, j the second variable B, and k the third variable C of this three-dimensional table • µ is the overall mean of the natural logarithm of the expected frequencies • λA i is the main effect for variable A • λB j is the main effect for variable B • λC k is the main effect for variable C • λAB ij is the two-way interaction effect for

variables A and B • λAC ik is the two-way interaction effect for variables A and C • λBC jk is the two-way interaction effect for variables B and C • λABC is the three-way interaction effect ijk for variables A, B, and C. 3.1.1 Example. Suppose one is interested in the relationship between heart disease, sex, and body weight. The continuous variable body weight is broken down into two discrete categories: not over weight and over weight. The variables heart disease and sex

are dichotomous. Then, the three-dimensional contingency looks like the results in the table below (where some fictitious numbers have been inserted in the table): Heart Disease Total no yes Body Weight

Sex

Not over weight

Male Female

Total Over weight

Male Female

Total

40 20

30 30

70 50

60

60

12

10 5

65 25

75 30

15

90

105

The basic strategy in loglinear modeling involves fitting models to the observed frequencies in the cross tabulation of categorical variables. The model can then be represented by a set of expected frequencies that may or may not resemble the observed frequencies. The most parsimonious model that fits the data is chosen and effect parameters for the variables and their interactions could be estimated (3). 3.2 Logistic Regression The logistic regression model is used when the response variable y of interest takes in two variables. Possible situations include studies in which subjects are alive or dead or have or do not have a particular characteristic (e.g., a specific disease). Denote the event y = 1 when the subject has the characteristic of interest and y = 0 when the subject does

LINEAR MODEL

not. In simple logistic regression, the subject has a single predictor variable x, which can take any form (continuous, discrete, or dichotomous). The logistic regression model relates then the probability P(y = 1) to the response via P(y = 1) =

exp(β0 + β1 x) 1 + exp(β0 + β1 x)

(7)

This model has a convenient representation in terms of the odds of the event y = 1 as odds(y = 1) =

P(y = 1) = exp(β0 + β1 x). (8) P(y = 0)

which means the log odds is simply the linear function β 0 + β 1 x, where the parameter β 1 is of primary interest. This often-called slope parameter controls the degree of association between the response and the predictor variable.

7

• β = (β 1 , . . ., β k ) is the vector of the k

unknown regression coefficients • Xi = (xi1 , . . ., xik ) is the vector of the

covariates for the patient i. Cox regression allows one to estimate the regression coefficients that best predict the observed survival. More details on estimation procedures are given in the article on Cox Regression. REFERENCES 1. F. Galton, Regression toward mediocrity in hereditary stature. J. Anthropologic. Inst. 1886; 15: 246–263. 2. M. Bland, An Introduction to Medical Statistics. Oxford: Oxford University Press, 1995. 3. D. Knoke and P. J. Burke, Log-Linear Models. Newberry Park, CA: Sage Publications, 1980. 4. D. R. Cox, Regression models and life tables (with discussion). J. Royal Stat. Soc. B 1972; 74: 187–220.

3.3 Cox Regression In many applications of survival analysis of clinical trials, the interest focuses on how covariates may effect the outcome. The proportional hazard regression model of Cox (4) has become the most important model for survival data. The basic endpoint of a patients, say i, is his/her survival Time T i . No specific distribution is assumed for this response variable, instead one considers his/her hazard function hi (t), which describes the instantaneous risk of failure of time t. Formally, hi (t) = lim

t→0

1 P(t ≤ T ≤ t + t|T ≥ t) (9) t

for example, the probability of failure of time t given no failure up to this time point. In its simplest form, but for many practical uses sufficient is a hazard function for a patient i given by hi (t) = h0 (t) exp(β1 xi1 + · · · + βk xik )

(10)

where • h0 (t) is a constant describing the hazard

function for patients with all covariates equal to 0

FURTHER READING A. Agresti, An Introduction to Categorical Data Analysis. New York: John Wiley & Sons, 1996. J. M. Bland and D. G. Altman, Regression towards the mean. Brit. Med. J. 1994; 308: 1499. P. McCullagh and J. A. Nelders, Generalized Linear Models. New York: Chapman and Hall, 1983. S. R. Searle, Linear Models. New York: John Wiley & Sons, 1971.

LOGISTIC REGRESSION

The logit transformation, defined in terms of π (x), is as follows:

STANLEY LEMESHOW




Ohio State University Columbus, OH, USA

g(x) = ln

 π (x) = β0 + β1 x. 1 − π (x)

(2)

DAVID W. HOSMER Jr The importance of this transformation is that g(x) has many of the desirable properties of a linear regression model. The logit, g(x), is linear in its parameters, may be continuous, and may range from −∞ to +∞ depending on the range of x. The second important difference between the linear and logistic regression models concerns the conditional distribution of the outcome variable. In the linear regression model we assume that an observation of the outcome variable may be expressed as y = E(Y|x) + ε. The quantity ε is called the error and expresses an observation’s deviation from the conditional mean. The most common assumption is that ε follows a normal distribution with mean zero and some variance that is constant across levels of the independent variable. It follows that the conditional distribution of the outcome variable given x is normal with mean E(Y|x), and a variance that is constant. This is not the case with a dichotomous outcome variable. In this situation we may express the value of the outcome variable given x as y = π (x) + ε. Here the quantity ε may assume one of two possible values. If y = 1, then ε = 1 − π (x) with probability π (x), and if y = 0, then ε = −π (x) with probability 1 − π (x). Thus, ε has a distribution with mean zero and variance equal to π (x)[1 − π (x)]. That is, the conditional distribution of the outcome variable follows a binomial distribution with probability given by the conditional mean, π (x).

University of Massachusetts, Amherst, MA, USA

The goal of a logistic regression analysis is to find the best fitting and most parsimonious, yet biologically reasonable, model to describe the relationship between an outcome (dependent or response variable) and a set of independent (predictor or explanatory) variables. What distinguishes the logistic regression model from the linear regression model is that the outcome variable in logistic regression is categorical and most usually binary or dichotomous. In any regression problem the key quantity is the mean value of the outcome variable, given the value of the independent variable. This quantity is called the conditional mean and will be expressed as E(Y|x), where Y denotes the outcome variable and x denotes a value of the independent variable. In linear regression we assume that this mean may be expressed as an equation linear in x (or some transformation of x or Y), such as E(Y|x) = β0 + β1 x. This expression implies that it is possible for E(Y|x) to take on any value as x ranges between −∞ and +∞. Many distribution functions have been proposed for use in the analysis of a dichotomous outcome variable. Cox & Snell (2) discuss some of these. There are two primary reasons for choosing the logistic distribution. These are: (i) from a mathematical point of view it is an extremely flexible and easily used function, and (ii) it lends itself to a biologically meaningful interpretation. To simplify notation, let π (x) = E(Y|x) represent the conditional mean of Y given x. The logistic regression model can be expressed as π (x) =

exp(β0 + β1 x) . 1 + exp(β0 + β1 x)

1 FITTING THE LOGISTIC REGRESSION MODEL Suppose we have a sample of n independent observations of the pair (xi , yi ), i = 1, 2, . . . , n, where yi denotes the value of a dichotomous outcome variable and xi is the value of the independent variable for the ith subject. Furthermore, assume that the outcome

(1)

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

LOGISTIC REGRESSION

variable has been coded as 0 or 1 representing the absence or presence of the characteristic, respectively. To fit the logistic regression model (1) to a set of data requires that we estimate the values of β 0 and β 1 , the unknown parameters. In linear regression the method used most often to estimate unknown parameters is least squares. In that method we choose those values of β 0 and β 1 that minimize the sum of squared deviations of the observed values of Y from the predicted values based upon the model. Under the usual assumptions for linear regression the least squares method yields estimators with a number of desirable statistical properties. Unfortunately, when the least squares method is applied to a model with a dichotomous outcome the estimators no longer have these same properties. The general method of estimation that leads to the least squares function under the linear regression model (when the error terms are normally distributed) is maximum likelihood. This is the method used to estimate the logistic regression parameters. In a very general sense the maximum likelihood method yields values for the unknown parameters that maximize the probability of obtaining the observed set of data. To apply this method we must first construct a function called the likelihood function. This function expresses the probability of the observed data as a function of the unknown parameters. The maximum likelihood estimators of these parameters are chosen to be those values that maximize this function. Thus, the resulting estimators are those that agree most closely with the observed data. If Y is coded as 0 or 1, then the expression for π (x) given in (1) provides (for an arbitrary value of β  = (β 0 , β 1 ), the vector of parameters) the conditional probability that Y is equal to 1 given x. This will be denoted Pr(Y = 1|x). It follows that the quantity 1 − π (x) gives the conditional probability that Y is equal to zero given x, Pr(Y = 0|x). Thus, for those pairs (xi , yi ), where yi = 1, the contribution to the likelihood function is π (xi ), and for those pairs where yi = 0, the contribution to the likelihood function is 1 − π (xi ), where the quantity π (xi ) denotes the value of π (x) computed at xi . A convenient way to express the contribution to the likelihood function for

the pair (xi , yi ) is through the term ξ (xi ) = π (xi )yi [1 − π (xi )]1−yi .

(3)

Since the observations are assumed to be independent, the likelihood function is obtained as the product of the terms given in (3) as follows: l(β) =

n 

ξ (xi ).

(4)

i=1

The principle of maximum likelihood states that we use as our estimate of β the value that maximizes the expression in (3). However, it is easier mathematically to work with the log of (3). This expression, the log likelihood, is defined as

L(β) = ln[l(β)]  = {yi ln[π (xi )] + (1 − yi )ln[1 − π (xi )]}. (5) To find the value of β that maximizes L(β) we differentiate L(β) with respect to β 0 and β 1 and set the resulting expressions equal to zero. These equations are as follows: n 

[yi − π (xi )] = 0

(6)

xi [yi − π (xi )] = 0,

(7)

i=1

and

n  i=1

and are called the likelihood equations. In linear regression, the likelihood equations, obtained by differentiating the sum of squared deviations function with respect to β, are linear in the unknown parameters, and thus are easily solved. For logistic regression the expressions in (6) and (7) are nonlinear in β 0 and β 1 , and thus require special methods for their solution. These methods are iterative in nature and have been programmed into available logistic regression software. McCullagh & Nelder (6) discuss the iterative methods used by most programs. In particular, they show that the

LOGISTIC REGRESSION

solution to (6) and (7) may be obtained using a generalized weighted least squares procedure. The value of β given by the solution to (6) and (7) is called the maximum likelihood ˆ Similarly, π(x estimate, denoted as β. ˆ i ) is the maximum likelihood estimate of π (xi ). This quantity provides an estimate of the conditional probability that Y is equal to 1, given that x is equal to xi . As such, it represents the fitted or predicted value for the logistic regression model. An interesting consequence of (6) is that n  i=1

yi =

n 

π(x ˆ i ).

i=1

That is, the sum of the observed values of y is equal to the sum of the predicted (expected) values. After estimating the coefficients, it is standard practice to assess the significance of the variables in the model. This usually involves testing a statistical hypothesis to determine whether the independent variables in the model are ‘‘significantly’’ related to the outcome variable. One approach to testing for the significance of the coefficient of a variable in any model relates to the following question. Does the model that includes the variable in question tell us more about the outcome (or response) variable than does a model that does not include that variable? This question is answered by comparing the observed values of the response variable with those predicted by each of two models; the first with and the second without the variable in question. The mathematical function used to compare the observed and predicted values depends on the particular problem. If the predicted values with the variable in the model are better, or more accurate in some sense, than when the variable is not in the model, then we feel that the variable in question is ‘‘significant’’. It is important to note that we are not considering the question of whether the predicted values are an accurate representation of the observed values in an absolute sense (this would be called goodness of fit). Instead, our question is posed in a relative sense. For the purposes of assessing the significance of an independent variable we compute

3

the value of the following statistic:  likelihood without the variable . likelihood with the variable (8) Under the hypothesis that β 1 is equal to zero, the statistic G will follow a chi-square distribution with one degree of freedom. The calculation of the log likelihood and this generalized likelihood ratio test are standard features of any good logistic regression package. This makes it possible to check for the significance of the addition of new terms to the model as a matter of routine. In the simple case of a single independent variable, we can first fit a model containing only the constant term. We can then fit a model containing the independent variable along with the constant. This gives rise to a new log likelihood. The likelihood ratio test is obtained by multiplying the difference between the log likelihoods of the two models by −2. Another test that is often carried out is the Wald test, which is obtained by comparing the maximum likelihood estimate of the slope parameter, βˆ1 , with an estimate of its standard error. The resulting ratio 

G = −2 ln

W=

βˆ1 ,  βˆ1 ) se(

under the hypothesis that β 1 = 0, follows a standard normal distribution. Standard errors of the estimated parameters are routinely printed out by computer software. Hauck & Donner (3) examined the performance of the Wald test and found that it behaved in an aberrant manner, often failing to reject when the coefficient was significant. They recommended that the likelihood ratio test be used. Jennings (5) has also looked at the adequacy of inferences in logistic regression based on Wald statistics. His conclusions are similar to those of Hauck & Donner. Both the likelihood ratio test, G, and the Wald test, W, require the computation of the maximum likelihood estimate for β 1 . For a single variable this is not a difficult or costly computational task. However, for large data sets with many variables, the iterative computation needed to obtain the maximum likelihood estimates can be considerable.

4

LOGISTIC REGRESSION

The logistic regression model may be used with matched study designs. Fitting conditional logistic regression models requires modifications, which are not discussed here. The reader interested in the conditional logistic regression model may find details in [4, Chapter 7]. 2 THE MULTIPLE LOGISTIC REGRESSION MODEL Consider a collection of p independent variables which will be denoted by the vector x = (x1 , x2 , . . . , xp ). Assume for the moment that each of these variables is at least interval scaled. Let the conditional probability that the outcome is present be denoted by Pr(Y = 1|x) = π (x). Then the logit of the multiple logistic regression model is given by g(x) = β0 + β1 x1 + β2 x2 + · · · + βp xp ,

Assume that we have a sample of n independent observations of the pair (xi , yi ), i = 1, 2, . . . , n. As in the univariate case, fitting the model requires that we obtain estimates of the vector β  = (β 0 , β 1 , . . . , β p ). The method of estimation used in the multivariate case is the same as in the univariate situation, i.e. maximum likelihood. The likelihood function is nearly identical to that given in (4), with the only change being that π (x) is now defined as in (10). There are p + 1 likelihood equations which are obtained by differentiating the log likelihood function with respect to the p + 1 coefficients. The likelihood equations that result may be expressed as follows: n  [yi − π (xi )] = 0

(9)

i=1

and

in which case π (x) =

3 FITTING THE MULTIPLE LOGISTIC REGRESSION MODEL

exp[g(x)] . 1 + exp[g(x)]

(10)

If some of the independent variables are discrete, nominal scaled variables such as race, sex, treatment group, and so forth, then it is inappropriate to include them in the model as if they were interval scaled. In this situation a collection of design variables (or dummy variables) should be used. Most logistic regression software will generate the design variables, and some programs have a choice of several different methods. In general, if a nominal scaled variable has k possible values, then k − 1 design variables will be needed. Suppose, for example, that the jth independent variable, xj has kj levels. The kj − 1 design variables will be denoted as Dju and the coefficients for these design variables will be denoted as β ju , u = 1, 2, . . . , kj − 1. Thus, the logit for a model with p variables and the jth variable being discrete is kj −1

g(x) = β0 + β1 x1 + · · · +

 u=1

βju Dju + βp xp .

n 

xij [yi − π (xi )] = 0,

i=1

for j = 1, 2, . . . , p. As in the univariate model, the solution of the likelihood equations requires special purpose software which may be found in many packaged programs. Let βˆ denote the solution to these equations. Thus, the fitted values for the multiple logistic regression model are π(x ˆ i ), the value of the expression in (13) computed using βˆ and xi . Before proceeding further we present an example that illustrates the formulation of a multiple logistic regression model and the estimation of its coefficients. 4 EXAMPLE To provide an example of fitting a multiple logistic regression model, consider the data for the low birth weight study described in Appendix 1 of Hosmer & Lemeshow (4). The code sheet for the data set is given in Table 1. The goal of this study was to identify risk factors associated with giving birth to a low birth weight baby (weighing less than 2500 g). In this study data were collected on

LOGISTIC REGRESSION

5

Table 1. Code Sheet for the Variables in the Low Birth Weight Data Set Variable

Abbreviation

Identification code Low birth weight (0 = birth weight ≥2500 g, 1 = birth weight <2500 g) Age of the mother in years Weight in pounds at the last menstrual period Race (1 = white, 2 = black, 3 = other) Smoking status during pregnancy (1 = yes, 0 = no) History of premature labor (0 = none, 1 = one, etc.) History of hypertension (1 = yes, 0 = no) Presence of uterine irritability (1 = yes, 0 = no) Number of physician visits during the first trimester (0 = none, 1 = one, 2 = two, etc.) Birth weight (g)

ID LOW

189 women; n1 = 59 of them delivered low birth weight babies and n0 = 130 delivered normal birth weight babies. In this example the variable race has been recoded using the two design variables shown in Table 2. FTV was recoded to 0 = some, 1 = none, and PTL was recoded to 0 = none, 1 = one or more. The two newly coded variables are called FTV01 and PTL01. The results of fitting the logistic regression model to these data are given in Table 3. In Table 3 the estimated coefficients for the two design variables for race are indicated in the lines denoted by ‘‘RACE 1’’ and ‘‘RACE 2’’. The estimated logit is given by ˆ g(x) = 0.545 − 0.035 × AGE − 0.015 × LWT + 0.815 × SMOKE + 1.824 × HT +0.702 × UI + 1.202 × RACE˜1 +0.773 × RACE 2 + 0.121 × FTV01 +1.237 × PTL01.

Table 2. Coding of Design Variables for RACE Design variable RACE

RACE 1

RACE 2

White Black Other

0 1 0

0 0 1

AGE LWT RACE SMOKE PTL HT UI FTV BWT

The fitted values are obtained using the estiˆ mated logit, g(x), as in (10).

5 TESTING FOR THE SIGNIFICANCE OF THE MODEL Once we have fit a particular multiple (multivariate) logistic regression model, we begin the process of assessment of the model. The first step in this process is usually to assess the significance of the variables in the model. The likelihood ratio test for overall significance of the p coefficients for the independent variables in the model is performed based on the statistic G given in (8). The only difference is that the fitted values, πˆ , under the model are based on the vector containing p ˆ Under the null hypoth+ 1 parameters, β. esis that the p ‘‘slope’’ coefficients for the covariates in the model are equal to zero, the distribution of G is chi-square with p degrees of freedom. As an example, consider the fitted model whose estimated coefficients are given in Table 3. For that model the value of the log likelihood is L = −98.36. A second model, fit with the constant term only, yields L = −117.336. Hence G = −2[(−117.34) − (−98.36)] = 37.94 and the P value for the test is Pr[χ 2 (9) > 37.94] < 0.0001 (see Table 3). Rejection of the null hypothesis (that all of the coefficients are simultaneously equal to zero) has an interpretation analogous to that in multiple linear regression; we may

6

LOGISTIC REGRESSION

Table 3. Estimated Coefficients for a Multiple Logistic Regression Model Using All Variables From the Low Birth Weight Data Set Number of obs. = 189 χ 2 (9) = 37.94 Prob > χ 2 = 0.0000

Logit estimates

Log likelihood = −98.36 Variable

Coeff.

Std. error

AGE LWT SMOKE HT UI RACE 1 RACE 2 FTV01 PTL01 cons

−0.035 −0.015 0.815 1.824 0.702 1.202 0.773 0.121 1.237 0.545

0.039 0.007 0.420 0.705 0.465 0.534 0.460 0.376 0.466 1.266

conclude that at least one, and perhaps all p coefficients are different from zero. Before concluding that any or all of the coefficients are nonzero, we may wish to look at the univariate Wald test statistics, se(βˆj ). These are given in the fourth Wj = βˆj / column (labeled z) in Table 3. Under the hypothesis that an individual coefficient is zero, these statistics will follow the standard normal distribution. Thus, the value of these statistics may give us an indication of which of the variables in the model may or may not be significant. If we use a critical value of 2, which leads to an approximate level of significance (two-tailed) of 0.05, then we would conclude that the variables LWT, SMOKE, HT, PTL01 and possibly RACE are significant, while AGE, UI, and FTV01 are not significant. Considering that the overall goal is to obtain the best fitting model while minimizing the number of parameters, the next logical step is to fit a reduced model, containing only those variables thought to be significant, and compare it with the full model containing all the variables. The results of fitting the reduced model are given in Table 4. The difference between the two models is the exclusion of the variables AGE, UI, and FTV01 from the full model. The likelihood ratio test comparing these two models is obtained using the definition of G given in (8). It has a distribution that is chi-square

z

Pr|z|

−0.920 −2.114 1.939 2.586 1.511 2.253 1.681 0.323 2.654 0.430

0.357 0.035 0.053 0.010 0.131 0.024 0.093 0.746 0.008 0.667

[95% conf. interval] −0.111 −0.029 −0.009 0.441 −0.208 0.156 −0.128 −0.615 0.323 −1.937

0.040 −0.001 1.639 3.206 1.613 2.248 1.674 0.858 2.148 3.027

with three degrees of freedom under the hypothesis that the coefficients for the variables excluded are equal to zero. The value of the test statistic comparing the models in Tables 3 and 4 is G = −2[(−100.24) − (−98.36)] = 3.76 which, with three degrees of freedom, has a P value of P[χ 2 (3) > 3.76] = 0.2886. Since the P value is large, exceeding 0.05, we conclude that the reduced model is as good as the full model. Thus there is no advantage to including AGE, UI, and FTV01 in the model. However, we must not base our models entirely on tests of statistical significance. Numerous other considerations should influence our decision to include or exclude variables from a model.

6 INTERPRETATION OF THE COEFFICIENTS OF THE LOGISTIC REGRESSION MODEL After fitting a model the emphasis shifts from the computation and assessment of significance of estimated coefficients to interpretation of their values. The interpretation of any fitted model requires that we can draw practical inferences from the estimated coefficients in the model. The question addressed is: What do the estimated coefficients in the model tell us about the research questions that motivated the study? For most models this involves the estimated coefficients for the independent variables in the model.

LOGISTIC REGRESSION

7

Table 4. Estimated Coefficients for a Multiple Logistic Regression Model Using the Variables LWT, SMOKE, HT, PTL01 and RACE From the Low Birth Weight Data Set Number of obs. = 189 χ 2 (6) = 34.19 Prob > χ 2 = 0.0000

Logit estimates

Log likelihood = 100.24 Variable

Coeff.

Std. error

LWT SMOKE HT RACE 1 RACE 2 PTL01 cons

−0.017 0.876 1.767 1.264 0.864 1.231 0.095

0.007 0.401 0.708 0.529 0.435 0.446 0.957

The estimated coefficients for the independent variables represent the slope or rate of change of a function of the dependent variable per unit of change in the independent variable. Thus, interpretation involves two issues: (i) determining the functional relationship between the dependent variable and the independent variable, and (ii) appropriately defining the unit of change for the independent variable. For a linear regression model we recall that the slope coefficient, β 1 , is equal to the difference between the value of the dependent variable at x + 1 and the value of the dependent variable at x, for any value of x. In the logistic regression model β 1 = g(x + 1) − g(x). That is, the slope coefficient represents the change in the logit for a change of one unit in the independent variable x. Proper interpretation of the coefficient in a logistic regression model depends on being able to place meaning on the difference between two logits. Consider the interpretation of the coefficients for a univariate logistic regression model for each of the possible measurement scales of the independent variable. 7 DICHOTOMOUS INDEPENDENT VARIABLE Assume that x is coded as either 0 or 1. Under this model there are two values of π (x) and equivalently two values of 1 − π (x). These values may be conveniently displayed in a 2 × 2 table, as shown in Table 5.

z

Pr|z|

−2.407 2.186 2.495 2.387 1.986 2.759 0.099

0.016 0.029 0.013 0.017 0.047 0.006 0.921

[95% conf. interval] −0.030 0.091 0.379 0.226 0.011 0.357 −1.781

−0.003 1.661 3.156 2.301 1.717 2.106 1.970

The odds of the outcome being present among individuals with x = 1 is defined as π (1)/[1 − π (1)]. Similarly, the odds of the outcome being present among individuals with x = 0 is defined as π (0)/[1 − π (0)]. The odds ratio, denoted by ψ, is defined as the ratio of the odds for x = 1 to the odds for x = 0, and is given by ψ=

π (1)/[1 − π (1)] . π (0)/[1 − π (0)]

(11)

The log of the odds ratio, termed log odds ratio, or log odds, is  π (1)/[1 − π (1)] = g(1) − g(0), ln(ψ) = ln π (0)/[1 − π (0)] which is the logit difference, where the log of the odds is called the logit and, in this example, these are  π (1) g(1) = ln 1 − π (1) 

and g(0) = ln

π (0) . 1 − π (0)

Using the expressions for the logistic regression model shown in Table 5 the odds ratio is    exp(β0 + β1 ) 1 1 + exp(β0 + β1 ) 1 + exp(β0 )   ψ =  1 exp(β0 ) 1 + exp(β0 ) 1 + exp(β0 + β1 )

8

LOGISTIC REGRESSION

Table 5. Values of the Logistic Regression Model When the Independent Variable is Dichotomous Independent variable X x=1 y=1 Outcome variable

y y=0

1 − π (1) =

Total

=

1 1 + exp(β0 + β1 ) 1.0

exp(β0 + β1 ) = exp(β1 ). exp(β0 )

Hence, for logistic regression with a dichotomous independent variable ψ = exp(β1 ),

x=0

exp(β0 + β1 ) π (1) = 1 + exp(β0 + β1 )

(12)

and the logit difference, or log odds, is ln(ψ) = ln[exp(β1 )] = β1 . This fact concerning the interpretability of the coefficients is the fundamental reason why logistic regression has proven such a powerful analytic tool for epidemiologic research. A confidence interval (CI) estimate for the odds ratio is obtained by first calculating the endpoints of a confidence interval for the coefficient β 1 , and then exponentiating these values. In general, the endpoints are given by

  βˆ1 ) . exp βˆ1 ± z1−α/2 × se( Because of the importance of the odds ratio as a measure of association, point and interval estimates are often found in additional columns in tables presenting the results of a logistic regression analysis. In the previous discussion we noted that the estimate of the odds ratio was ψˆ = exp(βˆ1 ). This is correct when the independent variable has been coded as 0 or 1. This type of coding is called ‘‘reference cell’’ coding. Other coding could be used. For example, the variable may be coded as −1 or +1. This type of coding is termed ‘‘deviation from means’’ coding. Evaluation of the

π (0) =

expβ0 1 + expβ0

1 − π (0) =

1 1 + expβ0

1.0

logit difference shows that the odds ratio is calculated as ψˆ = exp(2βˆ1 ) and if an investigator were simply to exponentiate the coefficient from the computer output of a logistic regression analysis, the wrong estimate of the odds ratio would be obtained. Close attention should be paid to the method used to code design variables. The method of coding also influences the calculation of the endpoints of the confidence interval. With deviation from means coding, the estimated standard error needed for con βˆ1 ), which fidence interval estimation is se(2  βˆ1 ). Thus the endpoints of the confiis 2 × se( dence interval are 

 βˆ1 ) . exp 2βˆ1 + z1−α/2 × 2 × se( In summary, for a dichotomous variable the parameter of interest is the odds ratio. An estimate of this parameter may be obtained from the estimated logistic regression coefficient, regardless of how the variable is coded or scaled. This relationship between the logistic regression coefficient and the odds ratio provides the foundation for our interpretation of all logistic regression results. 8 POLYTOMOUS INDEPENDENT VARIABLE Suppose that instead of two categories the independent variable has k > 2 distinct values. For example, we may have variables that denote the county of residence within a state, the clinic used for primary health care within a city, or race. Each of these variables has a fixed number of discrete outcomes and the scale of measurement is nominal.

LOGISTIC REGRESSION

Suppose that in a study of coronary heart disease (CHD) the variable RACE is coded at four levels, and that the cross-classification of RACE by CHD status yields the data presented in Table 6. These data are hypothetical and have been formulated for ease of computation. The extension to a situation where the variable has more than four levels is not conceptually different, so all the examples in this section use k = 4. At the bottom of Table 6 the odds ratio is given for each race, using white as the reference group. For example, for hispanic the estimated odds ratio is (15 × 20)/(5 × 10) = 6.0. The log of the odds ratios are given in the last row of Table 6. This display is typical of what is found in the literature when there is a perceived referent group to which the other groups are to be compared. These same estimates of the odds ratio may be obtained from a logistic regression program with an appropriate choice of design variables. The method for specifying the design variables involves setting all of them equal to zero for the reference group, and then setting a single design variable equal to one for each of the other groups. This is illustrated in Table 7. Use of any logistic regression program with design variables coded as shown in Table 7 yields the estimated logistic regression coefficients given in Table 8. A comparison of the estimated coefficients in Table 8 with the log odds in Table 6 ˆ shows that ln[ψ(black, white)] = βˆ11 = 2.079, ˆ ln[ψ(hispanic, white)] = βˆ12 = 1.792, and ˆ ln[ψ(other, white)] = βˆ13 = 1.386. In the univariate case the estimates of the standard errors found in the logistic regression output are identical to the estimates obtained using the cell frequencies from the contingency table. For example,

9

the estimated standard error of the estimated coefficient for design variable (1), βˆ11 , is 0.6325 = (1/5 + 1/20 + 1/20 + 1/10)1/2 . A derivation of this result appears in Bishop et al. (1). Confidence limits for odds ratios may be obtained as follows:  βˆij ). βˆij ± z1−α/2 × se( The corresponding limits for the odds ratio are obtained by exponentiating these limits as follows:  βˆij )]. exp[βˆij ± z1−α/2 × se(

9

CONTINUOUS INDEPENDENT VARIABLE

When a logistic regression model contains a continuous independent variable, interpretation of the estimated coefficient depends on how it is entered into the model and the particular units of the variable. For purposes of developing the method to interpret the coefficient for a continuous variable, we assume that the logit is linear in the variable. Under the assumption that the logit is linear in the continuous covariate, x, the

Table 7. Specification of the Design Variables for RACE Using White as the Reference Group RACE (code)

D1

White (1) Black (2) Hispanic (3) Other (4)

0 1 0 0

Design variables D2 0 0 1 0

D3 0 0 0 1

Table 6. Cross-Classification of Hypothetical Data on RACE and CHD Status for 100 Subjects CHD status

White

Black

Hispanic

Other

Total

Present Absent Total

5 20 25

20 10 30

15 10 25

10 10 20

50 50 100

ˆ Odds ratio (ψ) 95% CI ˆ ln(ψ)

1.0

8.0 (2.3, 27.6) 2.08

6.0 (1.7, 21.3) 1.79

4.0 (1.1, 14.9) 1.39

0.0

10

LOGISTIC REGRESSION

Table 8. Results of Fitting the Logistic Regression Model to the Data in Table 6 Using the Design Variables in Table 7 Variable

Coeff.

RACE 1 RACE 2 RACE 3 cons

2.079 1.792 1.386 −1.386

Variable

Odds ratio

RACE 1 RACE 2 RACE 3

8 6 4

Std. error 0.632 0.645 0.671 0.500

equation for the logit is g(x) = β 0 + β 1 x. It follows that the slope coefficient, β 1 , gives the change in the log odds for an increase of ‘‘l’’ unit in x, i.e. β 1 = g(x + 1) − g(x) for any value of x. Most often the value of ‘‘1’’ will not be biologically very interesting. For example, an increase of 1 year in age or of 1 mmHg in systolic blood pressure may be too small to be considered important. A change of 10 years or 10 mmHg might be considered more useful. However, if the range of x is from zero to one, as might be the case for some created index, then a change of 1 is too large and a change of 0.01 may be more realistic. Hence, to provide a useful interpretation for continuous scaled covariates we need to develop a method for point and interval estimation for an arbitrary change of c units in the covariate. The log odds for a change of c units in x is obtained from the logit difference g(x + c) − g(x) = cβ 1 and the associated odds ratio is obtained by exponentiating this logit difference, ψ(c) = ψ(x + c, x) = exp(cβ 1 ). An estimate may be obtained by replacing β 1 with its maximum likelihood estimate, βˆ1 . An estimate of the standard error needed for confidence interval estimation is obtained by multiplying the estimated standard error of βˆ1 by c. Hence the endpoints of the 100(1 − α)% CI estimate of ψ(c) are  βˆ1 )]. exp[cβˆ1 ± z1−α/2 cse( Since both the point estimate and endpoints of the confidence interval depend on the choice of c, the particular value of c should be clearly specified in all tables and calculations.

z

P|z|

3.288 2.776 2.067 −2.773

0.001 0.006 0.039 0.006

[95% conf. interval] 0.840 0.527 0.072 −2.367

3.319 3.057 2.701 −0.406

[95% conf. interval] 2.32 1.69 1.07

27.63 21.26 14.90

10 MULTIVARIATE CASE Often logistic regression analysis is used to adjust statistically the estimated effects of each variable in the model for differences in the distributions of and associations among the other independent variables. Applying this concept to a multiple logistic regression model, we may surmise that each estimated coefficient provides an estimate of the log odds adjusting for all other variables included in the model. The term confounder is used by epidemiologists to describe a covariate that is associated with both the outcome variable of interest and a primary independent variable or risk factor. When both associations are present the relationship between the risk factor and the outcome variable is said to be confounded. The procedure for adjusting for confounding is appropriate when there is no interaction. If the association between the covariate and an outcome variable is the same within each level of the risk factor, then there is no interaction between the covariate and the risk factor. When interaction is present, the association between the risk factor and the outcome variable differs, or depends in some way on the level of the covariate. That is, the covariate modifies the effect of the risk factor. Epidemiologists use the term effect modifier to describe a variable that interacts with a risk factor. The simplest and most commonly used model for including interaction is one in which the logit is also linear in the confounder for the second group, but with a

LOGISTIC REGRESSION

different slope. Alternative models can be formulated which would allow for other than a linear relationship between the logit and the variables in the model within each group. In any model, interaction is incorporated by the inclusion of appropriate higher order terms. An important step in the process of modeling a set of data is to determine whether or not there is evidence of interaction in the data. Tables 9 and 10 present the results of fitting a series of logistic regression models to two different sets of hypothetical data. The variables in each of the data sets are the same: SEX, AGE, and CHD. In addition to the estimated coefficients, the log likelihood for each model and minus twice the change (deviance) is given. Recall that minus twice the change in the log likelihood may be used to test for the significance of coefficients for variables added to the model. An interaction is added to the model by creating a variable that is equal to the product of the value of the sex and the value of age. Examining the results in Table 9 we see that the estimated coefficient for the variable SEX changed from 1.535 in model 1 to 0.979 when AGE was added in model 2. Hence, there is clear evidence of a confounding effect owing to age. When the interaction term ‘‘SEX × AGE’’ is added in model 3 we see that the change in the deviance is only 0.52 which, when compared with the chi-square distribution with one degree of freedom, yields a P value of 0.47, which clearly is not significant. Note that the coefficient for sex changed from 0.979 to 0.481. This is not surprising since the inclusion of an interaction term, especially when it involves a continuous variable, will usually produce fairly marked changes in the estimated coefficients of dichotomous variables involved in the interaction. Thus, when an interaction term is present in the model we cannot assess confounding via the change in a coefficient. For these data we would prefer to use model 2 which suggests that age is a confounder but not an effect modifier. The results in Table 10 show evidence of both confounding and interaction due to age. Comparing model 1 with model 2 we see that the coefficient for sex changes from 2.505 to 1.734. When the age by sex interaction is added to the model we see that the deviance

11

is 4.06, which yields a P value of 0.04. Since the deviance is significant, we prefer model 3 over model 2, and should regard age as both a confounder and an effect modifier. The net result is that any estimate of the odds ratio for sex should be made with respect to a specific age. Hence, we see that determining if a covariate, X, is an effect modifier and/or a confounder involves several issues. Determining effect modification status involves the parametric structure of the logit, while determination of confounder status involves two things. First, the covariate must be associated with the outcome variable. This implies that the logit must have a nonzero slope in the covariate. Secondly, the covariate must be associated with the risk factor. In our example this might be characterized by having a difference in the mean age for males and females. However, the association may be more complex than a simple difference in means. The essence is that we have incomparability in our risk factor groups. This incomparability must be accounted for in the model if we are to obtain a correct, unconfounded estimate of effect for the risk factor. In practice, the confounder status of a covariate is ascertained by comparing the estimated coefficient for the risk factor variable from models containing and not containing the covariate. Any ‘‘biologically important’’ change in the estimated coefficient for the risk factor would dictate that the covariate is a confounder and should be included in the model, regardless of the statistical significance of the estimated coefficient for the covariate. On the other hand, a covariate is an effect modifier only when the interaction term added to the model is both biologically meaningful and statistically significant. When a covariate is an effect modifier, its status as a confounder is of secondary importance since the estimate of the effect of the risk factor depends on the specific value of the covariate. The concepts of adjustment, confounding, interaction, and effect modification may be extended to cover the situations involving any number of variables on any measurement scale(s). The principles for identification and inclusion of confounder and interaction variables into the model are the same regardless

12

LOGISTIC REGRESSION

Table 9. Estimated Logistic Regression Coefficients, Log Likelihood, and the Likelihood Ratio Test Statistic (G) for an Example Showing Evidence of Confounding But no Interaction Model 1 2 3

Constant

SEX

AGE

−1.046 −7.142 −6.103

1.535 0.979 0.481

0.167 0.139

SEX × AGE

Log likelihood

0.059

−61.86 −49.59 −49.33

G 24.54 0.52

Table 10. Estimated Logistic Regression Coefficients, Log Likelihood, and the Likelihood Ratio Test Statistic (G) for an Example Showing Evidence of Confounding and Interaction Model 1 2 3

Constant

SEX

AGE

−0.847 −6.194 −3.105

2.505 1.734 0.047

0.147 0.629

of the number of variables and their measurement scales. Much of this article has been abstracted from (4). Readers wanting more detail on any topic should consult this reference. REFERENCES 1. Bishop, Y. M. M., Fienberg, S. E. & Holland, P. (1975). Discrete Multivariate Analysis: Theory and Practice. MIT Press, Boston. 2. Cox, D. R. & Snell, E. J. (1989). The Analysis of Binary Data, 2nd Ed. Chapman & Hall, London.

SEX × AGE

Log likelihood

G

0.206

−52.52 −46.79 −44.76

11.46 4.06

3. Hauck, W. W. & Donner, A. (1977). Wald’s Test as applied to hypotheses in logit analysis, Journal of the American Statistical Association 72, 851–853. 4. Hosmer, D. & Lemeshow, S. (2000). Applied Logistic Regression. 2nd Ed. Wiley, New York. 5. Jennings, D. E. (1986). Judging inference adequacy in logistic regression, Journal of the American Statistical Association 81, 471–476. 6. McCullagh, P. & Nelder, J. A. (1983). Generalized Linear Models. Chapman & Hall, London.

LOG-RANK TEST

The null hypothesis can also be stated using the hazard function. The null hypothesis is equivalent to comparing the hazard rates

ALAN HOPKINS Theravance Inc., South San Francisco, California

H0 : λ1 (t) = λ2 (t) = . . . = λK (t), for t ≥ 0 (1) The alternative hypothesis usually of interest is that the survival function for one group is stochastically larger or smaller than the survival functions for the other groups. H a : Sk ≥ Sk (t), or Sk (t) ≤ Sk (t), for some k, k with strict inequality for some t.

Calculation of the log-rank test is described, and an example is provided. Conditions under which the test is valid are discussed and testing of assumptions is addressed. Generalizations of the log-rank test are described including its relationship to the Cox regression model. Sample size calculation for the log-rank test is discussed, and computing software is described. Suppose T is a continuous, non-negative random variable representing survival times from point of randomization. The distribution of censored event times can be estimated by the survival function S(t) The survival function estimates the probability of surviving to time t so S(t) = Pr(T > t). The hazard function is directly related to the survival function through the relationship λ(t) = − d log S(t)/dt. The hazard rate is the conditional probability of death in a small interval [t, t + dt) given survival to the beginning of the interval. With censored data, we cannot observe events on all subjects, so we only observe a censoring time. Thus, for each subject, we have data on observation time, an indicator whether an event was observed at the last observation time, and treatment group. With this information, we can compare hazard functions in intervals containing observed death times and calculate a global test statistic for comparing the observed survival distributions among several groups. The log-rank test was originally proposed by Mantel (1), and it is equivalent under certain circumstances to the Cox (2) regression model.

0.0.2 Assumptions. The log-rank test is nonparametric in the sense that no underlying probability distribution is assumed for the survival function. The log-rank test assumes the censoring process is unrelated to survival times or to the treatment groups themselves (independent censoring) and that the survival times are from the same distribution for subjects recruited early or late in the clinical trial (stationarity). Observations are assumed to be independent. Special methods are required if recurrent events are observed on a single individual (e.g., multiple infections.) 0.0.3 Inference. Let t1 < t2 < · · · < tL represent the ordered distinct failure times in the combined groups of observations. At time tt , dij events are observed in the jth sample out of Rij individuals at risk just prior to ti . K
Here di = dij represents the total number j=1

of deaths at ti and Ri =

K


Rij the number of

j=1

subjects at risk at ti . We can represent the data at time ti as shown in Table 1. The test of hypothesis (1) is based on weighted deviations of the estimated hazard functions for each group from the overall estimated hazard rate among all data combined. If the null hypothesis is true, then an estimator of the expected hazard rate in the jth population under H0 is the pooled sample estimator of the hazard rate di /Ri . An estimate of the hazard rate for the jth sample

0.0.1 Hypothesis. Suppose we have K groups and we denote the survival functions associated with each group as S1 (t), . . . , SK (t). The null hypothesis can be expressed as H0 : S1 (t) = S2 (t) = . . . = SK (t), for t ≥ 0

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

LOG-RANK TEST Table 1. Layout and Notation for the K-Group Log-Rank Test Time ti

Group 1

Deaths Survivors

Group 2

...

di1

di2

...

diK

di

Ri2 − di2

...

RiK − diK

Ri − di

Ri1

Ri2

...

RiK

Ri

is dij /Rij . To compare survival distributions, we take a weighted average of deviations across all failure times. The test is based on statistics of the form: L  i=1

   di W(ti ) dij − Rij , j = 1, . . . , K Ri

where W(t) is a positive weight function. vj (t) is the sum over all event times of the difference in observed and conditionally expected events for group j. This quantity has a product hypermultinomial distribution with covariance matrix: Vjg =

At Risk

Ri1 − di1

Total

vj =

Group K

   L  Rig Rij Ri − di W(ti )2 di δjg − Ri Ri Ri − 1 i=1

j, g = 1, . . . , K where δ jg = 1 when j = g and 0 otherwise. Let v = (v1 , v2 , . . . , vK )T . Then a test statistic for hypothesis (1) is the quadratic form X 2 = vT V− v

(2)

where V− is a generalized inverse. The components of v are linearly dependent and sum to zero so the variance-covariance matrix has maximum rank K − 1. The overall test statistic can be constructed using any K − 1 components of v and corresponding elements of the variance-covariance matrix. Therefore, if the last row and column of V is deleted to give VK−1 and vK−1 = (v1 , v2 , . . . , vK−1 )T , then the overall log-rank test statistic is −1 vK−1 X 2 = vTK−1 VK−1

(3)

where VK−1 −1 is an ordinary inverse. The distribution of the weighted log-rank statistic is chi-squared with K − 1 degrees of freedom. Using W(ti ) = 1 gives the widely

used log-rank test. Alternative weights will be discussed in a later section. Since the log-rank statistic as presented here sums across multiple failure times, the tables used are not independent, which precludes use of standard methods to derive the asymptotic distribution of the statistic. The asymptotic properties of the log-rank test were rigorously developed using counting process techniques. For details of this approach, see Fleming and Harrington (3) or Kalbfleisch and Prentice (4). 0.0.4 A Special Case (K = 2) and W (t i ) = 1. Often a clinical trial consists of only two treatment groups. In this case, the computations are simplified. We may write the two-sample log-rank test as    L
di di1 − Ri1 Ri i=1 ZLR =     L R
Ri1 Ri − di i1 di 1− Ri Ri − 1 i=1 Ri which has approximately a standard normal distribution under the null hypothesis for large samples. 0.0.5 Relationship of the Log-rank Statistic to the Cox Regression Model. The log-rank test is closely related to the Cox proportional hazards regression model. Let zT = (z1 , . . . , zp ) represent p covariates on a given subject. In the case of the log-rank test, z would be indicator variables for treatment groups. The proportional hazards regression model is λ(t|z) = λ0 (t)exp(β T z), where λ0 (t) is the baseline hazard corresponding to zT = (0, . . . , 0) and β is a vector of regression coefficients. The likelihood for the Cox regression model is simply L

L(β) = 
i=1

exp(β T zi ) j∈Di

exp(β T zj )

(4)

LOG-RANK TEST

where Di is the set of subjects at risk at time ti . The efficient score for Equation (4) is given by U(β) = ∂/∂β log L(β), and its covariance by the inverse of I(β) = − ∂ 2 /∂β 2 log L(β). Then the score statistic is U’(0)I−1 (0)U(0), which has a chi-squared distribution with p − 1 degrees of freedom. This statistic is equivalent to the log-rank test when there are no tied survival times. 0.0.6 Power. The log-rank test is most powerful when the survival curves are proportional. This occurs when one survival function is consistently greater than the other over the study period. The log-rank test is the most powerful nonparametric test to detect proportional hazards alternatives. If the hazard functions cross, then there may be very little power to detect differences between the survival curves. One easy way to assess the proportionality assumption is to plot the Kaplan-Meier survival curves. If the survival curves cross, then the proportionality assumption is not met. Alternatively, a plot the estimated survival curves on a log(-log) scale gives a constant vertical shift of the two curves by an amount equal to the log of the hazards if the hazards are proportional. A more rigorous approach to checking the proportionality assumption is to use a statistical test based on a Cox regression model. Proportionality fails when there is an interaction between treatments and time. Introduction of a time-dependent interaction can be used to test formally for nonproportional hazards with the Cox regression model. Therneau and Grambsch (5) describe using residuals from Cox regressions to identify deviations from the proportional hazards assumption. 1 EXAMPLE: DISEASE-FREE SURVIVAL FOR ACUTE MYELOGENOUS LEUKEMIA AFTER BONE MARROW TRANSPLANTATION Klein and Moeschberger (6) provide a dataset containing 101 patients who received bone marrow transplantation after chemotherapy for acute myelogenous leukemia. Transplants were either allogenic (from the patient’s sibling) or autologous (from the patients own marrow harvested prior to chemotherapy).

3

The event time was based on relapse or death, whichever occurred first. The R software (7) package KMsurv contains this dataset called alloauto. Each patient in the dataset has a sequence number, a leukemia-free survival time (in months), an indicator for censoring (0 = yes, 1 = no), and an indicator for type of bone marrow transplant (1 = allogenic and 2 = autologous). There are 101 subjects in the dataset and 50 leukemia relapses. Of the 101 patients, 50 had allogenic transplants and 51 had autologous transplantation. An R script for this example is in Table 2. survfit calculates the Kaplan-Meier curve. The plot command gives the Kaplan-Meier curves shown in Fig. 1. The allogenic transplant survival is initially higher than the autologous transplants. This trend reverses itself as the survival functions cross at about 12 months casting doubt on the proportional hazards assumption. Vertical dashes on the survival functions represent censored observations. Finally, the survdiff command calculates the log-rank test shown in Table 3. Although there is separation of the two survival curves late in the time axis, the log-rank test does not yield a P < 0.05. Differences in the survival functions summed over time decrease the magnitude of the logrank statistic when the survival functions cross. The sum of the quantities (O-E)ˆ2/E in Table 3 is a conservative approximation to the actual log-rank chi-squared statistic and is produced in the output for information purposes only. 1.0.7 The Stratified Log-Rank Test. Sometimes one may know that survival is not proportional among all subjects but is related to a nuisance factor. Heterogeneous populations can sometimes be stratified into homogeneous groups for analysis purposes eliminating the nuisance source of variation. Stratified analysis can be applied to the log-rank test. This process is appropriate for situations where the proportional hazards assumption breaks down in known subgroups. For example, the hazard rate may be different for a disease outcome depending on disease burden at baseline. In that case, it may be possible to define strata within which the proportional hazards assumption is more

4

LOG-RANK TEST Table 2. R Script for Kaplan-Meier Plot and Log-Rank Test

0.8 0.6 0.4 0.2

allogenic autologous

0.0

Leukemia-Free Survival Probability

1.0

library(survival) data(alloauto, package = ‘‘KMsurv’’) my.fit <- survfit( Surv(time,delta) ∼ type, data = alloauto ) plot(my.fit, xlab = ‘‘Time (Months)’’, ylab = ‘‘Leukemia Free Survival Probability’’, lty = c(6, 1)) legend(5, 0.35, c(‘‘allogenic’’, ‘‘autologous’’), lty = c(1, 3)) survdiff( Surv(time,delta) ∼ type, data = alloauto )

0

10

20

30

40

50

Time (Months)

viable. The elements of the log-rank test can be computed separately for each stratum and then combined for an overall test. Let the strata be indexed h = 1, . . . , s and let v(h) and V(h) represent the stratum-specific components of the log-rank statistic. Then the stratified test statistic is expressed as (4): 

2 χK−1

s  = v(h) h=1

T  s  h=1

−1  V

(h)

s  v(h)



h=1

1.1 Choice of Weights The log-rank test is a special case of more general methods for comparing survival distributions with different weights. Careful selection

60

Figure 1. Kaplan-Meier survival curves for autologous and allogenic bone marrow transplants.

of weights can emphasize differences between certain regions of the survival curves. Several weighting schemes are summarized in Table 4. Weights equal to the number of subjects at risk Ri at each time ti was proposed by Gehan (8) for the two group setting and by Breslow (9) for multiple groups. W j (t) = Ri weights early portions of the survival functions more heavily than later portions. Tarone and Ware (10) proposed using weights that are a function of Ri such as √ Ri . Peto and Peto (11) proposed a weighting scheme based on an estimate of the common survival function. Anderson et al. (12) recommended slight modification to the Peto–Peto weights.

Table 3. Results of the Log-Rank Test for Bone Marrow Transplant Patients N Observed Expected (O-E) ˆ 2/E 50 22 24.2 0.195 51 28 25.8 0.182 Chisq = 0.4 on 1 degrees of freedom, p = 0.537 type = 1 type = 2

(O-E) ˆ 2/V 0.382 0.382

LOG-RANK TEST

5

Table 4. Common Weights for Weighted Log-Rank Statistics Weight

Test

1.0 Ri √ Ri

ˆS(t) =  1 −

Log-rank (1) Gehan-Breslow-Wilcoxon (8, 9) Tarone-Ware (10)

ti
di Ri +1



ˆ S(t)R i + 1)

i /(R ρ γ ˆ ˆ 1 − S(t) S(t)

Peto-Peto-Wilcoxon (11) Modified Peto (12) Gρ , Gρ,γ class (13, 3) ρ ≥ 0, γ ≥ 0

Table 5. Log-Rank Statistics for Bone Marrow Transplant Data Calculated Using SAS PROC LIFETEST Line No.

Test

1 2 3 4 5 6 7 8 9

Log-rank Wilcoxon Tarone Peto Modified Peto G (1,0) G (0,1) G (0,2) G (1,1)

Fleming and Harrington (13) proposed a general class of tests Gρ,γ with weight funcˆ i−1 )ρ [1 − S(t ˆ i−1 )]γ , ρ ≥ 0, tion W ρ,r (ti ) = S(t ˆ γ ≥ 0, where S(t) is the product-limit estimator based on the combined sample. Choice of ρ and γ can provide flexibility in selecting a region of the survival curves for weighting differences among the curves. Of course, W ρ,γ (ti ) = 1 when ρ = γ = 0 and we have the ordinary log-rank test. For ρ = 1 and γ = 0, more weight is given to early differences between the survival functions. For ρ = 0 and γ > 0, then more weight is given to departures between survival functions observed later during the observation period. Table 5 shows results for the bone marrow transplant data for various weighting schemes. Statistics were calculated using SAS procedure LIFETEST (14). The log-rank test lacks power with its equal weighting in this nonproportional hazards dataset. Tests in line numbers 2–6 are even less sensitive since they weight the early portion of the curves where there is the least difference. The G-class statistics that weight the later region of the survival curves (lines 7–9) give the

Chi-Square

DF

P-Value

0.3816 0.0969 0.0039 0.0000 0.0007 0.0008 4.2026 5.9276 2.9600

1 1 1 1 1 1 1 1 1

0.5368 0.7556 0.9501 0.9956 0.9791 0.9771 0.0404 0.0149 0.0853

smallest P-values for this dataset. The SAS code for calculating these statistics is given in Table 6. The code test = (all) calculates the statistics in lines 1–6 in Table 5. 1.2 Sample Size and Power The power of the log-rank test depends on the number of events during the course of a clinical study and not on the total number of subjects enrolled. Typically one estimates the number of events required and then makes assumptions about the accrual rate, length of the intake period, and the length of follow-up to estimate the number of subjects required to observe a specific number of events for a given power requirement. In the two-sample log-rank proportional hazards setting, assume the alternative hypothesis H A : S1 (t) = S(t)θ . For the twosided log-rank test with level α = 0.05 and power 1 − β, Schoenfeld (15) showed that the total sample size required is

n=

(zα/2 + zβ )2 (ln θ )2 p(1 − p)Pd

6

LOG-RANK TEST Table 6. SAS v9.1 Code to Calculate the P-Values for Weighted Log-Rank Statistics proc lifetest notable data = time time*delta(0); strata /group = type test proc lifetest notable data = time time*delta(0); strata /group = type test proc lifetest notable data = time time*delta(0); strata /group = type test proc lifetest notable data = time time*delta(0); strata /group = type test run;

where p is the proportion of subjects in group 1 and Pd represents the cumulative proportion of events in both groups by the end of the study. Generally the calculation of Pd will require assumptions about the survival distributions in each group. Pd could be estimated by assuming an exponential survival function or through simulation based on θ . In the real world of study design, the number of events observed depends on the rate of accrual, the length of the intake period, the dropout rate, and the length of follow-up. Lakatos (16) developed a general method for calculating sample size for the log-rank test that relaxes the proportionality assumption and allows arbitrary specification of the survival functions. Lakatos and Lan (17) reviewed the literature on different methods for estimating sample sizes and compared them with computer simulations. See also ‘‘Sample Size Calculation for Comparing Time-to-Event Data’’ in this encyclopedia. Use of a computer is recommended for complicated designs. The SAS procedure POWER (14) can calculate power and sample size for the log-rank test comparing two survival curves using the Lakatos method. The survival functions can be proportional hazards models, piecewise linear curves with proportional hazards, or arbitrary piecewise linear curves. The software allows specification of uniform accrual periods and a followup time. The EastSurv (18) program allows for calculation of sample size and power along with interim analysis planning. EastSurv permits

alloauto; = (all); run; alloauto; = ( fleming(0, 1) ); alloauto; = ( fleming(1, 1) ); alloauto; = ( fleming(0, 2) );

use of time-varying accrual patterns and modeling of dropout rates and piecewise exponential hazards. REFERENCES 1. N. Mantel, Evaluation of survival data and two new rank order statistics arising in its construction. Cancer Chemotherapy Rep. 1966; 50: 163–170. 2. D. R. Cox, Regression models and life tables. J. Roy. Stat. Soc. 1972; B34: 187–220. 3. T. R. Fleming and D. P. Harrington, Counting Processes and Survival Analysis. New York: Wiley, 1991. 4. J. D. Kalbfleisch and R. L. Prentice Statistical Analysis of Failure Time Data, Second Edition. New York: Wiley, 2002. 5. T. M. Therneau and P. M. Grambsch, Modeling Survival Data: Extending the Cox Model. New York: Springer, 2000. 6. J. P. Klein and M. L. Moeschberger, Survival analysis: Techniques for Censored and Truncated Data (Second Edition). New York: Springer, 2003. 7. R Development Core Team, R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2007. Available: http://www.Rproject.org. 8. E. A. Gehan, A generalized Wilcoxon test for comparing arbitrarily singly censored samples. Biometrika 1965; 52: 203–223. 9. N. Breslow, A generalized Kruskal-Wallis test for comparing K samples subject to unequal patterns of censorship. Biometrika, 1970; 57: 579–594.

LOG-RANK TEST 10. R. E. Tarone and J. Ware, On distributionfree tests for equality of survival distributions. Biometrika 1977; 64: 156–160. 11. R. Peto and J. Peto, Asymptotically efficient rank invariant test procedures (with discussion). J. Roy. Stat. Soc. A, 1972; 135: 186–206. 12. P. K. Anderson, O. Borgan, R. D. Gill, and N. Keiding, Linear nonparametric tests for comparison of counting processes with application to censored survival data (with discussion). Int. Stat. Rev. 1982; 50: 219–258. 13. D. P. Harrington and T. R. Fleming, A class of rank test procedures for censored survival data. Biometrika 1982; 69: 553–566. ® 14. SAS Institute Inc., SAS/STAT 9.1. User’s Guide. Cary, NC: SAS Institute Inc. 15. D. Schoenfeld, The asymptotic properties of nonparametric tests for comparing survival distributions. Biometrika 1981; 68: 316–319. 16. E. Lakatos, Sample sizes based on the logrank statistic in complex clinical trials. Biometrics 1988; 44: 229–241. 17. E. Lakatos and K. K. G. Lan, A comparison of sample size methods for the logrank statistic. Stat Med. 1992; 11: 179–191. 18. Cytel Software, Inc. EastSurv, User Manual. Cambridge, MA: Cytel, Inc., 2005.

CROSS-REFERENCES Censoring Kaplan-Meier Plots Cox proportional hazard model Hazard rate Hazard ratio Sample Size Calculation for Comparing Time-toEvent Data Stratified Analysis Survival Analysis

7

LONGITUDINAL DATA

1

HUNG-IR LI

and others before any treatment and then at weeks 2, 4, and 8 weekly during the 8-week treatment period.

Allergan Inc., Irvine, California

2.1 Design The study design sets the scope and features of the resulting longitudinal data. Prospectively designed longitudinal clinical trials describe in protocols the objectives and hypotheses of the trials, criteria for subject selection, the assignment of treatments to subjects (see also Randomization), and what, when, and how the outcome measures are collected. Most importantly, how the collected data including the background information such as medical history, prior medication, as well as the outcome measures or responses at the pretreatment and posttreatment visits will be analyzed need to be specified. Longitudinal data from a parallel design clinical trial such as the gabapentin study on PHN cited above will consist of 3 average daily pain scores for each patient in the treatment groups as specified in the study protocol. If a crossover design is used, each subject will receive different treatments in different periods with outcome measures repeated across time within each period (see also Crossover Design). The data will consist of a set of records for each treatment received by each subject. Since the total number of data points increases and each subject could serve as his own control in assessing the changes due to treatments, the crossover design usually requires a smaller set of subjects to collect an adequate amount of data for making valid scientific inferences. However, the multiple periods and washout time necessary in between periods may lengthen the total duration of the study.

DEFINITION

Longitudinal data are results from longitudinal studies and essentially consist of outcome measures or responses collected repeatedly over the time for each subject included in the study (see also Repeated Measures). This is in contrast to the conventional crosssectional studies in which subsets of subjects are selected from a population and evaluated all at one point of time. Therefore, longitudinal data link to multiple sets of cross-sectional data over a series of time points. 2 LONGITUDINAL DATA FROM CLINICAL TRIALS Longitudinal data could be collected from prospective or retrospective observational studies in epidemiologic or public health research to characterize the disease progression or risk factors for certain epidemics. Penninx et al. (1) reported a community-based study having evaluated 2847 subjects with or without major depression and having evaluated them for 4 years for the diagnosis of cardiac disease and cardiac mortality to assess the association between the cardiac mortality and major depression. Most of the longitudinal data in the clinical research and development settings are collected through prospectively designed longitudinal studies under specific regulatory requirements. For example, a randomized, controlled trial was conducted in 1996-1997 to evaluate the effectiveness of gabapentin for treatment of postherpetic neuralgia (PHN) as reported by Rowbotham et al. (2). A total of 229 patients found to have PHN were assigned in random (see also Randomization) to either gabapentin or placebo in an 8-week clinical trial. All patients are evaluated for the pain intensity

2.2 Time—Occasions of Measurements The desired study duration and the frequency of the measures/assessment repeated define the time metrics and/or visits. Days, weeks, months, and years are choices in clinical trials. These depend on the objectives of the longitudinal trials (e.g., quick onset or maintenance of the treatment effect), the nature of

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

LONGITUDINAL DATA

the target disease or medical condition (e.g., course of changes), and the regulatory agency requirements of clinically meaningful endpoints (e.g., reduction of weekly average pain score after 12 weeks of treatment on painful diabetic neuropathy). The overall cost and needed resources could well play a key role in the decision of time metrics, but the nature of the target disease and/or medical condition and the regulatory requirement of clinical meaningfulness are pivotal. For example, a phase III study on panic disorder may be required to have the weekly evaluation for 8 weeks instead of 4 weeks because the disorder itself has a cyclic fluctuation of panic attacks within a month. For pain-relief studies, the assessment may need to take place daily or weekly for the first period and then followed by monthly assessments to evaluate quick onset of symptom improvement as well as the maintenance of the treatment effectiveness. Selecting a sensible time metric so that the cadence is most useful for the outcome measures or responses is very important for the longitudinal studies. 2.3 Type of Responses/Measures /Outcomes Similar to the time metrics, the selection of outcome measures or responses is dictated by the questions to be answered by the longitudinal clinical trials (the objectives), the nature of the disease, and the regulatory requirement. These could be readings from a validated objective test such as blood pressure or QT interval from the EKG monitoring, patient reported scales, or diaries such as daily pain score on an 11-point Likert scale for pain associated with post-herpetic neuralgia. In general, they can be categorized as the following types that are pivotal to the selection of the statistical methods for analyses. The measures used in phase III longitudinal clinical trials are usually gold standards for the diagnosis of the disease and trajectories of clinical benefit (efficacy) or risk (safety) acceptable to regulatory agencies. Continuous: Systolic and diastolic blood pressures in a cardiopulmonary or hypertension interventional study are good examples of continuous measures. This is the most popular type of measure that statistical models were built around for years. The average pain

score in the gabapentin trial on PHN mentioned above is another example. Count: The occurrence of episodes within a certain period of time such as number of panic attacks daily in a panic disorder study falls into this type. Other examples include the exacerbation counts in a multiple sclerosis study. Categorical: Mortality is a typical dichotomous categorical response in oncology studies. The tracking of certain medical conditions or adverse events is dichotomous as well for risk assessment of medical treatments. The investigator-rated clinical global impression of change (worsened, no change, minimally improved, moderately improved, or much improved) in the gabapentin trial mentioned earlier is an ordinal categorical measure. Nominal categorical outcome measures or responses have multiple categories, but the categories are not ordered. Rating of a status statement with choices of true, false, or other is considered a nominal outcome measure but is less applicable for the evaluation of efficacy or safety in clinical trials. 3 ADVANTAGES People may wonder why clinical research should invest higher costs and more resources to do longitudinal investigations. What are the merits of choosing these designs over conventional cross-sectional studies? The need to assess changes over time for subjects with or without medical treatments is frequently the primary reason. In addition, longitudinal studies could also separate individual changes over time and the effect of baseline differences among subjects, sometime called cohort effects in epidemiology studies as cited in Pahwa and Blaire (3). In addition, longitudinal studies also result in more observations/data than cross-sectional studies with the same number of subjects. Zeger and Liang (4) used one simple example of a longitudinal study to demonstrate the increase in power and efficiency of the analysis compared with a cross-sectional study. The gain is more when the correlation is properly accounted for because each subject serves as his own control and the intersubject variability is reduced. The longitudinal data also

LONGITUDINAL DATA

allows statistical analyses that are robust to problems of model misspecification such as selection bias common on observational studies, according to Zeger and Liang (4). Jadhav and Gobburu (5) also have used data form pivotal trials (see Pivotal Trials and NDA) for treatment of Parkinson’s disease to explore the efficiency of longitudinal data analysis in comparison with a cross-sectional analysis. The longitudinal data analysis is concluded to be more powerful and less sensitive to different types of unbalanced data, whereas the cross-sectional analysis required either excluding the incomplete data (losing information) or data imputation for the missing date to make the data balanced. When considerable missing data occurred due to dropouts, the chance of claiming effectiveness falsely (see Type I Error) using the cross-sectional analysis with last observation carried forward (see LOCF) was inflated as concluded in the case comparison. 4

dividuals and thus become unbalanced as pointed out by Liang and Zeger (6). Little and Rubin (7) have characterized the missing data according to the following reasons into three mechanisms with mathematical and probability notations and have provided suggestions for statistical solutions. Diggle et al. (8) and Hedeker and Gibbons (9) have further elaborated these mechanisms with examples and simulations along with rich lists of references in various applications. • Missing Completely at Random (MCAR):

The occurrence of the specific missing data point is simply by chance. It could equally likely happen to any subject at any time and is independent of the data observed before and of the data observed after. The missing data caused by administrative errors or dropouts due to accidental death are good examples of MCAR. • Missing at Random (MAR): The occurrence of the specific missing data point is independent of the data that would have been observed. A good example is the missing values as a result of subjects being discontinued from the study when a laboratory test value is below or beyond a certain level as specified in the protocol. • Missing Not at Random (MNAR): The occurrence of the specific missing data point is dependent of the data that would have been observed. Missing values caused by dropouts due to lack of efficacy or adverse events fall into this type. This type of missing mechanism is considered informative and nonignorable in contrast to ignorable for MCAR and MAR.

CHALLENGES

Two major issues require more sophisticated statistical models and/or methods than some common approaches in analyzing longitudinal data. Correlated Data: The repeated measures collected from the same subject are correlated. In other words, the data are no longer independent to each other as required by the traditional linear regression model in assessing the association between the outcome measures or response and the explanatory factors, including the controlled medical treatment (see Regression). Ignoring the correlation will result in inefficient parameters and in inconsistent estimates of precision leading to invalid scientific inference. Missing Data: No matter how well the clinical trials are designed, conducted, and monitored, it is likely that one or more of the measures for some subjects are not repeated due to investigation site administrative error, subject noncompliance (missed visit, skipped questions, or invalid answers to questions), subjects are lost to follow-up, or subjects discontinue prematurely (see also Dropout). The longitudinal data collected are with an unequal number of observations across in-

3

5

ANALYSIS OF LONGITUDINAL DATA

The statistical methods for the analysis of longitudinal data need to be chosen based on the design of the longitudinal studies, the type of the outcome measures, the volume of the data, and the issues of the data that need to be addressed, as explained above, so that the results of the analyses capitalize on the

4

LONGITUDINAL DATA

richness of the longitudinal data, answer all intended questions, and apply to a more general population. Without appropriate methods, the inference from the longitudinal data could be invalid, misleading, and the data wasted. Zeger and Liang (4) provided an overview of methods for the analysis of longitudinal data in 1992. The most commonly used method is the regression approach in which the outcome measures or responses are related to the explanatory factors such as exposure to risk factors or medical treatment (see Regression). The full scope of the established statistical methods for longitudinal data analysis since that are suitable for different types of response/measures with the respective flexibility/limitations in addressing data correlation and missing data can be found in Fitzmaurice et al. (10), Diggle et al. (8), and Hedeker and Gibbons (9). As mentioned in Hedeker and Gibbons (9), the mixed-effect regression model (MRM) (see Mixed-effect Model) has become very popular for continuous longitudinal data. This approach models the data correlation within each subject to the regression of outcome measures or responses to explanatory factors together by assuming that the regression coefficients are different from subject to subject and are random values from a specified underlying distribution. Other explanatory factors included in the model are considered the same across subjects; thus, fixed effects and the model become a mixed-effect model. The data correlation within each subject is accounted for by each regression coefficient. These models also allow for unbalanced data due to the presence of missing data or unequally spaced measurements across time. Similarly, the mixed-effect logistic regression model is a popular choice for the analysis of dichotomous data, whereas the mixed-effect proportional odds model is quite common for the ordinal longitudinal data. Furthermore, mixed-effect regression models are developed for the nominal data and count data. It is important to note that for confirmatory longitudinal phase III studies (see Phase III Trials or Pivotal Trials), the selected statistical models/methods, similar to the selections of the measures/responses/

outcomes and time metrics, need consultation with the regulatory agency and to be prespecified in the study protocol in detail. It is also likely that simple approaches such as cross-sectional analyses for each time point with LOCF at the presence of missing data or analysis of covariance of change from baseline to the posttreatment time points may also be required along with the model-based analyses.

REFERENCES

1. B. W. J. H. Penninx, A. T. F. Beekman, A. Honig, D. J. H. Deeg, R. A. Schoevers, J. T. M. van Eijk, and W. van Tilburg, Depression and cardiac mortality: Results from a community-based longitudinal study. Arch Gen Psychiatry 2001; 58: 221–227. 2. M. Rowbotham, et al., Gabapentin for the treatment of postherpetic neuralgia: A randomized controlled trial. JAMA 1998; 280: 1837–1842. 3. P. Pahwa and T. Blaire, Statistical models for the analysis of longitudinal data. CACRC Newslett. April 2002. 4. S. L. Zeger and K. Y. Liang, An overview of methods for the analysis of longitudinal data. Stat. Med. 1992; 11: 1825–1839. 5. P. R. Jadhav and J. V. S. Gobburu, Modelbased longitudinal data analysis can lead to more efficient drug development: A case study. 2005. FDA Science, The Critical Path From Concept to Consumer, Innovative Approaches to Design and Evaluation of Clinical Trials, 11th Annual FDA Science Forum. 6. K. Y. Liang and S. L. Zeger, Longitudinal data analysis using generalized linear models. Biometrika 1986; 73: 13–22. 7. R. J. A. Little and D. B. Rubin, Statistical Analysis with Missing Data. 2nd ed. New York: Wiley, 2002. 8. P. J. Diggle, P. Heagerty, K. Y. Liang, and S. L. Zeger, Analysis of Longitudinal Data. 2nd ed. New York: Oxford University Press, 2002. 9. D. Hedeker and R. D. Gibbson, Longitudinal Data Analysis. Hoboken, NJ: Wiley, 2006. 10. G. M. Fitzmaurice, N. M. Laird, and J. H. Ware, Applied Longitudinal Analysis. Hoboken, NJ: Wiley, 2006.

LONGITUDINAL DATA

FURTHER READING T. W. Taris, A Primer in the Longitudinal Data Analysis. London, UK: Sage, 2000. J. H. Ware, Linear models for the analysis of longitudinal studies. Am. Stat. 1985; 39(2): 95–101.

CROSS-REFERENCES Adverse Event Clinical Trial/Study Crossover Design Dropout Mixed-effect Model Last Observation Cariied Forward (LOCF) New Drug Application (NDA) Phase III Trials Pivotal Trials Quality of Life Randomization Regression Repeated Measures Safety

5

MANUAL OF POLICIES AND PROCEDURES (MAPPS) The Manual of Policies and Procedures (MaPPs) comprises the approved instructions for internal practices and procedures followed by U.S. Food and Drug Administration’s Center for Drug Evaluation and Research (CDER) staff to help standardize the new drug review process and other activities. The MaPPs defines external activities, and it is available for the public to review to get a better understanding of office policies, definitions, staff responsibilities, and procedures.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/regulatory/applications/ ind page 1.htm#MaPPs) by Ralph D’Agostino and Sarah Karl.

1

MASKING

efforts. Although certain practices (sealed envelopes, tamper-evident packaging, etc.) are used in efforts aimed at both (preassignment) allocation concealment and (postassignment) blinding, such practices can confirm only that the process of allocation concealment or blinding has been ensured; they cannot assure us of the result or outcome of such efforts (7). Reporting the results of clinical trials, then, necessitates that both the blinding methods that were in place be described in detail and the results thereof also examined (7,8).

RAFE MICHAEL JOSEPH DONAHUE Vanderbilt University Medical Center, Nashville, Tennessee

Blinding, sometimes also called masking, is generally taken to mean the concealment of information about which of several treatments a subject in a clinical trial is receiving. As opposed to allocation concealment, a term used to describe the prevention of foreknowledge of the treatments that will be assigned to future subjects, blinding refers to efforts that take place after treatment assignment has been made (1). The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) guidelines note that blinding is an important means of reducing or minimizing the risk of biased study outcomes (2). Blinding in clinical trials is not a recent methodological advance, as researchers have used blinding since the late eighteenth century. Schulz and colleagues (3) report that Lavoisier and Franklin used blinding to examine claims made for mesmerism (the healing power of magnetism); Franklin is said to have used actual blindfolds on trial participants to conceal treatment information. In the early twentieth century, German researchers looked to blinding as a method of reducing bias, whereas in the United States and Britain, blinding was used as a way to reduce attrition. By the end of the 1930s, British and U.S. researchers began to acknowledge the value of blinding in avoiding bias (3). Kaptchuk (4) provides an historical account of blinding. Although blinding should help to reduce bias in clinical trials (5), its average effect seems weaker than the effect of allocation concealment (6).

2 GROUPS THAT MAY BE BLINDED AND BLINDING CONSEQUENCES FOR EACH Probably no fewer than four distinct groups of individuals can or should be blinded in a clinical trial: subjects, investigators (people directly involved in the day-to-day details of the trial), outcome assessors (people measuring outcomes, either subjective or objective, on the subjects and possibly also involved in day-to-day details of the trial), and data handlers (data managers, monitors, analysts, statisticians, and the like), although some sources actively consider only the first three groups. Still other sources consider primarily only subjects and clinicians as the active participants in the trial for whom blinding is an issue. Regardless of the classifications and names assigned to these groups, persons involved with the trial data have potential biases that can be managed (or attempted to be managed) via blinding. Subjects need to remain blinded for several reasons. Knowing that they have received a heralded new treatment, a standard treatment, a treatment with an unknown benefit, or a placebo can impact a subject’s psychological state and hence the evaluation of the treatment. Knowledge of the treatment can affect the psychological or physical responses by the subjects (3). Cooperation with the demands of the trial can also be impacted. Retention and compliance, both with visit schedules and dosing, can be influenced by knowledge of the treatment.

1 DISTINGUISHING BETWEEN METHODS AND RESULTS As an effort to reduce biases that can occur in clinical trials, a difference must be noted between the efforts or processes that are undertaken to create and maintain blinding and the actual results or outcomes of such

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

MASKING

Trial investigators, loosely defined as the individuals involved with carrying on the day-to-day details of the trial, can be influenced by knowledge of the treatments to which the subjects have been assigned. Attending physicians and nurses and anyone interacting with the participants throughout the trial are known to have their perceptions directly transferred to the subjects (9). Furthermore, their knowledge of the treatments may influence ancillary treatment for specific subjects, and/or promote or discourage continuation in the trial on the basis of the knowledge of the treatment assignment (3). A temptation exists for a caregiver to adjust care and attention in those subjects that he or she perceives to be at a disadvantage because of an inferior treatment (10). Outcome assessors, certainly those making subjective assessments, are at risk of making biased assessments if they are aware of the treatment the subject received. Interview-based scores, for example, a depression rating scale, would certainly be suspect if the interviewer and/or the assessor were aware of the subject’s treatment assignment. Obviously, the more objective the endpoint, the less observer bias can influence the results. At the extreme, an outcome such as death carries little room for bias; however, even death can be biased in a clinical trial if cause of death is required to be by a certain cause for the death to qualify as an outcome. Data handlers, if aware of treatment assignments, can also impart bias into the data. Which cases to review in an audit, which values to consider aberrant for purposes of clarification, and even which analyses to carry out on the data collection as a whole can be influenced by knowledge of the treatment assignment. Unblinded data review groups, such as data safety monitoring boards, can instill bias into a trial if they share knowledge they hold. Any positive or negative signal could impact future enrollment of certain types of subjects, thus biasing the later stages of a trial. 3

LEVELS OF BLINDING

Although the terms ‘‘open-label,’’ ‘‘singleblind,’’ and ‘‘double-blind’’ are used fre-

quently, they are often used without providing a precise definition; furthermore, the literature does not provide a consensus to their explicit definitions (3). Open-label trials are those trials in which all above-listed parties share complete knowledge of the treatment assignments. As such, these trials may be open to all the biases listed above. Single-blind trials are trials in which one group listed above (subject, investigators, outcome assessors, or data handlers) is kept without knowledge of the treatment assignments of the patients. Typically, this group will be the subjects, meaning that the investigators, outcome assessors, and data handlers all will have knowledge of the treatment assignments, but the subjects will not. Senn (11) presents arguments that singleblind placebo run-ins are unethical and scientifically and statistically unnecessary and adopts the motto ‘‘no randomization, no placebo,’’ which implies that placebos ought be used only when randomization has dictated their assignment. The term double-blind denotes a situation where two of the groups are blinded; however, it can also indicate a scenario where all the individuals involved in the trial are blinded. As such, some have held that this term can be misleading (3). However the term is used, description of exactly who was kept ignorant of treatment assignment (and how this was attempted and how successful those attempts were) should be noted in the report of the trial. The ICH definition of single-blinding refers to subjects being unaware; ICH doubleblinding refers to ‘‘subjects, investigators, monitors, and, in some cases, data analysts’’ as well (2). Triple-blinding as a term has been used to mean a trial where subject, investigators, and outcomes assessors are blinded, along with data analysts. The term ‘‘quadrupleblind’’ has been used rarely to mean all four groups (subject, investigators, outcome assessors, and data handlers) are kept ignorant (3). The Consolidated Standards of Reporting Clinical Trials (CONSORT) guidelines encourage the reporting of the blinding in the trial to specify ‘‘whether or not participants,

MASKING

those administering the interventions, and those assessing the outcomes were aware of group assignment [and] if not, how the success of masking was assessed’’ (8). As such, ambiguous descriptors, such as single-blind, double-blind, and triple-blind, are discouraged, and details concerning exactly who was kept ignorant, how they were kept that way, and how that was determined, are encouraged. 4 TECHNIQUES USED IN ATTEMPTS TO BLIND Double-blinding, at least under the definition that double-blinding includes subjects, investigators, and outcome assessors, is a goal of any trial, although such a goal might not always be attainable. In a comparator trial, treatments that match (with regard to method of treatment, size, shape, color, etc. of medications, etc.) make for the most effective attempt at blinding. In the case of two oral tablets, say one red and one blue, a doubledummy technique can be used: Matching placebo tablets, one red and one blue, both completely matching its active treatment can be constructed. A subject’s treatment is then a pair of tablets, either red active and blue placebo or red placebo and blue active. A similar technique can be used with many medications and treatments: Placebos to match the treatment (be they tablets, capsules, transdermal patches, injections, or even bandages or incisions) can be designed. Each subject then gets one active and one placebo of each type. In this scheme, of course, some players in the trial may necessarily be unblinded; for example, in a trial that compares a surgical treatment to an oral medication, the surgeon performing the procedure will certainly know if the surgery was real or not. In such a trial then, a blinded assessor is a must. Blinded readers in this sense and others (radiology or chart reviews, examinations of before-and-after photographs, etc.) are a practical way of maintaining blinding, at least at the level of evaluation. If an intervention cannot be blinded, then efforts must be made at least to mask the identity of the active treatment. As an

3

example, a trial that gives out health education pamphlets to the treated group should give out at least something to the control group, lest they come to know that they are actually the control group. This strategy can be used even when it is not possible to make the two groups look identical and would address at least some biases. Senn (11). discusses trials in which complete blinding is not possible, as in a case where effects of the treatment are readily discernable. Such a situation Senn refers to as veiling. In such a case, a subject might be able to determine that some treatments could be ruled out, but others might still be in play. Analysis of such designs requires special considerations discussed in the article. 5 MEASURING EFFECTIVENESS OF BLINDING Although valiant attempts can be made to keep all necessary parties in a clinical trial ignorant of treatment assignments, just attempting to maintain the blind does not guarantee the desired result. Measuring, or at least attempting to measure, the result of the efforts to maintain the blind needs to be performed. Investigators should assess the success of the blinding by directly asking the individuals who were to be blinded which intervention they think was administered. If the blinding was successful, then this determination should be no better than is achievable just by chance while guessing (3). Of course, side effects can provide hints about treatment assignment. Furthermore, not all individuals may feel free to express what they experienced or did to gain the knowledge they held; they might not want to expose some deliberate unblinding behavior. As such, measuring the success of the blinding effort will be met with some difficulties in interpretation (3). It is better, however, to report attempts at such interpretations and measurements, with all their foibles and frailties, than not to attempt them at all. At the least, investigators need to report any failure of the blinding procedure, such as use of nonidentical placebo or active preparations (3).

4

MASKING

When randomization is restricted, as is the case when treatments are randomized within small blocks of, say, six or eight patients at each site, premature unblinding can result in blown allocation concealment. Berger and Exner (12) discuss ways to test for selection bias in such a scenario, and no evaluation of the success of a masking scheme is complete without such an analysis. 6

UNBLINDING

Before a trial begins, procedures must be in place to dictate how unblinding will take place for both unanticipated situations (such as issues with patient safety) and for situations surrounding the planned analyses of the final study data. Typically, unblinding is carried out when a patient’s safety is at risk and knowledge of the treatment assignment is necessary for caring for the patient. If, however, simply ceasing study treatment is an option to care for the patient, then it is probably not necessary to unblind the treatment assignment. Of course, audit trails and appropriate documentation of all planned and unplanned unblindings should be created at the time unblinding occurs (1).

quality of controlled clinical trials. BMJ. 2001; 323: 42–46. 6. K. F. Schulz, I. Chalmers, R. J. Hayes, and D. G. Altman, Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995; 273: 408–412. 7. V. W. Berger and C. A. Christophi, Randomziation technique, allocation concealment, masking, and susceptibility of trials to selection bias. J. Mod. Appl. Stat. Methods. 2003; 2: 80–86. 8. D. Moher, K. F. Schulz, and D. G. Altman. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann. Intern. Med. 2001; 134: 657–662. 9. S. Wolf, Effects of suggestion and conditioning on action of chemical agents in human subject—pharmacology of placebos. J. Clin. Invest. 1950; 29: 100–109. 10. A. K. Akobeng, Understanding randomised controlled trials. Arch. Dis. Child. 2005; 90: 840–844. 11. S. J. Senn, A personal view of some controversies in allocating treatment to patients in clinical trials. Stat. Med. 1995; 14: 2661–2674. 12. V. W. Berger and D. V. Exner, Detecting selection bias in randomized clinical trials. Control. Clin. Trials. 1999; 20(4): 319–327.

FURTHER READING REFERENCES 1. P. M. Forder, V. J. Gebski, A. C. Keech, Allocation concealment and blinding: when ignorance is bliss. Med. J. Aust. 2005; 182: 87–89. 2. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH), ICH Guideline E8: General considerations for clinical trials. ICH web site. Available: www.ich.org. Accessed June, 22, 2006. 3. K. F. Schulz, I. Chalmers, and D. G. Altman, The landscape and lexicon of blinding in randomized trials. Ann. Intern. Med. 2002; 136: 254–259. 4. T. J. Kaptchuk, Intentional ignorance: a history of blind asessment and placebo controls in medicine. Bull. Hist. Med. 1998; 72: 389–433. ¨ 5. P. Juni, D. G. Altman, and M. Egger, Systematic reviews in health care: assessing the

H. J. Bang, L. Y. Ni, and C. E. Davis, Assessment of blinding in clinical trials. Control. Clin. Trials. 2004; 25(2): 143–156. V. W. Berger, Allocation concealment and blinding: when ignorance is bliss. Med. J. Aust. 2005; 183: 165. V. W. Berger, A. Ivanova, and M. Deloria-Knoll, Minimizing predictability while retaining balance through the use of less restrictive randomization procedures. Stat. Med. 2003; 22: 3017–3028. S. J. Day and D. G. Altman, Statistics notes: blinding in clinical trials and other studies. BMJ. 2000; 321: 504. R. Kunz and A. D. Oxman, The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ. 1998; 317: 1185–1190. S. J. Senn, Turning a blind eye—Authors have blinkered view of blinding. Brit. Med. J. 2004; 328(7448): 1135–1136.

MASKING

CROSS-REFERENCES Placebo Allocation concealment Randomization codes Randomization envelopes Randomization methods

5

MAXIMUM DURATION AND INFORMATION TRIALS

where zα and zβ are upper α and β quantiles of the standard normal distribution. This test is asymptotically equivalent to the familiar Wald test

KYUNGMANN KIM Biostatistics and Medical Informatics University of Wisconsin Madison, Wisconsin

Z= 


 where Var θˆ = I−1 f ixed . In typical clinical trials for chronic diseases such as cancer, patients are entered into the study during the accural period, (0, sa ), during which patients are enrolled serially, in a pattern known as staggered entry, and then are followed until the occurrence of an event of interest such as death of the patient or disease progression or until the time of study closure, subject to random loss to follow-up. The period from the time when the last patient enters the study to the time of study closure—that is, (sa ,sa + sf )— is known as the follow-up period, during which no new patients are entered; the length of this period is known as the minimum follow-up time for the study. The study duration consists of the accrual duration and the follow-up duration. In such clinical trials, a response variable is time to event, with possible censoring of events due to random loss to follow-up or study closure. Many clinical trials with time-to-event data as the primary outcome have been designed as duration trials, simply because studies are often planned in terms of calendar time. The difference between the calendar time and the information in chronic disease clinical trials in which time to event is a response variable leads to differences in how such studies are designed and analyzed. Lan and DeMets (1) were perhaps the first to formally recognize the difference between the two.

In clinical trials in which a response variable is observed more or less instantaneously, the statistical information of the test statistic is proportional to the number of study subjects, which is directly related to the duration of the study in calendar time and the rate by which subjects are enrolled and randomized into the study. However, in clinical trials in which a response variable is observed over follow-up as time to event, the statistical information is not directly related to the number of subjects but rather to the number of events of interest. Regardless of the type of response variable, the efficient score is a

U ∼ N (θ I, I) , where I is Fisher’s information and θ is the canonical parameter representing treatment effect according to the likelihood theory under the assumption of a relative large sample and a small effect size of treatment. A one-sided fixed-sample test of the null hypothesis H0 :θ = 0 indicating no treatment difference with a significance level α to detect the alternative hypothesis H 1 :θ = θ 1 > 0 with power 1 − β requires Pr(U ≥ cα |θ = 0) =α and Pr(U ≥ cα |θ =θ1 ) = 1 − β. This determines the information content or the information horizon of the study as If ixed =

2
zα + zβ θ12

θˆ − θ a
 ∼ N (0, 1) , ˆ Var θ

1 TWO PARADIGMS: DURATION VERSUS INFORMATION

(1)

and the critical value of the test as 
zα + zβ zα , cα = θ1

According to Lan and DeMets (1), there are two paradigms for the design of clinical trials with time-to-event data: duration trials and information trials. They are distinguished

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

MAXIMUM DURATION AND INFORMATION TRIALS

by the way the end of the study is defined. In the former, the study is concluded when data are accumulated over a fixed duration of follow-up on a specified number of subjects, a sample size in a traditional sense. In the latter, a study is concluded when a prespecified amount of statistical information has been obtained. For example, with the logrank test for comparison of time-toevent data in chronic disease clinical trials, the operating characteristics of the study do not depend directly on the number of subjects enrolled in the study, which is a function of the duration of the accrual period in calendar time and the enrollment rate of subjects into the study, but rather depend on the number of events of interest, which is directly proportional to the statistical information. Thus, the study design often specifies the duration of the accrual and follow-up periods to ensure that a necessary number of events are observed during the study. Either the accural duration or the follow-up duration is fixed, and the necessary duration of the other period is determined so that the required number of events are ultimately observed during the study. Determination of the study duration for a fixed-sample study has been investigated by Bernstein and Lagakos (2) and Rubinstein et al. (3). With the duration trial, the information of the test statistic at study closure is random, and there is no guarantee that the required amount of statistical information specified in equation 1 will be obtained if the duration design is strictly adhered to in the analysis. With the information trial, however, the statistical information specified for the design in equation 1 is obtained exactly, and as a result the operating characteristic of the statistical test can be maintained exactly as specified in the design. The calendar time of study closure will be random. 2 SEQUENTIAL STUDIES: MAXIMUM DURATION VERSUS INFORMATION TRIALS If it is desirable to monitor the data periodically during the course of a study, group sequential designs or designs based on triangular tests can be used. Group sequential methods such as by Pocock (4) and O’Brien

and Fleming (5) were developed to maintain the type I error probability at a desired overall significance level despite repeated significance tests. Both methods assume that the number of interim analyses is specified in advance and that the interim analyses are performed after equal increments of statistical information. For monitoring of time-toevent data, one needs the flexibility because these two assumptions are often not met. This flexibility can be achieved by using the error spending function introduced by Lan and DeMets (6). When computed at calendar time s, the efficient score is a

U (s) ∼ N (θ I (s) , I (s)) , as in the fixed-sample study. With the logrank test, which is the efficient score under the proportional hazards model, θ is the log hazard ratio of control λc to experimental λe ; that is, θ = log(λc /λe ), and I(s) is the asymptotic variance of the logrank statistic. The asymptotic variance I(s) is closely related to
the  expected number of events ε(s) by I (s  σz2 ε (s), where σz2 is the variance of treatment indicator Z. As will be shown later, the expression for ε(s) can be derived based on some parametric assumptions regarding the length of the accrual period in calendar time, the enrollment rate, and the distribution for time to event. Assume that patient accrual is uniform during the accrual period (0, sa ) with a constant accrual rate A, the average number of patients accrued per unit time; that allocation to treatment is by simple randomization, with possibly unequal allocation between two treatments, control (c) and experimental (e); that events occur with constant hazard rates λv ,v = c,e; and that random censoring occurs with common constant hazard rate ν. Finally, assume, at least tentatively, that accumulating data will be analyzed after equal increments of information for a maximum of K times. When designing a group sequential study, it is reasonable to assume a prespecified maximum number of analyses K at equal increments of statistical information. With a group sequential design, it is known that the information horizon of the study has to be

MAXIMUM DURATION AND INFORMATION TRIALS

inflated from that for the fixed-sample study as compensation for the possibility of early stopping for treatment difference. Given K,α, and β, and a group sequential design, the amount of required inflation in sample size is called the inflation factor F by Kim et al. (7). Given the information horizon for the corresponding fixed-sample study determined by equation 1, the maximum information, (i.e., the information content for the group sequential study), is determined by Imax = If ixed × F.

(2)

Given the maximum information for the group sequential design, the necessary maximum expected number of events to be observed by the end of study at calendar time sK (i.e., at the last analysis K), is determined by εmax  Imax /σz2 . Then the length of study sK = sa + sf is determined to satisfy ε(1) (sK ) = (1 − µz ) εe (sK ) + µz εc (sK ) = εmax , where µz is the mean of treatment indicator Z for the experimental treatment and εv (s) is the expected number of events by time s when all patients are given treatment v. In other word, the maximum duration of the study is −1 determined by sK = ε(1) (εmax ) subject to εmax −1 ≤ sa ≤ ε(1) (εmax ) . A These inequalities ensure that the accrual duration is long enough— but no longer than necessary— for the required maximum number of events. The expected number of events εv (s) can be evaluated by double integration with respect to the density function for time to event and the uniform density function for patient entry. For example, under exponential time with hazard rates λv , v = c, e, and exponential random censoring with common constant hazard rate ν, then the expected number of events by time s if all the patients in the study are given treatment v are εv (s) = A

λv λv

  exp{−λv (s − sa )+ } − exp(−λv s) s ∧ sa − , λv

3

where λv  = λv + ν, s∧&sa is the smaller of s and sa , and x+ = x if x is positive and 0 otherwise.Therole of exponential distribution is simply to provide a calculation for the expected number of events. The above formula can be generalized to other time to event distributions, and the arguments extend naturally to other parametric proportional hazards models after suitable transformation on the time scale. As noted above, once the study duration is fixed, the maximum expected number of events by the end of the trial can be estimated as ε(1) (sa + sf ). Hence, one may choose to fix the trial duration, sa + sf , or, equivalently, the total number of events to be observed. Although interim analyses are scheduled at regular intervals in calendar times, they depend on the information. By analogy with the fixed-sample study, there are also two paradigms for design of group sequential trials (1): the maximum duration trial in which the maximum duration of the study is fixed, or the maximum information trial in which the maximum information of the study is fixed. As such, the two designs again differ in how the end of the study is defined. Design procedure for maximum duration trials with time data has been investigated by Kim and Tsiatis (8), and that for maximum information trials has been proposed by Kim et al. (9). A maximum duration design specifies the end of the study in terms of the study duration, and a maximum information design specifies the end of the study in terms of the maximum information. In a maximum duration trial, the study is concluded either due to early stopping for treatment difference or when the follow-up reaches the prespecified calendar time. In a maximum information trial, the study is concluded either due to early stopping for treatment difference or when a prespecified maximum information is reached. Sequential clinical trials with time-toevent data are often designed as maximum duration trials in which a specified number of subjects enrolled during the accrual period are evaluated over the follow-up period. As such, the maximum duration—the accrual duration plus the follow-up duration—of a clinical trial is fixed at the design stage.

4

MAXIMUM DURATION AND INFORMATION TRIALS

Because the maximum information to be observed during the study is unknown at the time of an interim analysis, the information times have to be estimated. When monitoring a clinical trial with time-to-event data using the logrank test, the information time at an interim analysis is proportional to the maximum expected number of events to be observed during the study and is estimated by the number of events observed at the time of interim analysis divided by the maximum number of events expected by the end of the study. The denominator of this fraction is a random quantity and must be estimated. There are at least two candidates for the denominator, one under the null hypothesis of no treatment difference and the other under the specified alternative hypothesis; therefore, there are at least two candidates for the information time: tˆj =

Number of observed events at interim analysis , Maximum expected number of events Hj at the end of trial

j = 0, 1. This problem is due to the fact that the asymptotic variance of the logrank statistic depends on treatment effect, a situation similar to the comparison of proportions. This uncertainty about information horizon complicates interim analyses as it necessitates some adjustments in determining group sequential boundaries to maintain the type I error probability at a specified significance level. Kim et al. (9) proposed the following convention for estimation of the information time:  ε /ε if k < K and dk ≤ Dj , ˆtk,j = k max,j 1 otherwise where εk denotes the observed number of events at calendar time sk of the k-th interim analysis for k = 1, . . . ,K, and εmax,j denotes the information horizon under Hj for j = 0,1. By setting the information time equal to 1 at the last analysis, the type I error probability is always maintained using the error spending function. In maximum information trials, the information horizon is determined under the

specified alternative hypothesis to achieve a desired power given other design parameters, and the information time is estimated by tˆ1 and is always unbiased. The net effect is that computation of group sequential boundaries becomes straightforward; furthermore, not only the significance level but also the power of group sequential tests is maintained exactly as specified during the design of the study. Therefore, from a statistical point of view, a maximum information trial design is preferable. The end of the study is, however, defined as the random calendar time when the information horizon is realized. 3 AN EXAMPLE OF A MAXIMUM INFORMATION TRIAL For many years, radiotherapy has been the treatment of choice for patients with regional stage III non-small cell lung cancer, giving a median survival of 9 to 11 months and a 3-year survival rate of less than 10%. In attempts to improve survival in these patients, clinical researchers in the early 1980s considered the possibility that radiotherapy alone might not be sufficient to eradicate micrometastatic disease. At that time, there was also some evidence that platinum-based chemotherapy was beneficial in terms of survival in patients with more advanced disease. Therefore, various systemic approaches to the treatment of stage III patients were proposed, including chemotherapy in conjunction with radiotherapy. The Cancer and Leukemia Group B 8433 (CALGB 8433) study was developed in 1983tocompare the standard control treatment with radiotherapy alone with an experimental treatment using two courses of combination chemotherapy given before radiotherapy in these patients. The primary objective of the CALGB 8433 study was to compare survival with the experimental treatment with that with the control treatment. Under the proportional hazards assumption, survival data can be modeled based on the hazard function λ (t|z) = λ0 (t) exp (θ z) , where λ0 (t) represents the baseline hazard at time t, z is a treatment indicator, and

MAXIMUM DURATION AND INFORMATION TRIALS

θ is the log hazard ratio. The null hypothesis of no treatment difference is H0 :θ = 0, and the alternative hypothesis that the two treatments differ with respect to survival is H 1 :θ = 0. The sample size was obtained to achieve 80% power (1 − β) to detect a log hazard ratio, θ = 0.405, using the logrank test at a twosided significance level of α = 0.05. This log hazard ratio represents a 50% difference in median survival between the two treatments. Therefore, a total of 190 deaths on the two arms was required: εf ixed = =

2
4 zα/2 + zβ θ12 4 (1.96 + 0.84)2 = 190. (log 1.5)2

The fixed sample size was determined by assuming that the final analysis of the study would take place after 80% of the patients had died, so 190/0.8 or approximately 240 patients were required. Based on previous experience in the same patient population, about 60 patients were expected to be accrued each year. Therefore, the study was anticipated to have about 4 years of accrual with possibly 6 months to 1 year of additional follow-up to obtain 80% deaths among 240 patients. When this study was developed initially in 1983, this fixed sample size was used, and there was no provision for early stopping for treatment difference. The CALGB policies for interim monitoring for possible

early termination were amended in 1986, coinciding with the emergence of treatment differences in CALGB 8433. At the time of the first interim monitoring, a conservative error spending function α4∗ (t) = αt1.5 by Kim and DeMets (10) was chosen for formal sequential tests to take advantage of its flexible nature. This error spending function was known to generate group sequential boundaries similar to O’Brien and Fleming (5), but not quite as conservative early on. Also, because formal interim monitoring was going to be used, it was decided that the final analysis would be performed with more ‘‘information’’ than for the fixed-sample size analysis in order to maintain the same power. This group sequential test at 80% power has an inflation factor of F = 1.05 over the corresponding fixed-sample size design. Therefore, the maximum number of deaths required by the end of study was inflated to εmax = 190×1.05 = 200 in accordance with equation 2 and the proportionality between the number of events and the information. Table 1 summarizes the monitoring process in the study. Based on the the ground sequential test and the error spending function α4∗ (t), the study was closed in March of 1987. Although only 28% of the total number of deaths were obtained, 163 (68%) of the total number of patients were accrued by the time of study termination. More importantly, the results of this trial were published 2 years earlier than originally anticipated.

Table 1. Summary of the monitoring process in CALGB 8433. Nominal P-value Analysis date Sep 1985 Mar 1986 Aug 1986 Oct 1986 Mar 1987

5

Percent of information

Logrank P-value

Truncated O’Brien-Fleminga

Pocock

α4∗ (t)

5% 8% 18% 22% 29%

NA 0.021 0.0071 0.0015 0.0015b

0.0013 0.0013 0.0013 0.0013 0.0013

0.0041 0.0034 0.0078 0.0061 0.0081

0.0006 0.0007 0.0027 0.0026 0.0042

a The standard O’Brien-Fleming boundary would give a nominal P-value less than 0.0001 at each interim analysis shown. b The P-value of 0.0008 from the Cox model was used in the decision for early termination of the study. Note: NA, not applicable.

6

MAXIMUM DURATION AND INFORMATION TRIALS

REFERENCES 1. K. K. G. Lan and D. L. DeMets, Group sequential procedures: calendar versus information time. Stat Med. 1989; 8: 1191–1198. 2. D. Bernstein and S. W. Lagakos, Sample size and power determination for stratified clinical trials. J Stat Comput Simulat. 1978; 8: 65–73. 3. L. V. Rubinstein, M. H. Gail, and T. J. Santner, Planning the duration of a comparative clinical trial with loss to follow-up and a period of continued observation. J Chronic Dis. 1981; 34: 469–479. 4. S. J. Pocock, Group sequential methods in the design and analysis of clinical trials. Biometrika. 1977; 64: 191–199. 5. O’Brien, P. C. and T. R. Fleming, A multiple testing procedure for clinical trials. Biometrics. 1979; 35: 549–556. 6. K. K. G. Lan and D. L. DeMets, Discrete sequential boundaries for clinical trials. Biometrika. 1983; 70: 659–663.

7. K. Kim, A. A. Tsiatis, and C. R. Mehta, Computational issues in information-based group sequential clinical trials J Jpn Soc Comput Stat 2003; 15: 153–167. 8. K. Kim and A. A. Tsiatis, Study duration and power consideration for clinical trials with survival response and early stopping rule. Biometrics. 1990; 46: 81–92. 9. K. Kim, H. Boucher, and A. A. Tsiatis, Design and analysis of group sequential logrank tests in maximum duration versus information trials. Biometrics. 1995; 51: 988–1000. 10. K. Kim and D. L. DeMets, Design and analysis of group sequential tests based on the type I error spending rate function Biometrika 1987; 74: 149–154.

CROSS-REFERENCES

MAXIMUM TOLERABLE DOSE

point that is acceptable to the patient, based on several severity grading scales of adverse events, such as the Common Toxicity Criteria Grades developed by the National Cancer Institute of the United States or those developed by the World Health Organization. For instance, in oncology, DLT is usually defined as any nonhematological grade III or grade IV toxicity (except alopecia, nausea, vomiting, or fever, which can be rapidly controlled with appropriate measures); absolute neutrophil count < 500/ml for at least 7 days; febrile neutropenia (absolute neutrophil count < 500/ml for at least 3 days and fever above 38.5◦ C for 24 hours); or thrombocytopenia grade IV. Nevertheless, as stated above, definition of dose-limiting toxicities widely differs across Phase I (1).

SYLVIE CHEVRET Inserm France

The ‘‘Maximum Tolerable Dose’’ (MTD), also known as the ‘‘Maximum Tolerated Dose’’ or the ‘‘Maximally Tolerated Dose’’, is used both in animal toxicology and in early phases of clinical trials, mostly in life-threatening diseases such as cancer and AIDS. In toxicology, the MTD has been defined operationally as the highest daily dose of a chemical that does not cause overt toxicity in laboratory mice or rats. Similarly, in Phase I cancer clinical trials, the MTD of a new agent has been defined as the highest dose level of a potential therapeutic agent at which the patients have experienced an acceptable level of dose-limiting toxicity (DLT) or that does not cause unacceptable side effects. In both settings, these definitions refer on imprecise notions such as ‘‘overt,’’ ‘‘acceptable,’’ or ‘‘unacceptable’’ toxicity. Moreover, definition of dose-limiting toxicities has been found highly variable across published Phase I studies (1). This lack of consensus is observed also when dealing with the methods for establishing the MTD, with wide variations in the designs and the statistical methods of estimation. Finally, the concept of MTD itself has become criticized and even controversial. We summarize the main concepts underlying the choice of MTD as the main endpoint in cancer Phase I clinical trials and the main approaches used in establishing the MTD of a new drug. More discussion with regards to its currently reported limits is provided in the last section with some proposed alternate endpoints. 1

1.1 Underlying Assumptions Determining the optimal dose of a new compound for subsequent testing in Phase II trials is the main objective of cancer Phase I clinical trials. With cytotoxic agents, this dose typically corresponds to the highest dose associated with an acceptable level of toxicity, based on the underlying assumption that stems from the work of Skipper et al. (3) that the higher the dose, the greater the likelihood of drug efficacy. In addition to the relationship between dose and antitumor response, cytotoxic agents also exhibit a dose–toxicity relationship. Thus, dose-related toxicity is regarded as a surrogate for efficacy. In other words, it is assumed that dose-response curves for toxicity and efficacy are parallel or, simply expressed, ‘‘the more pain, the more gain.’’ These findings have yielded the concept of maximum tolerable or tolerated dose (MTD) as the main objective of Phase I trials. Of note, the recommended dose level for more Phase II trials is either considered as synonymous with the MTD or, in two thirds of the Phase I trials as reported recently (1), chosen as one dose level below the selected MTD.

BASIC CONCEPTS AND DEFINITIONS

The MTD is often defined as the dose that produces an ‘‘acceptable’’ level of toxicity (2); the dose that, if exceeded, would put patients at ‘‘unacceptable’’ risk for toxicity; or the dose that produces a certain frequency of (medically unacceptable) reversible, dose-limiting toxicity (DLT) within the treated patient population. DLT includes host effects up to the

2

HOW IS THE MTD ESTABLISHED?

In Phase I oncology trials conducted over the past few decades, the MTD has usually been

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

MAXIMUM TOLERABLE DOSE

estimated by the traditional 3 + 3 escalation rule, which traces back to the last 1960s (4). The maximum tolerated dose is determined thus by testing increasing doses on different groups of people until the highest dose with acceptable side effects is found (see ‘‘Phase I trials’’ for more details). 2.1 The MTD is Interpreted Differently According to the Design Numerous Phase I dose finding clinical trials are conducted everyday to find the ‘‘maximum tolerated dose’’ (MTD) of a cancer treatment. Although various innovative statistical designs for Phase I clinical trials have been proposed in the literature, the traditional 3 + 3 design is still widely used because of its algorithm-based simplicity in logistics for the clinical investigators to carry out. Actually, based on a recent review of Phase I trials of single agent cytotoxics published between 1993 and 1995, the MTD was usually defined as the dose level at which > 2/6 patients experienced DLT, but several studies required 3–4/6 patients (1). Such algorithm- or rulebased Phase I designs treat the MTD as being identifiable from the data so that in this setting, the MTD is a statistic rather than a parameter (5). Indeed, no statistical estimation after these trials has been maintained, although statistical quantities of interest have been recently estimated from such trial data (6). By design, the MTD estimate relies heavily on the actual cohort of patients treated and the order in which they enter the study, with poor properties. It seemed in the early 1990s that the MTD should be thought of as a population characteristic rather than as a sample statistic. The MTD as a percentile of the dose–toxicity relationship seemed the mostly commonly used definition of the MTD in so-called model-based designs. Most proposals consisted in establishing a mathematical model for the probability of DLT over the dose scale, iteratively fitted from the data after each patient inclusion using Bayesian inference (7,8). Comparisons show that the Bayesian methods are much more reliable than the conventional algorithm for selecting the MTD (9). NonBayesian methods were also proposed, with close properties (10).

3 EXTENDING THE MTD? 3.1 Maximum Tolerated Schedule Most Phase I clinical trials are designed to determine a maximum tolerated dose for one initial administration or treatment course of a cytotoxic experimental agent. Toxicity usually is defined as the indicator of whether one or more particular adverse events occur within a short time period from the start of therapy. However, physicians often administer an agent to the patient repeatedly and monitor long-term toxicity caused by cumulative effects. A new method for such settings has been proposed, the goal of which is to determine a maximum tolerated schedule (MTS) rather than a conventional MTD (11,12). 3.2 Overdose Control More recently, some researchers considered the percentile definition of MTD as inadequate in addressing the ethical question of protecting the patients from severe toxicity. This consideration leads to the suggestion by Babb and colleagues (13) that the MTD should be chosen with toxicity not exceeding the tolerable toxicity with a high probability, which imposes a safety constraint on the overdose control. 3.3 Most Successful Dose Objective responses observed in Phase I trials are important for determining the future development of an anticancer drug (14). Thus, much recent interest has developed in Phase I/II dose finding designs in which information on both toxicity and efficacy is used. These designs concern, for instance, dose finding in HIV in which both information on toxicity and efficacy are almost immediately available. Recent cancer studies are beginning to fall under this same heading in which toxicity can be evaluated quickly and, in addition, we can rely on biological markers or other measures of tumor response. Unlike the classic Phase I dose finding design in which the aim is to identify the MTD, the Phase I/II dose finding study aims to locate the most successful dose (MSD) (i.e., the dose that maximizes the product of the probability of seeing no toxicity together with the probability of seeing

MAXIMUM TOLERABLE DOSE

a therapeutic response). For a dose finding study in cancer, the MSD, among a group of available doses, is that dose at which the overall success rate is the highest (15,16). Close proposals have been also published, based on a bivariate modeling of toxicity and efficacy (17,18). 3.4 Patient-Specific Optimal Dose Because Phase I trials are small studies, the maximum tolerated dose of a new drug may not be established precisely for any individual. New paradigms for the clinical evaluation of new cancer therapies have been proposed. One proposal entails adjusting the search for the optimal dose on the basis of measurable patient characteristics to obtain a personalized treatment regimen (19). Accelerated titration (i.e., rapid intrapatient drug dose escalation) designs also seem to provide a substantial increase in the information obtained with regards to interpatient variability or cumulative toxicity (20). 3.5 Different Optimal Doses: The Example of Targeted Agents in Cancer Advances in molecular biology have led to a new generation of anticancer agents that inhibit aberrant and cancer-specific proliferative and antiapoptotic pathways. These agents may be cytostatic and may produce relatively minimal organ toxicity compared with standard cytotoxics. Thus, these new, targeted anticancer agents have posed challenges to the current Phase I paradigm of dose selection based on toxicity. Indeed, traditional trial designs and endpoints may not be adequate for developing contemporary targeted drugs. Notably, increasing the drug dose to toxicity seems unnecessary for drug effect, which makes the use of MTD as a surrogate of effective dose inappropriate in the Phase I setting. To accommodate these new drugs, the concept of an optimal biologic dose defined as a dose that reliably inhibits a drug target or achieves a target plasma concentration has been reported as desirable and appropriate for the Phase I study of mechanism-based, relatively nontoxic novel agents. This concept should rely on pharmacodynamic data in addition to toxicity (21).

3

In these early-phase dose finding clinical trials with monotone biologic endpoints, such as biological measurements, laboratory values of serum level, and gene expression, a specific objective is to identify the minimum dose that exhibits adequate drug activity and shifts the mean of the endpoint from a zero dose to the so-called minimum effective dose (22). Stepwise test procedures for dose finding have been well studied in the context of nonhuman studies in which the sampling plan is done in one stage (23). This development has fueled interest in alternatives to toxicity as a surrogate endpoint in Phase I clinical trials, although no consensus has been reached. Indeed, the optimal biologic dose rarely formed the basis of dose selection. This situation is exemplified in a recent overview of 60 Phase I trials that involved 31 single agents representative of the most common targets of interest in the oncology literature: 60% still used toxicity, whereas only 13% used pharmacokinetic data as endpoints for selection of the recommended Phase II dose (24). Finally, the selected dose should incorporate the fact that wide variations will be found in steady-state drug levels in patients (25). 4

CONCLUDING REMARKS

Phase I clinical trials are designed to identify an appropriate dose for experimentation in Phase II and III studies. Assuming that efficacy and toxicity are closely related, the highest dose level of a potential therapeutic agent at which the patients have experienced an acceptable level of dose-limiting toxicity defined the main objective of these trials. Efforts in Phase I studies over the past several years have focused on efficient estimation of maximum tolerated dose. Recently, the MTD has been controversial, in part because of difficulties in extrapolating findings to the whole treatment course and in applying population findings to individuals. Some extensions have been proposed to better address these issues. Otherwise, improved understanding of the biology of cancer has led to the identification of new molecular targets and the development of pharmacologic agents that hold

4

MAXIMUM TOLERABLE DOSE

promise for greater tumor selectivity than traditional cytotoxic agents. This development has avoided the use of the MTD as a surrogate endpoint for efficacy. Nevertheless, increased research efforts should be spent on the prospective evaluation and validation of novel biologic endpoints and innovative clinical designs so that promising targeted agents can be effectively developed to benefit the care of cancer patients. A need exists for improved definition of optimal biologic dose (26). Finally, allowing accelerated drug development through combined Phase I/II and Phase II/III clinical trial designs is a promising research area in the near future.

REFERENCES 1. S. Dent and E. Eisenhauer, Phase I trial design: Are new methodologies being put into practice? Ann. Oncol. 1996; 7(6): 561–566. 2. B. Storer, Phase I trials, in T. Redmond (ed.), Biostatistics in clinical trials, C.a.C. Chichester: John Wiley & Sons, 2001,:. pp. 337–342. 3. H. Skipper, F. J. Schabel, and W. Wilcox, Experimental evaluation of potential anticancer agents XIII: on criteria and kinetics associated with ‘‘curability’’ of experimental leukemia. Cancer Chemother. Rep. 1964; 35: 1. 4. M. Schneiderman, Mouse to man: statistical problems in bringing a drug to clinical trial. Proc. of the fifth Berkeley symposium on mathematical statistics and prob. in fifth Berkeley symposium on mathematical statistics and probability. Berkeley, CA: University of California Press, 1967. 5. W. F. Rosenberger and L. M. Haines, Competing designs for phase I clinical trials: a review. Stat. Med. 2002; 21(18): 2757–2770. 6. Y. Lin and W. Shih, Statistical properties of the traditional algorithm-based designs for phase I cancer clinical trials. Biostatistics. 2001; 2(2): 203–215. 7. J. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: a practical design for phase 1 clinical trials in cancer. Biometrics. 1990; 46(1): 33–48. 8. J. Whitehead and D. Williamson, Bayesian decision procedures based on logistic regression models for dose-finding studies. J. Biopharm. Stat. 1998; 8(3): 445–467.

9. P. Thall and S. Lee, Practical model-based dose-finding in phase I clinical trials: methods based on toxicity. Int. J. Gynecol. Cancer. 2003; 13(3): 251–261. 10. J. O’Quigley, and L. Shen, Continual reassessment method: a likelihood approach. Biometrics. 1996; 52(2): 673–684. 11. T. Braun et al., Simultaneously optimizing dose and schedule of a new cytotoxic agen. Clin. Trials. 2007; 4(2): 113–124. 12. T. Braun, Z. Yuan, and P. Thall, Determining a maximum-tolerated schedule of a cytotoxic agent. Biometrics. 2005; 61(2): 335–343. 13. J. Babb, A. Rogatko, and S. Zacks, Cancer phase I clinical trials: efficient dose escalation with overdose control. Stat. Med. 1998; 17(10): 1103–1120. 14. I. Sekine et al., Relationship between objective responses in phase I trials and potential efficacy of non-specific cytotoxic investigational new drugs. Ann. Oncol. 2002; 13(8): 1300–1306. 15. J. O’Quigley, M. Hughes, and T. Fenton, Dosefinding designs for HIV studies. Biometrics. 2001; 57(4): 1018–1029. 16. S. Zohar and J. O’Quigley, Optimal designs for estimating the most successful dose. Stat. Med. 2006; 25(24): 4311–4320. 17. P. F. Thall, E. H. Estey, and H.-G. Sung, A new statistical method for dose-finding based on efficacy and toxicity in early phase clinical trials. Investigational New Drugs. 1999; 17: 155–167. 18. Y. Zhou et al., Bayesian decision procedures for binary and continuous bivariate doseescalation studies. Pharm. Stat. 2006; 5(2): 125–133. 19. A. Rogatko et al., New paradigm in dosefinding trials: patient-specific dosing and beyond phase I. Clin. Cancer Res. 2005; 11(15): 5342–5346. 20. R. Simon et al., Accelerated titration designs for phase I clinical trials in oncology. J. Natl. Cancer Inst. 1997; 89(15): 1138–1147. 21. D. Kerr, Phase I clinical trials: adapting methodology to face new challenges. Ann. Oncol. 1994; 5(S4): 67–70. 22. S. Kummar et al., Drug development in oncology: classical cytotocixs and molecularly targeted agents. Br. J. Clin. Pharmacol. 2006; 62(1): 15–26. 23. M. Polley and Y. Cheung, Two-stage designs for dose-finding trials with a biologic endpoint using stepwise tests. Biometrics. 2007.

MAXIMUM TOLERABLE DOSE 24. W. Parulekarand E. Eisenhauer, Phase I trial design for solid tumor studies of targeted, noncytotoxic agents: theory and practice. J. Natl. Cancer Inst. 2004; 96(13): 990–997. 25. A. Adjei, The elusive optimal biologic dose in phase I clinical trials. J. Clin. Oncol. 2006; 24(25): 4054–4055. 26. B. Ma, C. Britten, and L. Siu, Clinical trial designs for targeted agents. Hematol. Oncol. Clin. North Am. 2002; 16(5): 1287–1305.

CROSS-REFERENCES Phase I trials Therapeutic Dose Range

5

METADATA

2.1 A Metadata Example The importance of metadata can be illustrated by the following example adapted from actual events. Consider the two datasets shown in Table 1. Each dataset contains the same variables but has different data values. With only cryptic variable names and no additional metadata, it would be almost impossible to determine what the data values represent without additional documentation. If we add context by stating that these datasets represent rocket burn instructions for placing a satellite in orbit and that the last three variables represent distance, speed, and force, then a knowledgeable rocket scientist may be able to infer what the data points could represent. In fact, that inference, based on inadequate metadata, led to a very costly mistake. The Mars Climate Orbiter was launched in December 1998 and was scheduled to go into orbit around Mars 9 months later. The $150 million satellite failed to reach orbit because the force calculations were ‘‘low by a factor of 4.45 (1 pound force = 4.45 Newtons), because the impulse bit data contained in the AMD file was delivered in lb-sec instead of the specified and expected units of Newton-sec’’ (2). In other words, a data file in English units (pounds) was input into a program that expected metric units (Newtons). Table 2 illustrates the same two datasets with additional metadata, including meaningful variable names, labels, and units. Note the units used in each dataset are now clearly identified in the column header metadata. Although it cannot be said with certainty that this metadata would have prevented the error, it would have increased the probability that the error would have been found by project personnel. The use of machine readable or ‘‘parsable’’ metadata would allow computer programs to be used to check for compatible units.

DAVID H. CHRISTIANSEN Christiansen Consulting Boise, Idaho

1

INTRODUCTION

Metadata (also ‘‘meta data’’ or ‘‘meta-data’’) are commonly defined as ‘‘data about data.’’ Clinical trial metadata are concerned with describing data originating from or related to clinical trials, including datasets and statistical analyses performed on the datasets. Clinical trial metadata may be in the form of a separate written document, may be linked electronically to a document or dataset, or may be integrated into a dataset as part of the definition of the data fields or variables. Metadata may be accessed by statisticians performing analyses on the data and by other scientists reviewing or using the data and results. Increasingly, metadata included in computerized datasets (machine-readable metadata) can also be used by statistical software and other computer applications to present or use the data in an appropriate manner, based on the metadata description. For machine-readable metadata to be used by a computer application, standards for the format and content must exist for both the metadata and the application that reads the metadata. Metadata are an important component of the documentation required for regulatory submissions and should provide a clear and concise description of the data collected and the analyses performed.

2

HISTORY/BACKGROUND

The term metadata was coined in 1969 by Jack E. Kelly. Although ‘‘Metadata’’ was copyrighted in 1986 by The Metadata Company (1), the generic ‘‘metadata’’ is commonly used in many disciplines, including computer science, database administration, geographic science, and clinical data management.

2.2 Geospatial Data Geographic science is an example of a discipline with well-defined metadata. In 1990, the Office of Management and Budget established the Federal Geographic Data

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

METADATA

Table 1. Example Datasets with No Metadata Dataset 1 Event

Time

D

S

F

Begin End

9/23/99 02:01:00 9/23/99 02:17:23

121,900,000

12,300 9,840

143.878

Dataset 2 Event

Time

D

S

F

Start Finish

19990923 05:01:00 19990923 05:17:23

196,200,000

5.5 4.4

640

Table 2. Mars Climate Orbiter Burn Instructions with Metadata Dataset 1 - Mars Orbit Insertion Burn in English Units Event

M/D/Y HH:MM:SS Pacific Daylight Time (Earth Receive Time, 10 min. 49 sec. Delay)

Begin End

9/23/99 02:01:00 9/23/99 02:17:23

Distance (miles) 121,900,000

Speed (miles/hr) 12,300 9,840

Force (Pounds) 143.878

Dataset 2 - Mars Orbit Insertion Burn in Metric Units Event Start Finish

YYYYMMDD EDT (Earth Receive Time, 10 min. 49 sec. Delay) 19990923 05:01:00 19990923 05:17:23

Committee as an interagency committee to promote the coordinated development, use, sharing, and dissemination of geospatial data on a national basis (3). In addition to detailed metadata and tools, the group also provides informational and educational materials that may be useful for motivating the development and use of metadata. For example, the U.S. Geologic Survey in a document titled ‘‘Metadata in Plain Language’’ (4) poses the following questions about geospatial data:

1. What does the dataset describe? 2. Who produced the dataset? 3. Why was the dataset created? 4. How was the dataset created? 5. How reliable are the data; what problems remain in the dataset? 6. How can someone get a copy of the dataset? 7. Who wrote the metadata?

Distance (km) 196,200,000

Speed (km/sec) 5.5 4.4

Force (Newtons) 640

Although these questions may have a different context and different emphasis in clinical trials, they can provide background information for our discussion of clinical trial metadata. 2.3 Research Data and Statistical Software Research data management refers to the design, collection, editing, processing, analyzing, and reporting of data that results from a research project such as a clinical trial. The characteristics and requirements for research data management activities are different from those of a commercial data management system. Because clinical trials are experimental by nature, the resulting datasets are unique, contain many different variables, and require a high degree of studyspecific documentation, including metadata. Conversely, commercial data systems, such as payroll or credit card billing, are characterized by very stable systems that perform well-defined functions on a relatively small

METADATA

3

number of standard variables. These differences make it difficult to use commercial data systems, such as databases and reporting systems, for research data. These unique requirements drove the development of statistical software specifically for research data over the last 30 years. The most commonly used systems all have some type of metadata at both the dataset and variable levels. These metadata allow the researchers conducting the studies to describe the structure and content of the datasets clearly. The ability to describe the data clearly and unambiguously is important in the analysis and reporting of the study results, both within an organization and for regulatory review and approval of new treatments.

3

2.4 Electronic Regulatory Submission

A dataset is a computer file structured in a predictable format that can be read and processed by a computer application or program, typically a statistical system such as SAS (SAS Institute, Cary, NC), S-Plus (Insightful Corporation, Seattle, WA), or SPSS (SPSS Corporation, Chicago, IL). In the discussion here, SAS terminology will be used, but similar concepts and attributes exist in all statistical languages and systems. Dataset-level metadata describes the content, structure, and use of the dataset by describing its physical and logical attributes. Some of these attributes relate to the dataset itself, whereas others are dependent on the context in which the dataset is used. The description and use of these attributes as they relate to regulatory submissions are shown below.

The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) (5) has developed a Common Technical Document (CTD) that ‘‘addresses the organization of the information to be presented in registration applications for new pharmaceuticals (including biotechnology-derived products) in a format that will be acceptable in all three ICH regions (Japan, Europe and United States)’’ (6). The CTD outline is, in fact, metadata for the regulatory submission. It provides information about the submission in a structured format that facilitates both the creation and review of the submission. Because the requirements for submission of clinical data vary from country to country, the CTD does not specifically address electronic submission of clinical data. The U.S. Food and Drug Administration (FDA), however, defined metadata requirements for clinical trial datasets submitted as part of its treatment approval processes in 1999 (7). The Clinical Data Interchange Standard Consortium (CDISC) enhanced the FDA metadata by adding metadata attributes at the dataset and variable levels (8). Since that time, the FDA and CDISC have collaborated on more standards, resulting in the FDA referencing the CDISC standard for clinical domain data as an acceptable format (9,10). The FDA and CDISC currently are working on a similar standard for analysis datasets (11,12).

DATASET METADATA

In the context of clinical trials and regulatory submissions, the most useful metadata refer to the datasets that contain the trial results. The datasets can be generally classified as those data collected during the execution of the trial (tabulation data or observed data) or data derived from the observed data for the purpose of display or analysis (analysis data). Dataset metadata can be classified at various levels as described below. The metadata attributes described are discussed by one or more of the following standards or organizations: CDISC Tabulation (10), CDISC Analysis (11), FDA (13), and SAS (14). 3.1 Dataset-Level Metadata

3.1.1 Dataset Name. – Unique dataset name for this file. FDA and CDISC have naming conventions for some clinical domain and analysis datasets. (FDA, CDISC, and SAS) 3.1.2 Description (Label). – A more detailed description of the content of the dataset. (FDA, CDISC, and SAS) 3.1.3 Location. – The relative physical file location in an electronic submission. (FDA and CDISC) 3.1.4 Structure. – The shape of the dataset or the level of detail represented by each

4

METADATA

row or record. Structure can range from very horizontal (one record per subject) to very vertical (one record per subject per visit per measurement). It is recommended that structure be defined as ‘‘one record per . . . ’’ rather than the ambiguous terms normalized or denormalized, horizontal or vertical, tall or short, skinny or fat, and so forth. (CDISC) 3.1.5 Purpose. – Definition of the type of dataset as tabulation or analysis. (CDISC) 3.1.6 Key Fields. – Variables used to uniquely identify and index records or observations in the dataset. (CDISC). 3.1.7 Merge Fields. – A subset of key fields that may be used to merge or join SAS datasets. (CDISC analysis) 3.1.8 Analysis Dataset Documentation. – Written documentation that includes descriptions of the source datasets, processing steps, and scientific decisions pertaining to creation of the dataset. Analysis dataset creation programs may also be included. (CDISC Analysis) 3.1.9 Rows. – The number of data records or observations in the dataset. (SAS) 3.1.10 Columns. – The number of variables or fields in the dataset. For most analysis datasets, each column represents the measurement value of some characteristic of the subject such as sex, age, and weight at a visit, for example. (SAS) 3.2 Variable-Level Metadata Each column or variable in a dataset has certain attributes that describe the content and use of the variable. These variable attributes are usually consistent for all observations within a dataset. That is, each column has the same attributes for all rows of the dataset. This rectangular data structure is required for statistical analysis by the vast majority of statistical analysis software and is the natural structure for CDISC Analysis datasets. Many CDISC Tabulation datasets, however, have a structure of one record per measurement. Because different measurements have

different attributes, metadata at the variable level is not adequate. See the section on value-level metadata below for a discussion of this issue. 3.2.1 Variable Name. – Unique name for the variable. A variable name should be consistent across datasets and studies within a regulatory submission. The data values of a variable should also be consistent in definition and units across all datasets within a submission. For example, if AGE is recorded in years in one study and months in another, then either AGE must be converted to, say, years in the second study or a different variable name, say AGEMON, must be used for age in months. (FDA, CDISC, and SAS) 3.2.2 Variable Label. – A more detailed description of the variable. This description may be used by some software and review tools. (FDA, CDISC, and SAS) 3.2.3 Type. – Description of how the data values for this variable are stored. Current conventions specify only a character string (CHAR) or a numeric value (NUM). These two types are consistent with SAS and other software, but additional types such as floating point, integer, binary, and date/time are used by some systems. (FDA, CDISC, and SAS) 3.2.4 Length. – The number of bytes allocated to store the data value. For CHAR variables, this number is the length of the character string. SAS and other software define this attribute as the number of bytes allocated to store the numeric value in some internally defined form, typically floating point hexadecimal. This length is not the number of digits used to display a number. See Format below. (SAS) 3.2.5 Format. – Description of how a variable value is displayed. Formats can be general, defining the width of a display field and the number of decimal digits displayed for a number. They can also provide code list values (1 = ‘‘Female’’; 2 = ‘‘Male’’) or display numerically coded values such as SAS dates in a human readable form. (FDA, CDISC, and SAS)

METADATA

CDISC Tabulation metadata defines this code list attribute as ‘‘Controlled Terms.’’ These terms are standard values for specific variables in certain clinical domains. 3.2.6 Role. – Description of how a variable may be used in a particular dataset. A variable may be assigned multiple roles. The role attribute is used by CDISC in two distinct ways. First, CDISC Tabulation datasets have a set of specific roles designed to describe a variable in the context of the proposed FDA JANUS database (15). • Identifier variables, usually keys,

identify the study, subject, domain, or sequence number of the observation. • Topic variables specify the focus of the observation, typically the name of a measurement or lab test in a one record per measurement structured dataset. • Timing variables specify some chronological aspect of a record such as visit number or start date. • Qualifier variables define a value with text, units, or data quality. Qualifiers are often used to store variables not otherwise allowed in tabulation datasets. Role attributes in CDISC Analysis datasets have a different emphasis than those in the tabulation models. Analysis roles focus on providing information useful to the statistical review rather than specification for the still-to-be-developed JANUS database. Because the primary goal of analysis roles is clear communication of the statistical analysis performed by the sponsor, the values of the role attribute are open-ended and can be extended as needed to meet this goal. The following roles have been identified by FDA and CDISC through the Analysis Dataset Model group (16). It should be noted that these definitions are still under development and may be subject to change. • Selection variables are frequently

used to subset, sort, or group data for reporting, displaying, or analysis. Common selection variables include treatment group, age, sex, and race. Specific study designs, drug indications,

5

and endpoints may have specific selection variables as well. For example, a hypertension trial may identify baseline blood pressure measurements as selection variables of interest. Flag variables identifying analysis populations such as ‘‘per protocol’’ or ‘‘intent to treat’’ are also commonly identified as selection variables. • Analysis variables relating to major study objectives or endpoints may be identified to assist reviewers. This identification may be especially useful in complex studies in which it may not be clear from the variable name which variable is the primary endpoint. • Support variables are identified as useful for background or reference. For example, the study center identifier may be used to group subjects by center, but the study name or investigator name would provide supporting information. • Statistical roles such as covariate, censor, endpoint, and so forth may also be useful for specific analyses and study designs. These roles may be added as needed to improve clear communication between the study authors and the reviewers.

3.2.7 Origin. – Describes the point of origin of a variable. CDISC Tabulation datasets allow CRF ‘‘derived’’ and ‘‘sponsor defined’’ values of origin. This attribute usually refers to the first occurrence of a value in a clinical study and does not change if a variable is added to another file such as an analysis dataset. 3.2.8 Source. – In a CDISC Analysis dataset, source provides information about how a variable was created and defines its immediate predecessor. For example, an analysis dataset may be created by merging two or more source datasets. The variable SEX from the demographics domain dataset DM would have a source of ‘‘DM.SEX’’. This convention of specifying the immediate predecessor defines an audit trail back to the original value, no matter how many generations of datasets are created. Derived variables in

6

METADATA

analysis datasets may include a code fragment to define the variable or may hyperlink to more extensive documentation or programs.

requires additional metadata for each value of the vital signs test as shown in Table 4. The value-level metadata defines the attributes needed to transpose the tabulation dataset into a CDISC Analysis dataset, typically with a structure of one record per subject. This dataset, shown in Table 5, contains the data points from the tabulation dataset, but in a different structure. Values of SEX and the derived variable Body Mass Index (BMI) have been added to illustrate that analysis dataset can include data from several sources. Note that now each measurement is represented by a variable, and each variable can have its own metadata attributes. This dataset structure is preferred by most statisticians and can be used directly by most statistical packages. ?>

3.3 Value-Level Metadata Many CDISC Tabulation (SDTM) datasets have a structure of one record per subject per time point per measurement. This structure means that different measurements having different attributes such as name, label, type, and format vary from record to record. To provide clear communication of the content and use of such a dataset, each measurement name (test code) must have its own metadata. For example, the CDISC Tabulation model defines a Vital Signs Dataset with measurements for height, weight, and frame size. Table 3 illustrates selected portions of such a dataset and its metadata. Some CDISC attributes have been changed or omitted to simplify the example. Note that the data values of HEIGHT, WEIGHT, and FRMSIZE are stored in the same column, and no useful metadata identifies format, variable type, role, or origin. This dataset is difficult to understand and cannot be used with statistical software without additional programming. This file structure

3.4 Item-Level Metadata Item-level metadata refers to the attributes of an individual data value or cell in a rectangular data matrix. That is, it refers to a value for the crossing of a specific variable (column) and observation (row). Item-level metadata are typically used to describe the quality of that particular measurement, for example, a partial date in which the day portion is missing and is imputed or a lipid measurement

Table 3. Vital Signs Tabulation Dataset Vital Signs Findings–1 Record/subject/measurement Name

USUBJID

VSTESTCD

VSTEST

Label Type Format Origin Role

Subject ID Char

VS Short Name Char

VS Name Char

Sponsor Identifier

CRF Topic

Sponsor Qualifier

00001 00001 00001

HEIGHT WEIGHT FRMSIZE

Height in cm Weight in kg Frame Size

VSSTRESN

VSSTRESC

Numeric Result Numeric ? CRF Qualifier

Character Result Char ? CRF Qualifier

165 56.1 Small

Table 4. Vital Signs Tabulation Value-Level Metadata Vital Signs Value-Level Metadata for VSTESTCD Value (VSTESTCD)

Label (VSTEST)

Type

Format or Controlled Terms

Origin

HEIGHT WEIGHT FRMSIZE

Height in cm Weight in kg Frame size

Num Num Char

3.0 5.1 Small, Medium, Large

VSSTRESN VSSTRESN VSSTRESC

METADATA

7

Table 5. Vital Signs Analysis Dataset Vital Signs Analysis - 1 Record/Subject Name

USUBJID

SEX

Label Type Format

Subject ID Char

Sex Char Female, Male

Origin Source

Role

Sponsor CRF VS.USUBJID DM.SEX

Identifier 00001

Qualifier, Selection Female

HEIGHT

WEIGHT

BMI

FRMSIZE

Num 3.0

Num 5.1

Num 5.2

CRF VS.VSSTRESN (where VS.VSTESTCD = ‘‘HEIGHT’’) Analysis

CFR VS.VSSTRESN (where VS.VSTESTCD = ‘‘WEIGHT’’) Analysis

Derived WEIGHT/ (0.01* HEIGHT)∗∗ 2

Analysis

Char Small, Medium, Large CRF VS.VSSTRESC (where VS.VSTESTCD = ‘‘FRMSIZE’’) Analysis

165

56.1

20.61

Small

in which a frozen sample was accidentally allowed to thaw. In each of these cases, a value exists, but it may be considered to be of lesser quality than a regular measurement. Because of the complex and expensive nature of clinical trials, it is not always practical to discard such measurements, nor is it scientifically valid to treat them as complete. The identification of these data items is especially important for accurate statistical analysis in regulatory submissions. In a discussion of quality assurance for submission of analysis datasets, one FDA statistical director stated that it was desirable to provide a ‘‘clear description of what was done to each data element: edits, imputations, partial missing . . . .’’ (17). Historically, the concept of status attributes for data items was used in the 1970s by the Lipids Research Clinic Program to identify missing values and record the editing status of each variable (18). Clinical research data management systems may also have similar features, but this information is not easily exported to statistical reporting and analysis programs. Currently, the most common method of identifying itemlevel quality involves the addition of a separate variable that contains the status value. These ‘‘status flags’’ are cumbersome to maintain and can increase the size of datasets considerably. The development of statistical datasets using the eXtensible Markup Language (XML) (19) have the potential to provide the more flexible structure required

for integrating item-level metadata into clinical datasets, but the tools required do not exist at this time. An audit file system for regulatory statistical reviewers was proposed by an FDA statistical director as a ‘‘file describing the changes or edits made during the data management or cleaning of the data,’’ that is, provide a link or audit trial between the original and submitted data values by providing metadata related to the edits, including the following attributes (20). • Patient, observation, visit, variable, and

other identifiers • Original and submitted values • Qualifiers describing the change such as

who, when, and why • Edit codes describing the action taken

such as empty (not recorded), completed, replaced, confirmed, suspicious but not collectable. It is interesting to note that many of the edit codes proposed by the FDA director are similar to the 1970s system described above. It should also be noted that current data management systems do have audit trails, but they typically cannot extract and submit the data in a form that is useable by a regulatory reviewer. 4

ANALYSIS RESULTS METADATA

Analysis results metadata define the attributes of a statistical analysis performed

8

METADATA

on clinical trial data. Analyses may be tables, listings, or figures included in a study report or regulatory submission. Analyses may also be statistical statements in a report, for example, ‘‘The sample size required to show a 20% improvement in the primary endpoint is 200 subjects per treatment arm’’ or ‘‘The active treatment demonstrated a 23% reduction in mortality (p = 0.023) as compared to placebo.’’ Analysis results metadata are designed to provide the reader or reviewer with sufficient information to evaluate the analysis performed. Inclusion of such metadata in FDA regulatory submissions was proposed in 2004 (21) and is included in the CDISC Analysis Data Model V 2.0 (22). By providing this information in a standard format in a predictable location, reviewers can link from a statistical result to metadata that describes the analysis, the reason for performing the analysis, and the datasets and programs used to generate the analysis. Note that analysis results metadata are not part of an analysis dataset but that one attribute of analysis results metadata describes the analysis datasets used in the analysis. • Analysis Name. – A unique identifier

•

•

•

•

for this analysis. Tables, figures, and listing may incorporate the name and number (Fig. 4 or Table 2.3). Conventions for this name may be sponsorspecific to conform to Standard Operating Procedures (SOPs). Description. – Additional text describing the analysis. This field could be used to search for a particular analysis or result. Reason. – Planned analyses should be linked to the Statistical Analysis Plan (SAP). Other reasons would include data driven, exploratory, requested by FDA, and so forth. Dataset(s). – Names of datasets used in the analysis. If datasets are part of a submission, then a link to the dataset location should be provided. Documentation. – A description of the statistical methodology, software used for computation, unexpected results, or any other information to provide the

reviewer with a clear description of the analysis performed. Links to the SAP, external references, or analysis programs may also be included.

5 REGULATORY SUBMISSION METADATA 5.1 ICH Electronic Common Technical Document In addition to the Common Technical document described earlier, the ICH has also developed a specification for an electronic Common Technical Document (eCTD), thus defining machine-readable metadata for regulatory submissions. This eCTD is defined to serve as an interface for industry-to-agency transfer of regulatory information while at the same time taking into consideration the facilitation of the creation, review, lifecycle management, and archival of the electronic submission (23). The eCTD uses XML (24) to define the overall structure of the document. The purpose of this XML backbone is two-fold: ‘‘(1) to manage meta-data for the entire submission and each document within the submission and (2) to constitute a comprehensive table of contents and provide corresponding navigation aids’’ (25). Metadata on submission level include information about submitting and receiving organization, manufacturer, publisher, ID and kind of the submission, and related data items. Examples for metadata on document level are versioning information, language, descriptive information such as document names, and checksums used to ensure accuracy. 5.2 FDA Guidance on eCTD Submissions The FDA has developed a guidance for electronic submission based on the ICH eCTD backbone. As discussed earlier, the ICH does not define detailed specification for submission of clinical data within the eCTD, but it does provide a ‘‘place-holder’’ for such data in the guideline E3 ‘‘Structure and Content of Clinical Study Reports’’ as Appendix 16.4 with the archaic term ‘‘INDIVIDUAL PATIENT DATA LISTINGS (US ARCHIVAL LISTINGS)’’ (26). The FDA eCTD specifies that submitted datasets should be organized as follows (27): Individual Patient Data Listings (CRTs)

METADATA • Data tabulations

– Data tabulations datasets – Data definitions – Annotated case report form • Data listing

– Data listing datasets – Data definitions – Annotated case report form

9

developments may include protocol authoring tools, Statistical Analysis Plan templates, eCRF and CRF automated database design, automated analysis and reporting, and submission assembly. This tool development by government, drug developers, and software providers will contribute to drug development and approval by enhancing the clear communication of the content and structure of clinical trial data and documents.

• Analysis datasets

– – – –

Analysis datasets Analysis programs Data definitions Annotated case report form

• Subject profiles • IND safety reports

The FDA Study Data Specification document (28) defines tabulation and analysis datasets and refers to the CDISC Data Definition Specification (Define.XML) for machinereadable dataset metadata in XML (29). This machine-readable metadata and the ICH eCDT are key elements in providing clear communication of the content and structure of clinical datasets and regulatory submissions. These metadata standards allow the regulatory agencies, software developers, and drug developers to create and use standard tools for creating, displaying, and reviewing electronic submissions and clinical datasets. The FDA has developed a viewer to use the ICH eCTD backbone to catalog and view the components of a submission, thus providing FDA reviewers with a powerful tool to view, manage, and review submissions. Software developers have used the Define.XML standard metadata to develop tools for compiling and viewing patient profiles and viewing tabulation datasets. SAS Institute has developed software to generate XML-based datasets with Define.XML metadata (30) and viewing tools for review and analysis. These first steps demonstrate the power of having clearly defined metadata for clinical research. The adoption and additional specification of these metadata standards will provide the basis for the development of a new generation of tools for review and analysis. Future

REFERENCES 1. U.S. Trademark Registration No. 1,409,260. 2. Mars Climate Orbiter Mishap Investigation Board Phase I Report, 1999:13. Available:ftp. hq.nasa.gov/pub/pao/reports/1999/MCO report. pdf. 3. The Federal Geographic Data Committee. Available:www.fgdc.gov/. 4. U.S. Geologic Survey, Metadata in Plain Language. Available:geology.usgs.gov/tools/ metadata/tools/doc/ctc/. 5. International Conference and Harmonisation. Available: www.ich.org. 6. International Conference and Harmonisation, Organization of the Common Technical Document for the Registration of Pharmaceuticals for Human Uses M4, 2004. Available: www.ich.org/LOB/media/MEDIA554.pdf. 7. Providing Regulatory Submissions in Electronic Format — NDAs, FDA Guidance, 1999. 8. D. H. Christiansen and W. Kubick, CDISC Submission Metadata Model, 2001. Available: www.cdisc.org/standards/SubmissionMetadata ModelV2.pdf. 9. FDA Study Data Specification, 2006. Available: www.fda.gov/cder/regulatory/ersr/Studydatav1.3.pdf. 10. CDISC Study Data Tabulation Model Version 1.1, 2005. Available: www.cdisc.org/models/sds/ v3.1/index.html. 11. CDISC Analysis Data Model: Version 2.0, 2006. Available: www.cdisc.org/pdf/ADaMdocument v2.0 2 Final 2006-08-24.pdf. 12. FDA Future Guidance List, p. 2. Available: www.fda.gov/cder/guidance/CY06.pdf. 13. Providing Regulatory Submissions in Electronic Format — Human Pharmaceutical Product Applications and Related Submissions Using the eCTD Specifications, FDA Guidance, 2006. Available: www.fda.gov/cder/ guidance/7087rev.pdf.

10

METADATA

14. SAS 9.1.3 Language Reference: Concepts. Cary, NC: SAS Institute Inc., 2005, pp. 475– 476. 15. JANUS Project Description, NCI Clinical Research Information Exchange, 2006. Available: crix.nci.nih.gov/projects/janus/. 16. D. H. Christiansen and S. E. Wilson, Submission of analysis datasets and documentation: scientific and regulatory perspectives. PharmaSUG 2004 Conference Proc., San Diego, CA, 2004: Paper FC04, p. 3. 17. S. E. Wilson, Submission of analysis datasets and documentation: regulatory perspectives. PharmaSUG 2004 Conference Proc., San Diego, CA, 2004. 18. W. C. Smith, Correction of data errors in a large collaborative health study. Joint Statistical Meetings presentation, Atlanta, GA, 1975. 19. SAS/ 9.1.3 XML LIBNAME Engine: User’s Guide. Cary, NC: SAS Institute Inc., 2004. 20. S. E. Wilson, Clinical data quality: a regulator’s perspective. DIA 38th Annual Meeting presentation, Chicago, IL, 2002: 20–21. Available: www.fda.gov/cder/present/DIA62002/default. htm. 21. D. H. Christiansen and S. E. Wilson, submission of analysis datasets and documentation: scientific and regulatory perspectives. PharmaSUG 2004 Conference Proc., San Diego, CA, 2004: Paper FC04, p. 5. 22. CDISC Analysis Data Model: Version 2.0, 2006, p. 22. Available: www.cdisc.org/pdf/ ADaMdocument v2.0 2 Final 2006-08-24. pdf. 23. ICH M2 EWG Electronic Common Technical Document Specification V 3.2, 2004, p. 1. Available: http://estri.ich.org/eCTD/eCTD Specification v3 2.pdf. 24. World Wide Web Consortium (W3C) Extensible Markup Language (XML). Available: http://www.w3.org/XML/.

25. ICH M2 EWG Electronic Common Technical Document Specification V 3.2, 2004: Appendix 1, p. 1-1. Available: http://estri.ich. org/eCTD/eCTD Specification v3 2.pdf. 26. ICH Structure and Content of Clinical Study Reports E3, p. 29. Available: http:// www.ich.org/LOB/media/MEDIA479.pdf. 27. Providing Regulatory Submissions in Electronic Format — Human Pharmaceutical Product Applications and Related Submissions Using the eCTD Specifications, FDA Guidance, 2006. Available: http://www.fda.gov/cder/ guidance/7087rev.pdf. 28. FDA Study Specifications. Available: http:// www.fda.gov/cder/regulatory/ersr/Studydatav1.3.pdf. 29. CDISC Case Report Tabulation Data Definition Specification (define.xml). Available: http://www.cdisc.org/models/def/v1.0/CRT DDSpecification1 0 0.pdf. 30. SAS 9.1.3 XML LIBNAME Engine: User’s Guide. Cary NC: SAS Institute Inc., 2004, pp. 27, 39–43.

CROSS-REFERENCES Biostatistics Statistical Analysis Plan Electronic Submission of NDA International Conference on Harmonisation (ICH) Good Programming Practice

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

of GRTs. These challenges generally arise from factors pertaining to the group-directed intervention and the nature of the group. The group-targeted interventions in GRTs are commonly prevention interventions (disease prevention or risk-factor prevention), and they are commonly targeted at relatively healthy populations. These two features have big consequences for GRTs. First, the rate of endpoint (disease, or risk factor) occurrence is relatively low. As a result, GRTs generally require a moderate number of groups (10–50) and typically have large numbers of study participants (1,000–10,000 +). GRTs are also typically long (5–20 + years), which makes follow-up of study participants and maintaining research relationships with collaborating organizations especially challenging. Also, the large size and long duration typical of most GRTs means an expensive trial and thus a low tolerance for degradation of scientific rigor. In addition, the healthy nature of the study population in most GRTs means that (1) research participation may have low salience or importance to study participants, and (2) the intervention must be designed to be easy to do. As a result of (1), there are special challenges for achieving high response rates for endpoint data collection, and for intervention compliance among those in the experimental organizations. As a result of (2), intervention contamination in the control organizations can easily happen, and so avoiding it is a challenge. It is critical for scientific rigor that these challenges be overcome. To do so, the challenges must first be recognized. Then, appropriate methods for addressing them must be planned in advance. Finally, these methods must be meticulously implemented throughout the trial. For each of the four requirements for trial execution—maintaining the research collaboration, maintaining the randomized assignment, achieving high location rate, and achieving high response rate—this article summarizes the challenges and same principles for addressing them and provides

ARTHUR V. PETERSON Jr. Member, Public Health Sciences, Fred Hutchinson Cancer Research Center, and Professor of Biostatistics, University of Washington.

1

INTRODUCTION

Just as with trials randomized by the individual, the randomized assignment in group-randomized trials (GRTs) provides two invaluable advantages that enable unambiguous conclusions to be reached from the trial: avoidance of selection and accidental biases (and consequent biases in the reported effect sizes), and provision of a basis for the statistical inferences that draw conclusions from the study. Moreover, the basis for statistical inference that randomization provides does not rely on any distributional or other assumptions. To realize the invaluable advantages that randomization enables, careful attention to trial conduct is needed from beginning to end of the GRT. For example, both sample size determination in the design phase at the start of the GRT and choice of statistical methods in the analysis phase at the end of the GRT must account for the intraclass correlation and the imprecision of its estimate. Likewise, achieving the scientific rigor enabled by the randomization in GRTs requires attention during the period of trial execution. There are four key requirements: (1) maintaining research collaboration with each recruited group for the duration of the trial, (2) maintaining the randomization assignment of intervention or control condition for each group throughout the trial, (3) achieving a high location rate for study participants at endpoint, and (4) achieving a high response rate at endpoint data collection. Special challenges to meeting these requirements are inherent to the very nature

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

examples of methods for applying these principles in the conduct of GRTs.

2 MAINTAINING THE RESEARCH COLLABORATIONS FOR THE DURATION OF THE TRIAL Maintaining the research collaborations with participating organizations is essential for maintaining scientific rigor. Because an organization’s dropping out of the study is not determined at random, but it is instead determined by selection due to circumstances, it introduces selection bias into the results and weakens the (randomization-generated) basis for statistical inference. Moreover, the consequences of one organization dropping out from a GRT are far more severe than one study participant dropping out from an individual-randomized trial. Because the number of randomized units in a GRT is small or moderate (10–50 organizations), compared with the typical 100–400 individuals in an individual-randomized trial, loss of even one organization can severely impact the integrity of the trial. Thus, strong efforts are needed throughout the trial to avoid organizational dropout and its associated degradation of scientific rigor. Presented in Table 1 are challenges inherent in GRTs for maintaining research collaboration with participating organizations and principles for addressing the challenges. Of special note is the need to minimize, by

design, the research burden on the organization. Making participation easy for the organization directly addresses challenge #1—participation in research is not a goal of collaborating organizations. It also makes it more possible to overcome challenges #2 and #4—changes in personnel and organizations’ priorities over time. Examples of methods for maintaining research collaboration include the following: (1) Learn about the organizations in advance; identify key stakeholders; and demonstrate knowledge and interest in stakeholders and in their organization, needs, priorities, and challenges. (2) Conduct well-organized, efficient, and highly motivating recruitment meeting(s) with key stakeholders in each collaborating organization. Not only does this meeting serve to motivate potential collaborators, but also it provides them with first-hand evidence that the research group is capable and competent. (3) Begin maintaining the collaboration during recruitment, for example, by emphasizing the research question and randomized design so that any decision to participate is an informed and lasting one, and by working as a team to prepare for collaboration in the event that the organization decides to join the study. (4) Make excellent customer service an integral part of all trial procedures. For example, respond quickly to organizations’ inquiries and requests, and do what you say you will do. (5) Be visible: Communicate regularly with collaborating organizations to update

Table 1. Maintenance of Research Collaborations with Participating Organizations Requirement: Maintain the research collaborations (support for all research activities) with all participating organizations for the duration of the trial Challenges 1. Participation in research is not a goal of collaborating organizations, which are busy with their own priorities and mandates. 2. Organizations have multiple stakeholders to be kept informed and satisfied. 3. Organizations’ priorities or circumstances change over time. 4. Turnover of personnel within the collaborating organizations over time.

Principles 1. Design the trial and activities to minimize the research burden on the organization. 2. Sell the research. 3. Emphasize the essential role of the organizations as research partners. 4. Meet their needs to be informed, motivated, valued, and to have expectations met. 5. Make maintaining the research collaborations the top priority for the research group.

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

progress, to alert them to upcoming activities, and to express appreciation for their support. Offer to present in-person to key stakeholders on the trial and its progress. (6) Incorporate the principles of maintaining collaborative relationships into all project activities and protocols. (7) Identify and develop simple, low-cost procedures, such as periodic progress-report letters and Secretary Day cards, that serve the sole purpose of building and maintaining collaborative relationships. (8) Coordinate and document all contacts with collaborating organizations. 3 MAINTAINING THE RANDOMIZED ASSIGNMENT The essential scientific advantages that randomization enables—avoidance of bias and a basis for statistical inference—depend on maintaining each organization’s randomized assignment to either experimental or control. Thus, strong efforts are warranted throughout the trial to minimize at all times the risk of straying from the randomized assignment. Conformance to the randomized assignment could be violated by (1) an organization’s nonacceptance, or poor understanding, of the concept of randomized assignment at the start of the collaboration, during recruitment; (2) an organization’s nonacceptance of the actual assignment when randomization is performed and the assignment is communicated; (3) intervention contamination in organizations assigned to the control condition; and (4) implementation noncompliance in organizations assigned to the experimental condition. Accordingly, maintaining the randomized assignment requires attention to each aspect. Presented in Table 2 are, for each of these four aspects, the challenges inherent in GRTs for maintaining the randomization and the principles for addressing the challenges. Of special note is the importance of learning about changes in key personnel, and motivating the new stakeholders about the crucial role of randomization. A change in key personnel could conceivably cause challenges to occur in all four aspects of maintaining the randomized assignment. GRT managers must be proactive to overcome such challenges.

3

Examples of methods for maintaining randomized assignment for the duration of the trial include (1) during recruitment, sell the randomized assignment as the crux of the experimental design and the key to the research’s ability to attain scientific rigor and unambiguous conclusions at the trial’s end. (For example, ‘‘Because of the randomized assignment, any difference in outcome between the control and experimental groups can be attributed to the intervention, and not to other factors that differ among participating organizations.’’) (2) Have the principal investigator be the one to notify the organization, so that the importance of the randomization is emphasized. (3) Emphasize at every verbal and written contact the importance of the research and the critical role of the randomized assignment in obtaining unambiguous study conclusions. (4) Control access to intervention materials. (5) Make provision whenever practical for control organizations to receive intervention materials after the trial is finished. (6) Avoid choosing as the experimental unit entities that communicate easily with each other, for example, classrooms within a school. (7) In all communications with key personnel in control organizations, reinforce the essential role of control organizations in the research and express gratitude for their support of this often less-than-glamorous role. (8) Try to design the intervention to also meet the existing goals of the organization. (9) Incorporate behavior change theory and principles into the provider training to motivate providers to want to implement the intervention. (10) Meet with implementers periodically to provide visible support and appreciation of their key role and to obtain their critical feedback on the quality of the intervention and its implementation. (11) Publicly acknowledge and thank implementers for their contributions. (12) Use both provider self-reports and project staff observations to monitor provider implementation. 4

LOCATING STUDY PARTICIPANTS

Because losses to follow-up (failure to obtain endpoint data from individual study participants in participating organizations) are

4

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

Table 2. Maintenance of the Randomized Assignment Requirement: Maintain the randomized assignment of experimental/control condition for each participating organization for the duration of the study Challenges

Principles

Aspect #1: Acceptance (at recruitment) of the concept of randomized assignment 1. Randomization is not the usual way organizations make decisions. 2. Each organization tends to have its own preference/hope for the experimental condition that would be best for them. 3. Benefits of the control condition may not be clearly evident to participating organizations. 4. Changes in organizations and key personnel may threaten the understanding and/or support of the randomized assignment.

1. Motivate randomization as the key element for the study’s scientific rigor. 2. Emphasize that it is unknown whether the intervention is effective or not; indeed, the whole purpose of the trial is to find that out. 3. Emphasize importance of both intervention and control organizations to the success of the research. 4. Learn about changes in key personnel and inform them promptly about the trial.

Aspect #2: Acceptance (at randomization) of the communication of the actual randomized assignment 1. Organization stakeholders of the randomization may anticipate the randomization result. 2. Changing personnel and circumstances could heighten preference.

5. Make the performance of the randomization a big deal (it is), and witnessed. 6. Inform organization immediately. 7. Apply principles 1-4 above, again.

Aspect #3: Avoidance of contamination in control organizations 1. Participation in study, even as control, may enhance interest in the scientific question, and hence the intervention. 2. Prevention interventions are necessarily easy to do, and so may be easy for control to adopt.

8. Minimize the opportunity for purloining the intervention. 9. Reinforce the essential role of control organizations in the research.

Aspect #4: Implementation compliance in intervention organizations 1. Implementers within the organizations may be reluctant to add to their existing activities. 2. Intervention activities may mean some extra work, often unrewarded by organizations.

selected (not random), each loss degrades the scientific integrity of the study: It introduces bias into the results and degrades the basis for inference that the randomization enabled. Thus, to maximize the scientific

10. Make the intervention easy to do, enjoyable, and useful to them. 11. Motivate and train providers to implement the intervention. 12. Maintain strong, collaborative relationships with implementers as research partners.

rigor of the study, informed and aggressive measures must be taken to minimize loss to follow-up. Because successfully following up a study participant requires both locating the study

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

participant and, once located, obtaining the endpoint data (compliant response to the endpoint survey), both need to be addressed and successfully accomplished. The first requirement is covered in this section; the second in the next section. Successfully locating a study participant is defined as obtaining an up-to-date address and/or telephone number (where the study participant can be contacted for collecting the trial’s endpoint data). Presented in Table 3 are the challenges in locating study participants and principles for addressing the challenges. Examples of methods for applying these principles (see Refs. 1–3) include (1) Collect at the start of the trial the names, addresses, and telephone numbers of study participants, their parents, and close friends or relatives who would know their whereabouts if they moved away. (2) First activities for updating addresses: periodic ‘‘softtracking’’ letters sent to study participants (which ask for no reply from the participant) that use the U.S. Postal Service’s ‘‘Address Service Requested’’ endorsement. (3) Use the National Change of Address Service to obtain

5

new addresses of study participants and parents, close friends, and relatives. (4) Contact parents, friends, and relatives, as needed. (5) Keep detailed records of all directory information changes and all contact attempts: date and time, whom contacted, relationship to study participant, and response/information obtained. (6) Use Directory Assistance (from more than one telephone company). (7) Use online people-search engines on the Internet. (8) Use Telephone Append & Verify, a service arranged by the U.S. Postal Service in cooperation with telephone companies to provide phone numbers that go with specific addresses. (9) If all else fails, use publicly or privately available databases. Deserving special emphasis is method (4)—Contact parents, friends, and relatives. Although timeconsuming, it is the most likely to yield accurate information. 5 HIGH RESPONSE RATE TO ENDPOINT SURVEY Together with achieving high location rates, achieving high response rates to the endpoint survey is necessary for achieving successful follow-up of study participants. In contrast

Table 3. Locating Study Participants at Endpoint Requirement: Locate at endpoint study participants enrolled at the start Challenges 1. Long-term nature of GRTs (prevention studies, rare endpoints) requires follow-up over many years. 2. Our society is highly mobile. 3. Participation is not a high priority in the lives of study participants. 4. Organizations may not keep track of new addresses of those who leave. 5. The USPS will forward mail to a new address for only 6 months following a move, and will provide a new address to a sender for only 12 months. 6. Telephone companies provide new telephone numbers for only a limited period of time (6 months-1 year). 7. Our society is skeptical about contact from unknown callers.

Principles 1. Collect locator information at the outset of the trial. 2. Update locator information periodically throughout the trial. 3. Use a friendly (yet professional) approach with study participants at all times in all circumstances. 4. Keep record for each study participant of all tracking attempts. 5. Don’t limit the number of tracking attempts. 6. Use multiple modes (mail, telephone, email, cell phones).

6

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

to the other requirements for the conduct of GRTs, the challenges for achieving high response rates are common to both grouprandomized trials and individual-randomized trials. The challenges, and principles and methods for addressing them, have been well established over many years [see Don Dillman’s useful text (4)]. Shown in Table 4 are the challenges inherent in GRTs and other trials for achieving high response rates for the endpoint survey, and principles for addressing these challenges. Some examples of methods for achieving these follow. To build trust, when contacting participants, always (1) provide name of research organization, (2) remind the participant about his/her past participation, and (3) assure participants that their participation in the survey is voluntary and confidential. To make surveys easy to do, (4) keep the questionnaire short <20 minutes to complete), design it to have an inviting appearance, and include a memorable cover design. (5) Keep the survey at a low reading level, with easy skip patterns, and with a salient first question to interest participants. (6) Make multiple contacts to initial nonresponders, using a variety of communication modes—mail, telephone, and others. To motivate participants, (7) emphasize the importance of the research and their personal participation; (8) use prepaid monetary incentives and, as needed, promised monetary incentives; and (9) develop a highly motivating cover letter that is short, to-thepoint, and signed. To be opportunistic, (10)

administer the survey whenever contact with a study participant is made (i.e., perform tracking and survey administration as one step whenever opportunity arises).

6 REPORTS OF SCIENTIFIC RIGOR Various trials have illustrated that the inherent challenges of GRTs can be addressed and overcome to achieve the critical requirements for rigorous trial conduct, even when working with multiple organizations. For example, Murray et al. (5) successfully followed up (located and surveyed) 93% of 7124 adolescent study participants 5–6 years after baseline. The Community Intervention Trial (6) successfully maintained collaboration with 22 sizeable (50,000–250,000) communities for 7 years. The Hutchinson Smoking Prevention Project (HSPP) GRT (7, 8), with 40 collaborating school districts and 8388 study participants, reported (1) maintenance of collaboration by 100% of the 40 school districts for the 12-year duration of the trial, (2) maintenance of the randomized assignment: (a) 100% acceptance of both the concept of randomization (at recruitment) and the actual randomized assignment of intervention vs. control (b) 100% acceptance of the actual random assignment (c) 100% avoidance of contamination in the control group (d) 99% + implementation, with 86% implementation fidelity in the experimental group, (3) 96.6% of study participants located 12 years after enrollment in the study, (4) 97.6%

Table 4. High Response Rate at Endpoint Survey Requirement: Obtain responses from located study participants Challenges

Principles

1. Research participation is not a high priority in the lives of study participants. 2. Most people in our society live busy lives.

1. Build trust. 2. Make survey completion easy, and interesting to do.

3. Our society is skeptical about contact from unknown callers.

3. Motivate participants to want to comply with requests for information. 4. Be pleasantly persistent. 5. Be opportunistic: administer survey concurrently with successfully locating study participant (‘‘Bird-in-the Hand’’ principle).

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

data collection response rate among those located Such reports provide encouraging evidence that proper attention to the challenges and principles for conducting GRTs, together with appropriate methods for conducting the trial, can achieve high levels of scientific rigor. 7

DISCUSSION AND CONCLUSIONS

GRTs are essential for investigating the effectiveness of group-delivered interventions that are so important for public health promotion and disease prevention. The important role and significant monetary costs of GRTs require special attention during both design and execution of the trial to achieve the scientific rigor needed to obtain unambiguous conclusions. Randomization enables scientific rigor, but it cannot guarantee it without the support of careful implementation of trial methods that overcome challenges inherent in GRTs. Essential for realizing the rigor that the design of the GRT enables are the methods used to conduct the trial. The goal of achieving scientific rigor during the conduct of GRTs seems achievable through a combination of (1) attention to the special challenges inherent in GRTs; (2) recognition of the requirements and principles for rigorous trial conduct; (3) advance planning to adopt proven methods, or as needed to develop new ones; and (4) commitment to rigor and meticulous execution of the methods. The importance of informed and persistent follow-up cannot be overemphasized. In this connection, it should be noted that various ‘‘attrition analyses’’— for example, demonstrating that the fraction of lost-to-follow-up is the same in the experimental and control conditions, or that the baseline characteristics of those lost to follow-up do not differ between experimental and controls— cannot mitigate the degradation of scientific rigor that loss-to-follow-up causes. Such attrition analyses cannot overcome the fundamental problem that the endpoints are unknown for those lost to follow-up, and they cannot be inferred from the data on those successfully followed, because of the selected nature of those lost to follow-up. Also, various imputation methods that attempt to minimize the

7

bias that results from loss-to-follow-up by imputing values for the unknown outcomes rely on unverifiable assumptions and should not be viewed as a substitute for aggressive follow-up of study participants. Some main themes are evident in the collection of principles and methods for conducting GRTs: (1) It is important to identify the challenges at the start of a GRT, and to develop strategies and methods for addressing them. (2) Motivating individuals is important for more than the intervention; it is useful wherever cooperation from people is needed: in recruiting organizations and maintaining their collaboration, in designing the (intervention) provider training, and in designing the research activities (tracking, data collection). (3) Dedicated customerservice in all phases of the trial has great benefits for enhancing the scientific rigor of the research. Both the cost and importance of GRTs are high. Thus, a high degree of attention to scientific rigor is warranted, both to realize the sizeable investments (in money, time, and scientific careers) that GRTs entail and, more important, to achieve unambiguous scientific conclusions needed to advance disease prevention and health promotion for our citizenry. How might the challenges, principles, and methods of GRTs change in the future? It can be expected that the challenges would not decrease, because they derive from characteristics inherent in GRTs: the prevention nature of the intervention, the healthy nature of the population, and the nature of organizations. Indeed, the challenges may well increase if our society continues in the direction of its current trends—being busy, mobile, and generally skeptical about requests for time and information. As for the principles for addressing the challenges, it can be expected that they will develop and change as additional experience with GRTs is gained, as new technologies develop, new services appear, old ones close, and new regulations are enacted, and as researchers continue to identify creative and effective solutions to the challenges encountered in the rigorous conduct of GRTs.

8

METHODS FOR CONDUCT OF RIGOROUS GROUP-RANDOMIZATION

REFERENCES 1. V. R. A. Call, L. B. Otto, and K. I. Spenner, Tracking Respondents: A Multi-Method Approach. Lexington, MA: Lexington Books, 1982. 2. P. L. Ellickson, Getting and keeping schools and kids for education studies. J. Community Psychol. 1994: 102–116. 3. P. L. Pirie, S. J. Thompson, S. L. Mann, A. V. Peterson, D. M. Murray, B. R. Flay, and J. A. Best, Tracking and attrition in longitudinal school-based smoking prevention research. Preventive Med. 1989; 18: 249–256. 4. D. A. Dillman, Mail and Internet Surveys: The Tailored Design Method. 2nd ed. New York: Wiley, 2000. 5. D. M. Murray, P. L. Pirie, R. V. Luepker, and U. Pallonen, Five- and six-year follow-up results from four seventh-grade smoking prevention strategies. J. Behav. Med. 1989; 12: 207–218. 6. The COMMIT Research Group, Community Intervention Trial for Smoking Cessation (COMMIT): I. Cohort results from a fouryear community intervention. Amer. J. Public Health 1995; 85: 183–192. 7. A. V. Peterson, K. A. Kealey, S. L. Mann, P. M. Marek, and I. G. Sarason, Hutchinson Smoking Prevention Project (HSPP): A longterm randomized trial in school-based tobacco use prevention. Results on smoking. J. Natl. Cancer Inst. 2000; 92: 1979–1991. 8. A. V. Peterson, S. L. Mann, K. A. Kealey, and P. M. Marek, Experimental design and methods for school-based randomized trials: Experience from the Hutchinson Smoking Prevention Project (HSPP). Controlled Clinical Trials 2000; 21: 144–165.

MICROARRAY

major contributions to clinical trials (9,10), basic science, cancer, diabetes, and nutrition to name just a few of the uses. In the future, we envision the impact of microarrays on research increasing as they are increasingly applied in Comparative Genomic hybridization (11), genotyping (12), phage display (13), tissue arrays (14), tiling arrays (15), and especially clinical trials. Microarrays have been a part of clinical trials essentially since microarrays have been developed. They have been used to differentiate lymphatic cancers, breast cancer, and small blue cell cancers to name a few. They have been used to identify new disease mechanisms for diseases such as diabetes (16) and have been used to predict response to drugs (17,18) and diagnosis diseases (19). However, many of these studies, especially the early ones, have not yielded reproducible or generalizable results (20), which is because of a lack of understanding of the needs for careful experimental design and conduct of microarray studies. Clinical trials and microarray studies on clinical samples have been a driving factor in the development and maturity of microarray technology. Whereas most microarray studies have been small(ish) basic science experiments, microarray studies on clinical trial samples, although less numerous, tended to have many samples that have provided sufficient power and information to identify the issues with microarrays, which we discuss in the rest of this article. The identification of these issues has allowed for them to be controlled for and reduced or eliminated. In addition, regulatory and U.S. Food and Drug Administration (FDA) requirements drove the development of the microarray quality control consortium (MAQC), which establishes the validity and reproducibility of array technology. Now that researchers understand the methods and techniques to run a microarray study to generate valid results, microarrays can provide great insights into clinical trials. Some potential uses include determining why certain people respond or do not respond to a

GRIER P. PAGE University of Alabama at Birmingham, Department of Biostatistics, School of Public Health,

XIANGQIN CUI University of Alabama at Birmingham, Department of Medicine, School of Medicine, Department of Biostatistics, School of Public Health, Birmingham, Alabama

In this review, we highlight some issues an investigator must consider when thinking about using expression arrays in a clinical trial. We present the material in a series of sections. The first section highlights previous uses of microarrays in clinical trials. The second section discusses expression microarrays, whereas other uses for microarrays are discussed in the next section. To generate high-quality microarray data for these applications requires careful consideration of experimental design and conduct. The next sections highlight the steps in the planning and analysis of a microarray study. The fourth section discusses defining the objectives of the study, followed by experimental design, data extraction from images, microarray informatics, single gene analysis, data annotation, analyses beyond a single gene, microarray result validation, and finally conclusions. 1

INTRODUCTION

Microarrays have been described as ‘‘the hottest thing in biology and medicine since the advent of the polymerase chain reaction a decade ago.’’ (Anonymous, 2001). This technology emerged circa 1996 (1,2) and had its first high-profile uses in 1998 and 1999 (3–5). What is ‘‘hot’’ is that microarrays allow the simultaneous measurement of mRNA expression levels from thousands of genes from samples. DNA microarray studies have already been published for variety of species including human (6), mouse (7), rat (8), and made

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

MICROARRAY

treatment, to understand the mechanisms of action of a drug or treatment, and to identify new drug targets and ultimately personalized medicine. However, microarrays are only successful with careful design conduct, analysis, and replication. This success is best exemplified by recent approval by the FDA of the first microarray based device, the MammaPrint (Molecular Profile Institute, Phoenix, AZ) array. 1.1 MammaPrint MammaPrint is the first commercial product approved (February 2007) by the FDA for disease prognosis. This product uses an agilentbased microarray that contains a 70-gene breast cancer signature to classify unfixed tumors from node negative women under age 61 as low or high risk for disease recurrence of the disease. The 70-gene breast cancer signature was initially developed by ‘t Veer in a 2002 paper in Nature. The initial study of 78 people identified a signature from a genome wide expression array (21). That signature was established in a study of 151 people (22). A diagnostic array was developed from the genome wide array. This array contains 232 probes that query the 70 signature genes, 289 probes for hybridization and printing quality control, as well as 915 normalization probes (23). This diagnostic array was validated in a study of 302 patients (24). The 70 gene profiles were developed in the initial study by identifying the genes most associated with good outcome called the ‘‘good prognosis template’’ and then validated with ‘‘leave one out cross validation.’’ For new samples, the 70 profile genes are correlated to the ‘‘good prognosis template’’ with a cosine correlation. A value higher than 0.4 results in a patient being assigned to the good-profile group, otherwise they are assigned to the poor profile group. MammaPrint was approved by the FDA because of several important steps the group took, which are explored in greater detail below. These steps included the use of large sample size, replication samples, a good statistical analysis plan, careful quality control, and the results of the MAQC.

2 WHAT IS A MICROARRAY? Usually, a microarray is a solid substrate (usually glass) on which many different sense or antisense (which depends on the technology) cRNAs or cDNAs have been spotted in specific locations, and some technologies attach the probes to beads. Normally, many different cDNAs are spotted onto a single array in a grid pattern, and the cDNAs are taken from genes known to code for poly A+ RNA or from EST sequences. First, RNA from a tissue sample is extracted and labeled with fluorescent dyes or radioactive nucleotides. This labeled RNA is then hybridized to the cRNA or cDNA immobilized on the array. The labeled RNA binds to its complementary sequence approximately proportional to the amount of each mRNA transcript in a sample. The amount of radioactivity of fluorescence can be measured, which allows estimation of the amount of RNA for each transcript in the sample. Alternative used of microarrays are described in the next section. 2.1 Types of Expression Microarrays A high-quality microarray study requires high-quality microarrays. The manufacture of high-quality spotted arrays (both long oligos and cDNA) is very time and resource intensive. Failure to generate high-quality arrays can lead to the arrays being no more than an expensive random number generator. The particulars on how to manufacture high-quality arrays are beyond the scope of this review (25). A variety of different microarray technologies, depending on the species being studied, may be available for use. In general, the technologies can be broken down into three general groups: short oligonucleotides (Affymetrix, NimbleGen), long oligonucleotides (Aglient, Illumina, Amersham), and ‘‘spotted’’ cDNA amplicons (NIA, Stanford, etc.). Each particular technology has its advantages and disadvantages. The short oligos have certain advantages in that they are manufactured to (usually) high standards. Because the arrays use the sum of several probes, the quantification of RNA levels seems to be robust. In addition, the probe sequences are known. Also, some

MICROARRAY

factors could be of disadvantage; specifically they are not particularly robust to polymorphisms. This factor can be a problem if one is working in an atypical species for which an array is not available. Short oligo arrays tend to be relatively expensive with a limited number of species available. However, NimbleGen seems to offer relatively inexpensive custom arrays for any species for which NimbleGen can be provided with sequence information. Long oligos arrays (50–80 mers) offer advantages in that they are robust to some polymorphisms, can often be used across related species, the sequence is known, the spots can be applied at a known concentration, and probes are single stranded. However, long oligos have a high cost of synthesizing and are nonrenewable, but they are often less expensive than short oligo platforms. The complete genome sequence is required to generate robust (unique) oligos with minimal cross-hybridization. Agilent has a very flexible format that allows for the development of highly customized arrays. cDNA amplicons or spotted arrays (4) use sets of plasmids of specific cDNAs, which may be full- or partial-length sequences to generate the sequence to be spotted on an array. The cDNA microarrays allow investigators to print any genes from any species for which a clone can be obtained. cDNA and some long oligo arrays microarrays allow for two samples (usually labeled Cy3 and Cy5) to be hybridized to a single array at a time. The array’s design can be changed rapidly, and the cost per array is low once the libraries have been established. Also, the cDNAs are a renewable supply. Several disadvantages of this method include a variable amount of DNA spotted in each spot and a high (10–20%) drop-out rates caused by failed PCR, failed spots, and possible contamination and cross-hybridization with homologous sequences. As a result, many cDNA arrays are of poor quality. Although microarray technology was originally developed to measure gene expression arrays (26), it has been modified for various other types of applications described in the next section.

3

2.2 Microarrays can Generate Reproducible Results After a series of high profile papers (27) to indicate that different microarray technologies did not generate reproducible results, the FDA has led a project entitled the MAQC project. The purpose of the MAQC project was to provide an assessment of the reproducibility, precision, and accuracy of a variety of human microarrays The MAQC is study of seven microarray platforms (Applied Biosystems, Affymetrix, Agilent, Eppendorf, GE Healthcare, Illumina, and NIC Operon array) with each being tested at 3–6 sites with at least three types of validation (taqMan, StaRTPCR, and QuantiGene. Approximately 150 investigators in seven organizations have been involved. In a series of papers in the September 2006 Nature Biotechnology (28–31), authors revealed that although a range of reproducibility was observed across platform types, microarrays, when conducted with high-quality experiment design, generate highly reproducible results both across platforms as well as to alternative technologies such as reverse transcription polymerase chain reaction (RT-PCR). The most reproducible technologies in both intrasite and intersite studied were those from Affymetrix, Illumina, and Agilent. These reports greatly increase the confidence of the scientific community in the reliability of microarrays and also encouraged the FDA to approve microarray-based prognostic tools. 3

OTHER ARRAY TECHNOLOGIES

Although microarrays were initially used for expression quantification, they are applied in the analysis of other types of data. All these types of arrays could be included potentially in a clinical trial. 3.1 Genotyping using Expression Microarrays Instead of using mRNA as hybridization target, DNA can be used to hybridize the arrays designed for measure gene expression (expression arrays), such as the Affymetrix expression chips that have redundant probe for each gene. Because only one copy is in

4

MICROARRAY

the genome for most genes, the hybridization difference between two DNA samples will just reflect the amount of DNA used in each sample for most genes. Therefore, redundant probes from the same gene should give similar fold changes across samples unless there is a polymorphism between one probe and the target in a sample. The fold change computed from the probe that contains the polymorphism will be different from those obtained from other probes in the same gene. This mechanism can be used to identify polymorphisms between samples, and the polymorphism revealed by this method is call single feature polymorphism (SFP) (32–35). Hybridizations with mRNA as target can also be used to identify DNA polymorphisms and genotype samples. Similarly, the presence of polymorphism between the two compared samples will cause the probe to show different fold change across samples compared with the nearby probes in the same gene (36). However, this strategy can be affected by some RNA expression properties, such as alternative splicing and RNA degradation. 3.2 Splicing Arrays When introns are removed (splicing) during transcription, different splicing products can be generated from the same gene because of the alternative usage of intron and exons (alternative splicing). Alternative splicing is becoming one of the most important mechanisms in control of gene expression by generating structurally and functionally distinct proteins. In addition, alternatively spliced transcripts may regulate stability and function of corresponding transcripts. It is believed that >50% of all human genes are subjected to alternative splicing (37). Although the machinery of splicing has been well known, how splice sites are selected to generate alternative transcripts remains poorly understood. For most alternative splicing events, their functional significance remains to be determined. In recent years, many researchers are characterizing alternative splicing using microarrays with probes from exons, introns, and intron–exon junctions. The basic idea is that the discordance among multiple probes

from the same gene indicates difference in transcripts themselves across samples. For example, skipping of an exon in one sample will cause the probes that hybridize to the skipped exon or the related exon–intron junctions to show different fold change compared with other probes from the same gene. Hu et al. (38) used a custom oligo array of 1600 rat genes with 20 pairs of probes for each gene to survey various rat tissues. A total of 268 genes were detected to have alternative splicing, and 50% of them were confirmed by RT-PCR. Clark et al. (39) monitored the global RNA splicing in mutants of yeast RNA processing factors using exon, intron, and exon-junction probes. Two of 18 mutants tested were confirmed by RT-PCR. In addition, they could cluster the mutants based on the indices for intron exclusion and junction formation to infer the function of associated mutant factors. In humans, Johnson et al. (40) analyzed the global alternative splicing of 10,000 multiexon genes in 52 normal tissues using probes from every exon–exon junction and estimated that at least 74% of human multiexon genes are alternatively spliced. They validated a random sample of 153 positives using RT-PCR and successfully confirmed 73. Other similar experiments were also conducted in humans to investigate alternative splicing in tumor cell line (41) and various tissues (42) but on a smaller scale. 3.3 Exon Array Most current designs of short oligo arrays have probes concentrated at the 3
end of genes because of the better EST support and commonly used reverse transcription methods starting from the 3
end. For sequenced genomes, the gene structure is annotated. Every exon instead of the 3
end can be represented on the array, which is called exon array. Exon array has a better representation of the genes. It provides not only information on the gene expression level but also an opportunity to detect alternative splicing by examining the relative expression of the exons across samples. If the relative expression of different exons of the same gene does not agree, then alternative splicing is indicated. It can detect some types of alternative

MICROARRAY

splicing such as exon skipping, but it is not sensitive to some types of alternative splicing, such as intron retention. In addition to detecting gene expression and alternative splicing, exon arrays can also be used to detect gene copy number difference through comparative genomic hybridization (43).

3.4 Tiling Array—Including Methylation Arrays The current knowledge of genes and gene expression is mainly based on the study of expressed mRNA and computational analysis of the EST collections. The full genome sequences of several organisms and the advance of microarray technology provides a mean, which are also called tiling arrays, to survey the whole genome for gene expression (44). The tiling arrays contain probes that cover the whole genome (or a few chromosomes) in an overlap, head-to-tail, or with small gaps. In theory, the expression of any region in the genome can be read out from the expression intensity of the probes. Tiling arrays have been developed for several organisms, including human (45,46), Drosophila (47), Arabidopsis (48,49), and rice (50). The results from the tiling experiments showed that tiling arrays can detect much more expressed regions in the genome than what have been known; however, because of the lack of reproducibility, many false positives may be observed (51,52). In addition, tiling arrays can be used to survey the genome for copy numbers using DNA instead of mRNA as hybridization target (53). The similar idea was also used to detect the epigenetic modification of the genome, DNA methylation (54,55), and histone acetylation (56). For detecting DNA methylations, arrays contain probes that correspond to each fragment of the methylation sensitive restriction enzymes. After digestion with these enzymes, the different fragments from hypermethylated or hypomethylated regions are enriched by specific adapters and PCR schemes. The enriched and samples are then hybridized to tiling arrays or arrays that cover just GpC islands in the genome (57).

5

3.5 SNP Chip A major task in dissecting the genetic factors that affect complex human disease is genotyping each individual in the study population. The most common polymorphism across individuals is the single nucleotide polymorphism (SNP). Special microarrays are specially designed for genotyping SNPs, such as Affymetrix 10k, 100k, 500k SNP chips, and Illumina SNP chips. These chips have probes centered on the SNP site and have perfect match probes for each allele as well as corresponding mismatch probes (58–60). Each array platform can genotype 10,000 or 500,000 SNP in a single hybridization. Although each hybridization is relatively expensive, the cost per genotype is low (61,62). 3.6 ChIP-on-Chip Microarrays can also be used to characterize the bindings of protein to DNA in defining gene regulation elements in the genome. The DNA sequences with proteins bound are enriched using chromatin immunoprecipitation (ChIP) and are then compared with regular DNA samples to detect the enrichment of some DNA elements. These DNA elements are associated with protein binding and tend to be involved in regulation of gene expression and other features related to the binding protein. The microarrays used in ChIP-chip studies are genomic arrays such as tiling arrays. However, because of the large number of arrays that it takes to cover the genome, some special arrays are designed to cover certain regions of the genome, such as the promoter regions and arrays, which use DNA from large probes (several kb or mb BAC clones). However, the resolution of the protein binding position decreases as the probe size increases. For reviews, see References 63–65. 3.7 Protein Arrays So far, the most application of microarrays in biological research are DNA arrays that analyze mRNA expression and DNA genotyping. However, the functional units for most genes are proteins instead of mRNA, which is just a messenger. Researchers have put forth a

6

MICROARRAY

large effort to increase the throughput for characterizing the assays with protein, such as the proteome technologies. Considerable efforts have been spent in developing protein arrays; however, because of the challenging nature of protein, there has been less success in producing and applying protein arrays. Nonetheless, the research and development of protein arrays is still an active field with the hope of similar success as DNA microarrays in the near future (66,67). Based on the specific use, two major types of protein arrays are available. One type is analytical array, in which antibodies are arrayed on solid surface to detect the corresponding proteins in the hybridization soup (68–70). Another type is more functional oriented to detect protein–protein, protein–DNA, and protein–small molecule interaction. For the protein–protein interaction arrays, proteins are arrayed on the surface, and they can interact and bind to their interaction partners in the hybridization solution (71,72). Protein–DNA interaction arrays are used for detecting the binding sites of some proteins in the genome (73,74). The protein–small molecule interaction arrays are used to identify substrates for protein (75), drug targets (76), and immune response to proteins (77–79). 4

DEFINE OBJECTIVES OF THE STUDY

In the next few sections, we will discuss the steps of a microarray study and what an investigator needs to be aware of before, during, and after starting a clinical trial with microarrays. The first step of any experiment, microarray or otherwise, is to define a hypothesis. Although it may seem perfectly obvious to state this, the objectives of a study are likely to be met if that study is designed in such a way that it is consistent with meeting those objectives, and a study can only be designed to meet the objectives if the objectives can be articulated clearly prior to initiating the study. It is not uncommon for investigators who have conducted a microarray study to be unable to state the objective(s) of their study clearly (e.g., to identify genes that are differentially expressed in

response to a treatment or genes that may be responsible for differential response to treatments). Often, investigators have inadvertently allowed themselves to go down the slippery slope from hypothesis-generating studies to studies that might be termed ‘‘objective generating.’’ By objective generating, we mean studies in which the investigator has initiated a microarray study without a clear conception of the exact objectives of the study in the sole hope that, by some mysterious process, the mass of data from the microarray study will make all things clear. We do not believe that such outcomes are likely. However, that is not to say that the experimental objectives may not include the generation of new objectives/hypotheses on interesting pathways/genes that may be highlighted as a result of the microarray study or be very broadly defined. Thus, we urge investigators to articulate clearly what they hope to obtain from microarray studies so that the studies can be designed to meet those objectives from the beginning; in other words, researchers should make a hypothesis. 5 EXPERIMENTAL DESIGN FOR MICROARRAY After stating a hypothesis, the most important step of an experiment is the experimental design. If an experiment is designed well, then the choice of analytical methods will be defined and the analysis plan will follow. Several essential principles must be in mind when designing a microarray experiment. These include avoidance of experimental artifact; elimination of bias via use of a simultaneous control group; randomization and (potentially) blinding; and reduction of sampling error via use of replication, balance design, and (where appropriate) blocking. 5.1 Avoidance of Experimental Artifacts Microarrays and most laboratory techniques are liable to nonbiological sources of variation including day, chip lot, reagent lot, day of extraction, clinic samples came from and personnel (post doc effects). In many cases, these sources of variation are larger than

MICROARRAY

the biological variation (80–84). If nonbiological and biological differences are confounded, then the experiment can be essentially meaningless. Thus, careful consideration and identification of all factors must be taken before starting a study. These factors must then be eliminated by making the experimental conduct more homogeneous or controlling by randomization and blocking. 5.2 Randomization, Blocking, and Blinding Although blocking can be used to control for measured or known confounding factor, such as the number of samples that can be run in a day, randomization of varieties/samples/groups and random sampling of populations is very useful for reducing the effect of unmeasured confounding factors (81,85,86), such as difference in weather and interruptions in sample delivery. Microarray experiments can require multiple levels of randomization and blocking to help minimize unanticipated biases from known factors. For example, if only four samples can be processed in a day and there are two experimental groups, then two samples from each treatment group can be run each day (blocking), but the two samples are randomly selected from all samples in a the experimental group. Proper randomization and blocking can greatly reduce the bias of studies. Blinding should, of course, be a part of the conduct of any clinical trial, but blinding may also be appropriate on array analysis at the sample collection and processing steps. Unintentional biases can be introduced by individuals collecting samples; for example, margins may be cleaned more carefully in one group compared to another. In addition, more care may be paid to the processing of one treatment group over another. All of these may cause bias, and if possible blinding should be used in microarray experiments. 5.3 Replication Individuals within a population vary, and all measurement tools such as microarrays measure with some error; thus, a single sample cannot be used to make generalizable inferences about a group or population. Replication of microarrays is needed at several levels.

7

5.3.1 Types of Replication. Replication in the context of microarrays can be incorporated at several levels: (R1) gene-to-gene; genes can be spotted multiple times per array, (R2) array-to-array; mRNA samples can be used on multiple arrays and each array is hybridized separately, and (R3) subject-tosubject; mRNA samples can be taken from multiple individuals to account for inherent biological variability. The first two types of replication are referred to as technical replication. The first type measures withinarray variation whereas the second type measures between-array variation. These types of replication are important for assessing the measurement error and reproducibility of microarray experiments and are extremely useful for improving precision of measurements. On the other hand, the third type of replication allows us to assess the biological variation within populations and thereby to make biologically interesting observations. R1 Technical replicates can not substitute for biological replicates (R3). Although R2 technical replicates have a specific role when the cost of samples is far larger than arrays, an experiment cannot be run only with R2 replicates and biological generalizable results cannot be obtained (87). 5.3.2 Replication, Power, and Sample Size. Sample size has a major impact on how confidently genes can be declared either differentially (sensitivity and power) or not differentially expressed (80,88) (specificity). Sample sizes can be determined in a variety of ways. One way is traditional statistical power analysis programs such as PS, which contains the following: power (1-beta), significance (alpha), a measure of variation (Standard deviation), and a detectable difference (delta). As an example at 80% power at a Bonferroni corrected significance level α = 0.05 to detect a 1/2 standard deviation (SD) reduction requires a sample size of over 250 per group, which is not normally achievable in microarray experimentations for budgetary reasons. Another approach we believe is more appropriate would be to choose sample size based on control of the false discovery rate (FDR) (89) and the expected discovery rate (EDR). The FDR is an estimate of the expected proportion of

8

MICROARRAY

genes declared significant that are in fact not differentially expressed [i.e., that are ‘‘false discoveries’’ (90,91)]. The EDR is the expected proportion of genes in which true differences between conditions exist that are found to be significantly different. This approach has been developed and applied in the PowerAtlas (92–94) (www.poweratlas.org). In addition, the PowerAtlas allows an investigator to either upload their own pilot data or to choose from among over 1000 public microarray experiments to use as pilot data for estimating sample size. 5.4 Practice, Practice, Practice No experimenter runs every step of a microarray experiment perfectly the first time. A learning curve is observed for all steps, and the learning process is a confounding factor. Training at all steps is necessary from sample collection to RNA processing, hybridization, and analysis. Thus, all the individual steps should be practiced before running an experiment, and new people who will handle samples need to be trained to sufficient standards before they run ‘‘real’’ samples. Resources spent on training are not wasted, and training samples should not be included in a ‘‘real’’ experiment. 5.5 Strict Experimental Practices Because microarray experiments are liable to many nonbiological sources of error, it is critical to conduct microarray studies that follow very strictly defined protocols. For example, know exactly what types of samples are and are not acceptable, what cut of the samples is needed, what protocol will be used to extract samples, what analyses will be used, what is good quality RNA, and so on before a study is started. Consider a microarray study like a clinical trial in which the researcher must perform a full disclosure of all steps before starting a clinical trial. Deviations from these protocols are strongly discouraged for fear of introducing biases. 6

DATA EXTRACTION

Once a microarray experiment has been conducted and the image of an array is obtained,

several steps must occur to convert the image to analyzable data, and the methods are specific to each technology. 6.1 Image Processing from cDNA and Long Oligo Arrays Image processing consists of three steps: addressing, segmentation, and information extraction. 6.1.1 Gridding/Addressing. Typically, a microarray is an ordered array of spots with constant separation between the row and column; grids or spots must be the same throughout the microarray. Addressing is the process of finding the location of spots on the microarray or assigning coordinates to each spot on the microarray. However, the spotting is rarely perfect; variations must be dealt with. Although software usually does a good job, manual review and occasional intervention results in better data, but it is a very time-consuming process. 6.1.2 Segmentation. Segmentation is the most important and also the most difficult part of the image analysis. In this process, each image pixel is classified as either signal or the background noise. The popular methods of segmentation used fixed circles, adaptive circles, adaptive shapes, or the histogram method. The first two methods provide the means to separate the circular spots from the background by clearly defining the boundaries of the spots. A variety of comparisons of the methods has been published with no clear winner (95,96). 6.1.3 Information Extraction. In the final step of image analysis, the mean and median values of the spot intensities and the background spot intensities are calculated. Usually, correlations between spot intensities, percentage of the spots without any signal, their distribution and signal-to-noise ratio (SNR) for each spot, and variation of the pixel intensities are also calculated. The spot intensities are then measured as the sum of intensities of all the pixels inside the spot.

MICROARRAY

6.2 Image Analysis of Affymetrix GeneChip Microarrays Affymetrix GeneChips are the most widely used oligonucleotide arrays. Unlike the other systems, the Affymetrix system represents each gene as 11–20 probe pairs. Each probe pair is composed of a 25 base pair perfect match (PM) probe that represents the gene’s known sequence and a mismatch (MM) probe that differs from the PM by the middle base. The expression level is some function of the averages of difference in intensities of PM and MM over the 20 sequences. Several algorithms have been developed for averaging the probe pairs to yield a final quantification. These include Dchip (97), GCRMA-EB and GCRMA-MLE (98), MAS5 (99), PDNN (100), and RMA (101,102). all of which have different measurement properties, and it is not yet clear which is best (103). Other technologies such as Illumina and NimbleGen have their own image analysis steps as well. 6.3 Normalization of DNA Data One of the early and near universal steps in microarray study is the use of a technique called either normalization or transformation. The normalization has at least two purposes: to adjust microarray data for effects that develop from variation in the technology rather than from biological differences between the RNA samples or between the printed probes (104) and ‘‘aiding in the analysis of the data by bending the data nearer the Procrustean bed of the assumptions underlying conventional analyses’’ (105), which will allow for reliable statistical and biological analyses. The former is really more adjusting for measured covariates such as dye biases whereas the later is the true meaning of normalization. Either way, a wide variety of methods has been developed for all meanings of normalization including several varieties of linear models (106), loess (104), quantilequantle (107), log2 , and others (108,109). Normalization is usually required in cDNA microarray experiments to reduce dye biases. This area still requires active research, and it is not clear which methods are appropriate for each chip and experimental design.

7

9

MICROARRAY INFORMATICS

For many investigators, microarrays will involve the highest data storage, analysis, and informatics hurdle they will face in their research careers. Microarrays generate huge amounts of data, which can make data storage and handling difficult. In addition, data reporting standards are being enforced by many journals for publications (110,111), and the NIH has started to make data sharing mandatory for certain grants. Microarray experiments generate volumes of data that many biological researchers may not be accustomed to. A single Affymetrix chip will generate about 50 MB of data. After initial processing, each chip will provide thousands to tens of thousands of numbers per array for analysis. After analysis, summary statistics, such as changes in expression and associated significance probabilities, will be available for all genes on the chips. Sorting through significance tests for tens of thousands of genes ‘‘manually’’ and trying to deduce a biological meaning is a Sisyphusian task because of the dimensionality of the data the speed at which new information is generated. These data can be overwhelming. Before an experiment has begun, consideration should be paid to how data are stored, viewed, and interpreted (112). 8

STATISTICAL ANALYSIS

Three types of single gene analyses are typically conducted on microarray data. Class prediction involves building models to predict which group to which samples should be assigned. This method is often used in clinical trials, for example, to develop profiles that predict poor prognosis of cancer (21) or to differentiate among pathologically similar samples (113). The second set of analyses is class discovery, which involves the unsupervised analysis of data to identify previously unknown relationships between genes or samples. The final type is class differentiation, which usually involves the inferential statistical analysis. 8.1 Class Prediction Analysis We use the term prediction to define the construction of a model that uses gene expression

10

MICROARRAY

experiments to classify objects into preexisting known classes, to develop indexes that can serve as a biomarker, or to predict to which class a sample should be assigned (114,115). Many methods can be used to construct such scores (116–118), and it is not clear which technique is best, but the goal of all methods is to find the best compromise between complexity and simplicity in the model. As the predicted model becomes more and more complex by using more and more sample information, the predicted ability in the sample in hand will increase; however, the sample data contain not only information about the true structure of the data but also it contains ‘‘noise’’ because of sample variation. Thus, great care must be taken in the model building (119–121). To build the models that will predict new samples well, one must build cross-validated models. Crossvalidating requires that one have sufficient data to hold some models from the estimation process so that one can subsequently check how well the prediction is using the held back data. For cross validation to be accurate, the held back data used in the cross validation must have not been used in the selection of the structure of the model used for prediction or which parameters go into that model (122,123). This validation has often been violated in microarray research. 8.2 Class Discovery Analysis Since Eisen et al. (4) first applied hierarchical clustering to microarray data analysis in 1998, cluster analysis has emerged as a prevalent tool for the exploration and visualization of microarray data. A variety of cluster methods are available, which include hierarchical and nonhierarchical methods. Among hierarchical methods are agglomerative and divisive methods. For the hierarchical methods, different ways can be used to measure distance between genes, which includes Person’s correlation, Euclidian distance, and Kendall’s Tau as well as a variety of methods for linking genes based on their distance including average, single, complete, and median. Several nonhierarchical methods exist including K-nearest neighbors, selforganizing Maps and related techniques such

a support vector machine and singular value decomposition. Each method and approach has its own positive and negative aspects that should be evaluated. Clustering is a commonly used tool for microarray data analysis; but unlike other statistical methods, no theoretical foundations provide the correct answer. This problem leads directly to several related criticisms of cluster analysis. First, the cluster algorithms are guaranteed to produce clusters from data, no matter what kind of data has been used. Two, different methods can produce drastically different results, and the search for the best choice among them has just begun (124,125). Third, no valid method is available to establish the number of clusters in nonhierarchical cluster analysis. Therefore, cautions are required for performing such analysis, and one should avoid overinterpreting the results; however, cluster analysis is good to provide exploratory descriptive analysis and concise displays of complex data. 8.3 Class Differentiation Analysis One of the main tasks in analyzing microarray data is to determine which genes are differentially expressed between two or more groups of samples. This type of analysis is a conventional hypothesis test. For making inference, virtually any statistical method can be used including t-test, analysis of variance, and linear models. A variety of Bayesian methods and information borrowing approaches such as Cyber-t (126) and SAM (90) has been developed. Because of the small sample sizes, it is often useful to employ variance shrinkage based methods for more robust estimation (127,128). 8.3.1 Adjusting for Multiple Testing. Because each microarray can contain thousands of genes, some adjustment for multiple testing is required to avoid many false positive results. One way is to control the family-wise error rate (FWE), which is the probability of wrongly declaring at least one gene as differentially expressed. A Bonferroni correction is a method to adjust P-values from independent tests. Permutation methods can be used to control FWE in presence of nonindependent tests (129). Another approach

MICROARRAY

to address the problem of multiple testing is the false discovery rate (FDR), which is the proportion of false positives among all genes initially identified as being differentially expressed (89,130). In addition, a variety of Bayesian and alternative FDR methods has been developed (91). 9

ANNOTATION

The gene-by-gene statistical analysis of microarray data is not the end of a study by any stretch of the imagination. The next step is the annotation of the expression data. The amount of information about the functions of genes is beyond what any one person can know. Consequently, it is useful to pull in information on what others have discovered about genes to interpret an expression study fully and correctly. A variety of tools such as array manufacturer’s web sites, KEGG (Kyoto Encyclopedia of Genes and Genomes)(131,132), Gene Index(133,134), Entrez Gene, Medminer (135), DAVID (Database for Annotation, Visualization and Integrated Discovery), and Gene Ontology (136). Each database and tool has slightly different data, and one should use multiple databases when annotating. Also be aware databases can be different with respect to the same information. 10 PATHWAY, GO, AND CLASS-LEVEL ANALYSIS TOOLS Analysis of microarray experiments should not stop at a single gene, but rather several approaches can be used to get a picture beyond a single gene. These tools are called by a variety of names including pathway analysis, gene class testing, global testing, entrez testing, or GO (Gene Ontology analysis). The goal of all these tools is to relate the expression data to other attributes such as cellular localization, biological process, molecular function or a pathway for individual genes or groups of related genes. The most common way to analyze a gene list functionally is to gather information from the literature or from databases that cover the whole genome. In recent years, many

11

tools have been developed to assess the statistical significance of association of a list of genes with GO annotations terms and new ones are released regularly (137). Extensive discussion has occurred of the most appropriate methods for the class level analysis of microarray data (138–140). The methods and tools are based on different methodological assumptions. Two key points must be considered: (1) does the method use gene sampling or subject sampling and (2) whether the method use competitive or self-contained procedures. The subject sampling methods are preferred, and the competitive versus selfcontained debate continues. Gene sampling methods base their calculation of the P-value for the gene set on a distribution in which the gene is the unit of sampling, whereas the subject sampling methods take the subject as the sampling unit. The latter, which is based on the subjects not the genes, is typically the unit of randomization in a study (141–143). Competitive tests, which encompass most existing tools, test whether a gene class, defined by a specific GO term or pathway or similar, is overrepresented in the list of genes differentially expressed compared to a reference set of genes. A self-contained test compares the gene set with a fixed standard that does not depend on the measurements of genes outside the gene set. Goeman et al. (144,145), Mansmann and Meister (141) and Tomfohr et al. (143) applied the selfcontained methods. These methods are also implemented in SAFE and Globaltest. Another important aspect of ontological analysis regardless of the tool or statistical method is the choice of the reference gene list against which the list of differentially regulated genes is compared. Inappropriate choice of reference genes may lead to false functional characterization of the differentiated gene list. Khatri and Draghici (146) pointed out that only the genes represented on the array should be used as reference list instead of the whole genome as it is a common practice. In addition, correct, up-to-date, and complete annotation of genes with GO terms is critical; the competitive and gene samplebased procedures tend to have better and more complete databases. GO allows annotation of genes at different levels of abstraction due to the directed acyclic graph structure

12

MICROARRAY

of the GO. In this hierarchical structure, each term can have one or more child terms as well as one or more parent terms. For instance, the same gene list is annotated with a more general GO term such as ‘‘cell communication’’ at a higher level of abstraction, whereas the lowest level provides a more specific ontology term such as ‘‘intracellular signaling cascade.’’ It is important to integrate the hierarchical structure of the GO in the analysis because various levels of abstraction usually give different P-values. The large number (hundreds or thousands) of tests performed during ontological analysis may lead to spurious associations just by chance. Correction for multiple testing is a necessary step to take. Other analyses look beyond single genes such as coexpression (147), network analysis (148,149), and promoter and transcriptional regulations (150,151). 11 VALIDATION OF MICROARRAY EXPERIMENTS A plethora of factors, which include biological and technical factors, inherent characteristics of different array platforms, and processing and analytical steps can affect results of a typical microarray experiment (152). Thus, several journals now require some sort of validation for a paper to be published. Sometimes, it is possible to confirm the outcome without doing any additional laboratorybased analysis. For example, array results can be compared with information available in the literature or in expression databases such as GEO (153). However, such in silico validation is not always possible or appropriate. Thus, other techniques such as RT-PCR, SAGE (154), and proteomics are used. However, many studies merely conduct technical validation of microarray results. This method may have been appropriate before the result of MAQC established the validity of expression studies. Thus, in our opinion, if microarray studies are well planned, then a valid and technical validation of the array results is not needed, but rather the verification that investigators should pursue should advance their hypotheses rather than arbitrarily technical validation of certain genes.

12 CONCLUSIONS When coupled with good experiment design, a high-quality analysis, and thorough interpretation, microarray technology has matured to the point where it can generate incredibly valuable information. Microarrays can be used to provide greater understanding of the disease being studied, to develop profiles to predict response to compounds, and to predict side effects or poor outcomes. In the near future, microarrays may be used determine which treatments a person may best respond to. We hope this article will help investigators in their use of microarrays. 13 ACKNOWLEDGEMENT The contents here were developed over many years in discussion with many investigators at UAB and around the world including David Allison, Steve Barnes, Lang Chen, Jode Edwards, Gary Gadbury, Issa Coulibaly, Kyoungmi Kim, Tapan Mehta, Prinal Patal, Mahyar Sabripour, Jelai Wang, Hairong Wei, Richard Weindruch, Stanislav Zakharkin, and Kui Zhang. The work could not have been conducted without their thoughts and input. GPP and XQ were supported by NIH grants AT100949, AG020681, and ES 012933 and GPP by NSF grant 0501890. REFERENCES 1. M. Chee, R. Yang, E. Hubbell, A. Berno, Z. C. Hunag, D. Stern, J. Winkler, D. J. Lockhart, M. S. Morris, and S. P. A. Fodor, Accessing genetic information with high-density DNA arrays. Science 1996; 274: 610–614. 2. D. J. Lockhart, H. Ding, M. Byrne, M. T. Follettie, M.V. Gallo, M. A. S. Chee, M. Mittmann, C. Wang, M. Kobayashi, H. Horton, and E. L. Brown, Expression monitoring by hybridization to high-density oligonucleotide arrays. Nat. Biotech. 1996; 14: 1675–1680. 3. C.-K. Lee, R. G. Kloop, R. Weindruch, and T. A. Prolla, Gene expression profile of aghing and its restriction by caloric restriction. Science 1999; 285: 1390–1393. 4. M. B. Eisen, P.T. Spellman, P.O. Brown, and D. Botstein, Cluster analysis and display of genome-wide expression patterns.

MICROARRAY Proc. Natl. Acad. Sci. U.S.A. 1998; 95: 14863–14868. 5. C. M. Perou, S. S. Jeffrey, R. M. van de, C. A. Rees, M. B. Eisen, D. T. Ross, A. Pergamenschikov, C. F. Williams, S. X. Zhu, J. C. Lee, D. Lashkari, D. Shalon, P. O. Brown, and D. Botstein Distinctive gene expression patterns in human mammary epithelial cells and breast cancers. Proc. Natl. Acad. Sci. U.S.A. 1999; 96: 9212–9217. 6. M. A. Ginos, G. P. Page, B. S. Michalowicz, K. J. Patel, S. E. Volker, S. E. Pambuccian, F. G. Ondrey, G. L. Adams, and P. M. Gaffney, Identification of a gene expression signature associated with recurrent disease in squamous cell carcinoma of the head and neck. Cancer Res. 2004; 64: 55–63. 7. Y. Higami, T. D. Pugh, G. P. Page, D.B. Allison, T. A. Prolla, and R. Weindruch, Adipose tissue energy metabolism: altered gene expression profile of mice subjected to longterm caloric restriction. FASEB J. 2003; 8: 415–417. 8. S. O. Zakharkin, K. Kim, T. Mehta, L. Chen, S. Barnes, K.E. Scheirer, R. S. Parrish, D. B. Allison, and G. P. Page, Sources of variation in Affymetrix microarray experiments. BMC Bioinformat. 2005; 6: 214. 9. J. C. Lacal, How molecular biology can improve clinical management: the MammaPrint experience. Clin. Transl. Oncol. 2007; 9: 203. 10. S. Mook, L. J. van’t Veer, Rutgers E.J., Piccart-Gebhart M.J., and F. Cardoso, Individualization of therapy using Mammaprint: from development to the MINDACT Trial. Cancer Genom. Proteom. 2007; 4: 147–155. 11. J. Zhao, J. Roth, B. Bode-Lesniewska, M. Pfaltz, P.U. Heitz, and P. Komminoth, Combined comparative genomic hybridization and genomic microarray for detection of gene amplifications in pulmonary artery intimal sarcomas and adrenocortical tumors. Genes Chromos. Cancer 2002; 34: 48–57. 12. K. L. Gunderson, F. J. Steemers, G. Lee, L. G. Mendoza, and M. S. Chee, A genomewide scalable SNP genotyping assay using microarray technology. Nat. Genet. 2005; 37: 549–554. 13. L. Cekaite, O. Haug, O. Myklebost, M. Aldrin, B. Ostenstad, M. Holden, A. Frigessi, E. Hovig, and M. Sioud, Analysis of the humoral immune response to immunoselected phage-displayed peptides by a microarray-based method. Proteomics 2004; 4: 2572–2582.

13

14. C. Gulmann, D. Butler, E. Kay, A. Grace, and M. Leader, Biopsy of a biopsy: validation of immunoprofiling in gastric cancer biopsy tissue microarrays. Histopathology 2003; 42:70–6. 15. T. C. Mockler, S. Chan, A. Sundaresan, H. Chen, S. E. Jacobsen, and J. R. Ecker, Applications of DNA tiling arrays for wholegenome analysis. Genomics 2005; 85: 1–15. 16. V. K. Mootha, C. M. Lindgren, K. F. Eriksson, A. Subramanian, S. Sihag, J. Lehar, P. Puigserver, E. Carlsson, M. Ridderstrale, E. Laurila, N. Houstis, M. J. Daly, N. Patterson, J. P. Mesirov, T. R. Golub, P. Tamayo, B. Spiegelman, E. S. Lander, J. N. Hirschhorn, D. Altshuler, and L. C. Groop, PGC-1alpharesponsive genes involved in oxidative phosphorylation are coordinately downregulated in human diabetes. Nat. Genet. 2003; 34: 267–273. 17. L. Cabusora, E. Sutton, A. Fulmer, and C. V. Forst, Differential network expression during drug and stress response. Bioinformatics 2005; 21: 2898–2905. 18. J. M. Naciff, M. L. Jump, S. M. Torontali, G. J. Carr, J. P. Tiesman, G. J. Overmann, and G. P. Daston, Gene expression profile induced by 17alpha-ethynyl estradiol, bisphenol A, and genistein in the developing female reproductive system of the rat. Toxicol. Sci. 2002; 68: 184–199. 19. Y. Tang, D. L. Gilbert, T. A. Glauser, A. D. Hershey, and F. R. Sharp, Blood gene expression profiling of neurologic diseases: a pilot microarray study. Arch. Neurol. 2005; 62: 210–215. 20. J. P. Ioannidis, Microarrays and molecular research: noise discovery? Lancet 2005; 365: 454–455. 21. L. J. ’t Veer, H. Dai, M. J. van de Vijver, Y. D. He, A. A. Hart, M. Mao, H. L. Peterse, K. K. van der, M. J. Marton, A. T. Witteveen, G. J. Schreiber, R. M. Kerkhoven, C. Roberts, P. S. Linsley, R. Bernards, and S. H. Friend, Gene expression profiling predicts clinical outcome of breast cancer. Nature 2002; 415: 530–536. 22. M. J. van de Vijver, Y. D. He, L. J. van’t Veer, H. Dai, A. A. Hart, D. W. Voskuil, G. J. Schreiber, J. L. Peterse, C. Roberts, M. J. Marton, M. Parrish, D. Atsma, A. Witteveen, A. Glas, L. Delahaye, T. van der Velde, H. Bartelink, S. Rodenhuis, E. T. Rutgers, S. H. Friend, and R. Bernards, A gene-expression signature as a predictor of survival in breast cancer. N. Engl. J. Med. 2002; 347: 1999–2009.

14

MICROARRAY

23. Glas A. M., A. Floore, L. J. Delahaye, A. T. Witteveen, R. C. Pover, N. Bakx, J. S. LahtiDomenici, T. J. Bruinsma, M. O. Warmoes, R. Bernards, L. F. Wessels, and L. J. van’t Veer, Converting a breast cancer microarray signature into a high-throughput diagnostic test. BMC Genom. 2006; 7: 278. 24. M. Buyse, S. Loi, L. van’t Veer, G. Viale, M. Delorenzi, A. M. Glas, M. S. d’Assignies, J. Bergh, R. Lidereau, P. Ellis, A. Harris, J. Bogaerts, P. Therasse, A. Floore, M. Amakrane, F. Piette, E. Rutgers, C. Sotiriou, F. Cardoso, and M. J. Piccart, Validation and clinical utility of a 70-gene prognostic signature for women with node-negative breast cancer. J. Natl. Cancer Inst. 2006; 98: 1183–1192.

31. W. Tong, A. B. Lucas, R. Shippy, X. Fan, H. Fang, H. Hong, M. S. Orr, T. M. Chu, X. Guo, P. J. Collins, Y. A. Sun, S. J. Wang, W. Bao, R. D. Wolfinger, S. Shchegrova, L. Guo, J. A. Warrington, and L. Shi, Evaluation of external RNA controls for the assessment of microarray performance. Nat. Biotechnol. 2006; 24: 1132–1139. 32. J. O. Borevitz, D. Liang, D. Plouffe, H. S. Chang, T. Zhu, D. Weigel, C. C. Berry, E. Winzeler, and J. Chory, Large-scale identification of single-feature polymorphisms in complex genomes. Genome Res. 2003; 13: 513–523.

25. W. Zhang, I. Shmulevich, and J. Astola, Microarray Quality Control. 2004. John Wiley & sons, Inc., Hoboken, NJ.

33. X. Cui, J. Xu, R. Asghar, P. Condamine, J. T. Svensson, S. Wanamaker, N. Stein, M. Roose, and T. J. Close, Detecting singlefeature polymorphisms using oligonucleotide arrays and robustified projection pursuit. Bioinformatics 2005; 21: 3852–3858.

26. M. Schena, D. Shalon, R. W. Davis, and P. O. Brown, Quantitative monitoring of gene expression patterns with a complementary DNA microarray. Science 1995; 270: 467–470.

34. N. Rostoks, J. Borevitz, P. Hedley, J. Russell, S. Mudie, J. Morris, L. Cardle, D. Marshall, and R. Waugh, Single-feature polymorphism discovery in the barley transcriptome. Genome Biol. 2005; 6:R54.

27. P. K. Tan, T. J. Downey, E. L. Spitznagel Jr, P. Xu, D. Fu, D. S. Dimitrov, R. A. Lempicki, B. M. Raaka, and M. C. Cam, Evaluation of gene expression measurements from commercial microarray platforms. Nucleic Acids Res. 2003; 31: 5676–5684.

35. E. A. Winzeler, C. I. Castillo-Davis, G. Oshiro, D. Liang, D. R. Richards, Y. Zhou, and D. L. Hartl, Genetic diversity in yeast assessed with whole-genome oligonucleotide arrays. Genetics 2003; 163: 79–89.

28. T. A. Patterson, E. K. Lobenhofer, S. B. Fulmer-Smentek, P. J. Collins, T. M. Chu, W. Bao, H. Fang, E. S. Kawasaki, J. Hager, I. R. Tikhonova, S. J. Walker, L. Zhang, P. Hurban, F. de Longueville, J. C. Fuscoe, W. Tong, L. Shi, and R. D. Wolfinger, Performance comparison of one-color and two-color platforms within the Microarray Quality Control (MAQC) project. Nat. Biotechnol. 2006; 24: 1140–1150.

36. J. Ronald, J. M. Akey, J. Whittle, E. N. Smith, G. Yvert, and L. Kruglyak, Simultaneous genotyping, gene-expression measurement, and detection of allele-specific expression with oligonucleotide arrays. Genome Res. 2005; 15: 284–291. 37. P. A. Sharp, The discovery of split genes and RNA splicing. Trends Biochem. Sci. 2005; 30: 279–281. 38. G. K. Hu, S. J. Madore, B. Moldover, T. Jatkoe, D. Balaban, J. Thomas, and Y. Wang, Predicting splice variant from DNA chip expression data. Genome Res. 2001; 11: 1237–1245.

29. L. Shi, L. H. Reid, W. D. Jones, R. Shippy, J. A. Warrington, S. C. Baker, P. J. Collins, F. de Longueville, E. S. Kawasaki, and K. Y. Lee, The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat. Biotechnol. 2006; 24: 1151–1161.

39. T. A. Clark, C. W. Sugnet, M. Ares Jr., Genomewide analysis of mRNA processing in yeast using splicing-specific microarrays. Science 2002; 296: 907–910.

30. R. Shippy, S. Fulmer-Smentek, R. V. Jensen, W. D. Jones, P. K. Wolber, C. D. Johnson, P. S. Pine, C. Boysen, X. Guo, E. Chudin, et al., Using RNA sample titrations to assess microarray platform performance and normalization techniques. Nat. Biotechnol. 2006; 24: 1123–1131.

40. J. M. Johnson, J. Castle, P. Garrett-Engele, Z. Kan, P. M. Loerch, C. D. Armour, R. Santos, E. E. Schadt, R. Stoughton, and D. D. Shoemaker, Genome-wide survey of human alternative pre-mRNA splicing with exon junction microarrays. Science 2003; 302: 2141–2144.

MICROARRAY 41. A. Relogio, C. Ben Dov, M. Baum, M. Ruggiu, C. Gemund, V. Benes, R. B. Darnell, and J. Valcarcel, Alternative splicing microarrays reveal functional expression of neuronspecific regulators in Hodgkin lymphoma cells. J. Biol. Chem. 2005; 280: 4779–4784. 42. K. Le, K. Mitsouras, M. Roy, Q. Wang, Q. Xu, S. F. Nelson, and C. Lee, Detecting tissue-specific regulation of alternative splicing as a qualitative change in microarray data. Nucleic Acids Res. 2004; 32:e180. 43. P. Dhami, A. J. Coffey, S. Abbs, J. R. Vermeesch, J. P. Dumanski, K. J. Woodward, R. M. Andrews, C. Langford, and D. Vetrie, Exon array CGH: detection of copy-number changes at the resolution of individual exons in the human genome. Am. J. Hum. Genet. 2005; 76: 750–762. 44. T. C. Mockler, S. Chan, A. Sundaresan, H. Chen, S. E. Jacobsen, and J. R. Ecker, Applications of DNA tiling arrays for wholegenome analysis. Genomics 2005; 85: 1–15. 45. P. Kapranov, S. E. Cawley, J. Drenkow, S. Bekiranov, R. L. Strausberg, S. P. Fodor, and T. R. Gingeras, Large-scale transcriptional activity in chromosomes 21 and 22. Science 2002; 296: 916–919. 46. D. Kampa, J. Cheng, P. Kapranov, M. Yamanaka, S. Brubaker, S. Cawley, J. Drenkow, A. Piccolboni, S. Bekiranov, G. Helt, H. Tammana, and T. R. Gingeras, Novel RNAs identified from an in-depth analysis of the transcriptome of human chromosomes 21 and 22. Genome Res. 2004; 14: 331–342. 47. M. Hild, B. Beckmann, S. A. Haas, B. Koch, V. Solovyev, C. Busold, K. Fellenberg, M. Boutros, M. Vingron, F. Sauer, J. D. Hoheisel, and R. Paro, An integrated gene annotation and transcriptional profiling approach towards the full gene content of the Drosophila genome. Genome Biol. 2003; 5:R3. 48. K. Yamada, J. Lim, J. M. Dale, H. Chen, P. Shinn, C. J. Palm, A. M. Southwick, H. C. Wu, C. Kim, M. Nguyen, et al., Empirical analysis of transcriptional activity in the arabidopsis genome. Science 2003; 302: 842–846. 49. V. Stolc, M. P. Samanta, W. Tongprasit, H. Sethi, S. Liang, D. C. Nelson, A. Hegeman, C. Nelson, D. Rancour, S. Bednarek, E. L. Ulrich, Q. Zhao, R. L. Wrobel, C. S. Newman, B. G. Fox, G. N. Phillips Jr, J. L. Markley, and M. R. Sussman, Identification of transcribed sequences in Arabidopsis thaliana by using high-resolution genome

15

tiling arrays. Proc. Natl. Acad. Sci. U.S.A. 2005; 102: 4453–4458. 50. L. Li, X. Wang, V. Stolc, X. Li, D. Zhang, N. Su, W. Tongprasit, S. Li, Z. Cheng, J. Wang, and X. W. Deng, Genome-wide transcription analyses in rice using tiling microarrays. Nat. Genet. 2006; 38: 124–129. 51. J. M. Johnson, S. Edwards, D. Shoemaker, and E. E. Schadt, Dark matter in the genome: evidence of widespread transcription detected by microarray tiling experiments. Trends Genet. 2005; 21: 93–102. 52. T. E. Royce, J. S. Rozowsky, P. Bertone, M. Samanta, V. Stolc, S. Weissman, M. Snyder, and M. Gerstein, Issues in the analysis of oligonucleotide tiling microarrays for transcript mapping. Trends Genet. 2005; 21: 466–475. 53. A. E. Urban, J. O. Korbel, R. Selzer, T. Richmond, A. Hacker, G. V. Popescu, J. F. Cubells, R. Green, B. S. Emanuel, M. B. Gerstein, S. M. Weissman, and M. Snyder, High-resolution mapping of DNA copy alterations in human chromosome 22 using high-density tiling oligonucleotide arrays. Proc. Natl. Acad. Sci. U.S.A. 2006; 103: 4534–4539. 54. A. Schumacher, P. Kapranov, Z. Kaminsky, J. Flanagan, A. Assadzadeh, P. Yau, C. Virtanen, N. Winegarden, J. Cheng, T. Gingeras, and A. Petronis, Microarray-based DNA methylation profiling: technology and applications. Nucleic Acids Res. 2006; 34: 528–542. 55. X. Zhang, J. Yazaki, A. Sundaresan, S. Cokus, S. W. L. Chan, H. Chen, I. R. Henderson, P. Shinn, M. Pellegrini, S. E. Jacobsen, and J. J S. Ecker, Genome-wide highresolution mapping and functional analysis of DNA methylation in arabidopsis. Cell 2006; 126: 1189–1201. 56. C. L. Liu, T. Kaplan, M. Kim, S. Buratowski, S. L. Schreiber, N. Friedman, and O. J. Rando, Single-nucleosome mapping of histone modifications in S. cerevisiae. PLoS Biol. 2005; 3:e328. 57. A. Schumacher, P. Kapranov, Z. Kaminsky, J. Flanagan, A. Assadzadeh, P. Yau, C. Virtanen, N. Winegarden, J. Cheng, T. Gingeras, and A. Petronis, Microarray-based DNA methylation profiling: technology and applications. Nucleic Acids Res. 2006; 34: 528–542. 58. G. C. Kennedy, H. Matsuzaki, S. Dong, W. Liu, J. Huang, G. Liu, X. Su, M. Cao, W.

16

MICROARRAY Chen, J. Zhang, et al., Large-scale genotyping of complex DNA. Nat. Biotech. 2003; 21: 1233–1237.

59. G. C. Kennedy, H. Matsuzaki, S. Dong, W. Liu, J. Huang, G. Liu, X. Su, M. Cao, W. Chen, J. Zhang, et al., Large-scale genotyping of complex DNA. Nat. Biotech. 2003; 21: 1233–1237. 60. H. Matsuzaki, S. Dong, H. Loi, X. Di, G. Liu, E. Hubbell, J. Law, T. Berntsen, M. Chadha, H. Hui, et al., Genotyping over 100,000 SNPs on a pair of oligonucleotide arrays. Nat. Methods 2004; 1: 109–111. 61. S. John, N. Shephard, G. Liu, E. Zeggini, M. Cao, W. Chen, N. Vasavda, T. Mills, A. Barton, A. Hinks, S. Eyre, et al., Whole-genome scan, in a complex disease, using 11,245 single-nucleotide polymorphisms: comparison with microsatellites. Am. J. Hum. Genet. 2004; 75: 54–64. 62. C. I. Amos, W. V. Chen, A. Lee, W. Li, M. Kern, R. Lundsten, F. Batliwalla, M. Wener, E. Remmers, D. A. Kastner, L. A. Criswell, M. F. Seldin, and P. K. Gregersen, High-density SNP analysis of 642 Caucasian families with rheumatoid arthritis identifies two new linkage regions on 11p12 and 2q33. Genes Immun. 2006; 7: 277–286. 63. Buck MJ, Lieb JD. ChIP-chip: considerations for the design, analysis, and application of genome-wide chromatin immunoprecipitation experiments. Genomics 2004; 83: 349–360. 64. J. Wu, L. T. Smith, C. Plass, and T. H. M. Huang, ChIP-chip comes of age for genomewide functional analysis. Cancer Res. 2006; 66: 6899–6902. 65. M. L. Bulyk, DNA microarray technologies for measuring protein-DNA interactions. Curr. Opin. Biotechnol. 2006; 17: 422–430. 66. C. S. Chen and H. Zhu, Protein microarrays. Biotechniques 2006; 40: 423, 425, 427. 67. P. Bertone and M. Snyder Advances in functional protein microarray technology. FEBS J. 2005; 272: 5400–5411. 68. B. B. Haab, M. J. Dunham, and P. O. Brown, Protein microarrays for highly parallel detection and quantitation of specific proteins and antibodies in complex solutions. Genome Biol. 2001; 2:RESEARCH0004. 69. A. Sreekumar, M. K. Nyati, S. Varambally, T. R. Barrette, D. Ghosh, T. S. Lawrence, and A. M. Chinnaiyan, Profiling of cancer cells using protein microarrays: discovery of novel radiation-regulated proteins. Cancer Res. 2001; 61: 7585–7593.

70. B. Schweitzer, S. Roberts, B. Grimwade, W. Shao, M. Wang, Q. Fu, Q. Shu, I. Laroche, Z. Zhou, V. T. Tchernev, J. Christiansen, M. Velleca, and S. F. Kingsmore, Multiplexed protein profiling on microarrays by rollingcircle amplification. Nat. Biotechnol. 2002; 20: 359–365. 71. H. Zhu, M. Bilgin, R. Bangham, D. Hall, A. Casamayor, P. Bertone, N. Lan, R. Jansen, S. Bidlingmaier, T. Houfek, et al., Global analysis of protein activities using proteome chips. Science 2001; 293: 2101–2105. 72. M. Arifuzzaman, M. Maeda, A. Itoh, K. Nishikata, C. Takita, R. Saito, T. Ara, K. Nakahigashi, H. C. Huang, A. Hirai, et al., Large-scale identification of protein-protein interaction of Escherichia coli K-12. Genome Res. 2006; 16: 686–691. 73. S. W. Ho, G. Jona, C. T. L. Chen, M. Johnston, and M. Snyder, Linking DNA-binding proteins to their recognition sequences by using protein microarrays. Proc. Natl. Acad. Sci. U.S.A. 2006; 103: 9940–9945. 74. D. A. Hall, H. Zhu, X. Zhu, T. Royce, M. Gerstein, and M. Snyder, Regulation of gene expression by a metabolic enzyme. Science 2004; 306: 482–484. 75. T. Feilner, C. Hultschig, J. Lee, S. Meyer, R. G. H. Immink, A. Koenig, A. Possling, H. Seitz, A. Beveridge, D. Scheel, et al., High throughput identification of potential arabidopsis mitogen-activated protein kinases substrates. Molec. Cell. Proteom. 2005; 4: 1558–1568. 76. H. Du, M. Wu, W. Yang, G. Yuan, Y. Sun, Y. Lu, S. Zhao, Q. Du, J. Wang, S. Yang, et al., Development of miniaturized competitive immunoassays on a protein chip as a screening tool for drugs. Clin. Chem. 2005; 51: 368–375. 77. A. Lueking, O. Huber, C. Wirths, K. Schulte, K. M. Stieler, U. Blume-Peytavi, A. Kowald, K. Hensel-Wiegel, R. Tauber, H. Lehrach, et al., Profiling of alopecia areata autoantigens based on protein microarray technology. Molec. Cell. Proteom. 2005; 4: 1382–1390. 78. W. H. Robinson, C. DiGennaro, W. Hueber, B. B. Haab, M. Kamachi, E. J. Dean, S. Fournel, D. Fong, M. C. Genovese, H. E. de Vegvar, et al., Autoantigen microarrays for multiplex characterization of autoantibody responses. Nat. Med. 2002; 8: 295–301. 79. A. Lueking, A. Possling, O. Huber, A. Beveridge, M. Horn, H. Eickhoff, J. Schuchardt, H. Lehrach, and D. J. Cahill, A nonredundant human protein chip for antibody

MICROARRAY

80.

81.

82.

83.

84.

85.

86.

87.

88.

89.

90.

91.

screening and serum profiling. Molec. Cell. Proteom. 2003; 2: 1342–1349. M. T. Lee, F. C. Kuo, G. A. Whitmore, and J. Sklar, Importance of replication in microarray gene expression studies: statistical methods and evidence from repetitive cDNA hybridizations. Proc. Natl. Acad. Sci. U.S.A. 2000; 97: 9834–9839. M. K. Kerr and G. A. Churchill, Statistical design and the analysis of gene expression microarray data. Genet. Res. 2001; 77: 123–128. K. Mirnics, Microarrays in brain research: the good, the bad and the ugly. Nat. Rev. Neurosci. 2001; 2: 444–447. K. R. Coombes, W. E. Highsmith, T. A. Krogmann, K. A. Baggerly, D. N. Stivers, and L. V. Abruzzo, Identifying and quantifying sources of variation in microarray data using high-density cDNA membrane arrays. J. Comput. Biol. 2002; 9: 655–669. Y. Woo, J. Affourtit, S. Daigle, A. Viale, K. Johnson, J. Naggert, and G. Churchill, A comparison of cDNA, oligonucleotide, and Affymetrix GeneChip gene expression microarray platforms. J. Biomol. Tech. 2004; 15: 276–284. D. Rubin, Practical implications of modes of statistical inference for causal effects and the critical role of the assignment mechanism. Biometrics 1991; 47: 1213–1234. M. K. Kerr and G. A. Churchill, Experimental design for gene expression microarrays. Biostatistics 2001; 2: 183–201. M. F. Oleksiak and G. A. Churchill, and D. L. Crawford, Variation in gene expression within and among natural populations. Nat. Genet. 2002; 32: 261–266. D. B. Allison and C. S. Coffey, Two-stage testing in microarray analysis: what is gained? J. Gerontol. A Biol. Sci. Med. Sci. 2002; 57:B189–B192. Y. Benjamini and Y. Hochberg, Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Royal Stat. Soc. Series B 1995; 57: 289–300. V. G. Tusher, R. Tibshirani, and G. Chu, Significance analysis of microarrays applied to the ionizing radiation response. Proc. Natl. Acad. Sci. U.S.A. 2001; 98: 5116–5121. D. Allison, G. Gadbury, M. Heo, J. R. Fernandez, C. K. Lee, T. A. Prolla, and R. Weindruch, A mixture model approach for the analysis of microarray gene expression data. Computat. Stat. Data Anal. 2002; 39: 1–20.

17

92. G. L. Gadbury, G. Xiang, J. Edwards, G. Page, and D. B. Allison, The role of sample size on measures of uncertainty and power. In D. B. Allison, J. W. Edwards, T. M. Beasley, and G. Page (eds.), DNA Microarrays and Related Genomics Techniques. Boca Raton, FL: CRC Press, 2005, pp. 51–60. 93. G. Gadbury, G. Page, J. Edwards, T. Kayo, R. Weindruch, P. A. Permana, J. Mountz, and D. B. Allison, Power analysis and sample size estimation in the age of high dimensional biology. Stat. Meth. Med. Res. 2004; 13: 325–338. 94. G. P. Page, J. W. Edwards, G. L. Gadbury, P. Yelisetti, J. Wang, P. Trivedi, and D. B. Allison, The PowerAtlas: a power and sample size atlas for microarray experimental design and research. BMC Bioinformat. 2006; 7: 84. 95. R. Nagarajan, Intensity-based segmentation of microarray images. IEEE Trans. Med. Imag. 2003; 22: 882–889. 96. Q. Li, C. Fraley, R. E. Bumgarner, K. Y. Yeung, and A. E. Raftery, Donuts, scratches and blanks: robust model-based segmentation of microarray images. Bioinformatics 2005. 97. C. Li and W. W. Hung, Model-based analysis of oligonucleotide arrays: model validation, design issues and standard error application. Genome Biol. 2001; 2: 32–35. 98. Z. Wu and R. A. Irizarry, Stochastic models inspired by hybridization theory for short oligonucleotide arrays. J. Comput. Biol. 2005; 12: 882–893. 99. E. Hubbell, W. M. Liu, and R. Mei, Robust estimators for expression analysis. Bioinformatics 2002; 18: 1585–1592. 100. L. Zhang, L. Wang, A. Ravindranathan, and M. F. Miles, A new algorithm for analysis of oligonucleotide arrays: application to expression profiling in mouse brain regions. J. Mol. Biol. 2002; 317: 225–235. 101. R. A. Irizarry, B. M. Bolstad, F. Collin, L. M. Cope, B. Hobbs, T. P. Speed, Summaries of Affymetrix GeneChip probe level data. Nucleic Acids Res. 2003; 31:e15. 102. R. A. Irizarry, B. Hobbs, F. Collin, Y. D. Beazer-Barclay, K. J. Antonellis, U. Scherf, T. P. Speed, Exploration, normalization, and summaries of high density oligonucleotide array probe level data. Biostatistics 2003; 4: 249–264. 103. K. Shedden, W. Chen, R. Kuick, D. Ghosh, J. Macdonald, K. R. Cho, T. J. Giordano, S. B. Gruber, E. R. Fearon, J. M. Taylor, and

18

MICROARRAY S. Hanash, Comparison of seven methods for producing Affymetrix expression scores based on False Discovery Rates in disease profiling data. BMC Bioinformat. 2005; 6: 26.

104. G. K. Smyth and T. Speed, Normalization of cDNA microarray data. Methods 2003; 31: 265–273. 105. J. Tukey, On the comparative anatomy of transformation. Ann. Mathemat. Statist. 1964; 28: 602–632. 106. R. D. Wolfinger, G. Gibson, E. D. Wolfinger, L. Bennett, H. Hamadeh, P. Bushel, C. Afshari, and R. S. Paules, Assessing gene significance from cDNA microarray expression data via mixed models. J. Comput. Biol. 2001; 8: 625–637. 107. B. M. Bolstad, R. A. Irizarry, M. Astrand, and T. P. Speed, A comparison of normalization methods for high density oligonucleotide array data based on variance and bias. Bioinformatics 2003; 19: 185–193. 108. B. P. Durbin, J. S. Hardin, D. M. Hawkins, and D. M. Rocke, A variance-stabilizing transformation for gene-expression microarray data. Bioinformatics 2002; 18(suppl 1):S105–S110. 109. B. P. Durbin and D. M. Rocke, Variancestabilizing transformations for two-color microarrays. Bioinformatics 2004; 20: 660–667. 110. C. A. Ball, G. Sherlock, H. Parkinson, P. Rocca-Sera, C. Brooksbank, HC. Causton, D. Cavalieri, T. Gaasterland, P. Hingamp, F. Holstege, et al., Standards for microarray data. Science 2002, 298: 539. 111. C. A. Ball, G. Sherlock, H. Parkinson, P. Rocca-Sera, C. Brooksbank, H. C. Causton, D. Cavalieri, T. Gaasterland, P. Hingamp, F. Holstege, et al., An open letter to the scientific journals. Bioinformatics 2002; 18: 1409. 112. K. H. Cheung, K. White, J. Hager, M. Gerstein, V. Reinke, K. Nelson, P. Masiar, R. Srivastava, Y. Li, J. Li, J. Li, et al., YMD: A microarray database for large-scale gene expression analysis. Proc. AMIA Symp. 2002; 140–144. 113. C. Baer, M. Nees, S. Breit, B. Selle, A. E. Kulozik, K. L. Schaefer, Y. Braun, D. Wai, and C. Poremba, Profiling and functional annotation of mRNA gene expression in pediatric rhabdomyosarcoma and Ewing’s sarcoma. Int. J. Cancer 2004; 110: 687–694. 114. R. L. Somorjai, B. Dolenko, and R. Baumgartner, Class prediction and discovery using

gene microarray and proteomics mass spectroscopy data: curses, caveats, cautions. Bioinformatics 2003; 19: 1484–1491. 115. M. D. Radmacher, L. M. McShane, and R. Simon, A paradigm for class prediction using gene expression profiles. J. Comput. Biol. 2002; 9: 505–511. 116. M. Ringner and C. Peterson, Microarraybased cancer diagnosis with artificial neural networks. Biotechnology 2003(suppl): 30–35. 117. U. M. Braga-Neto and E. R. Dougherty, Is cross-validation valid for small-sample microarray classification? Bioinformatics 2004; 20: 374–380. 118. C. Romualdi, S. Campanaro, D. Campagna, B. Celegato, N. Cannata, S. Toppo, G. Valle, and G. Lanfranchi, Pattern recognition in gene expression profiling using DNA array: a comparative study of different statistical methods applied to cancer classification. Hum. Mol. Genet. 2003; 12: 823–836. 119. R. Simon and M. D. Radmacher, and K. Dobbin, Design of studies using DNA microarrays. Genet. Epidemiol. 2002; 23: 21–36. 120. R. Simon, M. D. Radmacher, K. Dobbin, and L. M. McShane, Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. J. Natl. Cancer Inst. 2003; 95: 14–18. 121. R. Simon, Diagnostic and prognostic prediction using gene expression profiles in highdimensional microarray data. Br. J. Cancer 2003; 89: 1599–1604. 122. C. Ambroise and G. J. McLachlan, Selection bias in gene extraction on the basis of microarray gene-expression data. Proc. Natl. Acad. Sci. U.S.A. 2002; 99: 6562–6566. 123. R. Simon, M. D. Radmacher, K. Dobbin, and L. M. McShane, Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. J. Natl. Cancer Inst. 2003; 95: 14–18. 124. N. R. Garge, G. P. Page, A. P. Sprague, B. S. Gorman, and D. B. Allison, Reproducible clusters from microarray research: whither? BMC Bioinformat. 2005; 6(suppl 2):S10. 125. S. Datta and S. Datta, Comparisons and validation of statistical clustering techniques for microarray gene expression data. Bioinformatics 2003; 19: 459–466. 126. P. Baldi and A. D. Long, A Bayesian framework for the analysis of microarray expression data: regularized t -test and statistical inferences of gene changes. Bioinformatics 2001; 17: 509–519.

MICROARRAY 127. D. B. Allison, X. Cui, G. P. Page, and M. Sabripour, Microarray data analysis: from disarray to consolidation and consensus. Nat. Rev. Genet. 2006; 7: 55–65. 128. X. Cui, J. T. Hwang, J. Qiu, N. J. Blades, and G. A. Churchill, Improved statistical tests for differential gene expression by shrinking variance components estimates. Biostatistics 2005; 6: 59–75. 129. P. H. Westfall, D. V. Zaykin, and S. S. Young, Multiple tests for genetic effects in association studies. Methods Molec. Biol. 2002; 184: 143–168. 130. Y. Benjamini, D. Drai, G. Elmer, N. Kafkafi, and I. Golani, Controlling the false discovery rate in behavior genetics research. Behav. Brain Res. 2001; 125: 279–284. 131. M. Kanehisa, S. Goto, M. Hattori, K.F. Aoki-Kinoshita, M. Itoh, S. Kawashima, T. Katayama, M. Araki, and M. Hirakawa, From genomics to chemical genomics: new developments in KEGG. Nucleic Acids Res. 2006; 34:D354–D357. 132. X. Mao, T. Cai, J. G. Olyarchuk, and L. Wei, Automated genome annotation and pathway identification using the KEGG Orthology (KO) as a controlled vocabulary. Bioinformatics 2005; 21: 3787–3793. 133. Y. Lee, R. Sultana, G. Pertea, J. Cho, S. Karamycheva, J. Tsai, B. Parvizi, F. Cheung, V. Antonescu, J. White, et al., Crossreferencing eukaryotic genomes: TIGR Orthologous Gene Alignments (TOGA). Genome Res. 2002; 12: 493–502. 134. Y. Lee, J. Tsai, S. Sunkara, S. Karamycheva, G. Pertea, R. Sultana, V. Antonescu, A. Chan, F. Cheung, and J. Quackenbush, The TIGR Gene Indices: clustering and assembling EST and known genes and integration with eukaryotic genomes. Nucleic Acids Res. 2005; 33:D71–D74. 135. L. Tanabe, U. Scherf, L. H. Smith, J. K. Lee, L. Hunter, and J. N. Weinstein, MedMiner: an Internet text-mining tool for biomedical information, with application to gene expression profiling. Biotechnology 1999; 27: 1210–1217. 136. M. Ashburner, C. A. Ball, J. A. Blake, D. Botstein, H. Butler, J. M. Cherry, A. P. Davis, K. Dolinski, S. S. Dwight, J. T. Eppig, et al., Gene ontology: tool for the unification of biology (In Process Citation). Nat. Genet. 2000; 25: 25–29. 137. P. Khatri and S. Draghici, Ontological analysis of gene expression data: current tools,

138.

139.

140.

141.

142.

143.

144.

145.

146.

147.

148.

149.

19

limitations, and open problems. Bioinformatics 2005; 21: 3587–3595. J. J. Goeman and P. Buhlmann, Analyzing gene expression data in terms of gene sets: methodological issues 2. Bioinformatics 2007; 23: 980–987. I. Rivals, L. Personnaz, L. Taing, and M. C. Potier, Enrichment or depletion of a GO category within a class of genes: which test? Bioinformatics 2007; 23: 401–407. D. B. Allison, X. Cui, G. P. Page, and M. Sabripour, Microarray data analysis: from disarray to consolidation and consensus. Nat. Rev. Genet. 2006; 7: 55–65. U. Mansmann and R. Meister, Testing differential gene expression in functional groups. Goeman’s global test versus an ANCOVA approach. Methods Inf. Med. 2005; 44: 449–453. VK. Mootha, CM. Lindgren, KF. Eriksson, A. Subramanian, S. Sihag, J. Lehar, P. Puigserver, E. Carlsson, M. Ridderstrale, E. Laurila, et al., PGC-1alpha-responsive genes involved in oxidative phosphorylation are coordinately downregulated in human diabetes. Nat. Genet. 2003; 34: 267–273. J. Tomfohr, J. Lu, and T. B. Kepler, Pathway level analysis of gene expression using singular value decomposition. BMC Bioinformat. 2005; 6: 225. J. J. Goeman, S. A. van de Geer, F. de Kort, and H. C. van Houwelingen, A global test for groups of genes: testing association with a clinical outcome. Bioinformatics 2004; 20: 93–99. J. J. Goeman, J. Oosting, A. M. CletonJansen, J. K. Anninga, H. C. van Houwelingen, Testing association of a pathway with survival using gene expression data. Bioinformatics 2005; 21: 1950–1957. P. Khatri and S. Draghici, Ontological analysis of gene expression data: current tools, limitations, and open problems. Bioinformatics 2005; 21: 3587–3595. P. Zimmermann, M. Hirsch-Hoffmann, L. Hennig, and W. Gruissem, GENEVESTIGATOR. Arabidopsis microarray database and analysis toolbox. Plant Physiol. 2004; 136: 2621–2632. F. A. de la, P. Brazhnik, and P. Mendes, Linking the genes: inferring quantitative gene networks from microarray data. Trends Genet. 2002; 18: 395–398. S. Imoto, T. Higuchi, T. Goto, K. Tashiro, S. Kuhara, and S. Miyano, Combining microarrays and biological knowledge for estimating

20

MICROARRAY

gene networks via Bayesian networks. J. Bioinform. Comput. Biol. 2004; 2: 77–98. 150. Z. S. Qin, L. A. McCue, W. Thompson, L. Mayerhofer, C. E. Lawrence, and J. S. Liu, Identification of co-regulated genes through Bayesian clustering of predicted regulatory binding sites. Nat. Biotechnol. 2003. 151. B. Xing and M. J. van der Laan, A statistical method for constructing transcriptional regulatory networks using gene expression and sequence data. J. Comput. Biol. 2005; 12: 229–246. 152. D. Murphy, Gene expression studies using microarrays: principles, problems, and prospects. Adv. Physiol. Educ. 2002; 26: 256–270. 153. R. Edgar, M. Domrachev, and A. E. Lash, Gene Expression Omnibus: NCBI gene expression and hybridization array data repository. Nucleic Acids Res. 2002; 30: 207–210. 154. R. Tuteja and N. Tuteja, Serial analysis of gene expression (SAGE): unraveling the bioinformatics tools. BioEssays 2004; 26: 916–922.

MINIMUM EFFECTIVE DOSE (MINED)

1 INDIVIDUAL AND POPULATION DOSE RESPONSE CURVES

NEAL THOMAS and NAITEE TING

Figure 1 distinguishes between individual dose-response relationships (the three thinner curves representing three different individual subjects) and the (single, thicker) population average dose-response relationship. Because of intersubject variability, different subjects may respond to the same drug in different ways (2). The definitions of the MinED in the statistics literature focus on the population dose-response curve. The most common definition of the MinED is the lowest dose that produces a specified average difference in response with placebo or other comparator treatment. In the case of a binary outcome, the specified average response is a proportion. Bretz, Hothorn, and Hsu (3) define a MinED as the lowest dose that achieves a specified ratio of the mean response divided by the comparator mean response. None of the available literature appear to define the MinED with time-toevent outcomes analyzed by methods such as Cox regression. Figure 2 displays a typical monotonically increasing dose response curve (left side of the plot) with a hypothetical MinED and MaxED indicated by vertical lines. The MaxED is placed at a dose that nearly achieves the maximum mean response. There does not appear to be a broadly accepted definition of the MaxED, but it is common in pharmacometric practice to target no more than 90% of the maximum effect because dose-response curves often increase slowly to the maximum effect, so much higher doses are required to yield small additional improvements in efficacy. The MinED is the dose that produces a minimum clinically important difference (MCID) from the control. The curve on the right side of Figure 2 is a population dose-response curve for some measure of toxicity. A somewhat optimistic setting is displayed, with toxicity increasing for doses above the MinED. Because of subject variability around the average dose response, even in this setting some patients may not achieve the MinED without experiencing

Global Research and Development Pfizer Inc. New London, Connecticut

The International Conference on Harmonization (ICH) E4 Guideline for Industry (1), the primary source of regulatory guidance for dose response studies, provides a concise definition of a minimum effective dose (MinED or MED): Historically, drugs have often been initially marketed at what were later recognized as excessive doses (i.e., doses well onto the plateau of the dose-response curve for the desired effect), sometimes with adverse consequences (e.g., hypokalemia and other metabolic disturbances with thiazide-type diuretics in hypertension). This situation has been improved by attempts to find the smallest dose with a discernible useful effect or a maximum dose beyond which no further beneficial effect is seen, but practical study designs do not exist to allow for precise determination of these doses. Further, expanding knowledge indicates that the concepts of minimum effective dose and maximum useful dose do not adequately account for individual differences and do not allow a comparison, at various doses, of both beneficial and undesirable effects. Any given dose provides a mixture of desirable and undesirable effects, with no single dose necessarily optimal for all patients.

The definition of the MinED as the ‘‘smallest dose with a discernible useful effect’’ requires that we further define a discernible effect and a useful effect. The MinED is implicitly defined by population summaries, as indicated by the latter portion of the ICH passage. Methodological development of the MinED has focused on operational definitions of ‘‘discernible’’ and ‘‘useful,’’ and the distinction between population and individual effects. Although less commonly discussed, the guidelines assign similar importance to establishing a ‘‘maximum useful dose,’’ which is sometimes called the maximum effective dose (MaxED).

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

MINIMUM EFFECTIVE DOSE (MINED)

Response

2

Dose

Response

Figure 1. Individual and average dose-response curves.

Toxicity

Efficacy

MCID

MinED

MoxED Dose

Figure 2. Dose-response for efficacy and toxicity.

adverse events. In less optimistic settings, the MinED cannot be evaluated because of toxicities occurring at lower doses.

2 OPERATIONAL DEFINITIONS OF USEFUL EFFECT It is difficult to achieve consensus about the smallest magnitude of an effect that is useful. The useful effect is often based on

precedent or convention within a therapeutic area. There can be different sources for the precedent, including the estimated effect of a well-established therapy that has been accepted as useful. For example, in studying chronic eye diseases such as age-related macular degeneration or diabetic macular edema, a loss (or gain) of three lines (or, equivalently, 15 letters) in vision based on the Early Treatment Diabetic Retinopathy Study (ETDRS) chart is a common standard for

MINIMUM EFFECTIVE DOSE (MINED)

visual acuity (4). Another example would be a two-point mean difference on the International Prostate Symptoms Scale for a useful effect for benign prostate hyperplasia (5). Another useful concept is the minimum clinically important difference (MCID). This is the magnitude of the treatment response exceeding the placebo response that is large enough to be perceived by a subject. It is a commonly used concept in outcomes research. There are two general methods to study this issue (6, 7): (1) the anchor-based method, which correlates response of a clinical endpoint to responses to questions about perceptions of improvement (or worsening); and (2) the distribution-based method, which calculates the between-person standard deviation unit and uses a multiple of this unit as the MCID. A similar problem arises when selecting a tolerance interval width for noninferiority and equivalence trials, which typically arise when comparing two active drugs rather than a drug and placebo. Achieving consensus on acceptable difference between active drugs is also difficult in this setting, as has been discussed elsewhere (8, 9). 3 OPERATIONAL DEFINITIONS OF DISCERNIBLE EFFECTS A discernible effect has been interpreted as a statistically significant difference between a dose group and placebo by several investigators. Ruberg (10) provides another way of restating the ‘‘smallest dose with a discernible useful effect’’: ‘‘The MinED is the lowest dose producing a clinically important response that can be declared statistically, significantly different from the placebo response.’’ This definition of a discernible effect involving statistical significance depends on the size and design of the dose-response study. Two general approaches have been used to establish statistical differences: (1) pairwise testing of doses versus placebo, with adjustment for multiple comparisons; and (2) dose response modeling, with reporting sometimes based on Bayesian probabilities rather than P-values. Sequential applications of trend tests corrected for multiple comparisons are a compromise between the pairwise

3

test-based approaches and estimation-based approaches. The testing-based approaches appear to be more commonly used. Pairwise comparisons of each dose group to placebo can be performed that preserve type I error without requiring any assumptions about the ordering of dose groups. To maintain a prespecified type I error, the alpha level of each test must be adjusted (multiple comparison adjustment). Dunnett (11) is a highly referenced method in this setting. Numerous other multiple comparison methods have been proposed that have improved power for some implicitly defined dose-response curves (12–14). These methods are based on sequential hypothesis testing. They typically begin with tests for the highest dose. The lowest statistically significant dose that is also clinically useful is selected as the MinED. Testing (and estimation) methods have also been developed that assume an increasing order of response with dose but do not assume a specific form for the doseresponse curve. These methods, such as that of Williams (15, 16), are called isotonic inference, isotonic regression, or order-restricted inference (10, 17–20). Tests based on contrasts (weighted combinations of dose group mean responses) derived from dose-response curves have also been proposed (10, 21, 22). These approaches, like those from isotonic inference, can increase power to achieve statistical significance by using information about the likely shape of the dose-response curve. The contrast tests are also implemented as sequential tests, which proceed as long as statistical significance is achieved. The MinED is derived from the contrast that is statistically significant, and includes the lowest maximum dose. The maximum dose from the selected contrast is the MinED. Bauer (23) and Hsu and Berger (13) note that these methods can have inflated type I error for an ineffective high dose when the dose-response curve is not monotonic. Hsu and Berger (13) show that sequential tests that preserve the familywise error rate are in the form of directional pairwise comparisons with the comparator group.

4

MINIMUM EFFECTIVE DOSE (MINED)

As noted in Tamhane et al. (22), all of the testing procedures select a MinED from only those doses included in the dose-response study design; as a consequence, they can be very dependent on the selection the doses included in the design. They noted this fact and offered a different naming convention: ‘‘what any test procedure finds is a minimum detectable dose (MinDD).’’ Most of the procedures noted here require higher doses to be statistically significant before lower doses can be declared significant, and, as a consequence, they tend to report higher values for the MinED. The MinED can also be selected by inversion of an estimated dose-response curve to find a dose with expected response equal to a target value, typically the smallest useful dose (24). The inversion of the dose-response curve may be preceded by a global test for a dose-response trend. A significant global trend test, however, is not equivalent to statistical significance of individual doses, as is typically required in most testing approaches. Bretz, Pinheiro, and Branson (25) describe several criteria for selecting a MinED based on inversion of the dose-response curve. The MinED estimated from a dose-response curve is likely to be a dose not included in the study. Ruberg (10) describes another method of estimating the MinED using the logistic Emax curve. This approach is based on methods for determining the minimum detectable concentration in biological assays (26, 27). Unlike all of the other methods noted here, this approach is based on selecting a dose that could be consistently differentiated from placebo in future trials of a specified size. REFERENCES 1. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E4 DoseResponse Information to Support Drug Registration. Current Step 4 version, March 10, 1994. Available at: http://www.ich.org/LOB/ media/MEDIA480.pdf 2. N. Holford and L. Sheiner, Understanding the dose-effect relationship: clinical application of pharmacokinetic–pharmacodynamic models. Clin Pharmacokinet. 1981; 6: 429–453.

3. F. Bretz, L. Hothorn, and J. Hsu, Identifying effective and/or safe doses by stepwise confidence intervals for ratios. Stat Med. 2003; 22: 847–858. 4. U.S. Food and Drug Administration, Center for Drug Evaluation and Research. Joint meeting of the ophthalmic drugs subcommittee of the dermatologic and ophthalmic drugs advisory committee and the endocrine and metabolic drugs advisory committee, March 11, 1998. http://www.fda.gov/cder/foi/ adcomm/98/jnt doac emdac 031198 ag ques. pdf 5. F. Desgrandchamps, Importance of individual response in symptom score evaluation. Eur Urol. 2001; 40(Suppl 3): 2–7. 6. G. H. Guyatt, D. Osoba, A. W. Wu, K. W. Wyrwich, G. R. Norman, and the Clinical significance consensus meeting group. Methods to explain the clinical significance of health status measures. Mayo Clin Proc. 2002; 77: 371–383. 7. M. A. G. Sprangers, C. M. Moinpour, T. J. Moynihan, D. L. Patrick, D. A. Revicki, and the Clinical Significance Consensus Meeting Group. Assessing meaningful change in quality of life over time: a users’ guide for clinicians, Mayo Clin Proc. 2002; 77: 561–571. 8. Committee for Medicinal Products for Human Use (CHMP), European Medicines Agency. Guideline on the Choice of the Non-Inferiority Margin. London, UK, July 27, 2005. Available at: http://www.emea.europa.eu/pdfs/human/ ewp/215899en.pdf 9. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E10 Choice of Control Group and Related Issues in Clinical Trials. Current Step 4 version, July 20, 2000. Available at: http://www.ich.org/LOB/media/ MEDIA486.pdf 10. S. Ruberg, Dose response studies. II. Analysis and interpretation. J Biopharm Stat. 1995; 5: 15–42. 11. C. Dunnett, A multiple comparison procedure for comparing several treatments to control. J Am Stat Assoc. 1955; 50: 1096–1121. 12. Y. Hochberg, and A. Tamhane, Multiple Comparison Procedures. New York: Wiley, 1987. 13. J. Hsu and R. Berger, Stepwise confidence intervals without multiplicity adjustment for dose-response and toxicity studies. J Am Stat Assoc. 1999; 94: 468–482. 14. A. Tamhane and B. Logan, Multiple comparison procedures in dose response studies. In:

MINIMUM EFFECTIVE DOSE (MINED) N. Ting (ed.), Dose-Finding in Drug Development. New York: Springer, 2006, pp 172–183. 15. D. Williams, A test for differences between treatment means when several dose levels are compared with a zero dose level. Biometrics. 1971; 27: 103–117. 16. D. Williams, The comparison of several dose levels with a zero dose control. Biometrics. 1972; 28: 519–531. 17. T. Robertson, F. Wright, and R. Dykstra, Order Restricted Inference, New York: Wiley, 1988. 18. D. J. Bartholomew, Isotonic inference. In: Encyclopedia of Statistical Sciences, Vol. 4. New York: Wiley, 1983, pp 260–265. 19. C. Hirotsu, Isotonic inference. In: Encyclopedia of Biostatistics, Vol. 3. New York: Wiley, 1999, pp 2107–2115. 20. J. W. McDonald, Isotonic regression. In: Encyclopedia of Biostatistics, Vol. 3. New York: Wiley, 1999, pp 2115–2116. 21. S. Ruberg, Contrasts for identifying the minimum effective dose. J Am Stat Assoc. 1989; 84: 816–822. 22. A. Tamhane, Y. Hochberg, and C. Dunnett, Multiple test procedures for dose finding. Biometrics. 1996; 52: 21–37. 23. P. Bauer, A note on multiple testing procedures in dose finding. Biometrics. 1997; 53: 1125–1128. 24. T. G. Filloon, Estimating the minimum therapeutically effective dose of a compound via regression modelling and percentile estimation (Disc: p933-933). Stat Med. 1995; 14: 925–932.

5

25. F. Bretz, J. Pinheiro, and M. Branson, Combining multiple comparisons and modeling techniques in dose-response studies. Biometrics. 2005; 61: 738–748. 26. M. Davidian, R. J. Carroll, and W. Smith, Variance functions and the minimum detectable concentration in assays. Biometrika. 1988; 75: 549–556. 27. D. Rodbard, Statistical estimation of the minimal detectable concentration (‘‘sensitivity’’) of radioligand assays. Anal Biochem. 1978; 90: 1–12.

CROSS-REFERENCES

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

R&D promotion (i.e., guardian) is an important feature of the Japanese system. This reflects some historical mishaps related to blood products and human immunodeficiency virus in the 1990s, in which the Ministry was severely criticized for its unclear decisionmaking process. Japan has universal health insurance, and all clinical trials submitted for investigational new drug notification (IND), with the exception of phase I studies involving healthy volunteers, must be done under the health insurance scheme. The financial rules for reimbursement and copayment in IND-submitted trials are determined by the Health Insurance Bureau of the MHLW. The MHLW is responsible for national policy making and final decisions on IND and NDA approvals. Day-to-day scientific evaluation of actual INDs and NDAs, safety review and pharmaco-vigilance after marketing approval, and good manufacturing practice (GMP) inspections are handled by an external agency, the Pharmaceuticals and Medical Devices Agency (PMDA).

SHUNSUKE ONO Graduate School of Pharmaceutical Sciences University of Japan Tokyo, Japan

As an important component of pharmaceutical research and development (R&D) activities, clinical trials in Japan are implemented under various regulations of the Pharmaceutical Affairs Law (PAL) and related ordinances and guidelines issued by the Ministry of Health, Labour and Welfare (MHLW). 1 MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) The Ministry of Health, Labour and Welfare (MHLW), which was originally established in 1938, is a government agency responsible for public health, social security, and labor issues in Japan. In 2001, two ministries, the Ministry of Health and Welfare and the Ministry of Labour, were merged into the current MHLW in an effort to reorganize and streamline government ministries. The MHLW consists of the Minister’s Secretariat, 11 bureaus, and affiliated institutions, councils, local branch offices, and external organizations. For the updated organizational structure, see the Ministry’s website at http://www.mhlw.go.jp/english/index.html (in English). Issues related to pharmaceuticals and medical devices are under the control of two bureaus in the MHLW, the Pharmaceutical and Food Safety Bureau (PFSB) and the Health Policy Bureau (HPB). The PFSB is the office in charge of enactment, amendment, and implementation of the Pharmaceutical Affairs Law (PAL), the backbone of Japanese pharmaceutical regulation. The objectives of the PFSB are the implementation of regulatory rules and related guidelines rather than the promotion of R&D policies. A division in the HPB, the Research and Development Division, is responsible for R&D policy issues in health care. The philosophy of strict separation between regulations (i.e., police) and

2 PHARMACEUTICALS AND MEDICAL DEVICES AGENCY (PMDA) The PMDA is an incorporated administrative agency established in 2004. It is not a government agency per se but rather was established based on the law for the Incorporated Administrative Agency. It is therefore a quasi-governmental agency, and its objectives and operations are specifically determined by the law and related ordinances. The PMDA took over the operations and services of three previous agencies: the Pharmaceuticals and Medical Devices Evaluation Center (PMDEC), the Organization for Pharmaceutical Safety and Research (OPSR), and part of the Japan Association for the Advancement of Medical Equipment (JAAME) (1). 2.1 Objectives of the PMDA The PMDA has three major objectives. 1. Consultation, review, and related activities. The PMDA conducts scientific

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

review for submitted NDAs and INDs, and conveys its review results to the MHLW, the final decision-making agency, in the form of review reports. It also offers consultation services for the industry and clinical researchers on a fee-for-service basis (see section 2.4). 2. Postmarketing safety operations. The PMDA collects, analyzes, and disseminates information on the quality, efficacy, and safety of pharmaceuticals and medical devices on the market. 3. Adverse health-effect relief services. The PMDA provides services related to payment for medical expenses and disability pensions to patients affected by adverse drug reactions. These services were originally performed by the OPSR. Before the PMDA was established in 2004, review and consultation were provided in two separate organizations. The PMDEC undertook scientific review activities to evaluate NDAs and INDs submitted to the MHLW, and the OPSR provided consultation services for the industry for predetermined fees. The NDA and IND review activities are official responsibilities, for which delegation to the PMDA is stipulated by the PAL, but the consultation services are not directly based on the PAL. This is in apparent contrast to the regulatory apparatus in the United States, where the sponsors can ask for various types of meetings as part of the overall review (2). However, the basic services that the regulatory agencies in both countries offer to sponsors (i.e., scientific advice on a timely basis) are the same.

The PMDA website (http://www.pmda. go.jp/) provides more detailed information on the scope of its operations. 2.2 Organization of the PMDA Efficacy, safety, and quality issues of pharmaceuticals (including over-the-counter drugs) and medical devices are handled by eleven review-related offices in the PMDA. The Offices of New Drugs I, II, III, and IV, and the Offices of Biologics I and II are the offices responsible for conducting scientific review for NDAs and INDs of pharmaceutical products. In each office, several review teams are organized, each focusing on specific therapeutic categories (Table 1). As of October 2007, there were ten review teams operating in the PMDA. A review team consists of project managers and experts in chemistry, pharmacology and pharmacokinetics, toxicology, physicians, and biostatisticians. The total number of review staff including the reviewers and support staff involved in review activities is about 200, one-tenth of that of the U.S. Food and Drug Administration (as of 2007). Although the human resources for review have been expanding gradually, there is still a significant gap between the United States and Japan. The lack of sufficient review resources has often been attributed to insufficient opportunities for consultation services and delay in new drug approval times. 2.3 IND and NDA Review by the PMDA For each IND and NDA, a specific review team is assigned at the time of submission. Assignment is based on the therapeutic

Table 1. Review offices and therapeutic fields in the Pharmaceuticals and Medical Devices Agency (PDMA) PMDA office

Therapeutic area or type of products

New Drugs I (three teams)

Anticancer, anti-infective, dermatology, additives for pharmaceutical products, and gastroenterology Cardiovascular disease, Alzheimer disease, Parkinson disease, obstetrics and gynecology, and urology Central and peripheral nervous system, and ophthalmology Respiratory disease, allergy, and autoimmune disease, Blood products, etc. Vaccines and tissue engineering

New Drugs II (two teams) New Drugs III (one team) New Drugs IV (two teams) Biologics I (one team) Biologics II (one team)

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

3

area and class of the drug. When biological products are assigned to a nonbiologic team, several reviewers from the Office of Biologics always join the review team to scrutinize the quality issues. The focus of review is different between INDs and NDAs. The IND review is to check the validity of initiating clinical trials, paying particular attention to safety concerns. The PAL requires that the IND review be finished within 30 days. The scope of the NDA review, on the other hand, is much broader, and it commonly takes one or more years for nonpriority review products. Figure 1 presents the review process for typical pharmaceutical NDAs. Review results prepared by review teams as well as the summary documents (i.e., Common Technical Documents [CTD] Module 2) submitted by the drug companies are published publicly on an Internet website in Japanese (http://www.info.pmda.go.jp/).

clinical development plans with Japanese regulators. Any sponsor (i.e., drug companies or physicians in sponsor-investigator trials) can apply for a consultation. Applications must be submitted 3 months ahead of the meeting. Due to insufficient review capacities with the agency, applications in crowded therapeutic areas are prioritized according to a point table. Some applicants with low priority points (e.g., a trial for an additional new indication) sometimes must wait several months in the queue. Though the PMDA explains that this ‘‘rationing’’ of services is inevitable with its current lack of sufficient review resources, the industry insists that this situation should be improved as soon as possible by adopting a fee system as in the U.S. Prescription Drug User Fee Act (PDUFA). Several types of consultations are offered to satisfy the needs in different development stages. Common times of consultation include before phase I, pre-early phase II, pre-late 2.4 Consultation Services Provided by the phase II, after phase II, and before NDA PMDA submission. The sponsors can ask for complementary consultation(s) for a lower fee if The PMDA provides various consultation serfurther discussions or advice are considered vices on a fee-for-service basis. The fees vary, necessary. depending on the type of consultation. For The sponsor and PMDA discuss design example, the fee for a consultation after a issues (e.g., choice of endpoints, sample size, phase II trial (i.e., the planning stage for statistical analysis), conduct issues (e.g., good phase III studies) is about 6 million yen (U.S. $55,000). The consultation services were started clinical practice [GCP] compliance), and all other issues pertinent to the clinical trials of in 1997 to meet the needs of the industry. interest. To clarify questions and concerns, Before that time, there were no official opportunities in which the industry could discuss preliminary discussions can be held several scientific and regulatory issues in specific times by facsimile and over the phone. The

PAFSC Special members Support

PAC & Committee

Support

Consult

PMDA Compliance review Team review

Figure 1. New Drug Application (NDA) review process in Japan.

NDA submission

Expert meeting

Interview

Hearing

Review Report

Interview

Applicant (Pharmaceutical company)

MHLW PMSB

Advise

Final decision on approval

Approval

4

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

PMDA and the sponsor commonly hold a final meeting to conclude the discussions. In cases in which all the questions and concerns are settled before the final meeting, the meeting can be canceled. The results of discussions are documented by the reviewers of the PMDA. The sponsor can comment on the results before they are finalized. All consultation result documents must be attached to an NDA submission in CTD Module 1. 3 REGULATORY STRUCTURE FOR CLINICAL TRIALS Most clinical trials in Japan, both industry sponsored and investigator sponsored, are done under the regulation of the PAL. For clinical trials outside the scope of the PAL, the MHLW issued a general ethics guideline for clinical research in 2003. 3.1 Definition of Clinical Trials The PAL requires submission of IND notification for a clinical trial that could be part of a clinical data package in an NDA. By definition, ‘‘sponsor’’ would include the physician who plans to conduct the clinical trials as the sponsor-investigator, but most of the trials for which IND notifications are submitted are sponsored by pharmaceutical companies. The regulatory scheme of sponsor-investigator trials was introduced comparatively recently. The PAL was amended to incorporate the definition of sponsor-investigator trials in

2002, and domestic GCP guidelines were also amended in 2003 to reflect the changes in the PAL. Before that, there were, of course, clinical trials implemented spontaneously by clinical researchers. However, they were not given official regulatory status and thus were not allowed to be used for Japanese NDA submissions. Since the introduction of the sponsor-investigator trials, 21 sponsor-investigator IND notifications have been submitted (as of November 2006). In contrast to the IND-related requirements in the United States, the definition of the Japanese IND requirement puts more emphasis on the intention for future NDA submission of pharmaceutical products. This leads to limited IND submissions from academic researchers, who are not necessarily interested in pharmaceutical NDAs. The number of recently submitted IND notifications is shown in Figure 2. The reasons for the drastic changes observed are explained in section 3.3. The PAL stipulates that IND notifications for substances for which clinical trials are done for the first time in Japan should be reviewed by the PMDA within 30 days. A review team assigned to the IND notification checks the IND; if they identify critical problems that could harm study participants, they issue an order to hold the trial. In such a case, the sponsor cannot start the trial until the problems are settled. This mechanism of clinical hold is similar to that of the United States.

1400 1200

First INDs All INDs

1000 800 600 400 200

Ye ar 19 93 19 94 19 95 19 96 19 97 19 98 19 99 20 00 20 01 20 02 20 03 20 04

0

Figure 2. Annual Investigational New Drug (IND) notifications submitted to the Ministry of Health, Labour and Welfare (MHLW).

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

3.2 GCP and Related Regulations in Japan The conduct of clinical trials for which IND notifications are submitted must satisfy the requirements of the GCP guideline and other pertinent ordinances. The Japanese GCP guideline is a Ministerial ordinance under the PAL, and serious violations of the guideline could be eligible for criminal punishment. It should be noted that the scope of penalties in the GCP is only for the sponsors; all of the other participants on the investigator’s side (e.g., physicians as investigators or subinvestigators, heads of hospitals, and research nurses) are exempt from the penalties. This exemption from punishment is an outstanding feature of Japanese GCP regulation. However, physicians could be punished in sponsor-investigator trials if a severe violation was committed when they played the role of the sponsor. The PMDA is responsible for implementation of on-site GCP inspections (see section 3.5). Lack of punishment on the investigator’s side makes it virtually impossible to publish a blacklist of investigators who have been involved in serious GCP violations in the past. 3.3 Drastic Changes in Japanese Clinical Trial Market As Figure 2 shows clearly, a drastic decline in the number of domestic commercial trials has been observed. This was caused by several factors on both the demand and supply sides (3). The MHLW’s health insurance policies in 1980s to 1990s, which included tightfisted price-setting rules for newly approved drugs and historical price cutting for existing drugs, seemed to have a negative impact on R&D activities in Japan (4). In addition to the economic disincentives, the globalization of R&D activities also has reduced the demand for Japanese trials. The International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) E5 guideline, called the ‘‘bridging guideline,’’ was accepted by the United States, the European Union, and Japan, and was implemented in 1998; this accelerated the trend toward global use of foreign clinical data (5). At the same time, the

5

MHLW abolished the longstanding requirements for the inclusion of Japanese pharmacokinetic studies, a Japanese dose-response study, and at least one Japanese confirmatory phase III study in the clinical data package. All these changes significantly reduced the demand for Japanese trials. Drastic changes also occurred on the supply side of clinical trials. After several years of discussions by the ICH expert working group, ICH-GCP was implemented in Japan in 1997. The new GCP guideline, for the first time in Japanese history, introduced Western components of clinical trials such as intensive monitoring and auditing through source data verification, support staff such as research nurses and pharmacists, contract research organizations, and independent data monitoring committees. These changes on the supply side, in conjunction with the changes on the demand side, caused a reduction in the number of trials in the Japanese market. The cost of Japanese trials skyrocketed accordingly. Most academic institutions and national hospitals adopted similar feesetting tables, but the fees are far from similar because overhead costs vary significantly among institutions (see section 3.4). 3.4 Expensive Japanese Trials for the Industry The Japanese pharmaceutical R&D environment, and the clinical environment in particular, has been discussed in light of launch delays in Japan. Several health-care providers have been associated with the delay in the background. Because virtually all Japanese clinical trials (except typical phase I trials) are done under the universal health insurance scheme, investigators as well as study participants face significant insurance-related red-tape situations in contracts, in-house procedures, and payment. For the sponsor, the cost of Japanese trials (i.e., payment from a drug company to a hospital) is a serious concern. It has been reported that the payments per subject are much higher in Japan than in the United States, European countries, and most Asian countries. In Japan, the sponsors are not allowed to pay investigators directly because the investigators (i.e., physicians) are regarded as employees of the hospital.

6

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

Instead, clinical trial fees go directly to the hospitals. As previously mentioned, in many public and academic hospitals, a clinical trial fee for each trial is determined based on a point-table and a matrix to calculate the fee basis, taking into consideration the design and conditions of the trial. The basic components of point-tables are similar nationwide, but actual fees vary to a great extent because of variable overhead costs. In general, hospitals have much a stronger power over the fee-setting process because they are the customers for the drug companies’ marketed products. The close tie between R&D activities and postmarketing business characterizes Japanese clinical development. Also, the limited availability of clinical experts in some therapeutic fields makes it possible for them to exert pricing powers as monopolists or oligopolists in providing clinical trials. All these features seem to be associated with the exceptionally high prices of clinical trials in Japan. 3.5 Quality of Japanese Trials Until the ICH-GCP was introduced, Japanese trials had a generally poor reputation for quality. Even before the ICH-GCP introduction, GCP guidance had been issued in 1989, but it was not a legally binding ordinance based on the PAL. Western components of clinical trials such as source data verification (SDV) monitoring and auditing, research nurses, and strict rules for informed consent were introduced in Japan for the first time in 1997, along with the ICH-GCP. The actual reports of GCP inspections are not publicly available, but the PMDA routinely publishes summary findings for educational purposes and provides materials on the quality of Japanese trials (6). Before the introduction of the ICH-GCP, the most obvious deficiencies were errors in case report forms (CRFs). Some errors were serious (e.g., fraudulent discrepancies from medical records), but most of them were trivial such as typographical errors. Surprisingly, SDV was not officially done before the ICH-GCP in Japan because there was no regulatory basis to allow SDV with appropriate protection of personal information. The ICH-GCP

made it possible for drug companies to conduct intensive monitoring through SDV. Also, research nurses and pharmacists employed at clinical institutions started operations to support investigators and drug companies. These improvements in the research environment reduced CRF deficiencies drastically and thus increased the accuracy of the data. Some types of deficiencies have not decreased, however. According to the published summary of inspection results, the number of protocol deviations has not declined since the ICH-GCP introduction (6). Japanese investigators participate in clinical trials as employees of hospitals and within the restrictions of health insurance regulations. Their motivation is not necessarily high because clinical trials have historically been considered low-priority chores that their supervisors (e.g., professors, directors) order them to do to acquire research grants from drug companies. Academic incentives are not effective for many physicians because they know Japanese commercial trials are rarely accepted in prestigious medical journals. This mindset has been changing, but only gradually. Recent discussions on trial quality often focus on burdens related to handling documents. For example, there has been widespread misinterpretation of the scope of essential documents. Too much attention to paper handling is likely to increase the cost of monitoring activities and the cost of clinical trials, accordingly. 4 GOVERNMENT EFFORTS TO INVITE NEW DRUGS TO JAPAN For several reasons, some drugs already marketed in the United States and European Union are not introduced in Japan (4). Even when they are finally introduced in Japan, significant delays in market introduction are quite common. This situation, called ‘‘new drug lag,’’ has attracted a great deal of public attention, especially in cancer therapy. That many drugs and therapeutic regimens already approved in Western countries are unavailable or are not approved in Japan has caused Japanese policy-makers to establish regulatory schemes to bring those drug therapies to Japan.

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

4.1 Study Group on Unapproved Drugs (MHLW since January 2005) The Study Group on Unapproved Drugs started in January 2005 to oversee the situation of Japanese new drug lag, to evaluate the need for introducing unapproved drugs in Japan, and to lead these drugs to the clinical development stage. Ten formal meetings have been held since the group’s establishment, and development and approval strategies for 33 drugs have been discussed (as of November 2006). Of the 33 drugs, 20 drugs were cancer drugs, and nine were pediatric drugs. The study group also makes recommendations about clinical trials for safety confirmation that should be performed for drugs that have already been submitted for NDA because such trials could provide opportunities for patients to access these unapproved drugs during the NDA review. Compassionate use of investigational or unapproved drugs has not been legitimized in Japan. 4.2 Study Group on Cancer Combination Therapy (MHLW since January 2004) The Study Group on Cancer Combination Therapy specifically focuses on unapproved use (e.g., indications) or unapproved regimens of cancer drugs for which Japanese approval has already been obtained for some other indication(s) or regimen. Because drug companies would have to bear substantial costs to conduct clinical trials to expand these indications, they would rather maintain the indications unchanged and leave the decision on how to use the drugs to individual oncologists. As a result, many cancer drugs have been used for unapproved conditions or in unapproved regimens in Japan. Lack of flexibility in Japanese health insurance reimbursement decisions makes the situation more serious. For pharmaceutical products, reimbursement is strictly based on the conditions (i.e., indication, dose, regimen, patient type) of approval under the PAL. Japanese insurance bodies basically are not allowed to make their own decisions on reimbursement under the universal health-care insurance, a marked contrast to the diversified decisions by the U.S. insurance bodies.

7

To fill the gap between Japan and the United States/European Union, this MHLW study group investigates the current Japanese situation and also collects clinical evidence available in foreign countries. When the investigation shows that a given drug use is considered ‘‘standard’’ in the therapeutic field, the MHLW issues its approval based on those prior investigations and foreign evidence. In 2005, expanded indications were approved for 30 cancer drugs. 5 GOVERNMENT EFFORTS TO PROMOTE CLINICAL TRIALS IN JAPAN The MHLW and related government agencies have been implementing strategies to boost the number and quality of Japanese clinical trials. 5.1 Three-Year Clinical Trial Promotion Plan (MHLW and MEXT, 2003–2005) The MHLW and the Ministry of Education, Culture, Sports, Science and Technology (MEXT) jointly executed a set of programs to promote Japanese clinical trials, Three-Year Clinical Trial Promotion Plan. Setting up several large networks for clinical trials was one of the important objectives of this plan. Research grants to establish networks and implement some sponsorinvestigator trials as model programs were managed by the Center for Clinical Trials of the Japan Medical Association. As of September 2006, there are 1,212 clinical institutions participating in the networks. Training programs for clinical research coordinators (e.g., research nurses and pharmacists) were provided regularly under this plan. Several symposiums were also held to raise public awareness of clinical trials because it is believed that Japanese patients who receive medical services under the universal health insurance without substantial financial burdens are not necessarily interested in clinical trials. The national hospitals and research centers directly managed by the MHLW have been promoting harmonization of trial contract documents. These streamlining efforts also are intended to reduce the administrative costs of trials.

8

MINISTRY OF HEALTH, LABOUR AND WELFARE (MHLW) OF JAPAN

The MHLW plans to continue these activities for the next 5 years beginning with the fiscal year 2007. 5.2 Study Group on Faster Access to Innovative Drugs (MHLW, 2006) The MHLW convened an expert panel, the Study Group on Faster Access to Innovative Drugs, to discuss the bottlenecks to the introduction of new drugs in Japan in October 2006. This study group will focus on the principles of new drug approval, postmarketing safety measures, and enhancement of the PMDA review. It is expected that the discussion results will be reflected in upcoming amendments of existing regulations. REFERENCES 1. Y. Fujiwara and K. Kobayashi, Oncology drug clinical development and approval in Japan: the role of the pharmaceuticals and medical devices evaluation center (PMDEC). Crit Rev Oncol Hematol. 2002; 42: 145–155. 2. Center for Drug Evaluation and Research, U.S. Food and Drug Administration. Training and Communications: Formal meetings between CDER and CDER’s external constituents. MAPP 4512.1. Available at: http://web.archive.org/web/20050921092516 /http://www.fda.gov/cder/mapp/4512-1.pdf 3. S. Ono and Y. Kodama, Clinical trials and the new good clinical practice guideline in Japan. Pharmacoeconomics. 2000; 18: 125–141.

4. P. M. Danzon, The impact of price regulation on the launch delay of new drugs—evidence from twenty-five major markets in the 1990s. Health Econ. 2005; 14: 269–292. 5. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E5(R1) Ethnic Factors in the Acceptability of Foreign Clinical Data. Step 4 version, February 5, 1998. Available at: http://www.ich.org/LOB/media/ MEDIA481.pdf 6. K. Saito, Y. Kodama, S. Ono, M. Mutoh, S. Kawashima, and A. Fujimura, Quality of Japanese trials estimated from Good Clinical Practice auditing findings. Am J Ther. 2006; 13: 127–133.

FURTHER READING For more detailed information about Japanese regulations, see Japan Pharmaceutical Manufacturers Association (JPMA), Pharmaceutical Administration and Regulations in Japan, March 2006. Available from http://www.jpma.or.jp/english/parj/0607.html

CROSS-REFERENCES Good clinical practice (GCP) International Conference on Harmonization (ICH) Investigational New Drug Application (IND) Regulatory authorities Food and Drug Administration (FDA)

MIN TEST

the name by Gleser (2), and was further studied by Berger (3–5) and others. Saikali and Berger (6) have pointed out that the Min test and the simple IUT are alternative names for the same test. Note that the elementary tests are performed at the same α level as the desired overall size of the test, and the test is valid even if the individual test statistics are correlated. Here, validity means that the probability of a type 1 error is at most α. The conclusion that the test has the claimed probability of a type 1 error follows only if inference is restricted to accepting or rejecting the specified global null and alternative hypotheses previously discussed. No consideration in calculating the size of the test is given to other errors that could arise if other inferences are drawn. For example, suppose that the global null is not rejected but one or more of the component tests reject their elementary null Hoi . If it were desired to reach conclusions about the corresponding parameters, it would be necessary to take into account the possibility of additional errors in order to control the familywise error rate. There are many ways to control the multiplicity problem but reporting on significant findings in the computation of the components of the Min test is not one of them. After failing to find min[ti − ciα ] ≥ 0 for all i, the only statistically valid statement is that there is insufficient evidence to conclude that all of the parameters are positive. The test statistics ti can be parametric, such as a t-test, or it could be nonparametric, such as a rank test. The parameters appearing in the hypotheses may represent a single outcome measure obtained under different experimental conditions, or the parameters could arise from a multivariate outcome. Both cases occur when testing whether a combination comprising several treatments is efficacious.

EUGENE M LASKA MORRIS J MEISNER Nathan Kline Institute for Psychiatric Research New York University School of Medicine Orangeburg, New York

Multiple parameter testing arises in many if not most clinical trials. There are very few diseases for which treatments are directed at a single targeted endpoint. In many cases, the evidence required to demonstrate that a new treatment is effective requires the simultaneous comparison of many outcome measures. This translates to formal hypotheses involving inequalities among multiple parameters. One common case is testing whether the parameters are all positive. This arises, for example, when it is desired to show that a combination treatment is superior to each component; that a treatment is equivalent or noninferior to a control; or that a dose of a combination is synergistic. In such cases, the Min test is the test of choice. We introduce this test as well as present and discuss its properties and illustrative applications. 1

THE MIN TEST

The Min test is used to test a null hypothesis such as H o : υ i ≤ 0 for at least one i verses H 1 : υ i > 0 for all i, i = 1, 2, . . . , K. This is sometimes called the sign-testing problem. If ti is an α-level test of υ i ≤ 0 that rejects H oi : υ i ≤ 0 if the statistic is larger than its critical value ciα , then the Min test rejects the global null hypothesis Ho if the min[ti − ciα ] > 0. Alternatively, if pi is the observed P-value of the i-th test statistic, then the Min test rejects the null if the largest value of pi < α. This test is also called the simple intersection union test (simple IUT or the SIUT) because the form of the null hypothesis is the union of K ‘‘elementary’’ hypotheses, and the alternative is the intersection of the complement of the elementary hypotheses. The simple IUT was first described by Lehman (1), was given

1.1 Power The statistical power (probability of rejecting the null hypothesis) of an α-level Min test for K = 2 depends on the primary parameter δ, the minimum of υ 1 and υ 2 , and on the nuisance parameter γ , the difference between

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

MIN TEST

υ 1 and υ 2 . If the parameters υ i are means, given any fixed value of δ, the power of the α-level Min test increases as a function of the absolute value of γ . Therefore, given any value of the primary parameter, the α-level Min test has the smallest power when the nuisance parameter is zero, that is, when υ1 = υ2. In determining the sample size for a study, in the absence of prior knowledge to the contrary, a prudent statistician will assume that the means are equal to ensure that there is sufficient power for rejecting the null hypothesis. For example, for two normal random variables with effect sizes υ 1 /σ = υ 2 /σ = 0.5, to achieve 90% power to reject the null, the sample size per group is 84 based on a t-test and 97 per group based on a Wilcoxon test. If υ 1 /σ were larger than 0.5, then γ would be positive, and the power of the Min test would be greater than 0.90. The size of a test, α, is the supremum of the power taken over all values of the parameters that lie in the null space. For the sign-testing problem, where the parameters are the means of a bivariate normal, the parameter values that define α occur in the limit of (υ 1 , 0) and (0, υ 2 ) as υ 1 or υ 2 approaches positive infinity. A test is said to be biased if there is a parameter point in the alternative hypothesis space (so that the null should be rejected) for which the power to reject is smaller than the size of the test. Thus, biased tests reject the null more frequently when the null is true than when it is false. Unfortunately, for parameters in the alternative space that lie in the neighborhood of the origin, the size of the Min test is very small. Therefore, the test is biased, and the power is low near the origin. The power of the Min test has been considered under different distributional assumptions by many investigators, including Laska and Meisner (7, 8), Patel (9), Hung et al. (10), Hung (11), Wang and Hung (12), Horn et al. (13) Sidik and Jonkman(14), and Kong et al. (15, 16). The tables or formulas in their publications may be consulted to obtain sample sizes for specific circumstances. 1.2 Optimality Properties If the test statistics are restricted to the class of monotone functions of the test statistics

of the elementary hypotheses (T 1 , T 2 ), then, under some mild regularity conditions, the uniformly most powerful test of Ho is the Min test (1, 7). A test is said to be monotone in (T 1 , T 2 ) if (a, b) are values of the test statistics that lie in the critical region implies that (a’, b’) also lies in the critical region whenever a ≥ a and b ≥ b. That is, larger values of the test statistic provide greater evidence in support of rejection. The sign-testing problem was considered by Inada (17) for bivariate normal and Sasabuchi (18) for multivariate normal; they found that the Min test is the likelihood ratio test (LRT) under the assumption of equal sample sizes per group and known variance. They specified the null as H o :{υ 1 = 0, υ 2 > 0} or {υ 1 > 0, υ 2 = 0} rather than H o : υ 1 ≤ 0 or υ 2 ≤ 0, but later Berger (4) showed that the LRT is the same for both formulations of the null. When the sample sizes are unequal, however, Saikali and Berger (6) showed that the two tests are not the same. The rejection region of the LRT is a proper subset of the rejection region of the Min test, which is therefore uniformly more powerful. However, the Min test is biased and has low power when the parameters are close to the origin, and the size of the test α is attained only in the limit. In an effort to remedy this situation, Berger (4), Liu and Berger (19), and McDermott and Wang (20) enlarged the rejection region near the origin without changing the α level and obtained tests that are uniformly more powerful than the Min test. But this result does not come without a price. In particular, the new tests are not monotone. Laska and Meisner (7) have argued that in a regulatory setting, such a property would be unacceptable. Imagine the consequences of one randomized, controlled clinical trial with less evidence for rejection of the null being declared a pivotal study by the regulatory body while another study with more evidence does not reach the nominal level of statistical significance. Additionally, these ‘‘improved’’ new tests all contain points in the rejection region that are arbitrarily close to the origin, which of course is not good evidence against the null hypothesis. Perlman and Wu (21) have argued strongly against the use of these new tests. Hung (22) has cautioned that their value in increasing power must be carefully

MIN TEST

weighed against the shortcomings already discussed. 2 COMMON APPLICATIONS OF THE MIN TEST The Min test is useful in many areas of application, but its most common use has been in testing whether a combination treatment is efficacious. Such treatments are widely used, and, even though their components may be effective when used individually, it does not follow that the combination consisting of these components is useful. To market a combination treatment, the U.S. Food and Drug Administration (U.S. FDA 21CFR300.50) requires demonstration that ‘‘each component makes a contribution to the claimed effects . . . ’’ (23, 24). In addition, the European Agency for the Evaluation of Medical Products (EMEA) also requires that the benefit/risk assessment of the combination equals or exceeds the ratio of each of its components (25). 2.1 Combination Treatments: Single Endpoint There are many ways to interpret the concept of ‘‘contribution.’’ If all of the components of the combination treat the same symptom, such as pain, then each component ‘‘contributes’’ is interpreted as requiring that the combination be superior to each of its components in terms of effectiveness. Clearly, if the combination is not superior to one of its ingredients, then, other issues such as adverse events aside, the combination has no advantage over the ingredient. Suppose there are K components in the combination treatment. Then here, υ i is the difference between the mean effect of the combination and the mean effect of component treatment i. If ti is a one-sided test that rejects H oi : υ i ≤ 0 if the statistic is larger than its critical value ciα , then the Min test may be used to demonstrate that the components of a combination contribute to the claimed effects. 2.2 Combination Treatments: Multiple Endpoints A combination treatment may be designed to treat many symptoms, such as pain and

3

sleeplessness. For simplicity, suppose there are two components in the combination treatment, each of which may have an effect on J outcome measures. Let υ ij be the difference between the mean effect of the combination and the mean effect of component treatment i on the j-th outcome measure. The notion of contribution has multiple interpretations. Laska et al. (26) gave the definitions for uniformly best, comparable to the best, and admissible, and Tamhane and Logan (27) provided the definition for locally best: 1. Uniformly best. The combination treatment is superior to each of its component treatments on all J endpoints. 2. Locally best. For each endpoint, the combination treatment is at least as good as (noninferior to) the component treatment that is best for that endpoint and superior to each treatment on at least one endpoint. 3. Comparable to the best. For each endpoint, the combination treatment is at least as good as (noninferior to) the component treatment that is best for that endpoint. 4. Admissible. The combination treatment is superior to each component treatment on at least one endpoint. Laska et al. (26) gave a general theory for these hypothesis formulations and applied the theory when the random variables are multivariate normal and when the tests of the elementary hypotheses are rank tests. If tij is a one-sided test that rejects Hoij : υ ij ≤ 0 when the statistic is larger than its critical value cijα , then the Min test may be used to demonstrate that the combination is uniformly best. Under mild conditions, it too is uniformly most powerful (UMP)among the class of monotone functions of the test statistics tij . Tamhane and Logan (27) describe a test for demonstrating locally best. The alternative hypothesis is the intersection of the superiority alternative and the noninferior alternative. For each outcome j, a noninferiority margin ej must be chosen to meaningfully reflect clinical indifferences. If sij is

4

MIN TEST

an α/J-level test of υ ij ≤ 0 that rejects H oij : υ ij ≤ 0 if the statistic is larger than its critical value c*ijα , and tij is an α-level test of υ ij − ≤ ej , that rejects Hoij : υij − ≤ ej if the statistic is larger than its critical value cijα , then the test rejects Ho if simultaneously the Min test given by min[tij − cijα ] and max[sij − c∗ ijα/J ] are both positive. The test uses Roy’s (28) union intersection (UI) principle and the Min test if the endpoint on which the combination is superior is not specified in advance. If j∗ is the endpoint specified in advance for testing superiority, then the test is a Min test composed of min[tij − cijα ]>0 and max[sij∗ − c∗ ij∗ α ] > 0. To test the hypothesis of comparable to the best, Laska et al. (26) show that the Min test once again is a UMP test among the class of monotone functions of the test statistics tij . The component test tij rejects Hoij : υ − ≤ ej if the corresponding test statistic is larger than its critical value. These same tests were later used by Tamhane and Logan (27) in the noninferiority portion of their test for locally best. For the admissibility hypothesis, Laska et al. (26) compare the combination with each component treatment to determine if there is at least one endpoint where it is superior to that component. Inference about each component i involves J comparisons, so the size of these J tests must be adjusted to achieve an α-level test for the component. The proposed adjustment was based on the Bonferroni method, so the test for component i is max[sij − c∗ ijα/J ] > 0. The test in Laska et al. (26) then is min[max[sij − c∗ ijα/J ]] > 0. Westfall et al. (29) observed that the power of the test could be improved by replacing Bonferroni with Holm (30) and still more by using the Simes (31) and Hommel (32) approaches instead of Bonferroni. It is interesting to note that some of the definitions of contribution listed above have inclusion relations. If a combination is uniformly best, then it is locally best. Similarly, a locally best combination is comparable to the best, and it is admissible. There is no inclusion relationship between the latter two. Thus, in any application, based on closed testing, the admissibility hypothesis may be tested first; and if it is not rejected, the locally

best hypothesis may be tested; and if it is not rejected, then uniformly best can be tested. These tests are all conducted at the stated α level without adjustment for multiplicity. Alternatively, the comparable to the best hypothesis may be tested; and if not rejected, the locally best hypothesis may be tested; and if not rejected, uniformly best may be tested. This may not be prudent unless a specific endpoint for locally best is chosen in advance; otherwise, power is lost in seeking the endpoint where the combination is best. This power loss occurs because part of the overall type 1 error in testing locally best is spent in the contrast for each of the endpoints. Thus, if the goal is to test for uniformly best, there is no cost to either first testing for admissibility or first testing for comparable to the best. 2.3 Combination Treatments: Multiple Doses, Univariate Effect Particularly for antihypertensive treatments but for other indications as well, both the United States and the European Union require that dose-ranging studies be conducted, preferably using a factorial design. In such a design, patients are randomly allocated to receive one of the doses of drug A and one of the doses of drug B, which includes the possibility of the zero dose, placebo. Hung et al. (33, 34) considered the statistical problem of identifying which dose combinations have the property that both components make a contribution. These investigators recognized that such a design also yields information on the dose response of the combination. Just as for combination drugs treating multiple symptoms, the definition of ‘‘contributing’’ has many interpretations. Hung et al. (33) described contributing in both a weak and a strong sense. A combination exhibits global superiority in a weak sense if the average effect of the combination taken over all of the nonzero doses is superior to the average effect of each of the component treatments taken over its corresponding individual non-zero doses. A combination exhibits global superiority in a strong sense if there is at least one non-zero dose combination that is superior to both of its components. Notice that demonstration of weak superiority does

MIN TEST

not guarantee that there is a dose combination that is superior to its components. Hung et al. (10) developed two tests for global superiority. The α-level AVE test is an average of the Min test statistics examining the contribution of each combination relative to its components taken over all of the non-zero dose combinations under study. The MAX test examines the largest Min test statistic. Both are one-sided α-level tests. Hung (35) extended the tests to incomplete and unbalanced factorial designs in which not all dose combinations are studied and the sample sizes of the dose pairs are not equal. For the local problem of identifying which explicit dose pairs are contributing, they recommend using simultaneous Min tests adjusted according to the Hochberg (36) procedure to maintain the familywise error rate at α. Alternative multiple testing methods that protect the familywise error rate for identifying effective dose combinations were investigated by Buchheister and Lehmacher (37) and Lehmacher (38). They proposed procedures based on linear contrast tests and on closed testing procedures. 2.4 Synergy In some contexts, it is desirable to determine whether the components of a combination are additive, synergistic, or antagonistic at specific doses. These concepts specify whether the effect of the combination is, respectively, the same as, greater than, or less than expected on the basis of the effects of its components (39). Another concept is the notion of therapeutic synergy. A combination is said to be therapeutically synergistic if its effect is larger than the maximum effect achievable by any of its components over all doses in its therapeutic range. Laska et al. (40) proposed a simple approach to assessing synergy in a combination of two drugs that does not require modeling a response surface or marginal doseresponse curves. Their experimental design and test are based on the concept of an isobologram (39) to establish sufficient conditions for synergy at a specified dose (x1 , x2 ) of the combination. An isobole is the set of dose pairs from the two treatments, all of which have the same expected responses.

5

The design calls for studying the combination and a single dose of each drug. Suppose that x1e and x2e are the doses of drug 1 and drug 2, respectively, that produce the same level of response, say e. The potency ratio at e is defined to be ρ(e) = x2e /x1e . In many instances, ρ(e) is well studied, and an estimate r is available. Then, to investigate whether the combination is synergistic at (x1 , x2 ), N subjects are randomly assigned to each of three dose combinations: (x1 + x2 /r, 0), (0, r x1 + x2 ), and (x1 , x2 ). The Min test is used to see if the following two conditions hold simultaneously: 1. The response at the combination dose (x1 , x2 ) is greater than the response to drug 1 alone at dose (x1 + x2 /r, 0). 2. The response at the combination dose of interest is greater than the response to drug 2 alone at dose (0, r x1 + x2 ). If both these conditions are true, then synergy can be claimed at the combination dose of interest. Recently, Feng and Kelly (41) generalized this approach to three or more drugs and studied the power of the procedure. To test if a combination is therapeutically synergistic at a given dose pair (x1 , x2 ), its effect must be shown to be superior to the maximum effect achieved by its components at any dose. Suppose that xe1∗ and xe2∗ are the known doses of drug 1 and drug 2 that produce the maximum level of response, say e1 * and e2 *, respectively. Then, component i at dose xei∗ and the combination must be studied, and the Min test is an optimal test. 2.5 Bioequivalence, Equivalence and Noninferiority Two formulations of the same drug are said to be bioequivalent if they are absorbed into the blood and become available at the drug action site at about the same rate and concentration. Suppose υ T = mean blood concentration of a test treatment and υ S = mean blood concentration of a standard treatment. Then, to demonstrate bioequivalence the null hypothesis H0 : υT − υS ≥ δ or υT − υS ≤ −δ must be rejected in favor of HA : − δ < υT − υS < δ

6

MIN TEST

which is equivalent to HA : − δ < υT − υS and υT − υS < δ Here, δ > 0 is a prespecified tolerance limit. More generally, this representation of equivalence may be used for any clinically relevant outcome measure where δ demarks a zone of clinical indifference. Bioequivalence refers to measures of blood concentration of the drug, whereas equivalence refers to outcomes that measure the impact on the therapeutic target. The Min test may be used to test the null hypothesis. In the bioequivalence context, the test is often referred to as the TOST because it is composed of two one-sided α-level tests (42). Kong et al. (15) considered a multivariate setting and studied a method for demonstrating equivalence or noninferiority of two treatments on each endpoint under the assumption of multivariate normality. The test statistic is a Min test that takes into account a noninferiority margin ej for each outcome j that reflects meaningful clinical indifference. The form of the test and the form for testing whether a combination treatment is uniformly best are the same. After developing the distributional properties, the investigators used a simulation to examine the power of the test under different configurations of the means and covariance matrix. More recently, Kong et al. (16) considered a similar scenario in which the endpoints are distributed binomially.

REFERENCES 1. E. L. Lehmann, Testing multiparameter hypotheses. Ann Math Stat. 1952; 23: 541–552. 2. L. J. Gleser, On a theory of intersection-union tests [abstract]. Inst Math Stat Bull. 1973; 2: 233. 3. R. L. Berger, Multiparameter hypothesis testing and acceptance sampling. Technometrics. 1982; 24: 295–300. 4. R. L. Berger, Uniformly more powerful tests for hypotheses concerning linear inequalities and normal means. J Am Stat Assoc. 1989; 84: 192–199.

5. R. L. Berger, Likelihood ratio tests and intersection-union tests In: S. Panchapakesan and N. Balakrishnan (eds.), Advances in Statistical Decision Theory and Applications Boston: Birhauser, 1997, pp. 225–237. 6. K. G. Saikali and R. L. Berger, More powerful tests for the sign testing problem. J Stat Plan Inference. 2002; 107: 187–205. 7. E. M. Laska and M. Meisner, Testing whether an identified treatment is best. Biometrics. 1989; 45: 1139–1151. 8. E. M. Laska and M. Meisner, Testing whether an identical treatment is best: the combination problem In: Proceedings of the Biopharmaceutical Section of the American Statistical Association Alexandria, VA: American Statistical Association, 1986, pp. 163–170. 9. H. I. Patel, Comparison of treatments in a combination therapy trial. J Biopharm Stat. 1991; 1: 171–183. 10. H. M. Hung, G. Y. Chi, and R. J. Lipicky, Testing for the existence of a desirable dose combination. Biometrics. 1993; 49: 85–94. 11. H. M. Hung, Two-stage tests for studying monotherapy and combination therapy in twoby-two factorial trials. Stat Med. 1993; 12: 645–660. 12. S. J. Wang and H. M. Hung, Large sample tests for binary outcomes in fixed-dose combination drug studies. Biometrics. 1997; 53: 498–503. 13. M. Horn, R. Vollandt, and C. W. Dunnett, Sample size determination for testing whether an identified treatment is best. Biometrics. 2000; 56: 879–881. 14. K. Sidik and J. N. Jonkman, Sample size determination in fixed dose combination drug studies. Pharm Stat. 2003; 2: 273–278. 15. L. Kong, R. C. Kohberger, and G. G. Koch, Type I error and power in noninferiority/equivalence trials with correlated multiple endpoints: an example from vaccine development trials. J Biopharm Stat. 2004; 14: 893–907. 16. L. Kong, G. G. Koch, T. Liu, and H. Wang, Performance of some multiple testing procedures to compare three doses of a test drug and placebo. Pharm Stat. 2005; 4: 25–35. 17. K. Inada, Some bivariate tests of composite hypotheses with restricted alternatives. Rep Fac Sci Kagoshima Univ (Math Phys Chem). 1978; 11: 25–31. 18. S. Sasabuchi, A test of a multivariate normal mean with composite hypotheses determined by linear inequalities. Biometrika. 1980; 67: 429–439.

MIN TEST 19. H. Liu and R. L. Berger, Uniformly more powerful, one sided tests for hypotheses about linear inequalities. Ann Stat. 1995; 72: 23–55. 20. M. P. McDermott and Y. Wang, Construction of uniformly more powerful tests for hypotheses about linear inequalities. J Stat Plan Inference. 2002; 107: 207–217. 21. M. D. Perlman and L. Wu, The emperor’s new tests (with discussion). Stat Sci. 1999; 14: 355–369. 22. H. M. Hung, Combination drug clinical trial In: S. C. Chow (ed.), Encyclopedia of Biopharmaceutical Statistics, Revised and Expanded 2nd ed. New York: Marcel Dekker, 2003, pp. 210–213. 23. U.S. Food and Drug Administration, Department of Health and Human Services. Code of Federal Regulations, Title 21 Food and Drugs. Volume 5, Part 300.50: Fixedcombination prescription drugs for humans. Revised as of April 1, 1999. Available at: http://www.access.gpo.gov/nara/cfr/waisidx 99 /21cfr300 99.html 24. H. M. Leung and R. O’Neill, Statistical assessment of combination drugs: a regulatory view In: Proceedings of the Biopharmaceutical Section of the American Statistical Association Alexandria, VA: American Statistical Association, 1986, pp. 33–36. 25. European Agency for the Evaluation of Medicinal Products, Human Medicines Evaluation Unit, Committee for Proprietary Medicinal Products (CPMP). Note for Guidance on Fixed Combination Medicinal Products. CPMP/EWP/240/95. April 17, 1996. Available at: http://www.emea.europa.eu/ pdfs/human/ewp/024095en.pdf 26. E. M. Laska, D. I. Tang, and M. Meisner, Testing hypotheses about an identified treatment when there are multiple endpoints. J Am Stat Assoc. 1992; 87: 825–831. 27. A. C. Tamhane and B. R. Logan, A superiority-equivalence approach to one-sided tests on multiple endpoints in clinical trials. Biometrika. 2004; 91: 715–727. 28. S. N. Roy, On a heuristic method of test construction and its use in multivariate analysis. Ann Math Stat. 1953; 24: 220–238. 29. P. H. Westfall, S. Y. Ho, and B. A. Prillaman, Properties of multiple intersection-union tests for multiple endpoints in combination therapy trials. J Biopharm Stat. 2001; 11: 125–138. 30. S. Holm, A simple sequentially rejective test procedure. Scand J Stat. 1979; 6: 65–70.

7

31. R. J. Simes, An improved Bonferroni procedure for multiple tests of significance. Biometrika. 1986; 73: 751–754. 32. G. Hommel, A comparison of two modified Bonferroni procedures. Biometrika. 1988; 75: 383–386. 33. H. M. Hung, T. H. Ng, G. Y. Chi, and R. J. Lipicky, Response surface and factorial designs for combination antihypertensive drugs. Drug Inf J. 1990; 24: 371–378. 34. H. M. Hung, G. Y. Chi, and R. J. Lipicky, On some statistical methods for analysis of combination drug studies. Commun Stat Theory Methods. 1994; A23: 361–376. 35. H. M. Hung, Evaluation of a combination drug with multiple doses in unbalanced factorial design clinical trials. Stat Med. 2000; 19: 2079–2087. 36. Y. Hochberg, A sharper Bonferroni procedure for multiple tests of significance. Biometrika. 1988; 75: 800–802. 37. B. Buchheister and W. Lehmacher, Establishing desirable dose combinations with multiple testing procedures In: N. Victor, M. Blettner, L. Edler, R. Haux, P. Knaup-Gregori, et al. (eds.), Medical Informatics, Biostatistics and Epidemiology for Efficient Health Care and Medical Research Munchen, Germany: Urban & Vogel, 1999, pp. 18–21. 38. B. Buchheister and W. Lehmacher, Multiple testing procedures for identifying desirable dose combinations in bifactorial designs. GMS Med Inform Biom Epidemiol. 2006; 2 (2): Dec07. 39. M. C. Berenbaum, What is synergy?. Pharmacol Rev. 1989; 41: 93–141. 40. E. M. Laska. M. Meisner, and C. Siegel, Simple designs and model-free tests for synergy. Biometrics. 1994; 50: 834–841. 41. P. Feng and C. Kelly, An extension of the model-free test to test synergy in multiple drug combinations. Biometric J. 2004; 3: 293–304. 42. R. L. Berger and J. C. Hsu, Bioequivalence trials, intersection-union tests and equivalence confidence sets. Stat Sci. 1996; 11: 283– 319.

FURTHER READING Office of Combination Products, U.S. Food and Drug Administration Website, at http://www.fda.gov/oc/combination/

8

MIN TEST

CROSS-REFERENCES Bioavailability Combination therapy Multiple comparisons Multiple endpoints Noninferiority

MISSING DATA

a nonrandom process is nonignorable. Thus, under ignorable dropout, one can literally ignore the missingness process and nevertheless obtain valid estimates of, say, the treatment. The above definitions are conditional on including the correct set of covariates into the model. An overview of the various mechanisms, and their (non-)ignorability under likelihood, Bayesian, or frequentist inference, is given in Table 1. Let us first consider the case in which only one follow-up measurement per patient is made. When dropout occurs in a patient, leaving the investigator without follow-up measures, one is usually forced to discard such a patient from analysis, thereby violating the intention to treat (ITT) principle, which stipulates that all randomized patients should be included in the primary analysis and according to the randomization scheme. Of course, the effect of treatment can be investigated under extreme assumptions, such as, for example, a worst-case and a best-case scenario, but such scenarios are most often not really helpful. The focus of this article will be on analysis techniques for repeated measurements studies. Early work regarding missingness focused on the consequences of the induced lack of balance of deviations from the study design (2, 3). Later, algorithmic developments took place, such as the expectation-maximization algorithm (EM) (4) and multiple imputation (5). These have brought likelihood-based ignorable analysis within reach of a large class of designs and models. However, they usually require extra programming in addition to available standard statistical software. In the meantime, however, clinical trial practice has put a strong emphasis on methods such as complete case analysis (CC) and last observation carried forward (LOCF) or other simple forms of imputation. Claimed advantages include computational simplicity, no need for a full longitudinal model analysis (e.g., when the scientific question is in terms of the last planned measurement occasion only), and for LOCF, compatibility

GEERT MOLENBERGHS Universiteit Hasselt Center for Statistics Diepenbeek, Belgium

EMMANUEL LESAFFRE Catholic University of Leuven Leuven, Belgium

1

INTRODUCTION

Data from longitudinal studies in general, and from clinical trials in particular, are prone to incompleteness. As incompleteness usually occurs for reasons outside of the control of the investigators and may be related to the outcome measurement of interest, it is generally necessary to reflect on the process governing incompleteness. Only in special but important cases is it possible to ignore the missingness process. When patients are examined repeatedly in a clinical trial, missing data can occur for various reasons and at various visits. When missing data result from patient dropout, the missing data have a monotone pattern. Nonmonotone missingness occurs when there are intermittent missing values as well. The focus here will be on dropout. Reasons typically encountered are adverse events, illness not related to study medication, uncooperative patient, protocol violation, ineffective study medication, loss to follow-up, and so on. When referring to the missing-value, or nonresponse, process, we will use the terminology of Little and Rubin (1). A nonresponse process is said to be missing completely at random (MCAR) if the missingness is independent of both unobserved and observed data and missing at random (MAR) if, conditional on the observed data, the missingness is independent of the unobserved measurements. A process that is neither MCAR nor MAR is termed nonrandom (MNAR). In the context of likelihood inference, and when the parameters describing the measurement process are functionally independent of the parameters describing the missingness process, MCAR and MAR are ignorable, whereas

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

MISSING DATA

with the ITT principle. However, a CC analysis assumes MCAR, and the LOCF analysis makes peculiar assumptions about the (unobserved) evolution of the response, underestimates the variability of the response, and ignores the fact that imputed values are no real data. On the other hand, a likelihood-based longitudinal analysis requires only MAR, uses all data (obviating the need for both deleting and filling in data), and is consistent with the ITT principle. Furthermore, it can also be shown that the incomplete sequences contribute to estimands of interest (treatment effect at the end of the study), even early dropouts. For continuous responses, the linear mixed model is popular and is a direct extension of analysis of variance (ANOVA) and MANOVA approaches, but more broadly valid in incomplete data settings. For categorical responses and count data, so-called marginal (e.g., generalized estimating equations, GEEs) and randomeffects (e.g., generalized linear mixed-effects models, GLMMs) approaches are in use. Although GLMM parameters can be fitted using maximum likelihood, the same is not true for the frequentist GEE method, but modifications have been proposed to accommodate the MAR assumption (6). Finally, MNAR missingness can never be fully ruled out based on the observed data only. It is argued that, rather than going either for discarding MNAR models entirely or for placing full faith on them, a sensible compromise is to make them a component of a sensitivity analysis. 2

METHODS IN COMMON USE

We will focus on two relatively simple methods that have been and still are in extensive use. A detailed account of simple methods to handle missingness is given in Verbeke and Molenberghs (7, 8).

2.1 Complete Case Analysis A complete case analysis includes only those cases for analysis for which all measurements were recorded. This method has obvious advantages. It is very simple to describe, and because the data structure is as would have resulted from a complete experiment, standard statistical software can be used without additional work. Furthermore as the entire estimation is performed on the same subset of completers, there is a common basis for inference. Unfortunately, the method suffers from severe drawbacks. First, there is nearly always a substantial loss of information. The impact on precision and power is dramatic. Furthermore, such an analysis will only be representative for patients who remain on study. Of course a complete case analysis could have a role as an auxiliary analysis, especially if a scientific question relates to it. A final important issue about a complete case analysis is that it is only valid when the missingness mechanism is MCAR. However, severe bias can result when the missingness mechanism is MAR but not MCAR. This bias can go both ways, i.e., either overestimating or underestimating the true effect. 2.2 Last Observation Carried Forward A method that has received a lot of attention (9–11) is the last observation carried forward (LOCF). As noted, in the LOCF method, whenever a value is missing, the last observed value is substituted. For the LOCF approach, the MCAR assumption is necessary but not sufficient for an unbiased estimate. Indeed, it further assumes that subjects’ responses would have been constant from the last observed value to the endpoint of the trial. These conditions seldom hold (8). In a clinical trial setting, one might believe that the response profile changes as soon as a patient goes off treatment and even

Table 1. Overview of Missing Data Mechanisms Acronym

Description

Likelihood/Bayesian

Frequentist

MCAR MAR MNAR

missing completely at random missing at random missing not at random

ignorable ignorable non-ignorable

ignorable non-ignorable non-ignorable

MISSING DATA

that it would flatten. However, the constant profile assumption is even stronger. Therefore, carrying observations forward may bias estimates of treatment effects and underestimate the associated standard errors (8, 12–16). Further more this method artificially increases the amount of information in the data, by treating imputed and actually observed values on equal footing. Despite its shortcomings, LOCF has been the longstanding method of choice for the primary analysis in clinical trials because of its simplicity, ease of implementation, and the belief that the potential bias from carrying observations forward leads to a ‘‘conservative’’ analysis in comparative trials. An analysis is called conservative when it leads to no treatment difference, whereas in fact there is a treatment difference. However, reports of anti-conservative or liberal behavior of LOCF are common (17–21), which means that a LOCF analysis can create a treatment effect when none exists. Thus, the statement that LOCF analysis has been used to provide a conservative estimate of treatment effect is unacceptable. Historically, an important motivation behind the simpler methods was their simplicity. Indeed, the main advantage, shared with complete case analysis, is that complete data software can be used. However, with the availability of commercial software tools, such as, for example, the SAS procedures MIXED and NLMIXED and the SPlus and R nlme libraries, this motivation no longer applies. It is often quoted that LOCF or CC, although problematic for parameter estimation, produces randomization-valid hypothesis testing, but this is questionable. First, in a CC analysis, partially observed data are selected out, with probabilities that may depend on post-randomization outcomes, thereby undermining any randomization justification. Second, if the focus is on one particular time point, e.g., the last one scheduled, then LOCF plugs in data. Such imputations, apart from artificially inflating the information content, may deviate in complicated ways from the underlying data (17). Third, although the size of a randomizationbased LOCF test may reach its nominal size under the null hypothesis of no difference

3

in treatment profiles, there will be other regions of the alternative space where the power the LOCF test procedure is equal to its size, which is completely unacceptable. 3 AN ALTERNATIVE APPROACH TO INCOMPLETE DATA A graphical illustration is first provided, using an artificial example, of the various simple methods that have been considered, and then so-called direct likelihood analysis is discussed. 3.1 Illustration of Simple Methods Take a look at an artificial but insightful example, depicted in Fig. 1, which displays the results of the traditional methods, CC and LOCF, next to the result of an MAR method. In this example, the mean response is supposed to be linear. For both groups (completers and dropouts), the slope is the same, but their intercepts differ. Patients with incomplete observations dropped out half way through the study; e.g., because they reached a certain level of the outcome. It is obviously an MAR missingness mechanism. Using a method, valid under the MAR assumption, yields the correct mean profile, being a straight line centered between the mean profiles of the completers and incompleters. If one would perform a CC analysis, the fitted profile would coincide with the mean profile of the complete cases (bold line). Next, under LOCF, data are imputed (dashed line). The resulting fitted profile will be the bold dashed line. Clearly, both traditional methods produce an incorrect result. Furthermore, in a traditional available case analysis (AC), one makes use of the information actually available. One such set of estimators could be the treatment-specific mean at several designed measurement occasions. With a decreasing sample size over time, means later in time would be calculated using less subjects than means earlier in time. Figure 1 shows a dramatic instance of this approach, due to the extreme nature of this illustrative example. The key message is that such an approach cannot remove major sources of bias.

MISSING DATA

10

4

8

9

Unobserved

7

LOCF ‘data’

6

LOCF

4

MAR

5

Inc.Obs.

2

3

AC

1

CC

0

Comp.Obs. 0

1

2

3

4

5

6

7

8

9

10

Figure 1. Artificial situation, illustrates the results of the traditional MCAR methods—CC and LOCF—next to the result of the direct likelihood method.

3.2 Direct Likelihood Analysis For continuous outcomes, Verbeke and Molenberghs (8) describe likelihood-based mixedeffects models, which are valid under the MAR assumption. Indeed, for longitudinal studies, where missing data are involved, a mixed model only requires that missing data are MAR. As opposed to the traditional techniques, mixed-effects models permit the inclusion of subjects with missing values at some time points (both dropout and intermittent missingness). This likelihood-based MAR analysis is also termed likelihood-based ignorable analysis or, as used in the remainder of this article, a direct likelihood analysis. In such a direct likelihood analysis, the observed data are used without deletion nor imputation. In so doing, appropriate adjustments are made to parameters at times when data are incomplete, due to the within-patient correlation. Thus, even when interest lies, for example, in a comparison between the two treatment groups at the last occasion, such a full longitudinal analysis is a good approach, because

the fitted model can be used as the basis for inference at the last occasion. In many clinical trials, the repeated measures are balanced in the sense that a common (and often limited) set of measurement times is considered for all subjects, which allows the a priori specification of a ‘‘saturated’’ model. For example, a full group-by-time interaction for the fixed effects combined with an unstructured covariance matrix. Such a model specification is sometimes termed mixed-effects model repeated-measures analysis (MMRM) (11). Thus, MMRM is a particular form of a linear mixed model, relevant for acute phase confirmatory clinical trials, fitting within the direct likelihood paradigm. Moreover, this direct likelihood MMRM analysis of variance (ANOVA) and multivariate analysis of variance (MANOVA) approaches, but more generally valid when they are incomplete. This response is a strong answer to the common criticism that a direct likelihood method is making strong assumptions. Indeed, its coincidence with MANOVA for data without missingness shows that the assumptions

MISSING DATA

made are very mild. Therefore, it constitutes a very promising alternative for CC and LOCF. When a relatively large number of measurements is made within a single subject, the full power of random effects modeling can be used (8). The practical implication is that a software module with likelihood estimation facilities and with the ability to handle incompletely observed subjects manipulates the correct likelihood, providing valid parameter estimates and likelihood ratio values. A few cautionary remarks are warranted. First, when at least part of the scientific interest is directed toward the nonresponse process, obviously both processes need to be considered. Under MAR, both questions can be answered separately, which implies that a conventional method can be used to study questions in terms of the outcomes of interest, such as treatment effect and time trend, whereafter a separate model can be considered to study missingness. Second, likelihood inference is often surrounded with references to the sampling distribution (e.g., to construct measures of precision for estimators and for statistical hypothesis tests (22)). However, the practical implication is that standard errors and associated tests, when based on the observed rather than the expected information matrix and given that the parametric assumptions are correct, are valid. Thirdy, it may be hard to rule out the operation of an MNAR mechanism. This point was brought up in Section 1 and will be discussed further in Section 5. 4 ILLUSTRATION: ORTHODONTIC GROWTH DATA As an example, we use the orthodontic growth data, introduced by Potthoff and Roy (23) and used by Jennrich and Schluchter (24). The data have the typical structure of a clinical trial and are simple yet illustrative. They contain growth measurements for 11 girls and 16 boys. For each subject, the distance from the center of the pituitary to the maxillary fissure was recorded at ages 8, 10, 12, and 14. Figure 2 presents the 27 individual profiles. Little and Rubin (1) deleted 9 of the [(11 + 16) × 4] measurements, rendering 9

5

incomplete subjects, which even though it is a somewhat unusual practice has the advantage of allowing a comparison between the incomplete data methods and the analysis of the original, complete data. Deletion is confined to the age 10 measurements, and rougly speaking, the complete observations at age 10 are those with a higher measurement at age 8. Some emphasis will be placed on ages 8 and 10, the typical dropout setting, with age 8 fully observed and age 10 partially missing. The simple methods and direct likelihood method from Sections 2 and 3 are now compared using the growth data. For this purpose, a linear mixed model is used, assuming unstructured mean, i.e., assuming a separate mean for each of the eight age × sex combinations, together with an unstructured covariance structure, and using maximum likelihood (ML) as well as restricted maximum likelihood (REML). The mean profiles of the linear mixed model using maximum likelihood for all four datasets, for boys, are given in Fig. (3). The girls’ profiles are similar and hence not shown. Next to this longitudinal approach, a full MANOVA analysis and a univariate ANOVA analysis will be considered, i.e., one per time point. For all of these analyses, Table 2 shows the estimates and standard errors for boys at ages 8 and 10, for the original data and all available incomplete data, as well as for the CC and the LOCF data. First, the group means for the boys in the original dataset in Fig. (3) are considered; i.e., relatively a straight line is observed. Clearly, there seems to be a linear trend in the mean profile. In a complete case analysis of the growth data, the 9 subjects that lack one measurement are deleted, resulting in a working dataset with 18 subjects. This result implies that 27 available measurements will not be used for analysis, a severe penalty on a relatively small dataset. Observing the profiles for the CC dataset in Fig. (3), all group means increased relative to the original dataset but mostly so at age 8. The net effect is that the profiles overestimate the average length. For the LOCF dataset, the 9 subjects that lack a measurement at age 10 are completed by imputing the age 8 value. It is clear

6

MISSING DATA

Orthodontic Growth Data Profiles and Means 34

Distance

30

26

22

18

14 6

8

10

12

14

16

Age in Years Figure 2. Orthodontic growth data. Raw and residual profiles. (Girls are indicated with solid lines. Boys are indicated with dashed lines.)

22

24

Distance

26

28

Original Data CC LOCF Direct Likelihood (Fitted) Direct Likelihood (Observed)

8

10

12

14

Age Figure 3. Orthodontic growth data. Profiles for the original data, CC, LOCF, and direct likelihood for boys.

MISSING DATA

that this procedure will affect the apparently increasing linear trend found for the original dataset. Indeed, the imputation procedure forces the means at ages 8 and 10 to be more similar, thereby destroying the linear relationship. Hence, a simple, intuitively appealing interpretation of the trends is made impossible. In case of direct likelihood, two profiles can now be observed: one for the observed means and one for the fitted means. These two coincide at all ages except age 10. As mentioned, the complete observations at age 10 are those with a higher measurement at age 8. Due to the within-subject correlation, they are the ones with a higher measurement at age 10 as well, and therefore, the fitted model corrects in the appropriate direction. The consequences of this are very important. Although it is believed that the fitted means do not follow the observed means all that well, this nevertheless is precisely what should be observed. Indeed, as the observed means are based on a nonrandom subset of the data, the

fitted means take into account all observed data points, as well as information on the observed data at age 8, through the measurements that have been taken for such children, at different time points. As an aside, note that, in case of direct likelihood, the observed average at age 10 coincides with the CC average, whereas the fitted average does not coincide with anything else. Indeed, if the model specification is correct, then a direct likelihood analysis produces a consistent estimator for the average profile, as if nobody had dropped out. Of course, this effect might be blurred in relatively small datasets due to small-sample variability. Irrespective of the small-sample behavior encountered here, the validity under MAR and the ease of implementation are good arguments that favor this direct likelihood analysis over other techniques. Now compare the different methods by means of Table 2, which shows the estimates and standard errors for boys at age 8 and 10, for the original data and all available

Table 2. Orthodontic Growth Data. Comparison of Analyses Based on Means at Completely Observed Age 8 and Incompletely Observed Age 10 Measurement Method

Boys at Age 8

Boys at Age 10

Original Data Direct likelihood, ML Direct likelihood, REML MANOVA ANOVA per time point

22.88 (0.56) 22.88 (0.58) 22.88 (0.58) 22.88 (0.61)

23.81 (0.49) 23.81 (0.51) 23.81 (0.51) 23.81 (0.53)

All Available Incomplete Data Direct likelihood, ML Direct likelihood, REML MANOVA ANOVA per time point

22.88 (0.56) 22.88 (0.58) 24.00 (0.48) 22.88 (0.61)

23.17 (0.68) 23.17 (0.71) 24.14 (0.66) 24.14 (0.74)

Complete Case Analysis Direct likelihood, ML Direct likelihood, REML MANOVA ANOVA per time point

24.00 (0.45) 24.00 (0.48) 24.00 (0.48) 24.00 (0.51)

24.14 (0.62) 24.14 (0.66) 24.14 (0.66) 24.14 (0.74)

Last Observation Carried Forward Analysis Direct likelihood, ML Direct likelihood, REML MANOVA ANOVA per time point

7

22.88 (0.56) 22.88 (0.58) 22.88 (0.58) 22.88 (0.61)

22.97 (0.65) 22.97 (0.68) 22.97 (0.68) 22.97 (0.72)

8

MISSING DATA

incomplete data, as well as for the CC data and the LOCF data. Table 2 shows some interesting features. In all four cases, a CC analysis gives an upward biased estimate, for both age groups. This result is obvious, because the complete observations at age 10 are those with a higher measurement at age 8, as shown before. The LOCF analysis gives a correct estimate for the average outcome for boys at age 8. This result is not surprising because there were no missing observations at this age. As noted, the estimate for boys of age 10 is biased downward. When the incomplete data are analyzed, we see from Table 2 that direct likelihood produces good estimates. The MANOVA and ANOVA per time point analyses give an overestimation of the average of age 10, like in the CC analysis. Furthermore, the MANOVA analysis also yields an overestimation of the average at age 8, again the same as in the CC analysis. Thus, direct likelihood shares the elegant and appealing features of ANOVA and MANOVA for fully observed data, but it is superior with incompletely observed profiles. 5

SENSITIVITY ANALYSIS

When there is residual doubt about the plausibility of MAR, one can conduct a sensitivity analysis. Although many proposals have been made, this is still an active area of research. Obviously, several MNAR models can be fitted, provided one is prepared to approach formal aspects of model comparison with due caution. Such analyses can be complemented with appropriate (global and/or local) influence analyses (25). Another route is to construct pattern-mixture models, where the measurement model is considered, conditional upon the observed dropout pattern, and to compare the conclusions with those obtained from the selection model framework, where the reverse factorization is used (26, 27). Alternative sensitivity analyses frameworks are provided by Robins, et al. (28), Forster and Smith (29) who present a Bayesian sensitivity analysis, and Raab and Donnelly (30). A further paradigm, useful for sensitivity analysis, is so-called shared parameter models, where common latent or

random effects drive both the measurement process as well as the process governing missingness (31, 32). Nevertheless, ignorable analyses may provide reasonably stable results, even when the assumption of MAR is violated, in the sense that such analyses constrain the behavior of the unseen data to be similar to that of the observed data. A discussion of this phenomenon in the survey context has been given in Rubin, et al. (33). These authors first argue that, in well-conducted experiments (some surveys and many confirmatory clinical trials), the assumption of MAR is often to be regarded as a realistic one. Second, and very important for confirmatory trials, an MAR analysis can be specified a priori without additional work relative to a situation with complete data. Third, although MNAR models are more general and explicitly incorporate the dropout mechanism, the inferences they produce are typically highly dependent on the untestable and often implicit assumptions built in regarding the distribution of the unobserved measurements given the observed ones. The quality of the fit to the observed data need not reflect at all the appropriateness of the implied structure governing the unobserved data. Based on these considerations, it is recommended, for primary analysis purposes, the use of ignorable likelihood-based methods or appropriately modified frequentist methods. To explore the impact of deviations from the MAR assumption on the conclusions, one should ideally conduct a sensitivity analysis (8). 6 CONCLUSION In conclusion, a direct likelihood analysis is preferable because it uses all available information, without the need neither to delete nor to impute measurements or entire subjects. It is theoretically justified whenever the missing data mechanism is MAR, which is a more relaxed assumption than MCAR, necessary for simple analyses (CC, LOCF). There is no statistical information distortion, because observations are neither removed (such as in CC analysis) nor added (such as in LOCF analysis). Software is available, such that no

MISSING DATA

additional programming is involved to perform a direct likelihood analysis. It is very important to realize that, for complete sets of data, direct likelihood, especially with the REML estimation method, is identical to MANOVA (see Table 2). Given the classic robustness of MANOVA, and its close agreement with ANOVA per time point, this provides an extra basis for direct likelihood. Indeed, it is not as assumption-driven as is sometimes believed. This, in addition with the validity of direct likelihood under MAR (and hence its divergence from MANOVA and ANOVA for incomplete data), provides a strong basis for the direct likelihood method. 7

ACKNOWLEDGMENTS

The authors gratefully acknowledge support from Fonds Wetenschappelijk OnderzoekVlaanderen Research Project G.0002.98 ‘‘Sensitivity Analysis for Incomplete and Coarse Data’’ and from Belgian IUAP/PAI network ‘‘Statistical Techniques and Modeling for Complex Substantive Questions with Complex Data.’’ We are thankful to Eli Lilly for kind permission to use their data. REFERENCES 1. R. J. A. Little, and D. B. Rubin, Statistical Analysis with Missing Data. New York: John Wiley & Sons, 2002. 2. A. Afifi, and R. Elashoff, Missing observations in multivariate statistics I: Review of the literature. J. Am. Stat. Assoc. 1996; 61: 595–604. 3. H. O. Hartley, and R. Hocking, The analysis of incomplete data. Biometrics, 1971; 27: 7783–7808. 4. A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. Roy. Stat. Soc. Series B, 1977; 39: 1–38. 5. D. B. Rubin, Multiple Imputation for Nonresponse in Surveys. New York: John Wiley & Sons, 1987. 6. J. M. Robins, A. Rotnitzky, and L. P. Zhao, Analysis of semiparametric regression models for repeated outcomes in the presence of missing data. J. Am. Stat. Assoc. 1995; 90: 106–121.

9

7. G. Verbeke, and G. Molenberghs, Linear Mixed Models in Practice: A SAS-Oriented Approach. Lecture Notes in Statistics 126. New York: Springer-Verlag, 1997. 8. G. Verbeke, and G. Molenberghs, Linear Mixed Models for Longitudinal Data. New York: Springer-Verlag, 2000. 9. O. Siddiqui, and M. W. Ali, A comparison of the random-effects pattern mixture model with last observation carried forward (LOCF) analysis in longitudinal clinical trials with dropouts. J. Biopharm. Stat. 1998; 8: 545–563. 10. C. H. Mallinckrodt, W. S. Clark, R. J. Carroll, and G. Molenberghs, Assessing response profiles from incomplete longitudinal clinical trial data under regulatory considerations. J. Biopharm. Stat. 2003; 13: 179–190. 11. C. H. Mallinckrodt, T. M. Sanger, S. Dube, D. J. Debrota, G. Molenberghs, R. J. Carroll, W. M. Zeigler Potter, and G. D. Tollefson, Assessing and interpreting treatment effects in longitudinal clinical trials with missing data. Biol. Psychiatry, 2003; 53: 754–760. 12. R. D. Gibbons, D. Hedeker, I. Elkin, D. Waternaux, H. C. Kraemer, J. B. Greenhouse, M. T. Shea, S. D. Imber, S. M. Sotsky, and J. T. Watkins. Some conceptual and statistical issues in analysis of longitudinal psychiatric data. Arch. Gen. Psychiatry, 1993; 50: 739–750. 13. A. Heyting, J. Tolboom, and J. Essers. Statistical handling of dropouts in longitudinal clinical trials. Stat. Med. 1992; 11: 2043–2061. 14. P. W. Lavori, R. Dawson, and D. Shera. A multiple imputation strategy for clinical trials with truncation of patient data. Stat. Med. 1995; 14: 1913–1925. 15. C. H. Mallinckrodt, W. S. Clark, and R. D. Stacy. Type I error rates from mixed-effects model repeated measures versus fixed effects analysis of variance with missing values imputed via last observation carried forward. Drug Inform. J. 2001; 35(4): 1215–1225. 16. C. H. Mallinckrodt, W. S. Clark, and R. D. Stacy. Accounting for dropout bias using mixed-effects models. J. Biopharm. Stat. 2001; 11(1 & 2): 9–21. 17. M. G. Kenward, S. Evans, J. Carpenter, and G. Molenberghs. Handling missing responses: Time to leave Last Observation Carried Forward (LOCF) behind, Submitted for publication. 18. G. Molenberghs, H. Thijs, I. Jansen, C. Beunckens, M. G. Kenward, C. Mallinckrodt,

10

19.

20.

21.

22.

23.

24.

MISSING DATA and R. J. Carroll. Analyzing incomplete longitudinal clinical trial data. Biostatistics, 2004; 5: 445–464. C. H. Mallinckrodt, J. G. Watkin, G. Molenberghs, and R. J. Carroll. Choice of the primary analysis in longitudinal clinical trials. Pharm. Stat. 2004; 3: 161–169. R. J. A. Little, and L. Yau. Intent-to-treat analysis in longitudinal studies with dropouts. Biometrics 1996; 52: 1324–1333. G. Liu and A. L. Gould. Comparison of alternative strategies for analysis of longitudinal trials with dropouts. J. Biopharm. Stat. 2002; 12: 207–226. M. G. Kenward, and G. Molenberghs, Likelihood based frequentist inference when data are missing at random. Stat. Sci. 1998; 12: 236–247. R. F. Potthoff, and S. N. Roy, A generalized multivariate analysis of variance model useful especially for growth curve problems. Biometrika 1964; 51: 313–326. R. I. Jennrich, and M. D. Schluchter, Unbalanced repeated measures models with structured covariance matrices. Biometrics 1986; 42: 805–820.

25. G. Verbeke, G. Molenberghs, H. Thijs, E. Lesaffre, and M. G. Kenward, Sensitivity analysis for non-random dropout: A local influence approach. Biometrics 2001; 57: 7–14. 26. B. Michiels, G. Molenberghs, L. Bijnens, T. Vangeneugden, and H. Thijs. Selection models and pattern-mixture models to analyze longitudinal quality of life data subject to dropout. Stat. Med. 2002; 21: 1023–1041.

27. H. Thijs, G. Molenberghs, B. Michiels, G. Verbeke, and D. Curran, Strategies to fit pattern-mixture models. Biostatistics 2002; 3: 245–265. 28. J. M. Robins, A. Rotnitzky, and D. O. Scharfstein, Semiparametric regression for repeated outcomes with non-ignorable non-response. J. Am. Stat. Assoc. 1998; 93: 1321–1339. 29. J. J. Forster, and P. W. Smith, Model-based inference for categorical survey data subject to non-ignorable non-response. J. Roy. Stat. Soc. Series B 1998; 60: 57–70. 30. G. M. Raab, and C. A. Donnelly, Information on sexual behaviour when some data are missing. Appl. Stat. 1999; 48: 117–133. 31. M. C. Wu, and K. R. Bailey, Estimation and comparison of changes in the presence of informative right censoring: Conditional linear model. Biometrics 1989; 45: 939–955. 32. M. C. Wu, and R. J. Carroll, Estimation and comparison of changes in the presence of informative right censoring by modeling the censoring process. Biometrics 1988; 44: 175–188. 33. D. B. Rubin, H. S. Stern, and V. Vehovar, Handling ‘‘don’t know’’ survey responses: The case of the Slovenian plebiscite. J. Am. Stat. Assoc. 1995; 90: 822–828.

MONITORING

however, in exceptional circumstances, the sponsor may determine that central monitoring in conjunction with procedures such as investigators’ training and meetings, and extensive written guidance can assure appropriate conduct of the trial in accordance with GCP. Statistically controlled sampling may be an acceptable method for selecting the data to be verified. The monitor(s), in accordance with the sponsor’s requirements, should ensure that the trial is conducted and documented properly by carrying out the following activities when relevant and necessary to the trial and the trial site:

The purposes of trial monitoring are to verify that: • The rights and well-being of human sub-

jects are protected. • The reported trial data are accurate,

complete, and verifiable from source documents. • The conduct of the trial is in compliance with the currently approved protocol/amendment(s), with Good Clinical Practice (GCP), and with applicable regulatory requirement(s).

• Acting as the main line of communica-

tion between the sponsor and the investigator. • Verifying that the investigator has adequate qualifications and resources and these remain adequate throughout the trial period, and that the staff and facilities, which include laboratories and equipment, are adequate to conduct the trial in a safe and proper manner, and these remain adequate throughout the trial period. • Verifying, for the investigational product(s): • That storage times and conditions are acceptable, and that supplies are sufficient throughout the trial. • That the investigational product(s) are supplied only to subjects who are eligible to receive it and at the protocol specified dose(s). • That subjects are provided with necessary instruction on properly using, handling, storing, and returning the investigational product(s). • That the receipt, use, and return of the investigational product(s) at the trial sites are controlled and documented adequately. • That the disposition of unused investigational product(s) at the trial sites complies with applicable regulatory

Selection and Qualifications of Monitors • Monitors should be appointed by the

sponsor. • Monitors should be trained appropri-

ately and should have the scientific and/ or clinical knowledge needed to monitor the trial adequately. A monitor’s qualifications should be documented. • Monitors should be thoroughly familiar with the investigational product(s), the protocol, the written informed consent form and any other written information to be provided to subjects, the sponsor’s Standard Operation Procedures (SOP), GCP, and the applicable regulatory requirement(s). The sponsor should ensure that the trials are adequately monitored. The sponsor should determine the appropriate extent and nature of monitoring. The determination of the extent and the nature of monitoring should be based on considerations such as the objective, purpose, design, complexity, blinding, size, and endpoints of the trial. In general, a need exists for on-site monitoring, before, during, and after the trial; This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

MONITORING

requirement(s) and is in accordance with the sponsor’s authorized procedures. • Verifying that the investigator follows

the approved protocol and all approved amendment(s), if any. • Verifying that written informed consent

was obtained before each subject’s participation in the trial. • Ensuring that the investigator receives

the current Investigator’s Brochure, all documents, and all trial supplies needed to conduct the trial properly and to comply with the applicable regulatory requirement(s). • Ensuring that the investigator and the

investigator’s trial staff are adequately informed about the trial. • Verifying that the investigator and the

investigator’s trial staff are performing the specified trial functions, in accordance with the protocol and any other written agreement between the sponsor and the investigator/institution, and have not delegated these functions to unauthorized individuals. • Verifying that the investigator is enroll-

ing only eligible subjects. • Reporting the subject recruitment rate. • Verifying that source data/documents

and other trial records are accurate, complete, kept up-to-date, and maintained. • Verifying that the investigator provides

all the required reports, notifications, applications, and submissions, and that these documents are accurate, complete, timely, legible, dated, and identify the trial. • Checking the accuracy and complete-

ness of the Case Report Form (CRF) entries, source data/documents, and other trial-related records against each other. The monitor specifically should verify that: • The data required by the protocol are reported accurately on the CRFs and are consistent with the source data/ documents.

• Any dose and/or therapy modifications

are well documented for each of the trial subjects. • Adverse events, concomitant medications, and intercurrent illnesses are reported in accordance with the protocol on the CRFs. • Visits that the subjects fail to make, tests that are not conducted, and examinations that are not performed are clearly reported as such on the CRFs. • All withdrawals and dropouts of enrolled subjects from the trial are reported and explained on the CRFs. • Informing the investigator of any CRF

entry error, omission, or illegibility. The monitor should ensure that appropriate corrections, additions, or deletions are made, dated, explained (if necessary), and initialed by the investigator or by a member of the investigator’s trial staff who is authorized to initial CRF changes for the investigator. This authorization should be documented. • Determining whether all adverse events (AEs) are appropriately reported within the time periods required by GCP, the protocol, the IRB (Institutional Review Board)/IEC (Independent Ethic’s Committee), the sponsor, the applicable regulatory requirement(s), and indicated in the International Conference on Harmonisation (ICH) Guideline for Clinical Safety Data Management: Definitions and Standards for Expedited Reporting. • Determining whether the investigator is maintaining the essential documents. • Communicating deviations from the protocol, SOPs, GCP, and the applicable regulatory requirements to the investigator and taking appropriate action designed to prevent recurrence of the detected deviations. The monitor(s) should follow the sponsor’s established written SOPs as well as those procedures that are specified by the sponsor for monitoring a specific trial.

MONITORING

Monitoring Report • The monitor should submit a written report to the sponsor after each trial-site visit or trial-related communication. • Reports should include the date, site, name of the monitor, and name of the investigator or other individual(s) contacted. • Reports should include a summary of what the monitor reviewed and the monitor’s statements concerning the significant findings/facts, deviations and deficiencies, conclusions, actions taken or to be taken, and/or actions recommended to secure compliance. • The review and follow-up of the monitoring report by the sponsor should be documented by the sponsor’s designated representative.

3

MONOTHERAPY

therapy with therapy delayed for an appropriate length of time such that every patient has the opportunity to receive the drug and yet a sufficient comparison period exists to assess the effect on a short-term outcome (as well as the opportunity to compare the effect of the different durations imposed by the delayed start of treatment on longer-term outcomes). The comparisons of immediate versus delayed therapy may also be appropriate when it is unclear when to start therapy, a concept explored in the Concorde trial (3). Once an effective therapy is available for a disease, it becomes much more difficult to assess new therapies as it is usually no longer ethical to compare them with an untreated group (which may receive placebo). However, this is not always the case; for example, it may be appropriate to evaluate a new bronchodilator for acute asthma by comparison with a placebo as long as treatment is available if therapy is not effective. An alternative way to evaluate certain new monotherapy regimens is in a crossover trial. However, this approach is only practicable if the disease is chronic and returns to a similar baseline when therapy is stopped and no prolonged effect of the therapy exists after it is stopped. To evaluate a new monotherapy when an existing therapy exists and an untreated group is not ethically acceptable, the randomized comparison is usually with the existing standard therapy. The new therapy may be expected to be more effective than the existing therapy or of similar efficacy but with less toxicity, and this fact will influence whether the trial is designed as a superiority or equivalence (or non-inferiority trial). In either case, the trial will need to be larger than if the new drug was compared with no treatment, as the difference between the efficacy of the new and old drug is likely to be much smaller than between drug and no treatment. An example of a monotherapy trial comparing a new drug with existing therapy is the evaluation of the second antiHIV drug didanosine (ddI), which was compared directly with ZDV (4). Such trials are, wherever possible, blinded (or masked) either by making the two drugs indistinguishable

JANET DARBYSHIRE London, United Kingdom

1

DEFINITION

A trial that evaluates a monotherapy would most often test a single drug, but the definition could be extended to include other interventions such as immunotherapy or even a different modality such as radiotherapy that contain a single element. The issues around trial design are essentially the same and, therefore, this section focuses on monotherapy trials that evaluate a drug. The simplest and most efficient way to evaluate a new agent is to compare it with no treatment or a placebo in a classic randomized parallel group design, assuming that its safety and activity have already been demonstrated in Phase I and II trials. If the drug is to be clinically useful, the effect compared with no treatment will be relatively large and, therefore, the trial may not need to include many patients. The duration of the trial will be driven by the disease, in particular its natural history, and the primary outcome measure. A classic example of a new monotherapy was the drug zidovudine (ZDV), the first drug to be evaluated for the treatment of HIV infection. In patients with advanced HIV infection, ZDV was shown to be substantially and significantly better than placebo in preventing death over a short period (1). However, subsequent trials, also placebo-controlled, of ZDV at a much earlier stage of HIV infection showed that the effect was short-lived because of the emergence of resistance to ZDV (2). In a highly fatal disease or a disease with a high incidence of other clear-cut clinical endpoints, the benefits of the first potent therapy may even be clearly demonstrated without a trial, the often quoted ‘‘penicillinlike’’ effect. In such circumstances, it may be difficult to withhold a new drug that looks promising from individuals, and one approach could be to compare immediate

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

MONOTHERAPY

or, if this is not possible, employing the double dummy technique. The selection of the comparison therapy is crucial. For example, if a new antibiotic is being evaluated, the choice of an inappropriate comparator may make the new drug appear better than it really is. Alternatively, if the comparator happens to be more potent against the likely infecting organisms, a drug that is useful in other infections may appear ineffective. In certain circumstances, it may be necessary to assess a new drug that will ultimately be used only as part of a combination therapy regimen. In these circumstances, the new drug may be given in addition to the conventional therapy (which itself may be a single therapy or a combination). Again, the basic comparison is with a group that only received the conventional therapy and, wherever possible, the trial is blinded or masked. Many examples of this comparison exist, such as the ESPRIT trial, which is evaluating Interleukin-2 (IL-2) on a background of antiretroviral therapy (5) and the first trial of a drug from a new class in HIV infection, ritonavir, a protease inhibitor (6). One of the difficulties of assessing a drug by adding it to or substituting it for one component of a combination regimen, is that additive, synergistic, or antagonistic activity may exist between the drugs. Further, an active new drug may add little to a potent treatment and therefore be rejected as inactive. New drugs that will never be used as monotherapy, such as in HIV and tuberculosis (TB), are sometimes evaluated for the first time as monotherapy by using short-term studies based on laboratory markers. These studies are used to determine whether the new drug has any activity ideally comparing them with other drugs concurrently or alternatively using historical controls. For both TB and HIV, issues around such designs exist because of concerns about the emergence of drug resistance, and therefore, innovative designs are needed to address this increasingly important issue. An efficient approach to the evaluation of two drugs, which also explores the effect of giving them together, is the factorial design. This design relies on the assumption that there will not be an interaction between

them. If a possibility of an interaction exists, whether positive or negative, then it may be an important finding, and the factorial design is still the optimal approach to evaluating it. However, the sample size will need to be sufficiently large to adequately assess interactions. If monotherapy regimens fail, a number of options exist that may need to be explored depending on the disease and the available drugs and issues such as cross-resistance or interactions. For example, in early HIV trials, a new drug was often added to ZDV or, alternatively, patients were switched from one monotherapy to another. No single optimal approach to decide how to use new drugs exists, but a clear need exists to consider all the relevant disease and drug-related factors. REFERENCES 1. M. A. Fischl, D. D. Richmann, M. H. Grieco, M. S. Gottlieb, P. A. Volberding, O. L. Laskin et al., The efficacy of azidothymidine (AZT) in the treatment of patients with AIDS and AIDS related complex. A double-blind, placebo controlled trial. N. Engl. J. Med. 1987; 317: 185– 191. 2. HIV Trialists’ Collaborative Group, Zidovudine, didanosine, and zalcitabine in the treatment of HIV infection: meta-analyses of the randomised evidence. Lancet 1999; 353: 2014–2025. 3. Concorde Coordinating Committee, Concorde: MRC/ANRS randomised double-blind controlled trial of immediate and deferred zidovudine in symptom-free HIV infection. Lancet 1994; 343: 871–882. 4. J. O. Kahn, S. W. Lagakos, D. D. Richman, A. Cross, C. Pettinelli, S-H. Liou et al., A controlled trial comparing continued zidovudine with didanosine in human immunodeficiency virus infection. N. Engl. J. Med. 1992; 327: 581–587. 5. S. Emery, D. I. Abrams, D. A. Cooper, J. H. Darbyshire, H. C. Lane, J. D. Lundgren, and J. D. Neaton, The evaluation of subcutaneous Proleukin (interleukin-2) in a randomized international trial: rationale, design, and methods of ESPRIT. Controlled Clin. Trials 2002; 23: 198–220. 6. D. W. Cameron, M. Heath-Chiozzi, S. Danner, C. Cohen, S. Kravcik, C. Maurath et al., Randomised placebo controlled trial of ritonavir in advanced HIV-1 disease. Lancet 1998; 351(9102): 543–549.

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

in developed countries and two trials of interventions feasible in resource-limited settings (Table 1). For each trial, the objectives, study design, results, and conclusions are summarized, along with implications for future patients and questions for additional study.

DAVID E. SHAPIRO Center for Biostatistics in AIDS Research, Harvard School of Public Health Boston, Massachusetts

1 THE PEDIATRIC AIDS CLINICAL TRIALS GROUP 076 TRIAL

Mother-to-child transmission (MTCT) of human immunodeficiency virus type 1 (HIV) can occur through three major routes: through the placental barrier during gestation (in utero and antepartum), via contact with maternal bodily fluids during labor and delivery (intrapartum), and by ingestion of breast milk after delivery (postpartum); most MTCT is believed to occur close to the time of or during childbirth (1). Infants who have a positive HIV test within the first 72 hours of life are presumed to have been infected in utero, and those who are HIV-negative within the first 72 hours and HIV-positive thereafter are presumed to have been infected close to or during delivery (or via early breastfeeding, if the infant breastfeeds). Prevention of MTCT (PMTCT) is one of the most successful areas in HIV clinical research. Interventions to prevent MTCT have been developed for two different settings: the United States and other developed countries with access to medications, medical infrastructure, and safe replacement feeding so that breastfeeding can be discouraged; and resource-limited countries with limited access to medications and clean water, and where breastfeeding is critical (2). In the absence of intervention, the risk of MTCT is approximately 15–25% during pregnancy through delivery (3), and an additional 0.9% per month during breastfeeding (4). With combination antiretroviral therapy during pregnancy and delivery, MTCT risk can be reduced to below 2% in the absence of breastfeeding (5,6); several clinical trials of interventions to reduce MTCT during breastfeeding are ongoing (7). This article describes four important PMTCT randomized trials, including two trials of key interventions used

The seminal Pediatric AIDS Clinical Trials Group (PACTG) 076 trial, which was sponsored by the U.S. National Institutes of Health (NIH), was the first Phase III trial to establish the efficacy of an antiretroviral drug to prevent MTCT of HIV. As of October 2007, the PACTG 076 zidovudine (ZDV) regimen is still recommended in the United States and other developed countries (8). 1.1 Objectives Based on animal models of retroviral infection, it was hypothesized that ZDV might prevent MTCT either by reducing the circulating HIV viral load in the mother and thereby reducing exposure of the fetus to HIV in utero and during delivery, or by accumulating therapeutic concentrations in the fetus and infant that could provide protection during and after exposure to HIV, or both (9). Phase I studies in pregnant women suggested that ZDV was safe when used for short periods and that it crossed the placenta well (9). The primary objectives of the trial were to assess the efficacy and safety of ZDV for the prevention of MTCT of HIV. 1.2 Study Design The PACTG 076 study was a randomized, double-blind, placebo-controlled trial conducted in the United States and France from 1991 to 1994. HIV-infected pregnant women, who were between 14- and 34-weeks gestation, had CD4+ T-lymphocyte counts exceeding 200/mm3 and no indications for antiretroviral therapy were enrolled. The study regimen consisted of oral ZDV five times daily during pregnancy, intravenous ZDV during labor and delivery, and oral ZDV

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

Table 1. Summary of the Study Interventions in Four Important PMTCT Randomized Trials Study (site; years enrolled) PACTG 076 (US, France; 1991–4)

EMDC (Europe; 1993–1998)

Antepartum

Intrapartum

Postpartum: mother

Arm 1: ZDV (oral, Arm 1: ZDV 5x/day from 14 (intravenous) weeks gestation) Arm 2: placebo Arm 2: placebo

No antiretrovirals

Nonstudy antiretrovirals (mainly ZDV)

Non-study Non-study antiretrovirals antiretrovi(mainly ZDV) rals (mainly ZDV)

Arm 1: elective cesarean delivery

Postpartum: infant Arm 1: ZDV (oral, 4x/day for 6 weeks) Arm 2: placebo

Arm 2: vaginal delivery HIVNET 012 (Uganda; 1997–1999)

No antiretrovirals

Arm 1: NVP (oral, single-dose)

No antiretrovirals

Arm 2: ZDV (oral) Arm 3: placebo (stopped Feb. 1998) MASHI (Botswana; ZDV (oral, 2x/day 2001–2003) from 34 weeks gestation)∗

∗ In

First No antiretrorandomization∗∗ : virals∗ NVP (oral, single-dose) versus placebo ZDV (oral)

Arm 1: NVP (oral, single dose on day 2–3 of age) Arm 2: ZDV (oral, 2x/day for 1 week) Arm 3: placebo (stopped Feb. 1998)

Second randomization: Breastfeeding with ZDV (oral, 3x/day until age 6 months) versus formula feeding NVP∗∗ (oral, single-dose) plus ZDV (oral, 2x/day for 4 weeks)

the revised design, women received combination antiretroviral therapy if required per Botswana national guidelines. the initial design, the first randomization was to maternal/infant single dose NVP versus maternal/infant placebo.

∗∗ In

to the infant for 6 weeks. The women were followed until 6 months postpartum, and the infants were followed until 18 months of age. The primary efficacy outcome measure was the MTCT rate at 18 months of age, as estimated by the Kaplan-Meier method to permit inclusion of infants who had incomplete follow-up. The target sample size was 748 mother–infant pairs (636 assessable) to provide 80% power to detect a reduction in MTCT from 30% in the placebo arm to 20% in the ZDV arm, with a two-sided, 0.05 Type I error. The trial was to be monitored by an independent Data and Safety Monitoring Board (DSMB) at least annually for study progress and safety; three interim efficacy analyses were planned, with an O’Brien– Fleming stopping boundary. In

February 1994, the first interim efficacy analysis revealed a dramatic reduction in MTCT in the ZDV arm. The interim results were released immediately, enrollment was discontinued, all study participants who received the blinded study drug were offered ZDV, and study follow-up was completed as planned (9). 2 RESULTS In the final efficacy analysis based on complete follow-up of 402 assessable mother– infant pairs, MTCT rates were 7.6% and 22.6% in the ZDV and placebo groups, respectively (10), which represented a 66% reduction in overall MTCT risk and confirmed

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

the results of the interim efficacy analysis (9). ZDV significantly reduced MTCT both in utero and intrapartum (11). However, ZDV reduced maternal viral load only slightly (median reduction, 0.24 log), and after adjustment for the baseline maternal viral load and CD4+ cell count, the reduction in viral load from baseline to delivery was not significantly associated with MTCT risk (10). ZDV was well tolerated and minimal shortterm toxic effects were observed, other than significantly lower hemoglobin levels in ZDVexposed infants during the first 6 weeks of life (9,12). No differences between groups were observed with respect to adverse pregnancy outcomes, uninfected infant growth and immune parameters, or HIV disease progression (12,13). The development of viral mutations conferring resistance to ZDV was rare (14). 2.1 Conclusions and Questions for Additional Study The PACTG 076 results suggested that a maternal–infant ZDV regimen could dramatically reduce MTCT of HIV with little short-term toxicity in pregnant women with mildly symptomatic HIV disease. Within 2 months of the release of the initial PACTG 076 results, the U.S. Public Health Service (USPHS) issued interim guidance to support the use of the PACTG 076 ZDV regimen, and 4 months later, they issued more extensive guidance (15,16). Epidemiologic studies have subsequently demonstrated large decreases in MTCT with incorporation of the PACTG 076 regimen into general clinical practice (8). Two important questions for additional study included the regimen’s long-term safety and whether it would be efficacious in women with more advanced HIV disease. Long-term follow-up studies of the PACTG 076 mothers (17) and infants (18,19) observed no major adverse effects of ZDV within the first few years after delivery, although subsequently a possible association of in utero ZDV or ZDV/lamivudine (3TC) exposure with mitochondrial dysfunction among uninfected infants has been found in some studies but not others (8). The efficacy of ZDV among women with more advanced HIV disease was demonstrated in PACTG 185, which was a randomized trial in the United States of passive

3

immunization for the prevention of MTCT of HIV among pregnant HIV-infected women who had CD4+ counts below 500/mm3 and who were receiving ZDV for maternal health. Enrollment was discontinued after the first interim efficacy analysis revealed an unexpectedly low MTCT rate of 4.8%, which substantially increased the sample size required to achieve the primary objective (20). Questions for additional study regarding modifications of the PACTG 076 regimen were in opposite directions in developed and resource-limited countries. In developed countries, the primary question was whether intensifying the PACTG 076 regimen by adding other antiretrovirals to reduce viral load even more would increase efficacy. A nonrandomized study in France showed that adding 3TC to the PACTG 076 regimen starting at 32-weeks gestation could reduce MTCT to below 2% (21), and other studies subsequently observed similar results with other combination antiretroviral regimens (5,6). In resource-limited settings, however, the PACTG 076 regimen was already too complex and expensive to be feasible, so the key question was whether the regimen could be shortened or simplified, yet still reduce MTCT risk. The relative importance of the maternal and infant components of the ZDV regimen could not be determined from PACTG 076 because mothers and infants received the same treatment assignment. Interestingly, a 2 × 2 factorial design, which could have addressed this question, was considered for PACTG 076 but ultimately rejected because of concerns that the maternal and infant components would not have independent influence on MTCT risk, which would reduce the statistical power of the factorial design (22). Subsequently, a randomized, placebo-controlled trial in Thailand found that a simplified, short-course (SC) maternal ZDV regimen (oral ZDV twice daily starting from 36-weeks gestation and oral ZDV during labor) with no infant treatment could reduce MTCT by 50%, from 19% to 9% (23). Another Thai trial with a 2 × 2 factorial design compared long versus short maternal ZDV (starting from 28- or 36-weeks gestation) and long versus short infant ZDV (for 6 weeks or 3 days). At the first interimefficacy analysis, the short–short arm was discontinued because it was inferior to the

4

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

long–long arm (10.5% versus 4.1% MTCT), but at the end of the trial the long–long, long–short, and short–long arms had equivalent MTCT (6.5% versus 4.7% versus 8.6%); the rate of in utero MTCT was higher with short versus long antepartum ZDV, which suggested that longer treatment of the infant cannot substitute for longer treatment of the mother (24). A trial in Africa among breastfeeding HIV-infected women showed that a short-course combination regimen of ZDV and 3TC starting from 36-weeks gestation, orally during delivery, and for 1 week after delivery to the mother and infant, reduced MTCT at age 6 weeks by approximately 63% compared with placebo (25). 3 THE EUROPEAN MODE OF DELIVERY TRIAL Because most MTCT is thought to occur during labor and delivery, interventions at the time of delivery could potentially reduce MTCT risk substantially, especially for women who have little or no prenatal care, are diagnosed as HIV-infected late in pregnancy, or have high viral load near the time of delivery. In the early 1990s, when results of some observational studies but not others suggested that elective cesarean-section (ECS) delivery before membrane rupture and before labor could reduce MTCT compared with vaginal delivery, the European Mode of Delivery Collaboration (EMDC) conducted a randomized trial that demonstrated the efficacy of ECS (26). 3.1 Objectives It was hypothesized that ECS might reduce MTCT risk by avoiding direct contact with maternal vaginal secretions and infected blood in the birth canal and by reducing influx of maternal blood during uterine contractions. The primary objectives of the EMDC trial were to assess the relative risks and benefits of ECS versus vaginal delivery overall and in subgroups defined according to ZDV use and viral load (26). 3.2 Study Design HIV-infected pregnant women who were at 34–36-weeks gestation and had no indication for or contraindication to ECS were

enrolled in Italy from 1993 to 1998, and in France, the United Kingdom, Spain, Switzerland, and Sweden from 1995 to 1998. Women were randomized to ECS at 38-weeks gestation or vaginal delivery. Women assigned to the ECS group who went into labor before 38 weeks-gestation were delivered by cesarean section if labor was diagnosed before the start of the second stage. Women assigned to vaginal delivery waited for spontaneous labor unless a clinical decision was made for cesarean section. The primary efficacy outcome measure was MTCT by age 18 months. The original planned sample size was about 450 women, based on an anticipated MTCT rate of 15% in the vaginal delivery group and an estimated 50% reduction associated with ECS. With the publication of the PACTG 076 results, the assumed MTCT rate in the vaginal delivery group was decreased to 8%, which increased the required sample size to 1200 women. No interim analyses were planned. However, in March 1998, when the initially planned sample size was reached, and with increasing evidence from observational studies of a protective effect of ECS that was greater than previously suggested, an interim efficacy analysis was conducted and enrollment was discontinued because of a significant difference in MTCT between the ECS and vaginal delivery groups (26). 3.3 Results In all, 70% of women in the ECS group and 58% in the vaginal-delivery group received antiretroviral therapy during pregnancy, generally the PACTG 076 ZDV regimen. Overall, 88.3% of the women assigned to the ECS group delivered by cesarean section (4.3% of which were emergency procedures), and 11.7% of women delivered vaginally. In this study, 73.2% of the women assigned to the vaginal delivery group delivered vaginally and 26.8% delivered by cesarean section (54% of which were emergency procedures). In an intent-to-treat analysis, MTCT rates were 1.8% in the ECS group and 10.5% in the vaginal delivery group (P < 0.001). MTCT rates according to actual mode of delivery were 2.4% with ECS, 10.2% with vaginal delivery, and 8.8% with emergency cesarean section (after membrane rupture or

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

onset of labor). Few postpartum complications occurred and no serious adverse events occurred in either group (26).

3.4 Conclusions and Questions for Additional Study The results of both the intent-to-treat and as-delivered analyses suggested that ECS significantly lowers MTCT without a significantly increased risk of complications in women who received no antiretroviral therapy or only ZDV during pregnancy. Interpretation was somewhat complicated, however, because more women in the ECS group received ZDV, and substantial numbers of women did not deliver according to their assigned mode. Nonetheless, after publication of the results of the EMDC trial and an individual patient data meta-analysis of 15 prospective cohort studies that also observed a protective effect of ECS (27), the American College of Obstetrics and Gynecology (ACOG) recommended that ECS be offered to all HIVinfected pregnant women, and the ECS rate among HIV-infected women in the United States and Europe increased substantially (28). Because the EMDC trial was conducted before the advent of viral-load testing and combination antiretroviral therapy, one important question for additional study was whether ECS would be worthwhile among women with low viral loads or who receive combination antiretroviral regimens. Subsequent observational studies have suggested that MTCT risk is very low in such women, and current ACOG and USPHS guidelines recommend ECS only for women with viral load greater than 1000 copies per milliliter, for whom benefits with respect to reduction of transmission risk generally outweigh the increased risk of maternal and infant morbidity and the cost of ECS delivery (28). Another unanswered question was how soon after onset of labor or rupture of membranes the benefit of ECS is lost. These questions are unlikely to be answered by randomized clinical trials because of the large sample sizes required (28).

5

4 THE HIV NETWORK FOR PREVENTION TRIALS 012 TRIAL The PACTG 076 ZDV regimen, ECS, and even short-course ZDV or ZDV/3TC regimens were too expensive or complex for resourcelimited countries, and MTCT via breastfeeding remained a problem. The landmark, NIHsponsored HIV Network for Prevention Trials (HIVNET) 012 trial demonstrated the efficacy of a very simple and inexpensive regimen: a single dose of nevirapine (SD-NVP) given to the mother during labor and to the infant within 72 hours after birth (29). 4.1 Objectives It was hypothesized that giving SD-NVP to the woman during labor could protect the infant from infection during delivery and during the first 1–2 weeks of life via breastfeeding, because NVP has potent antiviral activity, is rapidly absorbed when given orally, passes quickly through the placenta, and has a long half-life in pregnant women and babies (29). The primary objectives were to determine the safety and rates of MTCT and infant HIV-free survival after exposure to SD-NVP or SC-ZDV during labor and the first week of life. 4.2 Study Design The HIVNET 012 trial was conducted in Uganda and enrolled HIV-infected pregnant women who were at greater than 32-weeks gestation and not currently receiving antiretroviral or HIV immunotherapy from November 1997 to April 1999 (29). The trial was originally designed as a 1500 patient, double-blind, placebo-controlled trial to determine the efficacy of SD-NVP and SC-ZDV (during labor and to the infant for 1 week), but enrollment to the placebo arm was discontinued after release of the Thai shortcourse ZDV trial results (23), after only 49 women had been enrolled in HIVNET 012. Enrollment into the open-label SD-NVP and SC-ZDV arms was continued to provide preliminary screening data on efficacy to select one of the two regimens for inclusion in a redesigned, future Phase III efficacy trial, in which the comparator would be a standard antiretroviral regimen to be chosen based

6

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

on the anticipated results of other continuing perinatal trials. The sample size of 500 assessable mother–infant pairs was chosen to provide 80% probability to choose SD-NVP or SC-ZDV correctly if the true difference in MTCT rates between arms were 0% or 8%, respectively. The primary efficacy endpoints were MTCT and HIV-free survival at 6–8 weeks, 14–16 weeks, and 18 months of age. The study was monitored by an independent DSMB and interim-efficacy analyses (with an O’Brien–Fleming stopping boundary) were to be performed approximately annually during the projected 3-year study duration. Postpartum follow-up was originally planned to be 18 months for infants and 6 weeks for mothers, but it was subsequently extended to 5 years (30). 4.3 Results In the final efficacy analysis based on 617 assessable mother–infant pairs, the estimated risks of MTCT in the SD-NVP and SC-ZDV groups were 11.8% and 20.0% by age 6–8 weeks, respectively (P = 0.0063) (30). The cumulative MTCT rates in both groups increased by ages 14–16 weeks and 18 months because of continued breastfeeding, but the absolute and relative reductions in MTCT risk with SD-NVP (8.2% and 41%, respectively) were sustained through age 18 months. Results for HIV-free survival were similar. Both regimens were well tolerated with few serious side effects (30). Mutations in HIV that conferred resistance to NVP were detected at 6–8 weeks postpartum in 25% of mothers (31) and 46% of assessable HIV-infected infants exposed to NVP, but these mutations were no longer detected after 12–24 months postpartum (32). 4.4 Conclusions and Questions for Additional Study The HIVNET 012 results suggested that SD-NVP was efficacious and safe. In most resource-limited countries, especially in subSaharan Africa, national PMTCT programs subsequently were built around the HIVNET 012 regimen (33). Extensive controversy about whether the results of HIVNET 012 were valid developed in 2002, after Boehringer Ingelheim (BI),

which is the manufacturer of NVP, decided to pursue a U.S. Food and Drug Administration (FDA) labeling change to include PMTCT using HIVNET 012 as a registrational trial (34). As a result, reviews of the safety data that were far more in-depth than would ordinarily occur for a NIH-sponsored trial that was not intended to support an FDA submission were conducted, including a pre-FDA inspection audit by an NIH contractor who found some deficiencies in study conduct. The findings led to a comprehensive and lengthy remonitoring effort by NIH, withdrawal of BI’s FDA application because of inability to meet time constraints, and ultimately a U.S. Institutes of Medicine review of HIVNET 012, which concluded that no reason could be found to retract the publications or alter the conclusions of the trial (34). Two important areas for additional study suggested by the HIVNET 012 results included the efficacy of combining SD-NVP with other regimens and the implications and prevention of NVP resistance after SDNVP. The efficacy of combining SD-NVP with other regimens depends on their duration and potency. When the mother receives antiretrovirals during pregnancy, adding SDNVP can increase the efficacy of short-course regimens: The PHPT-2 trial in Thailand showed that adding SD-NVP to SC-ZDV in a nonbreastfeeding population could reduce the MTCT risk to 2% (35); when this combination regimen was used in a breastfeeding setting, a somewhat greater MTCT risk of 6.5% was observed (36). However, the PACTG 316 trial suggested that SD-NVP does not seem to provide any additional efficacy when added to the standard antiretroviral regimens used in developed countries (at a minimum, the full PACTG 076 ZDV regimen, often combined with at least two other antiretrovirals). The trial was stopped for futility because of low MTCT rates of 1.6% with placebo and 1.4% with SD-NVP (6). The efficacy of SD-NVP plus SC-ZDV in infants of mothers who did not receive antiretrovirals during pregnancy was assessed in two clinical trials in Malawi; SD-NVP plus SC-ZDV provided greater efficacy than SD-NVP alone when the mother did not receive any antiretrovirals during labor (37) but not when the mother received SD-NVP during labor (38).

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

The high prevalence of NVP resistance mutations after SD-NVP is of concern because it could compromise the effectiveness of (1) SD-NVP for prevention of MTCT in subsequent pregnancies and (2) future antiretroviral treatment for HIV disease, which, in resource-limited settings, often includes drugs in the same class [non-nucleoside reverse transcriptase inhibitor (NNRTI)] as NVP (39). Initial data from secondary analyses or follow-up studies of clinical trials suggest that SD-NVP remains effective in subsequent pregnancies and NNRTI-based treatment may still be effective after SDNVP exposure, particularly if sufficient time has elapsed; randomized trials of the latter question in women and children are in progress (39). One approach to reducing NVP resistance after SD-NVP would be to avoid the maternal NVP dose; in the perinatal component of the 2 × 2 factorial MASHI trial in Botswana, equivalent MTCT rates with and without the maternal NVP dose were observed when mothers received SCZDV and infants received SD-NVP and SCZDV (40). Another approach that has been studied is adding additional antiretrovirals during and after delivery, under the hypothesis that NVP resistance emerges because NVP remains present in subtherapeutic concentrations for several days or weeks because of its long half-life (41). A clinical trial in South Africa showed that adding ZDV/3TC for 3 or 7 days after delivery to cover the NVP ‘‘tail’’ can reduce the prevalence of NVP resistance mutations after SD-NVP but did not seem to have an effect on MTCT. Randomized trials are ongoing to assess whether ‘‘tail therapy’’ for longer durations or with more potent antiretrovirals could reduce the prevalence of NVP resistance even more (39). 5

THE MASHI TRIAL

Breastfeeding accounts for up to half of all MTCT in resource limited-settings, but it also provides important benefits, such as protection against other causes of infant mortality and morbidity when replacement feeding such as infant formula or animal milk is not safely available (e.g., because of a lack of clean water), culturally acceptable,

7

or affordable (7). As of October 2007, several clinical trials of antiretroviral, immunologic, or behavioral interventions to reduce breastfeeding MTCT are ongoing but only a few have been completed (7). The postpartum component of the NIH-funded MASHI trial (42) is described to illustrate some key issues in this active area of research. 5.1 Objectives A previous randomized clinical trial of breastfeeding versus formula feeding in Kenya found that formula feeding could prevent an estimated 44% of MTCT without leading to excess infant mortality, and it was therefore associated with improved HIV-free survival (43). However, study participants were required to have access to clean water, which is often limited outside urban areas, and it did not receive any antiretroviral treatment or prophylaxis, which could provide protection from MTCT during breastfeeding (42). The primary objective of the postpartum component of the MASHI trial was to compare the efficacy and safety of formula feeding versus breastfeeding plus extended infant ZDV for the prevention of postpartum MTCT. 5.2 Study Design The MASHI trial was a 2 × 2 factorial randomized trial that enrolled HIV-infected women at 33–35-weeks gestation in Botswana from March 2001 to October 2003 (42). All women received SC-ZDV during pregnancy and labor, and all infants received 1 month of ZDV. The two randomization factors were (1) SD-NVP versus placebo and (2) breastfeeding plus infant ZDV until 6 months of age versus formula feeding. Initially, mothers and infants were both randomized to SD-NVP or placebo; however, almost midway through enrollment, after release of the PHPT-2 trial results (35), the MASHI design was revised to give all infants SD-NVP and randomize only the mothers to SD-NVP or placebo (42). This modification coincided with the availability of antiretroviral therapy in Botswana for qualifying HIV patients, so the original and revised designs can be viewed as two 2 × 2 factorial trials. The primary efficacy outcome measures were MTCT by

8

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS

age 7 months and HIV-free survival by age 18 months, and the primary safety outcome measure was the rate of adverse events by 7 months of age. The planned sample size was 1,200 mother-infant pairs, to provide 80% power to detect a 7% difference in MTCT by age 7 months and 90% power to detect a 10% difference in HIV-free survival by 18 months of age, based on reference rates of 17% and 79%, respectively, a two-sided type I error of 0.05, and an annual loss-to-follow-up rate of 10%. The trial was monitored by an independent DSMB and two interim efficacy analyses were planned, with an O’Brien–Fleming stopping boundary (42). 5.3 Results The results of the SD-NVP component of the trial were described previously. Breastfeeding with infant ZDV was associated with higher MTCT (9% versus 5.6%, P = 0.04) but lower mortality (4.9% vs. 9.3%, P = 0.003) by age 7 months compared with formula feeding, which resulted in comparable HIV-free survival at 18 months (84.9% vs. 86.1%, P = 0.60). The most common causes of infant deaths were diarrhea and pneumonia. A statistically significant interaction occurred between the feeding strategies and the original maternal/infant SD-NVP/placebo intervention, with a greater increase in MTCT with breastfeeding plus ZDV and SDNVP (P = 0.04), and a greater decrease in mortality with breastfeeding plus ZDV and placebo (P = 0.03); no significant interaction was observed in the revised study (42). 5.4 Conclusions and Questions for Additional Study The results of the MASHI trial suggested that formula feeding was associated with a lower risk of MTCT but a higher risk of early mortality compared with breastfeeding plus infant ZDV; that is, formula-fed infants escaped HIV infection but then died of other causes, which led to similar rates of HIV-free survival at 18 months in the two groups (42). The MASHI study demonstrated the risks of formula feeding but did not definitively support infant ZDV to prevent breastfeeding MTCT, which highlighted the need for additional study (42).

Prevention of breastfeeding MTCT in a safe manner remains one of the major challenges and the most active areas in MTCT research. Several randomized trials are ongoing or planned to assess the efficacy and safety of other interventions to prevent postpartum MTCT without adversely affecting infant or maternal health, which includes antiretroviral and immune interventions to reduce MTCT risk during breastfeeding, and interventions to make replacement feeding safer (7). The issues are complex and tension often exists between what is best for the mother and what is best for the baby. For example, many pregnant and postpartum HIV-infected women are asymptomatic and have relatively high CD4+ cell counts, and therefore would not require antiretroviral therapy for their HIV disease if they were not pregnant or breastfeeding; the safety of giving such mothers combination antiretrovirals for the duration of breastfeeding is unknown (33). A preventive infant vaccine would be a very attractive approach to both reducing the risk of MTCT while providing nutrition and protection against other infectious causes of morbidity and mortality via breastfeeding (44), but scientists believe that a vaccine is most likely at least 10 years away (45). REFERENCES 1. Kourtis AP, Lee FK, Abrams EJ, Jamieson DJ, Bulterys M. Mother-to-child transmission of HIV-1: timing and implications for prevention. Lancet Infect Dis. 2006; 6: 726–732. 2. Minkoff H. For whom the bell tolls. Am. J. Obstet. Gynecol. 2007; 197:S1–S2. 3. De Cock KM, Fowler MG, Mercier E, de Vincenzi I, Saba J, Hoff E, Alnwick DJ, Rogers M, Shaffer N. Prevention of mother-to-child HIV transmission in resource-poor countries: translating research into policy and practice. JAMA 2000; 283: 1175–1182. 4. Breastfeeding and HIV International Transmission Study Group, Coutsoudis A, Dabis F, Fawzi W, Gaillard P, Haverkamp G, Harris DR, Jackson JB, Leroy V, Meda N, Msellati P, Newell ML, Nsuati R, Read JS, Wiktor S. Late postnatal transmission of HIV-1 in breast-fed children: an individual patient data metaanalysis. J. Infect. Dis. 2004; 189: 2154–2166.

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS 5. Cooper ER, Charurat M, Mofenson L, Hanson IC, Pitt J, Diaz C, Hayani K, Handelsman E, Smeriglio V, Hoff R, Blattner W; Women and Infants’ Transmission Study Group. Combination antiretroviral strategies for the treatment of pregnant HIV-1-infected women and prevention of perinatal HIV-1 transmission. J. Acquir. Immune Defic. Syndr. 2002; 29: 484–494. 6. Dorenbaum A, Cunningham CK, Gelber RD, Culnane M, Mofenson L, Britto P, Rekacewicz C, Newell ML, Delfraissy JF, CunninghamSchrader B, Mirochnick M, Sullivan JL; International PACTG 316 Team. Two-dose intrapartum/newborn nevirapine and standard antiretroviral therapy to reduce perinatal HIV transmission: a randomized trial. JAMA 2002; 288: 189–198. 7. Kourtis AP, Jamieson DJ, de Vincenzi I, Taylor A, Thigpen MC, Dao H, Farley T, Fowler MG. Prevention of human immunodeficiency virus-1 transmission to the infant through breastfeeding: new developments. Am. J. Obstet. Gynecol. 2007; 197(Suppl 3):S113–S122. 8. Public Health Service Task Force. Recommendations for use of antiretroviral drugs in pregnant HIV-1 infected women for maternal health and interventions to reduce perinatal HIV-1 transmission in the United States. October 12, 2006 update. http://AIDSinfo.nih.gov. 9. Connor EM, Sperling RS, Gelber R, Kiselev P, Scott G, O’Sullivan MJ, VanDyke R, Bey M, Shearer W, Jacobson RL, Jimenez E, O’Neill E, Bazin B, Delfraissy J-F, Culnane M, Coombs R, Elkins M, Moye J, Stratton P, Balsley J; Pediatric AIDS Clinical Trials Group Protocol 076 Study Group. Reduction of maternal-infant transmission of human immunodeficiency virus type 1 with zidovudine treatment. Pediatric AIDS Clinical Trials Group Protocol 076 Study Group. N. Engl. J. Med. 1994; 331: 1173–1180. 10. Sperling RS, Shapiro DE, Coombs RW, Todd JA, Herman SA, McSherry GD, O’Sullivan MJ, Van Dyke RB, Jimenez E, Rouzioux C, Flynn PM, Sullivan JL. Maternal viral load, zidovudine treatment, and the risk of transmission of human immunodeficiency virus type 1 from mother to infant. Pediatric AIDS Clinical Trials Group Protocol 076 Study Group. N. Engl. J. Med. 1996; 335: 1621–1629. 11. Shapiro DE, Sperling RS, Coombs RW. Effect of zidovudine on perinatal HIV-1 transmission

12.

13.

14.

15.

16.

17.

18.

9

and maternal viral load. Pediatric AIDS Clinical Trials Group 076 Study Group. Lancet. 1999; 354: 156; author reply 157–158. Sperling RS, Shapiro DE, McSherry GD, Britto P, Cunningham BE, Culnane M, Coombs RW, Scott G, Van Dyke RB, Shearer WT, Jimenez E, Diaz C, Harrison DD, Delfraissy JF. Safety of the maternal-infant zidovudine regimen utilized in the Pediatric AIDS Clinical Trial Group 076 Study. AIDS. 1998; 12: 1805–1813. McSherry GD, Shapiro DE, Coombs RW, McGrath N, Frenkel LM, Britto P, Culnane M, Sperling RS. The effects of zidovudine in the subset of infants infected with human immunodeficiency virus type-1 (Pediatric AIDS Clinical Trials Group Protocol 076). J. Pediatr. 1999; 134: 717–724. Eastman PS, Shapiro DE, Coombs RW, Frenkel LM, McSherry GD, Britto P, Herman SA, Sperling RS. Maternal viral genotypic zidovudine resistance and infrequent failure of zidovudine therapy to prevent perinatal transmission of human immunodeficiency virus type 1 in pediatric AIDS Clinical Trials Group Protocol 076. J. Infect. Dis. 1998; 177: 557–564. Jamieson DJ, Clark J, Kourtis AP, Taylor AW, Lampe MA, Fowler MG, Mofenson LM. Recommendations for human immunodeficiency virus screening, prophylaxis, and treatment for pregnant women in the United States. Am. J. Obstet. Gynecol. 2007; 197(3 Suppl):S26–S32. Centers for Disease Control and Prevention. Recommendations of the US Public Health Service Task Force on the use of zidovudine to reduce perinatal transmission of human immunodeficiency virus. MMWR Recomm. Rep. 1994; 43: 1–20. Bardeguez AD, Shapiro DE, Mofenson LM, Coombs R, Frenkel LM, Fowler MG, Huang S, Sperling RS, Cunningham B, Gandia J, Maupin R, Zorrilla CD, Jones T, O’Sullivan MJ; Pediatrics AIDS Clinical Trials Group 288 Protocol Team. Effect of cessation of zidovudine prophylaxis to reduce vertical transmission on maternal HIV disease progression and survival. J. Acquir. Immune Defic. Syndr. 2003; 32: 170–181. Culnane M, Fowler M, Lee SS, McSherry G, Brady M, O’Donnell K, Mofenson L, Gortmaker SL, Shapiro DE, Scott G, Jimenez E, Moore EC, Diaz C, Flynn PM, Cunningham B, Oleske J. Lack of long-term effects of in utero exposure to zidovudine among uninfected children born to HIV-infected women. Pediatric

10

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS AIDS Clinical Trials Group Protocol 219/076 Teams. JAMA. 1999; 281: 151–157.

19. Hanson IC, Antonelli TA, Sperling RS, Oleske JM, Cooper E, Culnane M, Fowler MG, Kalish LA, Lee SS, McSherry G, Mofenson L, Shapiro DE. Lack of tumors in infants with perinatal HIV-1 exposure and fetal/neonatal exposure to zidovudine. J. Acquir. Immune Defic. Syndr. Hum. Retrovirol. 1999; 20: 463–467. 20. Stiehm ER, Lambert JS, Mofenson LM, Bethel J, Whitehouse J, Nugent R, Moye J Jr, Glenn Fowler M, Mathieson BJ, Reichelderfer P, Nemo GJ, Korelitz J, Meyer WA 3rd, Sapan CV, Jimenez E, Gandia J, Scott G, O’Sullivan MJ, Kovacs A, Stek A, Shearer WT, Hammill H. Efficacy of zidovudine and human immunodeficiency virus (HIV) hyperimmune immunoglobulin for reducing perinatal HIV transmission from HIV-infected women with advanced disease: results of Pediatric AIDS Clinical Trials Group protocol 185. J. Infect. Dis. 1999; 179: 567–575. 21. Mandelbrot L, Landreau-Mascaro A, Rekacewicz C, Berrebi A, Benifla JL, Burgard M, Lachassine E, Barret B, Chaix ML, Bongain A, Ciraru-Vigneron N, Crenn-Hebert C, Delfraissy JF, Rouzioux C, Mayaux MJ, Blanche S; Agence Nationale de Recherches sur le SIDA (ANRS) 075 Study Group. Lamivudine-zidovudine combination for prevention of maternal-infant transmission of HIV-1. JAMA. 2001; 285: 2083–2093. 22. Gelber RD, Lindsey JC, MaWhinney S. Clinical Trials to Reduce the Risk of Maternal-Infant Transmission of HIV Infection. In: AIDS Clinical Trials. Finkelstein DM, Schoenfeld DA, eds. 1995. John Wiley & Sons, New York. pp. 287–302. 23. Shaffer N, Chuachoowong R, Mock PA, Bhadrakom C, Siriwasin W, Young NL, Chotpitayasunondh T, Chearskul S, Roongpisuthipong A, Chinayon P, Karon J, Mastro TD, Simonds RJ. Short-course zidovudine for perinatal HIV-1 transmission in Bangkok, Thailand: a randomised controlled trial. Bangkok Collaborative Perinatal HIV Transmission Study Group. Lancet 1999; 353: 773–780. 24. Lallemant M, Jourdain G, Le Coeur S, Kim S, Koetsawang S, Comeau AM, Phoolcharoen W, Essex M, McIntosh K, Vithayasai V. A trial of shortened zidovudine regimens to prevent mother-to-child transmission of human immunodeficiency virus type 1. Perinatal HIV Prevention Trial (Thailand) Investigators. N. Engl. J. Med. 2000; 343: 982–991.

25. Petra Study Team. Efficacy of three shortcourse regimens of zidovudine and lamivudine in preventing early and late transmission of HIV-1 from mother to child in Tanzania, South Africa, and Uganda (Petra study): a randomised, double-blind, placebo-controlled trial. Lancet 2002; 359: 1178–1186. 26. No authors listed. Elective caesarean-section versus vaginal delivery in prevention of vertical HIV-1 transmission: a randomised clinical trial. The European Mode of Delivery Collaboration. Lancet 1999; 353: 1035–1039. 27. No authors listed. The mode of delivery and the risk of vertical transmission of human immunodeficiency virus type 1--a meta-analysis of 15 prospective cohort studies. The International Perinatal HIV Group. N. Engl. J. Med. 1999; 340: 977–987. 28. Jamieson DJ, Read JS, Kourtis AP, Durant TM, Lampe MA, Dominguez KL. Cesarean delivery for HIV-infected women: recommendations and controversies. Am. J. Obstet. Gynecol. 2007; 197(Suppl 3):S96–100. 29. Guay LA, Musoke P, Fleming T, Bagenda D, Allen M, Nakabiito C, Sherman J, Bakaki P, Ducar C, Deseyve M, Emel L, Mirochnick M, Fowler MG, Mofenson L, Miotti P, Dransfield K, Bray D, Mmiro F, Jackson JB. Intrapartum and neonatal single-dose nevirapine compared with zidovudine for prevention of mother-to-child transmission of HIV-1 in Kampala, Uganda: HIVNET 012 randomised trial. Lancet. 1999; 354: 795–802. 30. Jackson JB, Musoke P, Fleming T, Guay LA, Bagenda D, Allen M, Nakabiito C, Sherman J, Bakaki P, Owor M, Ducar C, Deseyve M, Mwatha A, Emel L, Duefield C, Mirochnick M, Fowler MG, Mofenson L, Miotti P, Gigliotti M, Bray D, Mmiro F. Intrapartum and neonatal single-dose nevirapine compared with zidovudine for prevention of mother-to-child transmission of HIV-1 in Kampala, Uganda: 18-month follow-up of the HIVNET 012 randomised trial. Lancet 2003; 362: 859–868. 31. Eshleman SH, Guay LA, Mwatha A, Brown ER, Cunningham SP, Musoke P, Mmiro F, Jackson JB. Characterization of nevirapine resistance mutations in women with subtype A vs. D HIV-1 6-8 weeks after single-dose nevirapine (HIVNET 012). J. Acquir. Immune Defic. Syndr. 2004; 35: 126–130. 32. Eshleman SH, Mracna M, Guay LA, Deseyve M, Cunningham S, Mirochnick M, Musoke P, Fleming T, Glenn Fowler M, Mofenson LM, Mmiro F, Jackson JB. Selection and fading of resistance mutations in women and infants receiving nevirapine to prevent HIV-1 vertical

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS transmission (HIVNET 012). AIDS 2001; 15: 1951–1957. 33. Dao H, Mofenson LM, Ekpini R, Gilks CF, Barnhart M, Bolu O, Shaffer N. International recommendations on antiretroviral drugs for treatment of HIV-infected women and prevention of mother-to-child HIV transmission in resource-limited settings: 2006 update. Am. J. Obstet. Gynecol. 2007; 197(Suppl 3):S42–S55. 34. Institute of Medicine of the National Academy of Sciences. Review of the HIVNET 012 Perinatal HIV Prevention Study. 2005. National Academies Press. Washington, DC. pp. 1–10. http://www.nap.edu/catalog.php?record id= 11264#toc.

41.

42.

35. Lallemant M, Jourdain G, Le Coeur S, Mary JY, Ngo-Giang-Huong N, Koetsawang S, Kanshana S, McIntosh K, Thaineua V; Perinatal HIV Prevention Trial (Thailand) Investigators. Single-dose perinatal nevirapine plus standard zidovudine to prevent mother-tochild transmission of HIV-1 in Thailand. N. Engl. J. Med. 2004; 351: 217–228. 36. Dabis F, Bequet L, Ekouevi DK, Viho I, Rouet F, Horo A, Sakarovitch C, Becquet R, Fassinou P, Dequae-Merchadou L, Welffens-Ekra C, Rouzioux C, Leroy V; ANRS 1201/1202 DITRAME PLUS Study Group. Field efficacy of zidovudine, lamivudine and singledose nevirapine to prevent peripartum HIV transmission. AIDS 2005; 19: 309–318. 37. Taha TE, Kumwenda NI, Gibbons A, Broadhead RL, Fiscus S, Lema V, Liomba G, Nkhoma C, Miotti PG, Hoover DR. Short postexposure prophylaxis in newborn babies to reduce mother-to-child transmission of HIV-1: NVAZ randomised clinical trial. Lancet 2003; 362: 1171–1177. 38. Taha TE, Kumwenda NI, Hoover DR, Fiscus SA, Kafulafula G, Nkhoma C, Nour S, Chen S, Liomba G, Miotti PG, Broadhead RL. Nevirapine and zidovudine at birth to reduce perinatal transmission of HIV in an African setting: a randomized controlled trial. JAMA 2004; 292: 202–209. 39. McConnell MS, Stringer JS, Kourtis AP, Weidle PJ, Eshleman SH. Use of single-dose nevirapine for the prevention of mother-tochild transmission of HIV-1: does development of resistance matter? Am. J. Obstet. Gynecol. 2007; 197(Suppl 3):S56–S63. 40. Shapiro RL, Thior I, Gilbert PB, Lockman S, Wester C, Smeaton LM, Stevens L, Heymann SJ, Ndung’u T, Gaseitsiwe S, Novitsky V, Makhema J, Lagakos S, Essex M. Maternal single-dose nevirapine versus placebo as

43.

44.

45.

11

part of an antiretroviral strategy to prevent mother-to-child HIV transmission in Botswana. AIDS. 2006; 20: 1281–1288. Cressey TR, Jourdain G, Lallemant MJ, Kunkeaw S, Jackson JB, Musoke P, Capparelli E, Mirochnick M. Persistence of nevirapine exposure during the postpartum period after intrapartum single-dose nevirapine in addition to zidovudine prophylaxis for the prevention of mother-to-child transmission of HIV-1. J. Acquir. Immune Defic. Syndr. 2005; 38: 283–288. Thior I, Lockman S, Smeaton LM, Shapiro RL, Wester C, Heymann SJ, Gilbert PB, Stevens L, Peter T, Kim S, van Widenfelt E, Moffat C, Ndase P, Arimi P, Kebaabetswe P, Mazonde P, Makhema J, McIntosh K, Novitsky V, Lee TH, Marlink R, Lagakos S, Essex M; Mashi Study Team. Breastfeeding plus infant zidovudine prophylaxis for 6 months vs formula feeding plus infant zidovudine for 1 month to reduce mother-to-child HIV transmission in Botswana: a randomized trial: the Mashi Study. JAMA. 2006; 296: 794–805. Nduati R, John G, Mbori-Ngacha D, Richardson B, Overbaugh J, Mwatha A, NdinyaAchola J, Bwayo J, Onyango FE, Hughes J, Kreiss J. Effect of breastfeeding and formula feeding on transmission of HIV-1: a randomized clinical trial. JAMA 2000; 283: 1167–1174. Fowler MG, Lampe MA, Jamieson DJ, Kourtis AP, Rogers MF. Reducing the risk of motherto-child human immunodeficiency virus transmission: past successes, current progress and challenges, and future directions. Am. J. Obstet. Gynecol. 2007; 197(Suppl 3):S3–S9. Stark K. Ending of HIV vaccine trial jolts industry. Philadelphia Inquirer. Oct 7, 2007. http://www.philly.com/philly/business/ 10296647.html.

FURTHER READING NIH AIDSinfo website. http://aidsinfo.nih.gov. UNAIDS website. http://www.unaids.org. CDC HIV/AIDS website. http://www.cdc.gov/hiv. IMPAACT website. http://impaact.s-3.com. I-Base website. http://www.i-base.info.

CROSS-REFERENCES AIDS Clinical Trials Group (ACTG) Human Immunodeficiency Virus (HIV) Trials Data and Safety Monitoring Board

12

MOTHER TO CHILD HUMAN IMMUNODEFICIENCY VIRUS TRANSMISSION TRIALS Interim Analysis Factorial Designs

MULTICENTER TRIAL

from the different centers performing functions in relation to a particular study protocol. The centers are clinics, coordinating centers, and other resource centers as needed, such as central laboratories and reading centers, for conduct of the trial. Broadly, the complement of multicenter trial is single center trial, defined as (1):

CURTIS L. MEINERT The Johns Hopkins University Center for Clinical Trials Bloomberg School of Public Health Baltimore, Maryland

1

DEFINITION

1. A trial performed at or from a single site: (a) Such a trial, even if performed in association with a coalition of clinics in which each clinic performs its own trial, but in which all trials focus on the same disease or condition (e.g., such a coalition formed to provide preliminary information on a series of different approaches to the treatment of hypertension by stress control or reduction); (b) A trial not having any clinical centers and a single resource center (e.g., the Physicians’ Health Study). 2. A trial involving a single clinic; with or without satellite clinics or resource centers. 3. A trial involving a single clinic and a center to receive and process data. 4. A trial involving a single clinic and one or more resource centers. ant: multicenter trial Usage note: Note that the usual line of demarcation between single and multicenter is determined by whether or not there is more than one treatment or data collection site. Hence, a trial having multiple centers may still be classified as a single-center trial if it has only one treatment or data collection site.

The term ‘‘multicenter trial’’ is variously defined: multicenter trial (1): 1. A trial involving two or more clinical centers, a common study protocol, and a data center, data coordinating center, or coordinating center to receive, process, and analyze study data. 2. A trial involving at least one clinical center or data collection site and one or more resource centers. 3. A trial involving two or more clinics or data collection sites. syn: collaborative trial (not recommended), cooperative trial (not recommended) ant: singlecenter trial Usage note: Preferred to collaborative trial or cooperative trial for reasons indicated in usage notes for those two terms. See single-center trial for comments on line of demarcation between single and multicenter trials. multicentre study (2): A study carried out at more than one study centre. multicentre trial (3): A clinical trial conducted according to a single protocol but at more than one site, and therefore carried out by more than one investigator.

Generally, in the language of trials, trials are assumed to be single center unless otherwise indicated. Typically, a multicenter trial involves two or more administratively distinct clinics as well as a coordinating center or data center and, perhaps, other resource centers as well. However, trials exist, such as the Physician’s Health Study (PHS; see Section 4 for sketch), that can be classified as single center or multicenter, depending on the characteristics used for classification. People in the PHS were recruited, enrolled, and followed by mail and telephone and, hence, no clinical centers existed. However, a coordinating center and a differentiated organizational structure did exist and, hence, from an organizational perspective, the trial can be classified as multicenter. (The trial is not indexed as multicenter by the National Library of Medicine (NLM) indexers and,

The National Library of Medicine (medical subject headings—annotated alphabetic list 2003) defines multicenter trials as controlled studies that are planned and carried out by several cooperating institutions to assess certain variables and outcomes in specified patient populations, for example, a multicenter study of congenital anomalies in children. A multicenter trial involves two or more administratively distinct study centers and a leadership structure involving investigators

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

MULTICENTER TRIAL

therefore, is regarded as single center under their system of classification.) 2

RELATED TERMS

Various other terms, as taken from Meinert (1), of relevance to multicenter structures are: multi-protocol: (adj) Of or relating to a research activity involving two or more protocols (e.g. as in a multi-study or program project. multi-site: (adj) 1. Relating to or having more than one site. 2. multicenter multi-study: (adj) Of, relating to, or consisting of multiple studies. ant: singlestudy rt: multi-trial Usage note: Use multi-trial if all studies are trials. See note for study n. multi-study: (n) A study having two or more separate and distinct studies performed under the same umbrella organizational structure; multi-trial when studies are trials. Usage note: Not to be confused with a study having a series of substudies. multi-study design: (n) A design involving two or more studies; performed on the same or different study populations. ant: single-study design rt: multi-trial design, multi-study structure Usage note: See multi-study. multi-study network: (n) A network of centers organized to conduct a series of studies. multi-study structure: (n) An umbrella organizational structure created and maintained to initiate and carry out a series of related studies involving the same or different study populations. ant: single-study structure Usage note: Use multi-trial if all studies are trials. See multi-trial structure and study n for added comments. multi-trial: (adj) Of, relating to, or consisting of multiple trials. ant: single-trial rt: multi-study multi-trial: (n) A study having two or more separate and distinct trials performed under the same umbrella organizational structure.

multi-trial design: (n) A design involving two or more trials performed on the same or different people and involving different treatment protocols. ant: single-trial design Usage note: See multi-trial structure. multi-trial structure: (n) An organizational structure created or maintained to initiate and carry out a series of related trials involving the same or different study populations. ant: singletrial structure Usage note: Most useful when characterizing past or existing structures as to observed function or planned intent. Note that a structure created as a single-trial structure may ultimately be classified as a multi-trial structure if it serves the purpose of directing two or more trials, even if not originally created for that purpose. program project: (n) A collection of interrelated research activities having a common organizational structure and bearing on the same general question or issue (e.g., a collection of basic and applied research projects aimed at providing a better understanding of atherosclerosis, especially such a collection funded from a single source). sponsor: (n) 1. A person or agency that is responsible for funding a designated function or activity; sponsoring agency. 2. A person or agency that plans and carries out a specified project or activity. 3. The person or agency named in an Investigational New Drug Application or New Drug Application; usually a drug company or person at such a company, but not always (as with an INDA submitted by a representative of a research group proposing to carry out a phase III or phase IV drug trial not sponsored by a drug company). 4. A firm or business establishment marketing a product or service. study: (adj) Of or relating to one or something being evaluated or studied. Usage note: Used primarily as a general descriptor, as in study candidate, when other more precise terms are not appropriate or have undesirable connotations

MULTICENTER TRIAL

(as in trial candidate). See usage note for study n for additional notes. study: (n) 1. An experimental or nonexperimental investigation or analysis of a question, process, or phenomenon. 2. Any one of a variety of activities involving the collection, analysis, or interpretation of data. 3. Clinical trial, especially in a setting where there is a desire or need to de-emphasize the experimental nature of the investigation. 4. An investigation involving both a trial and nonexperimental investigation (as in the Coronary Artery Surgery Study, comprised of a clinical trial and a followup study) (4, 5). Usage note: Widely and loosely used. Avoid in favor of more informative, less generic, terms whenever possible (e.g. use trial rather than study when appropriate). As a label, limit use to defined sets of activities involving data collection not ordinarily characterized with a more informative, design-specific term (such as trial or followup study), or where a general term is needed to characterize a collection of activities involving a number of different designs (e.g. referring to a mix of trials and observational studies). Avoid as a synonym for a specified kind of investigation having a recognized term (as in referring to an investigation as a trial in the title or abstract of a paper and study in the body of the paper). umbrella organizational structure: (n) [multicenter multistudies] An organizational structure created to support and direct a series of separate and distinct studies; multi-study structure.

3

HISTORY

The accomplishment of scientific goals by collaboration involving disparate groups and disciplines is hardly unique to trials. Increasingly, as goals expand and projects become more complex, the only viable route to accomplishment is through the ‘‘pooling’’ of effort. Evidence of this trend extends to virtually all sciences, marked most recently by human genome projects.

3

Multicenter trials started to come into prominence in the 1940s. An early example is a trial of patulin, a metabolic product of Penicillium patulum Bainer, as a possible preventive for the common cold. It was carried out in the fall of 1943 and spring of 1944 and involved a study population of more than 100,000 persons assembled from multiple factory and postal worksites in the United Kingdom (6). One of the first multicenter trials involving a chronic disease was of streptomycin treatment of pulmonary tuberculosis (see Section 4 for sketch of this trial and others). One of the largest multicenter trials ever undertaken was carried out largely in 1954 and was done to test effectiveness of the Salk vaccine in preventing polio. All told, the study involved some 1.8 million children recruited from schools across the United States, with 650,000 of them being randomized to receive injections of the vaccine or matching placebo injections and the others serving as a nonrandomized comparison group. To a major degree, emergence of the multicenter trial coincides with transition of a society burdened with infectious diseases to one afflicted with chronic diseases and from trials involving small sample sizes and short-term treatment and follow up to trials involving large sample sizes and long-term treatment and follow up. The realization that even small benefits of treatment in reducing mortality or morbidity translates into huge reductions when projected to the general population served to underscore the need for sample sizes far beyond the reach of any one study site. By the 1960s, it was clear that the industrialized world was facing an ‘‘epidemic’’ of heart disease and of increasing rates of cancer of most types. It was clear, as well, that if treatments were to be tested for their potential in reducing or reversing the rise, that it would have to be via large-scale long-term multicenter trials. This reality gave rise to working parties in the United Kingdom and collaborative groups in the United States in the late 1950s and early 1960s to undertake multicenter treatment and prevention trials for cancer and heart disease. The ‘‘multicenter era’’ came into prominence in the late 1970s and 1980s in

4

MULTICENTER TRIAL

the United States due, in no small measure, to the efforts of Donald Fredrickson (1924–2002; Director of NIH 1975–1981) and Tom Chalmers (1917–1995). Both men were major proponents of trials and especially the long-term multicenter trial. The mindset underlying trials has progressed, albeit slowly, from one in which trials were seen as a measure of last resort to one in which they are seen increasingly as the option of choice. No doubt exists that single center trials are administratively less complicated than multicenter trials; but it is also true that, on average, multicenter trials are more robust and more rigorously designed and conducted than their single center counterparts. The move to multicenter trials has been accomplished by an ever more active role of sponsors in initiating multicenter trials. The NIH started to assume the role of initiator in the 1970s with the advent of requests for applications (RFAs; grant funding) and requests for proposals (RFPs; contract funding) for multicenter trials. To a large degree, the traditional investigator-initiated multicenter trial, at least in the United States via NIH funding, has become ‘‘endangered’’ with the advent in the mid-1990s of the requirement that investigators need permission to submit proposals in excess of $500,000 per year (direct cost). Permission is unlikely if the proposed trial is not high on the agenda of an institute. The transition to a ‘‘global economy’’ led to the creation of the International Conference of Harmonisation (ICH) in 1990 (www.ich.org). This economy has caused drug firms to take progressively more directive approaches to initiating and conducting trials they sponsor and has caused them to want study sites located in various parts of the world. For a comprehensive compilation of landmark trials published over time, visit the James Lind Library website at: jameslindlibrary.org/trial records/published. html.

4 EXAMPLES Streptomycin Treatment of Pulmonary Tuberculosis Trial Period of conduct: 1947–1948 Study population: Patients with acute bilateral pulmonary tuberculosis and unsuitable for collapse therapy Trial type: Treatment trial Locale of conduct: United Kingdom Sponsor/funding: UK Medical Research Council Sample size: 109 (not including 2 people who died in the preliminary observation period) Test treatment: Bed rest (minimum of 6 months) + streptomycin (intramuscular; 2g/day in 4 injections at 6 hour intervals) Control treatment: Bed rest (minimum of 6 months) Outcome measure: Death; radiographic changes as determined by masked readers Enrolling sites: 6 Method of assignment: Randomized blocks; stratified by clinic and gender; numbered envelopes Assignment ratio (observed): 55(S):52(C) Finding/conclusion: Streptomycin treatment beneficial in reducing the risk of death and in effecting improved chest radiographs Primary publication: Streptomycin in Tuberculosis Trials Committee, Streptomycin treatment of pulmonary tuberculosis. Br. Med. J. 1948; 2: 769–782.

MULTICENTER TRIAL

5

University Group Diabetes Program (UGDP)

Physicians’ Health Study (PHS)

Period of conduct: 1960–1975 Study population: Adults (males and females) with non-insulin-dependent, adult-onset, diabetes Trial type: Treatment/prevention trial Locale of conduct: United States Sponsor/funding: National Institutes of Health Sample size: 1,027 Test treatments (4): Antidiabetic diet + 1.5 gm tolbutamide/day; diet + 100 mg phenformin/day, diet + 10, 12, 14, or 16 units of insulin lente/day depending on body surface; diet + insulin lente daily dosed to control blood sugar Control treatment: Antidiabetic diet + placebo matched to tolbutamide or phenformin (no placebo injections) Outcome measure: Renal, CV, eye, and peripheral vascular morbidity and all cause mortality Clinics: 12 Coordinating center: University of Minnesota, School of Public Health, through 1963; University of Maryland, School of Medicine, thereafter Method of assignment: Randomized blocks; stratified by clinic and gender; centrally administered Assignment ratio: 1:1:1:1:1 Finding/conclusion: Tolbutamide and phenformin treatments stopped due to safety concerns; insulin treatments no more effective than placebo in prolonging life or reducing morbidity Primary publications: University Group Diabetes Program Research Group, A Study of the effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes: I. Design, methods and baseline characteristics. Diabetes 1970; 19(Suppl 2): 747–783. University Group Diabetes Program Research Group, A Study of the effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes: II. Mortality results. Diabetes 1970; 19(Suppl 2): 785–830. University Group Diabetes Program Research Group, A Study of the effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes: V. Evaluation of Phenformin therapy. Diabetes 1975; 24(Suppl 1): 65–184.

Period of conduct: 1982–1995 Study population: Male physicians, aged 40 to 84, registered in the American Medical Association, free of overt evidence of heart disease Trial type: Prevention trial Locale of conduct: United States Sponsor/funding: National Institutes of Health Sample size: 22,071 Test treatments: Aspirin (325 mg/day) and beta carotene (50 mg/day) Test treatment groups: Aspirin and beta carotene (5,517), aspirin and beta carotene placebo (5,520), aspirin placebo and beta carotene (5,519) Control treatment: Aspirin placebo and beta carotene placebo (5,515) Period of treatment: 46–77 months depending on when enrolled Method of treatment administration and followup: Mail and telephone Primary outcome measure: Fatal and nonfatal MI Study control center: Harvard Medical School Method of assignment: Blocked randomization Finding/conclusion: In regard to aspirin: Regular doses of aspirin were found to significantly reduce the risk of both fatal and nonfatal MI Primary publication: Steering Committee of the Physicians’ Health Study Research Group, Final report of the aspirin component of the ongoing physicians’ health study. NEJM 1989; 321:129–135.

6

MULTICENTER TRIAL

Pediatric AIDS Clinical Trial Group: Protocol 076 (ACTG 076) Period of conduct: 1991–1994 Study population: Pregnant, HIV-infected, women; 14 to 34 weeks gestation on entry Trial type: Prevention trial Locale of conduct: Primarily United States Sponsor/funding: National Institutes of Health Sample size: 477 Test treatment: Zidovudine (100 mg orally 5 times daily plus intrapartum zidovudine (2 mg/kg of body weight given intravenously for 1 hour followed by 1 mg/kg/hr until delivery; plus zidovudine for the newborn (2 mg/kg daily every 5 hours for 6 weeks, beginning 8 to 12 hours after birth Control treatment: Matching placebo Primary outcome measure: HIV infection of newborn Study clinics: 59 Method of assignment: Randomization, stratified by gestation time (2 groups) Finding/conclusion: ‘‘In pregnant women with mildly symptomatic HIV disease and no prior treatment with antiretroviral drugs during the pregnancy, a regimen consisting of zidovudine given ante partum and intra partum to the mother and to the newborn for six weeks reduced the risk of maternal-infant transmission by approximately two thirds’’ Primary publication: E. M. Connor, R. S. Sperling, R. Gelber, P. Kiseley, G. Scott, et al. for the Pediatric AIDS Clinical Trials Group Protocol 076 Study Group, Reduction of maternal-infant transmission of human immunodeficiency virus type 1 with zidovudine treatment. NEJM 1994; 331:1173–1180.

Asymptomatic Carotid Surgery Trial (ACST) Period of conduct: 1993–2003 Study population: Asymptomatic patients with substantial carotid narrowing Trial type: Prevention trial Locale of conduct: 30 different countries Sponsor/funding: Medical Research Council of the United Kingdom Sample size: 3,120 Test treatment: Immediate carotid endarterectomy (CEA) (1,560) Control treatment: Deferral of CEA (1,560) Period of followup: Variable depending on time of randomization; continuous from randomization to common closing date Primary outcome measure: Stroke (fatal or nonfatal) Study clinics: 126 (located in 30 different countries) Coordinating center/data center: Clinical Trial Service Unit, Oxford, United Kingdom Method of assignment: Minimized randomization using age, sex, history of hypertension, and several other variables Finding/conclusion: ‘‘In asymptomatic patients younger than 75 years of age with carotid diameter reduction about 70% or more on ultrasound, immediate CEA halved the net 5-year stroke risk from about 12% to about 6%. Half this benefit involved disabling or fatal stokes.’’ Primary publication: MRC Asymptomatic Carotid Surgery Trial (ACST) Collaborative Group, Prevention of disabling and fatal stroke by successful carotid endarterectomy in patients without recent neurological symptoms: randomised controlled trial. Lancet 2004; 363:1491–1502.

5 ORGANIZATIONAL AND OPERATIONAL FEATURES The key feature of multicenter trials is a differentiated organizational structure involving at least two administratively distinct centers and a defined infrastructure structure serving to bind centers and associated personnel into a cohesive whole. The key elements in the organizational and operational structure of multicenter trials are as follows: Center Director: The scientific head of a study center. Executive Committee (EC): A committee in multicenter studies responsible for

MULTICENTER TRIAL

direction of the day-to-day affairs of the study. One of the key committees in the organizational structure of a multicenter trial. Usually consists of the officers of the study and perhaps others selected from the steering committee and typically headed by the chair or vice-chair of the steering committee and reporting to that committee. Principal Investigator (PI): Broadly, the scientific head of study; generally best avoided in favor of more operationally explicit terminology in multicenter trials because of potential for confusion in favor of terms such as Center Director and Study Chair. Traditionally, in single center trials, the PI is the person in charge of the trial and is usually also the one responsible for enrollment and study of patients in the trial. As a result, the term is often reserved in multicenter trials for persons heading study clinics, leaving those heading the coordinating center or other resource center with some lesser designation implying ‘‘nonprincipal’’ investigatorship. However, even if uniformly applied to all center directors, the term is still best avoided, except in settings where the term is used to refer to a single individual (e.g., the one in investigator-initiated trials who is the recognized head by virtue of initiative in the funding effort). The term should be avoided in egalitarian settings in which, in effect, multiple ‘‘principal’’ investigators exist such as those represented in sponsor or governmentinitiated trials. Research Group: The entire set of personnel involved in the conduct of a research project; in multicenter trials includes center directors and support staff, representatives from the sponsoring agency, and study committee members. syn: investigative team, investigative group, study group (not a recommended syn), study staff (not a recommended syn) Steering Committee (SC): A committee of an organization responsible for directing or guiding the activities of that

7

organization. In multicenter trials, the committee responsible for conduct of the trial and to which other study committees report. Usually headed by the study chair and consisting of persons designated or elected to represent study centers, disciplines, or activities. One of the key committees in multicenter structures. Study Center: An operational unit in the structure of a study, especially a multicenter structure, separate and distinct from other such units in the structure, responsible for performing specified functions in one or more stages of the study (e.g., a clinical center or resource center, such as coordinating center). Study Chair: Chair of the steering committee. Study Officers: The set of persons holding elected or designated offices in a study; in multicenter trials, generally the study chair and vice-chair and the heads or directors of key resource centers, such as the coordinating center, and project office. Treatment Effects Monitoring Committee: A standing committee in the structure of most trials responsible for the periodic review of accumulated data for evidence of adverse or beneficial treatment effects during the trial and for making recommendations for modification of a study treatment, including termination, when appropriate. One of the key committees in the organizational structure of a multicenter trial. Usually constituted such that voting privileges are restricted to members not directly involved in the execution of the trial and not associated with participating centers or sponsors of the trial. syn: Data Monitoring Committee (DMC), Data and Safety Monitoring Committee (DSMC) and Safety Monitoring Committee (SMC); sometimes also Ethical Committee or Ethics Committee but not recommended usage because of likely confusion with institutional review boards (IRBs) and the like.

8

6

MULTICENTER TRIAL

COUNTS

The table below gives counts of all trials, randomized clinical trials, all multicenter clinical trials, and randomized multicenter clinical trials from 1980 forward, as indexed by the National Library of Medicine and counted via PubMed (August 2003). The indexing for multicenter trials was largely nonexistent prior to 1980 and probably spotty in the early 1980s. For example, the UGDP publications, while indexed as ‘‘clinical trial,’’ are not indexed as ‘‘multicenter trial’’. Randomized multicenter trials published since 1980 account for 9% of all randomized trials and 52% of all multicenter trials. About 40% of all randomized multicenter trials are in cancer or cardiovascular disease. The counts of single and multicenter trials (based on indexing by NLM) published in BMJ, Lancet, JAMA, and NEJM in 2002 and the 25th, 50th (median), and 75th percentile points of the sample distributions for the two classes of trials is given in the table below. Not surprisingly, multicenter trials are larger than single center trials. 7

ACKNOWLEDGMENTS

Thanks to Susan Tonascia, Ann Ervin, and Betty Collison for help in producing this piece. REFERENCES 1. C. L. Meinert, Clinical Trials Dictionary: Terminology and Usage Recommendations. Baltimore, MD: The Johns Hopkins Center for Clinical Trials, 1996. 2. S. Day, Dictionary for Clinical Trials. Chichester: Wiley, 1999. 3. International Conference on Harmonisation, E6 Good Clinical Practice. Washington, DC: U.S. Department of Health and Human Services, Food and Drug Administration, Center for Drug Evaluation and Research (CDER), and the Center for Biologics Evaluation and Research (CBER), April 1996. 4. CASS Principal Investigators and their associates, Coronary Artery Surgery Study (CASS): a randomized trial of coronary artery bypass surgery: comparability of entry characteristics

and survival in randomized patients and nonrandomized patients meeting randomization criteria. J. Am. Coll. Cariol. 1984; 3: 114–128. 5. CASS Principal Investigators and their associates, National Heart, Lung, and Blood Institute Coronary Artery Surgery Study (CASS): a multicenter comparison of the effects of randomized medical and surgical treatment of mildly symptomatic patients with coronary artery disease, and a registry of consecutive patients undergoing coronary angiography. Circulation 1981; 63(monograph 79)(Part II): I1–I-81. 6. Patulin Clinical Trials Committee, Clinical trial of patulin in the common cold. Lancet 1944; 2: 373–375.

FURTHER READING B. G. Greenberg (chair), A Report from the Heart Special Project Committee to the National Advisory Heart Council: Organization, Reviews and Administration of Cooperative Studies, 1967. Controlled Clin. Trials 1988; 9: 137–148. C. R. Klimt, Principles of multi-center clinical studies. In: J. P. Boissel and C. R. Klimt (eds.), Multi-Center Controlled Trials. Principles and Problems. Paris: INSERM, 1979. Cancer Research Campaign Working Party, Trials and tribulations: thoughts on the organization of multicentre clinical studies. Br. Med. J. 1980; 280: 918–920. J. Y. Lee, J. E. Marks, and J. R. Simpson, Recruitment of patients to cooperative group clinical trials. Cancer Clin. Trials 1980; 3: 381–384. J. M. Lachin, J. W. Marks, J. L. Schoenfield, and the NCGS Protocol Committee and the National Cooperative Gallstone Study Group, Design and methodological considerations in the National Cooperative Gallstone Study: a multicenter clinical trial. Controlled Clin. Trials 1981; 2: 177–229. C. L. Meinert, Organization of multicenter clinical trials. Controlled Clin. Trials 1981; 1: 305–312. Coronary Drug Project Research Group, The Coronary Drug Project: methods and lessons of a multicenter clinical trial. Controlled Clin. Trials 1983; 4: 273–541. S. J. Pocock, Clinical Trials: A Practical Approach. New York: Wiley, 1983. D. G. Weiss, W. O. Williford, J. F. Collins, and S. F. Bingham, Planning multicenter clinical trials: a biostatistician’s perspective. Controlled Clin. Trials 1983; 4: 53–64.

MULTICENTER TRIAL

All trials Rz

Total 1980–85 1986–90 1991–95 1996–00 2001 2002 Total

27,806 39,973 65,534 96,667 19,254 18,469 267,703

5,388 11,625 16,777 25,006 5,790 5,761 70,347

19.4 29.1 25.6 25.9 30.1 31.2 26.3

No. of trials Single center Multi-center BMJ Lancet JAMA NEJM

29 62 27 31 149

Multicenter trials Total Rz % Rz

% Rz

35 47 38 54 174

501 3,039 2,901 4,229 961 825 12,456

194 1,091 1,473 2,547 602 525 6,432

38.7 35.9 50.8 60.2 62.6 63.6 51.6

9

RZ MC by disease Cancer CV 25 219 213 334 77 72 940

81 344 428 583 130 103 1,669

Sample size Single center 25% Mdn 132 41 84 41 82

C. L. Meinert and S. Tonascia, Clinical Trials: Design, Conduct, and Analysis. New York: Oxford University Press, 1986. C. L. Meinert, In defense of the corporate authorship for multicenter trials. Controlled Clin. Trials 1993; 14: 255–260. L. M. Friedman, C. D. Furberg, and D. L. Demets, Fundamentals of Clinical Trials, 3rd ed. Multicenter Trials. New York: Springer, 1998, pp. 345–356. B. O’Brien, L. Nichaman, J. E. H. Brouhe, D. L. Levin, P. C. Prorok, and J. K. Gohagan for the PLCO Project Team, Coordination and management of a large multicenter screening trial: the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial. Controlled Clin. Trials 2000; 21: 310S–328S. Asthma Clinical Research Network, The Asthma Clinical Research Network. Controlled Clin. Trials 2001; 22: 117S–251S.

272 204 230 123 219

75%

25%

Multicenter Mdn

75%

1,090 1,004 1,382 287 775

211 201 481 239 281

540 920 1,182 446 690

1,859 2,309 2,475 1,232 2,037

MULTINATIONAL (GLOBAL) TRIAL

multinational trial it is important to carefully consider such factors as the number of countries and ratios of patients per center and centers per country. Several investigators have discussed some of the practical difficulties associated with multinational trials (1–3). One important difficulty is that medical practice can vary among countries. There might be different diagnostic criteria or differences in medical terminology that lead to a different understanding of what constitutes the disease in question or how a study endpoint is defined. For example, if the study endpoint is hospitalization for the disease in question, there may be substantial differences among countries in the criteria for a hospital admission. In addition, the standard of care for that hospital admission may vary considerably among countries, and the variations in the standard of care may interact with the study treatment, leading to the potential for countryby-treatment interaction. Another aspect of medical practice relates to the choice of the control treatment. When the standard of care for the disease varies from country to country, a decision must be made between choosing a common control treatment across the protocol and choosing the control treatment deemed most relevant within each country. The logistical aspects of providing study supplies for a multinational trial might also be somewhat difficult. Import/export policies for experimental treatments vary from country to country, and they may be time consuming to navigate. Even when the treatments are locally available, the formulations might differ, leading to questions regarding equivalent bioavailability. Another logistical difficulty has to do with the analysis of laboratory samples. Differences in laboratory assays can make a pooled analysis of local laboratory results difficult to interpret. The use of a central laboratory solves this problem, but it may be difficult to ship biologic samples across national borders. The International Conference on Harmonization (ICH) was formed to develop common regulatory standards for clinical research across Europe, North America, and Japan. Despite considerable progress, there

STEVEN M. SNAPINN Amgen, Inc. Thousand Oaks, California

Multicenter trials have long been common in clinical research (see also the article on multicenter trials). The main reasons for conducting a trial at multiple centers is the ability to recruit a large number of patients in a relatively short period of time, and the ability to generalize the results more broadly. Multinational trials are those in which the centers are in multiple countries; Suwelack and Weihrauch (1) define them as ‘‘prospective clinical trials with investigational centers from more than one country with a common protocol foreseeing the combination of clinical data from all centers for a joint evaluation.’’ The reasons for conducting a multinational trial are similar to those for a multicenter trial; however, there are also some special considerations. 1

PRACTICAL ISSUES

The need to recruit a large number of patients is often the main driving force for multinational trials, but there may be particular reasons to recruit patients from specific parts of the world. For example, the disease might be particularly prevalent in specific countries, or in some cases it might be necessary to recruit diseased patients who are treatment-na¨ıve, necessitating selection of countries where access to the treatment is currently limited. It can also be advantageous to demonstrate the safety and efficacy of a therapy under a variety of conditions to gain confidence that the conclusions of the trial are generalizable. For this reason, it might be desirable to conduct the trial in countries with different lifestyles, diets, or cultures. However, it should be recognized that these differences, as well as differences in patient characteristics, may lead to increased variability, which may decrease the power of the trial or complicate the ability to reach a clear conclusion. For this reason, when designing a

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

MULTINATIONAL (GLOBAL) TRIAL

are still several legal and regulatory aspects to consider. For example, different countries may have different requirements for informed consent or for reporting serious adverse events to health authorities. When submitting the study protocol for institutional review board review, different countries may assess potential risks differently and so may require conflicting changes. Cultural differences may also cause some difficulties. When the Case Report Form (CRF) needs to be translated into the local languages, there is always the potential for misunderstandings. This is particularly true for the collection of information on patientreported outcomes, including information on quality of life, where nuances in the wording of questions can have a major effect. Although much information on the CRF can be collected through checkboxes, there is often the need to collect some information as free text, which naturally leads to additional potential for misunderstanding. 2 COUNTRY-BY-TREATMENT INTERACTION Multinational trials may provide broadly generalizable results, but this requires that the results be consistent across patient subsets. However, substantial country-by-treatment interaction has been noted in several clinical trials, leading to difficulties in interpretation. One such study was the Metoprolol CR/XL Randomized Intervention Trial in Congestive Heart Failure (MERIT-HF), described by Wedel et al. (4), which compared the βblocker metoprolol with placebo with respect to total mortality in 3991 patients with heart failure. The trial was stopped early by the steering committee due to a highly significant reduction in total mortality (hazard ratio = 0.66; P = 0.00009). After completion of the trial, the investigators carried out a number of subgroup analyses. Although the overall result was consistent among most subgroups, there was one notable exception: Among the 1071 patients in the United States, the hazard ratio was 1.05. After carefully examining the potential causes of the interaction, the investigators cautioned against overinterpretation of subgroup results and concluded that the best estimate of the treatment

effect for any subgroup, including country subgroups, should be the overall observed effect. O’Shea and DeMets (5) and O’Shea and Califf (6) reviewed other cardiovascular trials that have found country-by-treatment interactions. For example, the Beta-blocker Heart Attack Trial (BHAT) compared propranolol with placebo in 3837 patients who had survived a myocardial infarction; the overall result was highly positive (the mortality rates were 9.8% with placebo and 7.2% with propranolol), but there was wide variation in the size and direction of the effect among study centers. The Flolan International Randomized Survival Trial (FIRST) compared epoprostenol with placebo in patients with heart failure; although mortality was greatly reduced by epoprostenol among patients from Europe, there was no apparent benefit in patients from North America. The Platelet Glycoprotein IIb/IIIa in Unstable Angina: Receptor Suppression Using Integrilin Therapy (PURSUIT) trial compared the platelet inhibitor eptifibatide with placebo in nearly 11,000 patients with an acute coronary syndrome; there was an absolute reduction of 3.3% in the primary endpoint among 3827 patients in North America, but the reductions in other regions ranged only from 1.0% to −1.4%. The Global Utilization of Streptokinase and t-PA for Occluded Coronary Arteries I (GUSTO-I) trial studied the effects of four thrombolytic strategies in 41 thousand patients and found a statistically significant country-by-treatment interaction when comparing the U.S. and non-U.S. patients. Finally, the Global Use of Strategies To Open Occluded Coronary Arteries IV (GUSTO-IV) trial studied the use of abciximab in 7800 patients with an acute coronary syndrome; though there was no overall effect of abciximab, the drug appeared to be beneficial among patients in North America. Although it is not clear if these interactions are due to chance, O’Shea and Califf (7) discuss some of the differences among countries that might lead to interactions like these. Examining the results of over a dozen cardiovascular trials, they found significant differences in patient characteristics. One notable difference was that patients enrolled in the United States were heavier and taller

MULTINATIONAL (GLOBAL) TRIAL

than other patients, and were more likely to have diabetes. They also found important differences in the use of concurrent therapies. For example, compared with patients from other countries, patients in the United States were consistently more likely to be taking a β-blocker. In PURSUIT, women from the United States were considerably more likely to be taking hormone replacement therapy. In addition, patients from the United States in the acute coronary syndrome trials presented to the hospital earlier than patients in other countries. 3

COST EFFECTIVENESS EVALUATION

Some clinical trials include a health economic or cost-effectiveness evaluation (see also the article on cost-effectiveness analysis). Although the situation is conceptually similar to that of the evaluation of clinical effects, the generalizability of costeffectiveness results is more controversial (8–10). Differences in medical practice and patient characteristics have the potential to cause country-by-treatment interaction with regard to clinical effects, but the biologic effect of the treatment is expected to be consistent among countries. With respect to economic data, on the other hand, it is often assumed that systematic differences among countries will prevent pooling the results. This is due to perceived differences in practice patterns, payment systems, and relative prices of resources. Despite the common assumption that economic results will vary among countries, it is reasonable to include an assessment of the degree of similarity. Cook et al. (9) discussed the issues involved in combining economic data from multinational trials, and pointed out that one can use the same tools to assess interaction for clinical and economic endpoints. They concluded that, in the absence of interaction, the pooled estimate of cost effectiveness should be considered representative of the participating countries. Reed et al. (10) summarized a workshop held on this topic and discussed the terminology for multination economic evaluations; this terminology depends on whether pooled

3

or country-specific estimates of clinical efficacy and resource utilization are used. Also concerned by between-country heterogeneity in costs, Pinto et al. (11) proposed univariate and multivariate shrinkage estimators for costs and effects from multinational trials. REFERENCES 1. D. Suwelack and T. R. Weihrauch, Practical issues in design and management of multinational trials. Drug Inf J. 1992; 26: 371–378. 2. H. Maier-Lenz, Implementating multicenter, multinational clinical trials. Drug Inf J. 1993; 27: 1077–1081. 3. H. T. Ho and S. C. Chow, Design and analysis of multinational clinical trials. Drug Inf J. 1998; 32: 1309S–1316S. 4. H. Wedel, D. DeMets, P. Deedwania, B. Fagerberg, S. Goldstein, et al., for the MERIT-HF Study Group. Challenges of subgroup analyses in multinational clinical trials: experiences from the MERIT-HF trial. Am Heart J. 2001; 142: 502–511. 5. J. C. O’Shea and D. L. DeMets, Statistical issues relating to international differences in clinical trials. Am Heart J. 2001; 142: 21–28. 6. J. C. O’Shea and R. M. Califf, International differences in treatment effects in cardiovascular clinical trials. Am Heart J. 2001; 141: 875–880. 7. J. C. O’Shea and R. M. Califf, International differences in cardiovascular clinical trials. Am Heart J. 2001; 141: 866–874. 8. S. D. Sullivan, B. Liljas, M. Buxton, C. J. Lamm, P. O’Byrne, et al., Design and analytic considerations in determining the costeffectiveness of early intervention in asthma from a multinational clinical trial. Control Clin Trials. 2001; 22: 420–437. 9. J. R. Cook, M. Drummond, H. Glick, and J. F. Heyse, Assessing the appropriateness of combining economic data from multinational clinical trials. Stat Med. 2003; 22: 1955–1976. 10. S. D. Reed, K. J. Anstrom, A. Bakhai, A. H. Briggs, R. M. Califf, et al., Conducting economic evaluations alongside multinational clinical trials: toward a research consensus. Am Heart J. 2005; 149: 434–443. 11. E. M. Pinto, A. R. Willan, and B. J. O’Brien, Cost-effectiveness analysis for multinational clinical trials. Stat Med. 2005; 24: 1965–1982.

4

MULTINATIONAL (GLOBAL) TRIAL

CROSS-REFERENCES Case report form Cost-effectiveness analysis Foreign clinical data International Conference on Harmonization (ICH) Multicenter trial

MULTIPLE COMPARISONS

µ1 , . . . ,µk . Fisher’s least significant difference (LSD) procedure (1) declares the pair µi and µj different if the t-statistic that compares µi and µj is significant at level 0.05 and the F-statistic that compares all means is also significant at level 0.05. Fisher’s LSD protects the FWE under the global null hypothesis µ1 = . . . = µk because, to declare at least one pair different, we must reach a P-value of 0.05 or less for the F-statistic, and that has probability 0.05 under the global null hypothesis. On the other hand, suppose that µ1 = . . . = µk−1 = µ, but µk is so much larger than µ that the F-statistic is virtually guaranteed to reach statistical significance. In that case, Fisher’s LSD is tantamount to pairwise comparisons of means at level 0.05. But the FWE for 0.05-level pairwise comparisons of the equal means µ1 , . . . ,µk−1 is clearly larger than 0.05. Therefore, Fisher’s LSD provides weak control but not strong control of the FWE.

MICHAEL A. PROSCHAN National Institute of Allergy and Infectious Diseases Bethesda, Maryland

Multiple comparisons arise in several ways in clinical trials; common examples include multiple arms, multiple endpoints, subgroup analyses, and monitoring over multiple time points. At first, it seems that trying to answer several questions in the same trial is very efficient, especially if another large trial to answer the same questions is unlikely. The problem is that if the comparisonwise error rate (also known as the per-comparison error rate)—the expected proportion of false positives—is 0.05, then the chance of at least one false positive, which is known as the familywise error rate (FWE) or the experimentwise error rate, can be substantially greater than 0.05. In fact, with enough comparisons, the FWE can be close to 1. To control the FWE at level 0.05, one must require stronger evidence for each individual comparison; this is called a multiple comparison adjustment.

2 CRITERIA FOR DECIDING WHETHER ADJUSTMENT IS NECESSARY Whether and how much multiplicity adjustment is needed is controversial. Some have argued that such adjustments are never needed (2), whereas others have used several case studies to argue that without adjusting for all comparisons made in a clinical trial, one cannot be confident of the results (3). The view of many in clinical trials is somewhere between these two positions, namely that adjustment is needed in certain settings but not in others. It is important to note that multiplicity adjustment is needed only when a trial is considered successful when at least one comparison is statistically significant, not when all comparisons must be statistically significant to declare a success. For example, regulatory agencies such as the Food and Drug Administration (FDA) require proof that a combination drug is superior to each of its constituents. If two constituents exist, A and B, then the null hypotheses are H0A : the combination is no better than constituent A, and H0B : the combination is no better than constituent B. To declare success, we must reject

1 STRONG AND WEAK CONTROL OF THE FWE Two different levels of control of the FWE exist. The first level, which is called weak control, means that under the global null hypothesis that each separate null hypothesis is true, the probability of at least one type 1 error is no greater than α. But the global null hypothesis is a very strong assumption; it is more likely that some null hypotheses are true and others are not. We would like the probability of rejecting at least one true null hypothesis to be α or less, where that probability is computed without necessarily assuming that the other null hypotheses are true. If this holds, then the FWE is said to be controlled strongly. To understand why strong control is indeed stronger than weak control, consider a trial with k arms in which we are interested in all pairwise comparisons of means

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

MULTIPLE COMPARISONS

both H0A and H0B ; it is not sufficient to reject at least one of H0A and H0B . If each comparison is made at level 0.05, then the probability of a successful trial—the probability of the intersection of the separate rejection events—will be no greater than 0.05. Therefore, no multiple comparison adjustment is needed in this situation (4). A similar setting is when a drug must show benefit on several different characteristics to be declared beneficial. A helpful way to distinguish whether adjustment is needed is to ask whether a successful trial requires rejection of hypothesis 1 AND hypothesis 2 . . . .AND hypothesis k. If so, then no adjustment is needed. If AND is replaced by OR, then adjustment may be needed. In the sequel, we restrict attention to the latter setting. Proschan and Waclawiw (5), who observed that some settings exist in which nearly everyone agrees that a multiple comparison adjustment is needed, tried to determine what those settings had in common. One factor is the sheer number of comparisons. Situations in which the number of comparisons is often larger than two include interim monitoring and subgroup analyses. Another factor for deciding whether a multiple comparison adjustment is needed is whether one stands to benefit from the multiplicity. For instance, even if investigators have a good reason for changing the primary endpoint in the middle of a trial, their decision will be met with skepticism, especially if no treatment effect existed for the original endpoint but an effect existed for the new primary endpoint. Situations in which one stands to benefit from unadjusted multiple comparisons include monitoring, subgroup analyses, multiple primary endpoints, and comparison of multiple doses of a drug to a placebo. By contrast, no single entity stands to gain from multiple unadjusted comparisons of drugs made by different manufacturers. The most important and difficult factor is the relatedness of the questions being asked and their answers (6, p. 31–35). Clinical trialists tend to feel more compelled to adjust for a family of related (e.g., heart disease and stroke) than unrelated (e.g., cancer and heart disease) hypotheses. Situations in which the questions are related include interim monitoring (in which case the same question is

asked at different times), subgroup analyses, and comparisons of different doses of the same drug to a control. One could debate the relatedness of comparisons of drugs made by different manufacturers to the same control group. One might consider that whether drug A is better than control is a completely separate question than whether drug B is better than control. In fact, the two questions could have been answered in separate trials, in which case no multiple comparison adjustment would have been made (2, 7). But the test statistics in separate trials are statistically independent, so learning that a type 1 error was made in one trial does not erode our confidence in conclusions of other trials. Learning that a type 1 error was made in a trial that compares multiple treatments to the same control does erode our confidence in the results of comparisons of the other treatments with control because it calls into question whether the common control arm was aberrant. Thus, having the same control de facto makes the answers to the questions related (8). In other cases, the questions and answers are more clearly unrelated. Sometimes, two completely different interventions that could have been tested in separate trials are, for efficiency reasons, tested in the same trial using a factorial design. The Women’s Angiographic Vitamin and Estrogen trial (9) in post-menopausal women compared hormone replacement with placebo and vitamins with placebo with respect to diameters of coronary arteries measured using an angiogram. It was considered unlikely that an interactive effect of the two interventions would occur, so a factorial design was used. Unlike comparisons of several arms to the same control, comparisons of main effects in a factorial design with no interaction are asymptotically statistically independent. Therefore, multiple comparison adjustments are often not used for main effect comparisons in a factorial trial (8). Table 1 shows the multiplicity criteria (columns) applied to common clinical trial settings (rows). Darker rows indicate settings that suggest a greater need for adjustment. Monitoring and subgroups are the darkest, and this is consistent with the general feeling that these situations call for some sort of

MULTIPLE COMPARISONS

3

Table 1. Guidelines applied to four common sources of multiplicity in clinical trials Large Number of Comparisons?

Single Entity Benefits?

Related Family?

Multiple Arms Doses vs. same control Different drugs, same control Different drugs, factorial Primary Endpoints Related Unrelated Subgroups Monitoring It usually applies. It may or may not apply, depending on the circumstances. The reader should fill in the gray squares with white or black ones according to the circumstances of the specific application. It usually does not apply.

adjustment (specific adjustment techniques are discussed later). Multiple primary endpoints and multiple doses of a drug compared with the same control are also dark rows, suggesting that separate comparisons at level 0.05 are probably not appropriate. 3 IMPLICIT MULTIPLICITY: TWO TAILED TESTING Before moving to specific adjustment methods in different clinical trial settings, we apply the relatedness criterion to an issue we do not usually associate, although we should, with multiple comparisons—two-tailed testing. Consider, for example, a trial that compares a drug with placebo with respect to mortality. Although the alternative hypothesis for a two-tailed test is that the mortality rate differs between treatment and placebo, we always want to know the direction of the effect. Therefore, we are really trying to answer two separate questions with respect to mortality: (1) Is treatment better than placebo? (2) Is treatment worse than placebo? The two conclusions and their implications could not be more dissimilar, so why do researchers routinely lump them by controlling the probability of any difference? It makes more sense to specify error rates for the two types of errors separately; we may or may not want to use α = 0.025 for each. Consider the Cardiac Arrhythmia Suppression Trial (CAST), which tested the

hypothesis that suppression of cardiac arrhythmias in arrhythmic patients with a previous heart attack would reduce the rate of sudden arrhythmic deaths and cardiac arrests. When it was discovered that two drugs, encainide and flecainide, actually increased the rate of sudden death and cardiac arrest compared with placebo (10), investigators eliminated these two drugs and continued with the third drug, moricizine. The continuation trial was called CAST II (11). Investigators used level 0.025 for declaring superiority and level 0.05 for declaring harm in CAST II, even though the combined type 1 error rate, 0.025 + 0.05 = 0.075, exceeds the conventional level of significance in clinical trials. Because of the asymmetry of harm and benefit coupled with the experience of CAST 1, it made sense to exceed the conventional two-tailed error rate of 0.05 4 SPECIFIC MULTIPLE COMPARISON PROCEDURES We briefly review some common multiplicity adjustment techniques in clinical trials. Readers should consult Miller (6) or Hochberg and Tamhane (12) for a full treatment of multiple comparison methods. 4.1 Multiple Arms Certain multiarmed trials exist in which a multiple comparison adjustment is typically not made. One example is when an arm

4

MULTIPLE COMPARISONS

is added for the purpose of ‘‘validating’’ a trial. For example, the purpose of the Glucosamine/Chondroitin Arthritis Intervention Trial (13) was to determine whether glucosamine and/or chondroitin were superior to placebo for the relief of pain of osteoarthritis of the knee. It was already known that celecoxib relieves pain, so a celecoxib arm was included to validate the trial; if the celecoxib and placebo arms did not differ, then some would question the validity of the trial. Because celecoxib is being included for a very different purpose, no reason exists to lump the comparison of celecoxib to placebo with the comparisons of glucosamine and/or chondroitin to placebo. In the section, ‘‘Criteria for deciding whether adjustment is necessary,’’ we cited another case in which adjustment for multiple comparisons is not typically used—a factorial trial in which an interaction is considered very unlikely, such as when the mechanisms of action and/or outcomes of interest are different. Such a trial is often thought of as two separate trials combined for efficiency reasons only. In many trials, more interest is focused on certain comparisons than on others, as pointed out by Cook and Farewell (14) and Hughes (15). For instance, interest often centers on comparisons with a common control arm. The mere fact that comparisons are made with the same control group was cited in the section on ‘‘Criteria’’ as a reason to consider a multiplicity adjustment, especially if more than two comparisons are made. Dunnett (16) developed a classic method for doing so. Let Ti be the t-statistic that compares arm i to the control, but replace the pooled variance of the two arms by the pooled variance across all k arms and use (k + 1)(n − 1) instead of 2(n − 1) degrees of freedom. Let Pi be the corresponding P-value, i = 1,2, . . . ,k. Treatment i is declared different from the control if Pi ≤ ck , where ck is selected so that P(P1 ≤ ck or P2 ≤ ck or . . . or Pk ≤ ck ) = α. It is easy to see that Dunnett’s method protects the FWE not only under the global null hypothesis, but also when only some means differ from control. Thus, Dunnett’s method provides strong control of the type 1 error rate for comparisons with control.

A more powerful modification of Dunnett’s method that still strongly controls the type 1 error rate is the following step down method (see Example 4.2 of Reference 12). Order the P-values so that P(1) < P(2) < . . . < P(k) . Step 1 tests whether any of the k treatments differ from control by comparing P(1) with ck . If P(1) > ck , then stop and declare no treatment superior to control, whereas if P(1) ≤ ck , then declare the associated treatment different from control and proceed to step 2. If we proceed to step 2, then we either already made a type 1 error or we did not. If we did not, then at most k − 1 type 1 errors are possible, so we ‘‘step down’’ and compare the remaining k − 1 treatments to control. That is, we compare P(2) with ck−1 , where ck−1 is the critical value for P-values that compare k − 1 active arms to control. If we find, after stepping down, that P(2) > ck−1 , then we stop and declare no remaining treatments different from control. On the other hand, if P(2) ≤ ck−1 , then we declare that treatment different from control, and then ‘‘step down’’ again by comparing P(3) to ck−2 , and so on. The step down version of Dunnett’s method clearly has more power than the original, but it has a drawback. Suppose that the k different treatments are made by different companies. If one company’s drug is extremely good, then another company’s drug will not have to meet the same burden of proof to be declared superior to the control. This failure to use a ‘‘level playing field’’ for all comparisons may be objectionable. The section entitled, ‘‘Crtieria for deciding whether adjustment is necessary’’ mentioned another setting that involves comparisons with the same control—that of different doses of the same drug. Table 1 suggests that it is inappropriate to compare each dose to control at level α as the primary analysis. But, often the first question is whether the drug worked, and the dose-response relationship is examined only if the answer is yes. One useful method is a hierarchical procedure whereby one first establishes that the comparison of the active arms combined to the control is significant at level α, then compares each dose to control at level α.

MULTIPLE COMPARISONS

4.2 Multiple Endpoints The multiple endpoint problem is complex because the endpoints might all be primary or a combination of primary and secondary. First consider multiple primary endpoints. As mentioned in the section on Criteria and as suggested by Table 1, if the endpoints are thought to be unrelated, then one could defend not using any adjustment, although most statisticians would feel uneasy if the study had more than two primary endpoints. An attractive method to adjust within a group of similar endpoints is to use a perpatient summary that combines the different endpoints. An example is O’Brien’s rank test (17), which first ranks patients on each of the k outcomes, and then averages ranks across outcomes for a given patient. These per-patient summary measures are then analyzed using a t-test or permutation test. Cook and Farewell (14) note that this method works well when many outcome variables reasonably measure the same phenomenon that treatment is expected to benefit. Follmann (18) showed that if relatively few endpoints exist and treatment has a strong effect on one outcome but not others, then the Bonferroni method, which uses significance level α/k for each endpoint, is attractive. It controls the FWE strongly and is very simple to use, but it is conservative. Bonferroni’s method works well when the number of endpoints is not too large and the test statistics are not highly correlated. For example, Table 2 shows the FWE for two, five, or ten z-statistics with correlation ρ when no adjustment is used (upper panel) and when the Bonferroni adjustment is used (lower panel). The type 1 error rate is substantially inflated with no multiple comparison adjustment, even if ρ is high. The FWE using the Bonferroni adjustment is only slightly less than 0.05 for two uncorrelated z-statistics, which indicates very slight conservatism. As the correlation becomes close to 1, the FWE is about half of 0.05 because the two test statistics are essentially one as the correlation approaches 1. Nonetheless, even for a fairly high correlation of 0.7, the degree of conservatism is not excessive. The conservatism is much more substantial with a greater number of comparisons. For example, with 10

5

z-statistics whose pairwise correlations are all 0.7, the actual type 1 error rate is 0.029 instead of 0.05. More powerful variants of the Bonferroni procedure maintain strong control of the FWE. One variant (19) is a step-down procedure similar to the step-down version of Dunnett’s method described above. If the smallest of the k P-values is less than or equal to α/k, then we declare that endpoint significant and proceed to the second step, whereas if the smallest P-value exceeds α/k, then we stop. If we proceed to the second step, then we compare the second smallest P-value to α/(k − 1). If the second smallest P-value exceeds α/(k − 1), then we stop; if it is less than or equal to α/(k − 1), then we declare that endpoint significant and compare the third smallest P-value to α/(k − 2), and so on. Any time we fail to reach statistical significance, we stop testing, so any larger P-values cannot be declared significant. This procedure, which still strongly controls the type 1 error rate, is more powerful than the classical Bonferroni method, which requires each P-value to be α/k or less to be declared significant. Another Bonferroni variant is Hochberg’s (20) modification of Simes’ (21) method, though as Dmitrienko and Hsu (22) point out, Hochberg’s procedure does not control the FWE in all settings. The Hochberg modification first compares the largest P-value to α; if the largest P-value is α or less, then all endpoints are declared significant. If the largest P-value is larger than α, then the second largest P-value is compared with α/2. If the second largest P-value is α/2 or less, then it and smaller P-values are declared significant. If the second largest P-value exceeds α/2, then the third largest P-value is compared with α/3, and so on. Finally, if all other P-values are larger than their thresholds, then the smallest P-value is declared significant if its P-value is α/k or less. For example, with two endpoints, if both attain P-values of 0.05 or less, then they are both declared significant; if one P-value exceeds 0.05, then the other is still declared significant if its P-value is 0.025 or less. The discussion thus far has not dealt with secondary endpoints. It is difficult to formulate a single strategy that covers all cases

6

MULTIPLE COMPARISONS

Table 2. FWE of unadjusted (upper panel) and Bonferroni-adjusted (lower panel) two-tailed tests for two, five, or ten comparisons using Z-statistics with the same pairwise correlation, ρ FWE with no adjustment ρ k 2 5 10

0 .098 .226 .401

.10 .097 .224 .394

.20 .096 .218 .377

.30 .095 .209 .351

.40 .093 .197 .321

.50 .091 .183 .287

.60 .088 .167 .251

.70 .083 .148 .213

.80 .078 .127 .173

.90 .070 .102 .128

1 .050 .050 .050

.70 .043 .035 .029

.80 .040 .030 .023

.90 .036 .023 .017

1 .025 .010 .005

FWE with Bonferroni adjustment ρ 2 5 10

0 .049 .049 .049

.10 .049 .049 .049

.20 .049 .048 .047

.30 .048 .047 .045

.40 .048 .045 .042

because, as D’Agostino (23) points out, there are many different purposes for secondary endpoints. One purpose might be, after showing that the treatment works, to understand its mechanism of action, in which case a multiplicity adjustment is probably not needed. In other situations, it is less clear how much adjustment is needed. Some people would be content to use level α for each secondary endpoint, arguing that the designation ‘‘secondary outcome’’ instills in the reader the proper amount of caution when interpreting results. Others suggest adjusting for all secondary outcomes or all secondary outcomes plus the primary outcome (24) using the Bonferroni method. The problem is that trials are usually powered for the primary outcome, so even with no multiple comparison adjustment, power for secondary outcomes may be low. To undermine power even more by adjusting for all secondary outcomes makes it very unlikely to reach statistical significance. A reasonable middle ground is to consider any secondary outcome finding suggestive if it reaches level α when no adjustment is made and more definitive if it remains significant after adjustment for multiplicity. 4.3 Subgroup Analyses The subgroup row of Table 1 is completely dark, which suggests that some sort of multiplicity adjustment or at least cautionary language is needed for subgroup conclusions,

.50 .047 .042 .039

.60 .045 .039 .034

especially if the overall effect is not significant. That is what happened in the rpg120 HIV Vaccine Study (25), which was the first Phase 3 trial of an HIV vaccine in humans. No significant difference in HIV acquisition was found between vaccine and placebo arms overall, although an effect seemed to exist in nonwhite participants. One is tempted to conclude that the trial would have reached statistical significance if it had enrolled only nonwhites. The problem is that by chance alone we can often find one or more subgroups in which a benefit of treatment seems to exist, and other subgroups in which no benefit seems to exist. Peto (26) showed that with two equally-sized subgroups constituted completely at random, if the overall effect reaches a P-value of about 0.05, then the probability is about 1/3 that one subgroup will have a statistically significant treatment effect that is more than three times as large as the other subgroup’s treatment effect, which fails to reach statistical significance. Thus, one can be misled by subgroup results even in the simplest possible setting of only two equally sized subgroups. In practice, many subgroups of varying sizes often exist, which compounds the problem. ISIS-2 Investigators (27) illustrated the difficulty in interpreting subgroup effects in a factorial trial of aspirin and streptokinase on mortality. The trial showed a strong benefit of aspirin. In an effort to show how misleading subgroup results can be, investigators

MULTIPLE COMPARISONS

facetiously noted a slightly adverse effect of aspirin in patients born under the astrological signs of Gemini or Libra, but a strong positive effect of aspirin for patients of other astrological signs (P < 0.00001). The ISIS2 example highlights that the magnitude of multiplicity can be larger than it seems. Astrological sign sounds like it should comprise 12 components, but the example above combined two noncontiguous signs, Gemini and Libra. It is hard to know the actual magnitude of multiplicity because if the authors could not demonstrate their point by combining two astrological signs, then they may have tried combining three or more signs. The same sort of phenomenon has arisen from post-hoc combinations of different classes of heart failure, for example. Similarly, if subgroups are formed by somewhat arbitrary cutpoints on a numeric scale, then one may be inclined to experiment with different cutpoints until one is found that highlights subgroup differences. The true extent of multiplicity may be unknowable, in which case it is impossible to separate a real effect from the play of chance. The discussion so far has stressed the need for extreme caution in interpreting subgroup effects, but too much caution will cause us never to find real subgroup effects. After all, even with no adjustment for multiplicity, power in subgroups is often low. For this reason, a widely accepted compromise is first to test whether a subgroup by treatment interaction exist. If the interaction is not statistically significant, then one estimates the treatment effect in the subgroup by the overall treatment effect. If the interaction test is significant, then one reports the treatment effects observed in the different subgroups. It is important to keep in mind that although the treatment effect may differ numerically across subgroups (a quantitative interaction), it is unusual for the treatment to benefit one subgroup and harm another (a qualitative interaction) (26). 4.4 Interim Monitoring The monitoring row of Table 1 is completely dark, which suggests that adjustment is needed for monitoring. A popular and flexible method is to use a spending function

7

α*(t), which dictates the cumulative amount of type 1 error to be spent by time t of the trial, where t is measured in terms of relative information; t = 0 and 1 at the beginning and end of the trial, respectively. Making α*(1) = α ensures that the FWE will be controlled. The boundary at a given interim analysis depends on its time and the times of previous interim analyses, and it is computed using numerical integration. Neither the number nor the timing of interim analyses need be specified in advance. The properties of spending function boundaries depend on the particular spending function selected. Desirable spending functions spend relatively little alpha early, and then make up for that by steeply increasing for t close to 1. This causes early boundaries to be large, and the boundary at the end of the trial to be close to what it would have been without monitoring. A more thorough treatment of spending functions and other monitoring boundaries may be found in References 28 and 29. This article focused mostly on phase III clinical trials. Generally speaking, less emphasis is placed on multiplicity adjustment in earlier phase trials because such trials are used to decide whether more definitive phase III testing is justified rather than to declare the treatment beneficial.

REFERENCES 1. G. W. Snedecor and W. G. Cochran, Statistical Methods, 7th ed. Ames, IA: The Iowa State University Press, 1980. 2. K. Rothman, No adjustments are needed for multiple comparisons. Epidemiology 1990; 1: 43–46. 3. L. A. Moy´e, Multiple Analyses in Clinical Trials. New York: Springer-Verlag, 2003. 4. E. M. Laska and M. J. Meisner, Testing whether an identified treatment is best. Biometrics 1989; 45: 1139–1151. 5. M. A. Proschan and M. A. Waclawiw, Practical guidelines for multiplicity adjustment in clinical trials. Control. Clin. Trials 2000; 21: 527–539. 6. R. G. Miller, Simultaneous Statistical Inference. New York: Springer-Verlag, 1981. 7. D. B. Duncan, Multiple range and Multiple F-tests. Biometrics 1955; 11: 1–42.

8

MULTIPLE COMPARISONS 8. M. Proschan and D. Follmann, Multiple comparisons with control in a single experiment versus separate experiments: why do we feel differently? Am. Stat. 1995; 49: 144–149. 9. D. D. Waters, E. L. Alderman, J. Hsia, et al., Effects of hormone replacement therapy and antioxidant vitamin supplements on coronary atherosclerosis in postmenopausal women: a randomized controlled trial. J. Am. Med. Assoc. 2002; 288: 2432–2440.

10. Cardiac Arrhythmia Suppression Trial Investigators. Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N. Engl. J. Med. 1989; 321: 406–412. 11. Cardiac Arrhythmia Suppression Trial II Investigators. Effect of the antiarrhythmic agent moricizine on survival after myocardial infarction. N. Engl. J. Med. 1992; 327: 227–233. 12. Y. Hochberg and A. C. Tamhane, Multiple Comparison Procedures. New York: Wiley, 1987. 13. D. O. Clegg, D. J. Reda, C. L. Harris, et al., Glucosamine, chondroitin sulfate, and the two in combination for painful knee osteoarthritis. N. Engl. J. Med. 2006; 354: 795–808. 14. R. J. Cook, and V. T. Farewell, Multiplicity considerations in the design and analysis of clinical trials. J. R. Stat. Soc. A 1996; 159: 93–110. 15. M. D. Hughes, Multiplicity in clinical trials. In: P. Armitage, and T. Colton, eds. Encyclopedia of Biostatistics. New York: Wiley, 1998. 16. C. Dunnett, A multiple comparisons procedure for comparing several treatments with a control. J. Am. Stat. Assoc. 1957; 50: 1096–1121. 17. P. C. O’Brien, Procedures for comparing samples with multiple endpoints. Biometrics 1984; 40: 1079–1087. 18. D. A. Follmann, Multivariate tests for multiple endpoints in clinical trials. Stats. Med. 1995; 14: 1163–1176. 19. S. Holm, A simple sequentially rejective multiple test procedure. Scand. J. Stats. 1979; 6: 65–70. 20. Y. Hochberg, A sharper Bonferroni procedure for multiple tests of significance. Biometrika 1988; 75: 800–802. 21. R. J. Simes, An improved Bonferroni procedure for multiple tests of significance. Biometrika 1986; 73: 751–754. 22. A. Dmitrienko and J. C. Hsu, Multiple testing in clinical trials. In: S. Kotz, N. Balakrishnan,

C. B. Read, B. Vidakovic, and N. Johnson, eds. Encyclopedia of Statistical Sciences. New York: Wiley, 2006. 23. R. B. D’Agostino, Controlling alpha in a clinical trial: the case for secondary endpoints. Stats. Med. 2000; 19: 763–766. 24. C. E. Davis, Secondary endpoints can be validly analyzed, even if the primary endpoint does not provide clear statistical significance. Control. Clin. Trials 1997; 18: 557–560. 25. The rgp120 HIV Vaccine Study Group. Placebo-controlled, phase III trial of a recombinant glycoprotein 120 vaccine to prevent HIV-1 infection. J. Infect. Dis. 2005; 191: 654–665. 26. R. Peto, Clinical trials. In P. Price and K. Sikora, eds. Treatment of Cancer. London: Chapman and Hall, 1995. 27. ISIS-2 Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet 1988; 2: 349–360. 28. C. Jennison and B. W. Turnbull, Group Sequential Methods with Applications to Clinical Trials. Boca Raton, FL: Chapman and Hall/CRC Press, 2000. 29. M. A. Proschan, K. K. Lan, and J. T. Wittes, Statistical Monitoring of Clinical Trials: A Unified Approach. New York: Springer, 2006.

MULTIPLE ENDPOINTS

(2) multivariate global tests. Multiple testing approaches, such as the well-known Bonferroni procedure, make individual assessments for each endpoint under investigation. They are particularly advantageous if the endpoints can be classified before the study according to some hierarchy, as this natural ordering can then be explicitly used in the multiple testing approach. In contrast, global tests aim at assessments across all endpoints based on measuring the distance between multivariate populations. These methods include, for example, the classical multivariate analysis of variance (MANOVA) F-tests, which are particularly powerful if treatment differences are expressed through the combined effect of the endpoints. In the following, the main attributes underpinning multiple testing and global testing methodologies are briefly described.

FRANK BRETZ and MICHAEL BRANSON Norvatis Pharma AG Basel, Switzerland

1

INTRODUCTION

A common problem in pharmaceutical research is the comparison of two treatments for more than one outcome measure, which in the clinical context are often referred to as endpoints. A single observation or measurement is often not sufficient to describe a clinically relevant treatment benefit. In respiratory studies, for example, several endpoints (such as FEV1 , respiratory symptoms, and health-related quality of life) are frequently considered to determine a treatment-related benefit. In cardiovascular trials, possible endpoints include time to myocardial infarction, congestive heart failure, stroke, and so on. In such instances, the experimenter is interested in assessing a potential treatment effect while accounting for all multiple outcome measures. The aspects of a highly multidimensional and complex syndrome are usually assessed by means of various symptoms or ordinal items in scales. In order to map these observations on one (or a few) ordinal scale(s) that best represents the severity (or improvement) of the disease, one can try to reduce the dimensionality by combining the univariate projections of this syndrome to a single (or a few) measure(s) of efficacy. This approach is also in agreement with the ICH E9 guideline (1), in which it is recommended to use a single (primary) endpoint, where possible, thus reflecting the need to efficiently reduce the dimensionality. If, nevertheless, certain aspects cannot be combined into one scale or index because they describe inherently different dimensions, or if various measures need to be taken in order to capture the entire benefit range, multiple endpoints are unavoidable. Statistical test procedures for the analysis of multiple endpoints can roughly be classified into (1) multiple testing approaches and

2

MULTIPLE TESTING METHODS

If not stated otherwise, the comparison of two treatments j = 1, 2 with k > 1 endpoints being measured on nj patients are considered. No assumptions are made about the distribution of the measurements in this section. The interest lies in testing the k null hypotheses H i : θ 1i = θ 2i (the two treatments do not differ in effect) against the one-sided alternative hypotheses K i : θ 1i > θ 2i (treatment 1 is better than treatment 2), where θ ji denotes the mean effect of treatment j for endpoint i = 1, . . ., k. Note that the following results are directly extendable to the two-sided testing situation. A main concern when testing multiple hypotheses is the increased likelihood of rejecting incorrectly at least one true null hypothesis [familywise error rate, FWER; see Hochberg and Tamhane (2) for a detailed overview]. In particular, within the regulated environment in which drug development takes place, it is often required to control the FWER at a prespecified level α-frequently, but not restricted to, 2.5% onesided. A standard approach in this context is the Bonferroni procedure. Let pi denote the P-value for the ith hypothesis, Hi , as

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

MULTIPLE ENDPOINTS

obtained from applying an appropriate twosample test. If pi < α/k, H i can be rejected at FWER ≤ α, which follows from Bonferroni’s inequality 
 {pi < α/k} FWER = P i∈T  P(pi < α/k) ≤ α ≤ i∈T

where T denotes the set of all true null hypotheses. Several improvements of the Bonferroni procedure exist. The closure principle (CP) of Marcus et al. (3) is a general and powerful multiple testing procedure, which includes many of these improvements as special cases. For simplicity, its use is illustrated for k = 3 endpoints (see Fig. 1). Starting from the set of hypotheses of interest H 1 , H 2 , H 3 , all possible intersection hypotheses H12 , H 23 , H 13 , H 123 are created, where Hij = Hi ∩ Hj , 1 ≤ i, j ≤ 3, are the intersection hypotheses of two hypotheses at a time. For example, H23 states that endpoints 2 and 3 do not differ for both treatments. The overall intersection hypothesis H 123 is the global null hypothesis of no treatment effect across all three endpoints. According to the CP, a hypothesis H i is rejected at FWER α if Hi itself and all hypotheses formed by intersection with H i are each rejected at (local) level α. For example, in order to reject H1 in Fig. 1, one has to reject H 1 itself as well as H12 , H 13 , H 123 , where the choice of the individual level-α tests is free. Application of the Bonferroni approach to each hypothesis, for example, leads to the stepwise procedure of Holm (4): Let p(1) ≤ . . . ≤ p(k) denote the ordered unadjusted P-values with the associated hypotheses H (1) , . . ., H (k) . Then, H (i) is rejected if p(j) < α/(k − j + 1), j = 1, . . ., i (i.e., if all hypotheses H(j) preceding H (i) are also rejected). As a result of its stepwise nature, Holm’s procedure is more powerful than the Bonferroni procedure. A second important application of the CP is the allocation of the FWER α to a preordered sets of hypotheses (5). Chi (6), for example, advocated the definition of relevant decision paths before conducting the clinical trial so that the inherent multiplicity problem caused by multiple endpoints is reduced

by sequentially testing different sets of endpoints in a clinically relevant and meaningful way. It is, therefore, common practice in clinical trials to classify the endpoints into primary, co-primary, and secondary endpoints (6) and to test them in a prespecified order of importance, thus reflecting the need to efficiently reduce the multiplicity. Different strategies of allocating the FWER α between the sets of hypotheses are possible (6, 7), most of which can ultimately be derived by applying the CP. Fixed sequence methods test the individual endpoints in a prespecified order, each at level α, where the nonrejection of a hypothesis at any step stops the test procedure (5). The fixed sequence approach is only applicable, when a complete hierarchy of the endpoints is available and a hypothesis is only of interest and thus tested if the higher prioritized hypotheses have all been rejected beforehand (8). Gatekeeping methods first test a preliminary hypothesis and, in case of a rejection, continue testing additional hypotheses (9). This approach may be required if a single primary endpoint is to be tested for significance before further secondary endpoints are to be analyzed. More general gatekeeping procedures are introduced in (10). An important point to note is that of requiring statistical significance for all primary (multiple) endpoints before being able to claim ‘‘success.’’ In this context, no formal adjustment of the FWER is necessary (i.e., each endpoint is tested separately at level α) (8). What should be carefully evaluated and understood is the impact on the power for such a hypothesis testing strategy. Other multiplicity adjustments exist that improve on Bonferroni’s inequality, in particular the Simes test (11) and its stepwise extension (12). All of the methods considered so far, however, do not account for the inherent correlation between different endpoints. Exploiting the association between the endpoints can considerably improve the methods above. Under the standard ANOVA assumptions and if the correlations are known (or at least well approximated), the endpointspecific t-tests are jointly multivariate t distributed (13) and efficient integration routines can be used for the design and analysis

MULTIPLE ENDPOINTS

3

H123

Figure 1. CP hypothesis tree for three endpoints

H12

H13

H23

H1

H2

H3

of related clinical trials (14). If the correlations are unknown (which is the typical case in practice), a standard alternative is resampling the data to approximate the joint distribution of the test statistics (15, 16). Different resampling schemes are available, each of which has its own advantages. The key idea is to permute, a large number of times, the entire patient-specific observation vectors such that the complete information for a single patient, across the endpoints, is maintained. Such procedure retains the (stochastic) dependencies between the endpoints under the null hypothesis when no real differences exist among the treatments. For each resampling step, the test statistics are computed based on the resampled data. Then, the originally observed test statistics are compared with the resampled test statistics via the resampling distribution, where extreme values indicate a significant treatment effect. The resampling methods are available for a variety of multiple comparison procedures, in particular for the CP based on t-statistics (15). In addition, extensions to the comparison of multiple treatments are possible and software is readily available such as PROC MULTTEST in SAS.

3

MULTIVARIATE GLOBAL TESTS

The classic approach for comparing two multivariate normal populations with an unknown common covariance matrix is Hotelling’s T 2 test (17)

T2 =

n1 n2 (Y1 − Y2 ) S−1 (Y1 − Y2 ) n1 + n2

where Yj denotes the vector of sample means,  nj S = j l=1 (Yjl − Yj )(Yjl − Yj ) /ν denotes the sample covariance matrix with ν = n1 + n2 − 2, and Yjl denotes the observation vector for patient l under treatment j = 1, 2. Under the null hypothesis of no treatment effect across all endpoints, (n1 + n2 − k − 1)T 2 is k(n1 + n2 − 2)Fν1 , ν2 distributed with ν 1 = k and ν 2 = n1 + n2 − k − 1 degrees of freedom. Note that T 2 can be regarded as a multivariate generalization of the twosample t test to the multivariate setting. The test statistic T 2 is the squared maximum of univariate t test statistics of all linear combinations of the endpoints. In addition, the T 2 test has several optimality properties. Among others, it is uniformly the most powerful of all tests invariant to scale transformations (18). Several extensions to situations with more than two treatments exist, the most notable being Wilks’ , Pillai’s trace, Hotelling-Lawley trace, and Roy’s maximum root, all of which are approximately F distributed (18). All of the afore-mentioned global tests were developed for the two-sided alternative and thus lack power in practical applications involving one-sided directional decisions. A variety of one-sided tests in the multivariate setting exists, although many problems remain unsolved [see Tamhane and Logan (19) for a recent review of these methods]. Kudˆo (20) derived the exact likelihood ratio (LR) test for the test problem H: θ = 0 vs. K: θ ≥ 0 with at least one θ i > 0, when the covariance structure is known. Perlman (21) extended the LR test to situations with an unknown covariance structure. However, the exact distribution of the LR test is not free

4

MULTIPLE ENDPOINTS

of the unknown nuisance parameters. Moreover, the LR test is biased, so that research to circumvent these difficulties is still ongoing (22, 23). O’Brien (24) proposed a different solution by restricting the alternative space K, where the standardized treatment differences (θ 1i − θ 2i )/σ i , i = 1, . . ., k, are assumed to be all positive and of the same magnitude. Using ordinary least square (OLS) and generalized least square (GLS) methods, O’Brien (24) showed that the resulting LS statistics are standardized weighted sums of the individual −1 t statistics ti = (Y 1i − Y 2i )/ s2i (n−1 1 + n2 ),

where Y ji , j = 1, 2, and s21 denote, respectively, the mean response and pooled sample variance of endpoint i = 1, . . ., k. More  specifi ˆ cally, the OLS statistic is tOLS = i ti / 1
R1, ˆ where R denotes the pooled sample correlation matrix. Alternatively, the GLS method  ˆ −1 t/ 1
R ˆ −1 1 weights the t statistGLS = 1
R ˆ where t is the vector of t tics according to R, statistics from above. Note that by construction, tGLS may include negative weights for certain correlation structures so that it is possible to reject H in favor of positive treatment effects when, in fact, the opposite is true (25). As the exact distribution of tOLS and tGLS is unknown, approximations have to be used. O’Brien (24) proposed the use of a t distribution with ν(= n1 + n2 − 2) degrees of freedom. For small sample sizes, the LS tests can be either conservative or liberal, depending on the parameter constellations. Improved small sample approximations can be found in Logan and Tamhane (26). For large sample sizes, the limiting standard normal distribution can be used. ¨ ¨ In contrast, Lauter (27) and Lauter et al. (28) derived exact one-sided level-α tests without restricting the multivariate alternative region. They gave conditions under which choosing data-dependent weight vectors w, lead to the test statistics   wi ti n1 n2 tw =  i n1 + n2 w
Rw ˆ being t distributed with ν degrees of freedom under H. Let Y i denote the total variablewise mean over both samples. A common choice is then to set wi = νii−1 , where

 nj νii = 2j=1 l=1 (Yijl − Y i )2 is the ith diagonal element of the total product matrix, leading to the standardized score (SS) test. Logan and Tamhane (26) compared the power of the OLS and the SS tests. They found out by analytical methods that (1) if a single endpoint exists with a positive effect, the OLS test is more powerful than the SS test; and (2) if all endpoints have a positive effect with the same magnitude (which is the underlying assumption of the LS tests), both tests are equally powerful. Moreover, the authors conducted a simulation study for a variety of scenarios, showing that both the OLS test and the SS test behave similar, in terms of power, throughout most of the alternative region. Alternatively to the SS test, the first principal component calculated from the total product matrix can be used, leading to the PC test (27, 28). The PC test has higher power than the OLS test over a wide range of the alternative, such as, for example, if some variables have an effect and others not.

4 CONCLUSIONS In this article, some of the existing methods to analyze multiple endpoints have briefly been reviewed. The main application of these methods are clinical trials focusing on relatively few efficacy endpoints, where a strong control of the type I error rate is mandatory. It is important to reiterate that the multiple testing methods, as discussed above, are P-value based approaches, providing a high degree of flexibility in tailoring the testing strategy to suit the particular application. Such procedures are not restricted to particular data types, and are applicable for the analysis of, for example, normal, binary, count, ordinal, and time-to-event endpoints. An interesting and evolving method in this context is the use of the CP in adaptive designs, which allows the user to select and confirm endpoints within a single trial (29, 30). A further approach is to apply multivariate tests to the intersection hypotheses of the CP (31). Such a hybrid approach thus combines the advantages of multiple testing methods (assessment of the individual hypotheses) and multivariate global test

MULTIPLE ENDPOINTS

(taking the multivariate nature of data into account.) Other applications may require different statistical methods than reviewed in this article. Longitudinal data, for example, may include (nonlinear) mixed-effects models to describe the stochastic dependencies between the time points (32). The analysis of multivariate time-to-event outcomes has been the topic of much research, and standard implementations based on a counting-process formulation (33, 34) can be easily implemented in standard statistical software (35). Safety (adverse event) data may require analysis using novel multiple testing techniques based on controlling the false discovery rate (36, 37) or by, for example, using hierarchical Bayes models as proposed by Berry and Berry (38). Finally, high-dimensional screening studies with thousands of endpoints, such as gene expression profiling, for example, often use completely different statistical tools again (39, 40). REFERENCES 1. ICH Tripartite Guideline E9. (1998). International conference on harmonization; guidance on statistical principles for clinical trials. (online). Available: http://www.fda.gov/80/cder/guidance/. 2. Y. Hochberg and A. C. Tamhane, Multiple Comparison Procedures. New York: Wiley, 1987. 3. R. Marcus, E. Peritz, and K. R. Gabriel, On closed testing procedures with special reference to ordered analysis of variance. Biometrika 1976; 63: 655–660. 4. S. Holm, A simple sequentially rejective multiple test procedure. Scand. J. Statist. 1979; 6: 65–70. 5. W. Maurer, L. A. Hothorn, and W. Lehmacher, Multiple comparisons in drug clinical trials and preclinical assays: a-priori ordered hypotheses. In: J. Vollman (ed.), Biometrie in der Chemische-Pharmazeutichen Industrie, vol. 6. Stuttgart, Germany: Fischer Verlag, 1995. 6. G. Chi, Multiple testings: Multiple comparisons and multiple endpoints. Drug Inform. J. 1998; 32: 1347S–1362S. 7. P. H. Westfall and A. Krishen, Optimally weighted, fixed sequence and gatekeeper multiple testing procedures. J. Stat. Plan. Infer. 2001; 99: 25–40.

5

8. CPMP Points to Consider. (2002). PtC on multiplicity issues in clinical trials. CPMP/EWP/908/99. (online). Available: http://www.emea.eu.int/pdfs/human/ewp/ 090899en.pdf. 9. P. Bauer, J. R¨ohmel, W. Maurer, and L. A. Hothorn, Testing strategies in multi-dose experiments including active control. Stat. Med. 1998; 17: 2133–2146. 10. A. Dmitrienko, W. W. Offen, and P. H. Westfall, Gatekeeping strategies for clinical trials that do not require all primary effects to be significant. Stat. Med. 2003; 22: 2387–2400. 11. R. J. Simes, An improved Bonferroni procedure for multiple tests of significance. Biometrika 1986; 73: 751–754. 12. Y. A. Hochberg, Sharper Bonferroni procedure for multiple tests of significance. Biometrika 1988; 75: 800–802. 13. F. Bretz, A. Genz, and L. A. Hothorn, On the numerical availability of multiple comparison procedures. Biometric. J. 2001; 43: 645–656. 14. A. Genz and F. Bretz, Comparison of methods for the computation of multivariate probabilities. J. Comput. Graph. Stat. 2002; 11: 950–971. 15. P. H. Westfall and S. S. Young, Resamplingbased multiple testing. New York: Wiley, 1993. 16. J. F. Troendle, A permutational step-up method of testing multiple outcomes. Biometrics 1996; 52: 846–859. 17. H. Hotelling, The generalization of Student’s ratio. Ann. Math. Stat. 1931; 2: 360–378. 18. T. W. Anderson, An Introduction to Multivariate Statistical Analysis, 3rd ed. New York: Wiley, 2003. 19. A. C. Tamhane and B. R. Logan, Multiple endpoints: an overview and new developments. Technical Report, 43. Milwaukee, WI: Division of Biostatistics, Medical College of Wisconsin, 2003. 20. A. Kudˆo, A multivariate analogue of the oneside test. Biometrika 1963; 15: 403–418. 21. M. D. Perlman, One-sided testing problems in multivariate analysis. Ann. Stat. 1969; 40: 549–567. 22. M. D. Perlman and L. Wu The Emperor’s New Tests (with discussion). Statistical Science 1999; 14: 355–381. 23. M. S. Srivastava, Methods of multivariate statistics. New York: Wiley, 2002. 24. P. C. O’Brien, Procedures for comparing samples with multiple endpoints. Biometrics 1984; 40: 1079–1089.

6

MULTIPLE ENDPOINTS

25. S. J. Pocock, N. L. Geller, and A. A. Tsiatis, The analysis of multiple endpoints in clinical trials. Biometrics 1987; 43: 487–498. 26. B. R. Logan and A. C. Tamhane, On O’Brien’s OLS and GLS tests for multiple endpoints. In: Y. Benjamini, F. Bretz, and S. Sarkar (eds.), New Developments in Multiple Comparison Procedures. IMS Lecture Notes - Monograph Series 47. 2004, pp. 76–88. ¨ 27. J. Lauter, Exact and F test for analyzing studies with multiple endpoints. Biometrics 1996; 52: 964–970. ¨ 28. J. Lauter, E. Glimm, and S. Kropf, Multivariate tests based on left-spherically distributed linear scores. Ann. Stat. 1998; 26: 1972–1988.

33. R. L. Prentice, B. J. Williams, and A. V. Peterson, On the regression analysis of multivariate failure time data. Biometrika 1981; 68: 373–379. 34. L. J. Wei, D. Y. Lin, and L. Weissfeld, Regression analysis of multivariate incomplete failure time data by modeling marginal distributions. J. Amer. Stat. Assoc. 1993; 84: 1065–1073. 35. T. M. Therneau and P. M. Grambsch, Modeling survival data: Extending the Cox Model. New York: Springer, 2000. 36. Y. Benjamini and Y. Hochberg, Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Royal Stat. Soc. B 1995; 57: 289–300.

29. M. Kieser, P. Bauer, and W. Lehmacher, Inference on multiple endpoints in clinical trials with adaptive interim analyses. Biometric. J. 1999; 41: 261–277. 30. G. Hommel, Adaptive modifications of hypotheses after an interim analysis. Biometric. J. 2001; 43: 581–589. 31. W. Lehmacher, G. Wassmer, and P. Reitmeir. Procedures for two-sample comparisons with multiple endpoints controlling the experimentwise error rate. Biometrics 1991; 47: 511–521.

37. D. V. Mehrotra and J. F. Heyse, Use of the false discovery rate for evaluating clinical safety data. Stat. Meth. Med. Res. 2004; 13: 227–238 38. S. M. Berry and D. A. Berry, Accounting for multiplicities in assessing drug safety: a three-level hierarchical mixture model. Biometrics 2004; 60: 418–426. 39. T. Speed, Statistical Analysis of Gene Expression Microarray Data. Boca Raton, FL: CRC Press, 2003.

32. J. Pinheiro and D. Bates, Mixed-Effects Models in S and S-PLUS. New York: Springer, 2000.

40. G. Parmigiani, E. S. Garett, R. A. Irizarry, and S. L. Zeger, The Analysis of Gene Expression Data. New York: Springer, 2003.

MULTIPLE EVALUATORS

and measure agreement? What is the overall level of agreement? How much bias and variance is there among evaluators? In short, the goal of an agreement study is two-fold. The first goal is to determine if the measurements from multiple evaluators agree with each other. If not, then the second goal is to identify where the difference(s) occur and correct/calibrate them if possible. The agreement problem covers a broad range of data, and examples can be found in different disciplines. The scale of a measurement can be continuous, binary, nominal, or ordinal. The rest of this article is organized as follows. In the next section, the approaches for assessing agreement of continuous data are reviewed. We classify these approaches into three categories. The first category is the hypothesis testing-type approach; the second category is an index approach; and the third category is the interval-type approach. In the section entitled, ‘‘Agreement for categorical data,’’ the approaches for assessing agreement of categorical data are reviewed. A summary is provided in the last section.

JASON J. Z. LIAO and ROBERT C. CAPEN Merck Research Laboratories, West Point, Pennsylvania

1

INTRODUCTION

In medical and other related sciences, clinical or experimental measurements usually serve as a basis for diagnostic, prognostic, therapeutic, and performance evaluations. The measurement can be from multiple systems, processes, instruments, methods, raters, and so forth, but for the sake of simplicity, we refer to them as ‘‘evaluators’’ throughout this article. As technology continues to advance, new methods/instruments for diagnostic, prognostic, therapeutic, and performance evaluations become available. Before a new method or a new instrument is adopted for use in measuring a variable of interest, its agreement relative to other similar evaluators needs to be assessed. Measurements of agreement are needed to assess the reliability of multiple raters (or the same rater over time) in a randomized clinical trial (RCT). These measurements of agreement can be used for assessing the reliability of the inclusion criteria for entry into an RCT, validating surrogate endpoints in a study, determining that the important outcome measurements are interchangeable among the evaluators, and so on. An agreement study involving multiple evaluators can happen in all phases of drug development or other medical-related experimental settings. A typical design for an agreement study involves sampling n subjects from the population, sampling d evaluators from a population of evaluators, and obtaining r replicate determinations/evaluator. Each evaluator is ‘‘blinded’’ to all others to ensure independence of measurement error; the measurements represent the range of ratings proposed to be used interchangeably in the RCT. Various questions regarding agreement can be posed, such as the listings by Bartko (1). Can the measurements from evaluators be used interchangeably? How does one define

2

AGREEMENT FOR CONTINUOUS DATA

Let Y ij be the continuous measurement made by the j-th evaluator on the i-th subject (i = 1, . . . , n, and j = 1, . . . , d). The goal is to assess the agreement among the measurements made by the d evaluators. In the case of d = 2, the paired measurements of a perfect agreement from the two evaluators fall exactly on the identity line Y 2 = Y 1 , for example, the 45◦ line through the origin. The existing approaches can be classified into three categories. An appropriate transformation, such as logarithm, of the data may be recommended and used to better meet the assumptions under a specified approach. Therefore, Y ij are the final chosen reportable values used in the agreement evaluations. 2.1 Hypothesis Testing Approach The first approach of this type is the paired t-test to compare the mean value of two measurements for d = 2, and the F-test to compare the mean value of d(d > 2) mea-

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

MULTIPLE EVALUATORS

surements (1). The second approach in this category is a hypothesis of the parameters from, for example, different types of regression settings (2–9). All these hypothesis-type approaches heavily depend on the residual variance, which can reject a reasonably good agreement when the residual errors are small (more precision) but fail to reject a poor agreement when the residual errors are large (less precision) (10). Other critiques of these approaches can be found in Reference 11. Thus, any kind of approach based on a hypothesis testing strategy is not recommended for assessing agreement. 2.2 An Index Approach Two commonly used approaches comprise this category: the intraclass correlation coefficient (ICC) and the concordance correlation coefficient (CCC). Fleiss (12) gave an overview of ICC as a measure of agreement among multiple evaluators. In defining ICC, a useful two-way random effects model is given by Yij = µ + si + rj + eij , i = 1, . . . , n, and j = 1, . . . , d,

(1)

where µ is a fixed parameter and where si , rj , and eij are independent random effects with mean 0 and variances σs2 , σr2 , and σe2 , respectively. Usually d = 2. The term si is the subject effect, rj is the systematic bias (i.e., the systematic error), and eij is the measurement error. To assess the agreement among multiple evaluators by using the concept of intraclass correlation coefficient, the index is defined as ρu =

σs2 σs2 + σr2 + σe2

(2)

In Equation (1), the denominator of ρ u is the variance of Y ij (unconditionally on j), and the numerator of ρ u is the covariance between Y ij1 and Y ij2 (unconditionally on j1 = j2). Thus, ICC defined in Equation (2) is the correlation between any two arbitrary measurements made on the same subject.

Another measuring index is the ICC defined as follows: ρc =

σs2

σs2 + σe2

(3)

ρ c in Equation (3) does not take systematic error into account in assessing agreement under Equation (1). Contrasting this with the ICC defined in Equation (2), the denominator of ρ c in Equation (3) is the variance of Y ij conditionally on j. Thus, ρ c is the correlation between Y ij1 and Y ij2 conditionally on j1 = j2, which is the usual product–moment correlation between two specific measurements made on the same subject. Note that ρ u = ρ c if we assume a model without systematic error, that is σr2 = 0 in Equation (1), which is also referred to as the one-way random effects model (13). The one-way random model is as follows: Yij = µ + si + eij , i = 1, . . . , n, and j = 1, . . . , d,

(4)

Again, ICC from Equation (4) is the simple correlation between Y ij1 and Y ij2 . Therefore, we can estimate the ICCs using the Pearson correlation coefficient. However, the estimation of ICCs based on applying analysis of variance (ANOVA) is the most commonly used and recommended approach (13). Among the various methods proposed for calculating the confidence intervals of the ICCs, the one proposed by Cappelleri and Ting (14) almost exactly maintains the nominal coverage probability for typical agreement studies (15). As pointed out by Rousson et al. (16), the main difference between these two ICCs is that the value of ρ u is attenuated when systematic error exists, whereas ρ c is not. Another difference is that ρ u does not depend on the ordering of the d measurements, whereas ρ c does. For assessing interevaluator agreement, Rousson et al. (16) recommended using ρ u because a systematic bias clearly indicates a failure in agreement and should be taken into account. In contrast, they recommended using ρ c for test–retest agreement because the first and the second

MULTIPLE EVALUATORS

trial in this situation are clearly not interchangeable. In assessing agreement for two evaluators, Lin (10) claimed that we should divide the assessment into two steps. First, a precision step assesses how close the observations are to the regression line. Second, an accuracy step assesses how close the regression line and the target line (the line of identity) are to each other. Using squared differences under the assumption that the observations are from a bivariate normal distribution and µ2 , and covariance matrix with mean µ1 
σ12 ρσ1 σ2 , Lin (10) proposed a concorρσ1 σ2 σ22 dance correlation coefficient (CCC) for d = 2 (denoted as CCC = ρCa ) as follows: CCC = 1 −

E(Y2 − Y1 )2 E[(Y2 − Y1 )2 |ρ = 0]

= ρCa =

2ρσ1 σ2 σ12 + σ22 + (µ1 − µ2 )2

(5)

This index combines the measure of precision (ρ) and measure of accuracy 
Ca =

2σ1 σ2 σ12 +σ22 +(µ1 −µ2 )2

together. Lin (10) esti-

mated his CCC by replacing the parameters with the corresponding samples values and recommended using the Fisher  Z- transformation Z(r) = tanh−1 (r) = 12 ln 1+r 1−r for inference. However, some concerns for using CCC exist (17–20). Note that the CCC is a special case of the formula for ICC (19, 21). Therefore, two improved CCCs have been proposed in the literature (22, 23). Many critiques argue against using the index approaches. In assessing agreement, all indices assumed observations from a distribution (usually normal) with a fixed mean and constant covariance. However, the fixed mean assumption is often not satisfied in practice (24–26). In addition, usually a single index is not good enough to measure the degree of agreement (27). When an index indicates a poor agreement, no indication shows what is wrong. When poor agreement occurs, it is important to determine the degree of bias (fixed and/or proportional biases). No current agreement index can provide this information. As pointed out by Bland and Altman (11, 28), a correlation method is very sensitive

3

to the range of the measurements available in the sample. The greater this range, the higher the correlation. In addition, it is not related to the actual scale of measurement or to the size of error that might be clinically allowable. Related to this point, Liao and Lewis (20) gave examples where nested experiments lead to conflicted conclusions. Because any index approach is very sensitive to sample heterogeneity, therefore, Atkinson and Nevill (18) suggested no index approach should be used to assess agreement. 2.3 An Interval Approach The commonly used approach in this category was proposed for d = 2 evaluators by Bland and Altman (28). Let Di , Y i2 − Y i1 , i = 1, . . . , n. Assuming the Di s are approximately normally distributed, Bland and Altman (11, 28) proposed using the 95% interval of the observed differences [D − 2SD , D + 2SD ]

(6)

which they called ‘‘limits of agreement’’ to n  measure agreement, where D = n1 Di and

S2D

=

1 n−1

n 

i=1

(Di −

D)2 .

These limits are

i=1

then compared with scientifically acceptable boundaries. The interval in Equation (6) will ensure that 95% of all differences will fall into it under the assumption of normality. This method does not depend on the range of the sample, and the limits of agreement give some indication whether the discrepancy is acceptable in practice by comparing the limits with a scientifically justifiable boundary. As a supplement, they also proposed a mean–difference graphic that plots the difference Di against the mean of the two measurements M i = (Y i1 + Y i2 )/2 along with the 95% limits of the difference, for example, the limits of agreement. They claimed that this graphical tool could be used to investigate the assumptions and possible trends in systematic error (bias) and/or in measurement error, thus, leading to a possible transformation of the responses being compared. These trends can be tested using an appropriate technique, such as the Spearman rank correlation between |Di | and M i .

4

MULTIPLE EVALUATORS

The Bland and Altman approach is a favorite of medical researchers with over 11,000 citations. However, some concerns about this method exist. First, it creates a problem of interpretation when a mixture of fixed, proportional bias and/or proportional error occurs (29). Second, it is only good for the test–retest situation (16). Third, the level and type of bias between the two sets of measurements cannot be fully assessed, and covariate information cannot be incorporated into this approach (25, 26). Fourth, the mean–difference plot gives artifactual bias information in measurements differing only in random error (30). A new approach that overcomes these concerns was proposed in References 25 and 26. 3

AGREEMENT FOR CATEGORICAL DATA

rect for such marginal disagreement (35), it is preferable that it be investigated as a potential source of interevaluator difference (36). Existing approaches for categorical data generally focus on assessing agreement through the calculation of a kappa-like index or by modeling the pattern of agreement in the data. 3.1 Measuring Agreement between Two Evaluators 3.1.1 Kappa. Suppose two evaluators are to classify n subjects into m mutually exclusive and exhaustive categories. Following Banerjee et al. (32), let pkk be the proportion of subjects placed into category k(k = 1, 2, . . . , m) by both evaluators and define m  pkk as the observed proportion of p0 = k=1

Since the development of the kappa coefficient (31), a considerable amount of research has been performed in the area of interevaluator agreement for categorical data (see, for example, Reference 32). Cohen originally developed kappa as a measure of chancecorrected agreement between two evaluators for nominal ratings (two or more ratings that cannot be logically ordered, such as positive/negative, schizophrenic/bi-polar/ neurotic/depressed, etc.). However, kappa itself is affected by the prevalence of the illness (33), an issue that will be discussed in more detail in the section entitled, ‘‘Issues in kappa.’’ Understanding the underlying assumptions behind the methods as well as their limitations is critical when analyzing categorical agreement data. For example, in the development of kappa, Cohen assumed independence between the evaluators. When quantifying the level of agreement, this assumption might be questionable when the evaluations from one clinician are known in advance by a second clinician, or if the evaluations are done over time, without blinding, by the same clinician. A well-publicized shortcoming of kappa is its inability to differentiate between two components of disagreement (34). Although it is possible to cor-

agreement. A portion of this overall agreement will be due to chance alone. Call this portion pc . Kappa is defined as the achieved agreement beyond chance relative to the potential agreement beyond chance (33). p0 − pc (7) κ= 1 − pc Cohen (31) defined chance agreement in a natural way as pc =

m 

pk· p·k

(8)

k=1

where pk· is the proportion of subjects placed into the kth category by the first evaluator and p·k is the proportion of subjects placed into the kth category by the second evaluator. When disagreements between evaluators are not equally weighted, the use of kappa is problematic. 3.1.2 Weighted Kappa. Cohen (37) generalized the definition of kappa to include situations when disagreements are not equally weighted. Let njk be the number of subjects in the (j, k)th cell of an m × m table. The total m  number of subjects is n = njk . Define j, k = 1

MULTIPLE EVALUATORS

wjk to be the weight corresponding to the (j, k)th cell. Then, weighted kappa, which measures the proportion of weighted agreement corrected for chance (32), is defined as κw =

p0w − pcw 1 − pcw

(9)

where p0w =

m m  1  wjk njk = wjk pjk n j, k = 1

pcw =

1 n2

m  j, k = 1

j, k = 1

wjk nj· n·k =

m 

wjk pj· p·k (10)

j, k = 1

Three common choices for the weights are provided in Equation (11). Weights can also be selected based on disagreement (37).  wjk =

1, j = k (Unweighted) 0, j =  k

wjk = 1 −

|j − k| (Linear Weights) (m − 1)2

wjk = 1 −

(j − k)2 (Quadratic Weights) (m − 1)2 (11)

Hypothesis testing and confidence interval construction follow similarly to the unweighted case (38–40), although a large number of subjects are generally required to construct confidence intervals for weighted kappa even for a modest number of categories (32). Note that for 2 × 2 tables, use of either linear or quadratic weights is equivalent to calculating unweighted kappa. For more than two categories, each weighing scheme assigns less weight to agreement as the categories get farther apart. The greatest disparity in ratings is assigned an agreement weight of 0. In general, for more than two categories the use of quadratic weights is common (33). 3.1.3 Agreement Measures for Ordinal Data. In many settings, the potential ratings that can be assigned to a subject form a natural ordering. Such ordinal scales often arise because a truly continuous variable is discretized. For example, the degree of

5

motion of a particular joint, which could be measured, might instead be categorized as ‘‘unrestricted,’’ ‘‘slightly restricted,’’ ‘‘moderately restricted,’’ and ‘‘highly restricted’’ (33). Although the kappa statistics mentioned above can, in theory, be extended to ordinal data, doing so will generally result in a loss of statistical power (41, 42) and could lead to misleading results (43). Various alternative methods to kappa exist for assessing agreement for ordered categories (44). The approach developed by Rothery (45) is a nonparametric analog to the ICC and has a straightforward interpretation (44). 3.2 Extensions to Kappa and Other Approaches for Modeling Patterns of Agreement 3.2.1 Extensions to Kappa. Since the development of kappa more than 45 years ago, most research has focused on the case of two evaluators using a binary scale to classify a random sample of n subjects. Both Banerjee et al. (32) and Kraemer et al. (43) provide nice discussions on various extensions to kappa. We refer the reader to their work for more details. Extension to the case of multiple evaluators was treated in Reference 46. Fleiss assumed that each of n subjects was rated by an equal number (d > 2) of evaluators into one of m mutually exclusive and exhaustive nominal categories, where, for each subject, the set of evaluators was randomly selected from a ‘‘population’’ of evaluators. Davies and Fleiss (47) developed a chance-corrected statistic for measuring agreement assuming, among other things, that a common set of evaluators rated all subjects. Kraemer (48) allowed for a different number of ratings per subject as well as for the same evaluator to place a subject into more than one category. By considering the placement of each observation in the rank ordering of the m categories, Kraemer could derive a kappa-like statistic as a function of the Spearman rank correlation coefficient. Kraemer et al. (43) warned that the use of kappa when more than two nominal categories exist is suspect regardless of the number of evaluators. 3.2.2 Modeling Patterns of Agreement. Up to this point, the focus has been centered on

6

MULTIPLE EVALUATORS

assessing agreement through the calculation of a single index. Alternatively, one can model the pattern of agreement in the data. Various sophisticated strategies have been developed for doing this modeling, and a discussion of them is beyond the scope of this article. The most common approaches include the use of log-linear models (49–51), latent-class models (50, 52–56), and generalized estimating equations (57–60). 3.3 Issues with Kappa Although the use of kappa statistics is widespread, appearing in the psychology, education, physical therapy, medical, and psychiatry literature (to name a few), its usefulness as a metric to gauge interevaluator agreement is not universally accepted (61). A thorough discussion of what kappa is designed to do is provided in Kraemer et al. (43). In this section, we will describe some more common issues surrounding the use of kappa. It is not meant to be an exhaustive list. 1. Trait prevalence. When the prevalence of a disease or trait is rare, the kappa statistic can be misleading. Feinstein and Cicchetti (62) discussed this paradox and ways to resolve it. Sim and Wright (33) defined a prevalence index in the context of a 2 × 2 table as |p11 − p22 |. When the prevalence index is high, chance agreement is very likely and kappa will be attenuated as a result. 2. Evaluator bias. Bias measures the extent that two evaluators disagree on the proportion of ‘‘positive’’ (or ‘‘negative’’) cases (33). This bias is different than bias relative to a ‘‘gold standard.’’ However, a small level of bias, as determined by the bias index of Sim and Wright (33), does not imply a large value for kappa. In fact, any time the proportion of subjects in the (1,2) cell is the same as the proportion of subjects in the (2,1) cell, their bias index will be 0. A calculation of both the bias and prevalence index should accompany any calculation of kappa. 3. Kappa does not distinguish among types and sources of disagreement. By placing less weight on agreement for

categories that are more separated, this issue can be, to some degree, overcome. Still, different weighting schemes will lead to different values for kappa, so using any type of ad hoc rule for determining the strength of agreement (see, for example, Reference 63) is inappropriate. 4. Nonindependent ratings. An important assumption in the development of kappa is that evaluators generate ratings independently of one another. The independence assumption is often satisfied in practice by appropriate care in the design of the agreement study. Dependent ratings will generally lead to inflated values of kappa. Two typical settings where the independence assumption is questionable are where (1) one evaluator is aware of the ratings of the other evaluator and (2) the same evaluator rates the same set of subjects, without blinding, on two different occasions. In the latter setting, Streiner and Norman (64) suggest that a time interval of 2–14 days is reasonable but depends on the attribute being evaluated. 5. Ordinal data. Calculating kappa on data that are continuous but have been categorized is a practice that should be avoided. It is much better to preserve the original scale and apply measures appropriate for continuous data to assess agreement (see the section entitled ‘‘Agreement for continuous data’’). When it is impossible to obtain the continuous measure or when the data are ordinal in their original form, the weighted form of kappa is often used to assess agreement. This approach is questionable because it is generally not as powerful as other measures of agreement specifically developed for ordinal data (e.g., see Reference 45). 4 SUMMARY AND DISCUSSION In this article, we have reviewed approaches for assessing measurement agreement among multiple evaluators. For continuous data, the index approach and the interval approach are

MULTIPLE EVALUATORS

commonly used for assessing the agreement. Because of its simplicity and its intuitive and practical interpretability, we particularly recommend using the interval approach instead. For the common situation of two evaluators and two nominal categories, we primarily discussed kappa and weighted kappa statistics and mentioned several issues surrounding their usefulness to assess agreement. Trying to describe the degree of agreement among two or more evaluators by a single index is overly optimistic. Even for the simplest case of two evaluators and binary categories, besides reporting the value for a kappa statistic, one should report the proportion of overall agreement and the proportions of agreement specific to each category. For multiple evaluators, the strategy for two evaluators can be adopted for pairs of evaluators. If the underlying attribute is continuous, one can use log-linear models, latent class models, or generalized estimating equations to investigate, among other things, interevaluator agreement. A similar strategy can be used for the case of nominal or ordinal scale data, although for ordinal data, one should avoid the use of kappa. The design of an agreement study is very important. At the very least, design aspects that must be considered include the procedures for selecting the subjects and the evaluators. The population from which the subjects are selected must be well–defined, and the evaluators must be well trained/validated. We advocate collecting r replicate evaluations for each subject by each evaluator. If the r replicate evaluations are to be completed over time, then the time interval must be chosen to minimize the possibility of correlated ratings while still ensuring that the attribute being observed has not meaningfully changed in the population or sample. A consequence of violating the latter requirement is that the ratings would become time-dependent (i.e., not interchangeable). Other important issues regarding the general design of agreement studies are discussed in References 11, 13, 44, 65, and 66. For the particular issue of sample size, see References 15, 67, and 68, for continuous data, and see References 33, 69, and 70 for categorical data.

5

7

ACKNOWLEDGMENTS

The authors thank Dr. Christy Chuang-Stein and two referees for their valuable comments and suggestions that improved this article. REFERENCES 1. J. J. Bartko, Measurements of agreement: a single procedure. Stat. Med. 1994; 13: 737–745. 2. W. E. Deming, Statistical Adjustment of Data. New York: Wiley, 1943. 3. U. Feldmann, B. Schneider, H. Klinkers, and R. Haeckel, A multivariate approach for the biometric comparison of analytical methods in clinical chemistry. J. Clin. Chem. Clin. Biochem. 1981; 19: 121–137. 4. U. Feldmann and B. Schneider, Bivariate structural regression analysis: a tool for the comparison of analytic methods. Methods Informat. Med. 1987; 6: 205–214. 5. H. Passing and W. A. Bablok, A new biometrical procedure for testing the equality of measurements from two different analytical methods. J. Clin. Chem. Clin. Biochem. 1983; 21: 709–720. 6. E. L. Bradley and L. G. Blackwood, Comparing paired data: a simultaneous test of means and variances. Am. Stat. 1989; 43: 234–235. 7. K. Linnet, Estimation of the linear relationship between the measurements of two methods with proportional errors. Stat. Med. 1990; 9: 1463–1473. 8. B. Zhong and J. Shao, Testing the agreement of two quantitative assays in individual means. Commun. Stat. Theory Meth. 2002; 31: 1283–1299. 9. B. Zhong and J. Shao, Evaluating the agreement of two quantitative assays with repeated measurements. J. Biopharmaceut. Stat. 2003; 13: 75–86. 10. L. I-K. Lin, A concordance correlation coefficient to evaluate reproducibility. Biometrics 1989; 45: 255–268. 11. J. M. Bland and D. G. Altman, Statistical methods for assessing agreement between two methods of clinical measurement. Stat. Methods Med. Res. 1999; 8: 135–160. 12. J. L. Fleiss, The Design and Analysis of Clinical Experiments. New York: John Wiley & Sons, 1986. 13. N. J.-M. Blackman, Reproducibility of clinical data I: continuous outcomes. Pharmaceut. Stat. 2004a; 3: 99–108.

8

MULTIPLE EVALUATORS

14. J. C. Cappelleri and N. Ting, A modified large sample approach to approximate interval estimation for a particular intraclass correlation coefficient. Stat. Med. 2003; 22: 1861–1877. 15. Y. Saito, T. Sozu, C. Hamada, and I. Yoshimura, Effective number of subjects and number of raters for inter-rater reliability studies. Stat. Med. 2006; 25: 1547–1560. 16. V. Rousson, T. Gasser, and B. Seifert, Assessing intrarater, interrater and test-retest reliability of continuous measurements. Stat. Med. 2002; 21: 3431–3446. 17. R. Muller and P. Buttner, A critical discussion of intraclass correlation coefficients. Stat. Med. 1994; 13: 2465–2476. 18. A. Atkinson and A. Nevill, Comment on the use of concordance correlation to assess the agreement between two variables. Biometrics 1997; 53: 775–777. 19. C. A. E. Nickerson, A note on ‘a concordance correlation coefficient to evaluate reproducibility.’ Biometrics 1997; 53: 1503–1507. 20. J. J. Z. Liao and J. Lewis, A note on concordance correlation coefficient. PDA J. Pharmaceut. Sci. Technol. 2000; 54: 23–26. 21. J. L. Carrasco and L. Jover, Estimating the generalized concordance correlation coefficient through variance components. Biometrics 2003; 59: 849–858. 22. J. J. Z. Liao, An improved concordance correlation coefficient. Pharmaceut. Stat. 2003; 2: 253–261. 23. M. P. Fay, Random marginal agreement coefficients: rethinking the adjustment for chance when measuring agreement. Biostatistics 2005; 6: 171–180. 24. J. J. Z. Liao, Agreement for curved data. J. Biopharmaceut. Stat. 2005; 15: 195–203. 25. J. J. Z. Liao, R. C. Capen, and T. L. Schofield, Assessing the reproducibility of an analytical method. J. Chromat. Sci. 2006a; 44: 119–122. 26. J. J. Z. Liao, R. C. Capen, and T. L. Schofield, Assessing the concordance of two measurement methods. ASA Proceedings on Section of Biometrics CD, 2006b. 660–667. 27. R. A. Deyo, P. Diehr, and D. L. Patrick, Reproducibility and responsiveness of health status measures: statistics and strategies for evaluation. Control. Clin. Trial 1991; 12: 142S–158S. 28. J. M. Bland and D. G. Altman, Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 2: 307–310.

29. J. Ludbrook, Comparing methods of measurement. Clin. Exp. Pharmacol. Physiol. 1997; 24: 193–203. 30. W. G. Hopkins, Bias in Bland-Altman but not regression validity analyses. Sportscience 2004; 8: 42–46. 31. J. Cohen, A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 1960; 20: 37–46. 32. M. Banerjee, M. Capozzoli, L. McSweeney, D. Sinha, Beyond kappa: a review of interrater agreement measures. Canadian J. Stat. 1999; 27: 3–23. 33. J. Sim and C. C. Wright, The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys. Ther. 2005; 85: 257–268. 34. T. P. Hutchinson, Kappa muddles together two sources of disagreement: tetrachoric correlation is preferable. Res. Nurs. Health 1993; 16: 313–315. 35. R. Zwick, Another look at interrater agreement. Psychol. Bull. 1988; 103: 374–378. 36. J. Cohen, Weighted kappa: nominal scale agreement with provisions for scaled disagreement or partial credit. Psychol. Bull. 1968; 70: 213–220. 37. J. L. Fleiss, J. Cohen, and B. S. Everitt, Large sample standard errors of kappa and weighted kappa. Psychol. Bull. 1969; 72: 323–327. 38. D. A. Bloch and H. C. Kraemer, 2 × 2 Kappa coefficients: measures of agreement or association. Biometrics 1989; 45: 269–287. 39. D. V. Cicchetti and J. L. Fleiss, Comparison of the null distributions of weighted kappa and the C ordinal statistic. Appl. Psychol. Meas. 1977; 1: 195–201. 40. J. L. Fleiss and D. V. Cicchetti, Inference about weighted kappa in the non-null case. Appl. Psychol. Meas. 1978; 2: 113–117. 41. A. Donner and M. Eliasziw, Statistical implications of the choice between a dichotomous or continuous trait in studies of interobserver agreement. Biometrics 1994; 50: 550–555. 42. E. Bartfay and A. Donner, The effect of collapsing multinomial data when assessing agreement. Internat. J. Epidemiol. 2000; 29: 1070–1075. 43. H. C. Kraemer, V. S. Periyakoil, and A. Noda, Kappa coefficient in medical research. Stat. Med. 2002; 21: 2109–2129. 44. N. J.-M. Blackman, Reproducibility of clinical data II: categorical outcomes. Pharmaceut. Stat. 2004b; 3: 109–122.

MULTIPLE EVALUATORS 45. P. Rothery, A nonparametric measure of intraclass correlation. Biometrika 1979; 66: 629–639. 46. J. L. Fleiss, Measuring nominal scale agreement among many raters. Psychol. Bull. 1971; 76: 378–382. 47. M. Davies and J. L. Fleiss, Measuring agreement for multinomial data. Biometrics 1982; 38: 1047–1051. 48. H. C. Kraemer, Extension of the kappa coefficient. Biometrics 1980; 36: 207–216. 49. M. A. Tanner and M. A. Young, Modeling agreement among raters. J. Am. Stat. Assoc. 1985; 80: 175–180. 50. A. Agresti, Modelling patterns of agreement and disagreement. Stat. Methods Med. Res. 1992; 1: 201–218. 51. P. Graham, Modelling covariate effects in observer agreement studies: the case of nominal scale agreement. Stat. Med. 1995; 14: 299–310. 52. W. R. Dillon and N. Mulani, A probabilistic latent class model for assessing inter-judge reliability. Multivar. Behav. Res. 1984; 19: 438–458. 53. M. Aickin, Maximum likelihood estimation of agreement in the constant predictive model, and its relation to Cohen’s kappa. Biometrics 1990; 46: 293–302. 54. J. S. Uebersax, and W. M. Grove, Latent class analysis of diagnostic agreement. Stat. Med. 1990; 9: 559–572. 55. Y. Qu, M. Tan, and M. H. Kutner, Random effects models in latent class analysis for evaluating accuracy of diagnostic tests. Biometrics 1996; 52: 797–810. 56. S. L. Hui and X. H. Zhou, Evaluation of diagnostic tests without gold standards. Stat. Methods Med. Res. 1998; 7: 354–370. 57. S. R. Lipsitz, and G. M. Fitzmaurice, Estimating equations for measures of association between repeated binary responses. Biometrics 1996; 52: 903–912. 58. J. M. Williamson, A. K. Manatunga, and S. R. Lipsitz, Modeling kappa for measuring dependent categorical agreement data. Biostatistics 2000; 1: 191–202. 59. N. Klar, S. R. Lipsitz, and J. G. Ibrahim, An estimating equations approach for modelling kappa. Biomet. J. 2000; 42: 45–58. 60. E. Gonin, S. R. Lipsitz, G. M. Fitzmaurice, and G. Molenberghs, Regression modelling of weighted κ by using generalized estimating equations. Appl. Stat. 2000; 49: 1–18.

9

61. J. S. Uebersax, Diversity of decision-making models and the measurement of interrater agreement. Psychol. Bull. 1987; 101: 140–146. 62. A. R. Feinstein and D. V. Cicchetti, High agreement but low kappa I: the problems of two paradoxes. J. Clin. Epidemiol. 1990; 43: 543–548. 63. J. R. Landis and G. G. Koch, The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–174. 64. D. L. Streiner and G. R. Norman, Health Measurement Scales: A Practical Guide to their Development and Use, 3rd ed. Oxford, UK: Oxford University Press, 2003. 65. H. C. Kraemer, Evaluating Medical Tests. Objective and Quantitative Guidelines. Newbury Park, CA: Sage, 1992. 66. P. Graham and R. Jackson, The analysis of ordinal agreement data: beyond weighted kappa. J. Clin. Epidemiol. 1993; 9: 1055–1062. 67. D. G. Bonett, Sample size requirements for estimating intraclass correlations with desired precision. Stat. Med. 2002; 21: 1331–1335. 68. J. J. Z. Liao, Sample size calculation and concordance assessment for an agreement study. ENAR presentation, 2004. 69. A. Donner, Sample size requirements for the comparison of two or more coefficients of interobserver agreement. Stat. Med. 1997; 15: 1157–1168. 70. S. D. Walter, M. Eliasziw, and A. Donner, Sample size and optimal designs for reliability studies. Stat. Med. 1998; 17: 101–110.

CROSS-REFERENCES Interrater reliability Reproducibility Kappa statistic weighted Kappa

MULTIPLE RISK FACTOR INTERVENTION TRIAL (MRFIT)

1 TRIAL DESIGN 1.1 Overview The objective of MRFIT was to test whether a multifactor intervention would result in lower CHD mortality among men, baseline ages 35–57 years, at higher risk for CHD as measured by three established major risk factors: blood pressure (BP), serum cholesterol, and cigarette smoking. Eligible highrisk men were randomized to either the ‘‘Special Intervention’’ (SI) group, which targeted cessation of smoking, lowering serum cholesterol, and lowering BP, or to the ‘‘Usual Care’’ (UC) group; the latter were referred to their personal physicians or a community care clinic and did not receive any study intervention.

LYNN E. EBERLY School of Public Health, University of Minnesota, Minneapolis, Minnesota

JEREMIAH STAMLER Feinberg School of Medicine, Northwestern University, Chicago, Illinois

LEWIS H. KULLER School of Public Health, University of Pittsburgh, Pittsburgh, Pennsylvania

JAMES D. NEATON School of Public Health, University of Minnesota, Minneapolis, Minnesota

1.2 Design of the Multifactor Intervention for the SI Group

The Multiple Risk Factor Intervention Trial (MRFIT) was a nationwide randomized trial on primary prevention of coronary heart disease (CHD) death, sponsored by the National Heart, Lung, and Blood Institute within the United States National Institutes of Health, and it was conducted during 1972–1982. The study cohort consisted of men (baseline ages 35–57 years) at higher risk for CHD death, but with no clinical evidence of cardiovascular disease (CVD); women were not included because of substantially lower CHD rates. The intervention tested was multifactor: intensive counseling for smoking cessation, dietary advice particularly to lower serum cholesterol, and stepped-care pharmacologic treatment for hypertension (primarily with diuretics). The primary outcome was CHD mortality, with an observation period of 6 years minimum and 7 years on average (1,2). Since the end of active follow-up in 1982, the 361,662 men who were screened during MRFIT recruitment, which includes the 12,866 who were randomized, have been followed for mortality date and cause through Social Security Administration and National Death Index searches (3,4).

1.2.1 Cigarette Smoking. Each cigarette smoker was counseled individually by a MRFIT physician immediately after randomization. Dosage reduction (low tar, low nicotine cigarettes) was not recommended (5,6). 1.2.2 Nutrition. The nutrition intervention did not stipulate a structured diet, but instead it aimed to encourage lifelong shopping, cooking, and eating patterns concordant with saturated fat intake < 10% of calories, dietary cholesterol < 300 mg/day, polyunsaturated fat ∼ 10% of calories, with a fare moderate in total fat and calories to facilitate prevention and control of obesity and elevated BP (7). 1.2.3 Hypertension. Antihypertensive drugs were prescribed using a stepped-care protocol beginning with the use of an oral diuretic (either hydrochlorothiazide or chlorthalidone); other drugs were sequentially added if the BP goal (either a 10 mm Hg reduction in diastolic BP, or diastolic BP no more than 89 mm Hg) was not reached. Before drug prescription, weight loss was attempted for overweight men (8).

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

MULTIPLE RISK FACTOR INTERVENTION TRIAL (MRFIT)

1.2.4 Simultaneous Efforts. Shortly after randomization, each man was invited (with spouse or friend) to a 10-week series of weekly group discussions covering all three risk factors; common educational materials were used at all sites. After this initial phase, individual counseling by an intervention team (behavioral scientist, nutritionist, nurse, physician, and general health counselor) and measurement of risk factors was provided every 4 months; specific risk factor goals were set for each person. Men could be examined at more frequent intervals based on their changes in risk factors and intervention team recommendations (1,9). 1.3 Sample Size and Power Calculations The MRFIT design stipulated recruitment of at least 12,000 men ages 35 to 57 years; the men had to be at elevated risk of CHD mortality but without clinical signs of CVD at baseline. The randomized sample size for MRFIT of 12,866 was estimated to provide 88% power to detect a 26.6% reduction in risk of CHD death over 6 years (29.0/1000 men in UC compared with 21.3/1000 men in SI) using a one-sided test of proportions and a type I error of 5%. Behind this calculation were several steps with key assumptions about level of CHD risk in the UC group and anticipated intervention effects. First, a multiple logistic regression model for 6-year CHD death—as a function of serum cholesterol, diastolic BP, and cigarettes smoked/day—was developed using Framingham Heart Study data. Second, variables that represented the screening levels of diastolic BP, serum cholesterol, and cigarettes/day for the MRFIT randomized men were entered into the logistic risk model to project the 6-year CHD death rate of 29.0/1000 men in UC. Third, a reduction of the CHD death rate to 21.3/1000 men in SI was projected based on the following assumptions about reductions of screening risk factor levels for SI men: (1) a 10% reduction in serum cholesterol for those at a level of 220 mg/dL or more (no reduction for others); (2) a 10% reduction in diastolic BP for those at a level of 95 mm Hg or more (no reduction for others); (3) a 25% reduction for smokers of 40 or more cigarettes/day, a 40% reduction for

smokers of 20–39/day, and a 55% reduction for smokers of <20/day. It was also assumed that reductions would be observed for UC smokers of 5%, 10%, and 15%, respectively, but with no reductions in cholesterol or BP for UC men. Additional assumptions were made about levels of non-adherence in the SI group, and duration of sustained intervention required to achieve the projected reductions in risk factor levels (1,10). 2 TRIAL SCREENING AND EXECUTION Continual attention to quality assurance was heavily emphasized in all phases of MRFIT (11). 2.1 Determination of Risk and Trial Eligibility Trial eligibility, determined over three screening visits (S1, S2, S3), was based primarily on diastolic BP, cigarettes smoked/day, and serum cholesterol, among other factors. At S1, BP was measured, after the screenee had been seated for 5 minutes, three times at 2-minute intervals using a standard mercury sphygmomanometer; the average of the second and third readings was used for screening. Casual serum samples were divided into two aliquots, one of which was analyzed at a local laboratory that had been standardized by the Centers for Disease Control; the other was retained for analysis by the Central Laboratory at the Institute for Medical Sciences in San Francisco. In all, 336,117 of the 361,662 men screened were excluded at S1 for age (<35 or >57 years old), expected geographic mobility, for extremely high risk (history of either heart attack or diabetes that required medication, serum cholesterol 350+ mg/dL, or diastolic BP 115+ mm Hg), or for ‘‘lower risk status.’’ Lower risk status was determined from a risk score, based on the Framingham logistic regression model for 6-year CHD death, using each man’s S1 cholesterol, diastolic BP, and cigarettes/day. Lower risk status was defined as below the 85th percentile of risk score, later changed to below the 90th percentile. Thus, men had to have a risk score in the upper 10–15% to be considered for inclusion (4,9,10).

MULTIPLE RISK FACTOR INTERVENTION TRIAL (MRFIT)

Of the remaining 25,545 men, 9,754 were excluded at S2 for body weight ≥150% of desirable weight (0.9 times average weight for men of the same height in the 1960–1962 National Health Survey), heavy drinking, angina pectoris (from Rose questionnaire), history or resting ECG evidence of myocardial infarction (MI), use of lipid-lowering drugs, untreated diabetes based on a 75 g glucose tolerance test, diabetes treatment with insulin or oral antihyperglycemics, illness/ disability incompatible with participation, prescribed diets incompatible with the intervention, diastolic BP 120+ mm Hg, or for not attending S2. An additional 1,680 men were eligible for but did not attend S3, and they were excluded. The remaining 12,866 men consented to participation and were randomized, 6,428 to SI and 6,438 to UC (4,9,10). 2.2 Recruitment and Randomization Recruitment was carried out in 1973–1975 by 22 clinics in 18 cities across the United States. S1 screenings were held in civic, religious, and public health facilities, places of employment, shopping centers, and by doorto-door recruitment. S2 and S3 screenings were held in study clinics. After eligibility and consent to participate had been established, clinic coordinators called the coordinating center for the randomization assignment. Allocation to treatment arm was stratified by clinic and balanced in blocks of size four or six (1,12).

3

database. Blood samples were processed by the Central Laboratory; periodic quality control was implemented by the Centers for Disease Control Lipid Standardization Laboratory. During-trial follow-up was above 90% through 6 years for each of the SI and UC groups. The last day of active follow-up was 28 February 1982 (1,9,13,14). 2.4 Mortality and Morbidity Ascertainment Four key endpoints, each considered singly, were of interest: CHD mortality (the primary endpoint), all-cause mortality, nonfatal MI or CHD mortality, and CVD mortality (the secondary endpoints). Clinics learned of deaths through contact with family or friends of the participants, routine clinic follow-up of missed visits, responses to change-of-address postcards sent semiannually to all participants, and searches of publicly accessible files of deceased persons. Cause of death was determined by a Mortality Review Committee (a three-member panel of cardiologists independent of any MRFIT clinic and blinded to treatment arm and interim results) by reviewing clinic and hospital records, nextof-kin interviews, death certificates, and autopsy reports. Nonfatal MI ascertainment was based on follow-up ECG findings and hospitalization records (1,15).

2.3 During-Trial Follow-Up

3 FINDINGS AT THE END OF INTERVENTION 3.1 Baseline Differences and During Trial Changes in Risk Factors

At or about each anniversary of randomization, SI and UC participants returned to their clinical center for assessment of risk factor levels and morbidity status. A 24-hour dietary recall was repeated at four (UC) or five (SI) of the first six follow-up visits by nutritionists trained in standardized procedures. Data collection forms were reviewed and edited at each clinic by a clinic coordinator, then mailed to the coordinating center for data entry, editing, and analysis. Each form received a unique log number, and it was double-keyed and checked against a data dictionary for out of range values and logic errors; samples of each form type were later keyed a third time and checked against the

The SI and UC groups were not statistically different at baseline (S1, S2, or S3) in levels of numerous risk factors and related measures. Significant differences were observed between the two groups at each annual followup visit, with larger reductions in risk factor levels observed in the SI group. Average diastolic BP was reduced from 91.0 at S3 to 80.5 mm Hg at visit 6 in SI and from 90.9 to 83.6 mm Hg in UC. The percent of smokers was reduced from 59.3% at S3 to 32.3% at visit 6 in SI and from 59.0% to 45.6% in UC. Average plasma cholesterol was reduced from 240.3 at S2 to 228.2 mg/dL at visit 6 in SI and from 240.6 to 233.1 mg/dL in UC (P < 0.01 for each of these comparisons) (1).

4

MULTIPLE RISK FACTOR INTERVENTION TRIAL (MRFIT)

3.2 Primary and Secondary Outcomes With an average follow-up of 7 years through 28 February 1982, there were 265 deaths among SI men (138 CVD, 115 CHD) and 260 deaths among UC men (145 CVD, 124 CHD). All-cause death rates were 41.2 per 1000 men (19.3/1000 for CVD, 17.9/1000 for CHD) in SI, and 40.4 per 1000 (22.5/1000 for CVD, 19.3/1000 for CHD) in UC. Thus, for the primary outcome, there was a 7.1% lower CHD death rate among SI men [relative difference in proportion dead (pUC -pSI )/pUC = 7%, 90% CI (−15%, 25%)]. The observed CHD death rate of 17.9/1000 in SI men was lower than the rate of 21.3/1000 for which the study had been powered, but the observed CHD death rate of 19.3/1000 in UC men was substantially lower than the Framinghamprojected rate of 29.0/1000 (1). The total death rate was 2.1% higher for SI than UC, whereas the CVD death rate was 4.7% lower for SI than UC. A slight CHD and total mortality advantage for SI, which developed around year 2, waned by year 5. The combined outcome of CHD mortality or acute MI was lower in the SI men (395 events) compared to the UC men (431 events) [relative difference 8%, 95% CI (−5%, 20%)] (1,15). Additional analyses (done for the first time in the 1990s) considered four combined morbidity/mortality endpoints in accordance with approaches in general use by then in randomized trials. The combined endpoints, based on during trial data through 28 February 1982, included a clinical cardiac event endpoint: earliest of significant serial change in ECG, clinical non-fatal myocardial infarction (MI), congestive heart failure (CHF), ECG-diagnosed left ventricular hypertrophy (LVH), or CHD death. The SI/UC hazard ratio for this endpoint was 0.87 [95% CI (0.76, 0.99), adjusted for age at screening, baseline screening factors, and baseline presence of resting and/or exercise ECG abnormalities], a 13% reduction in risk for the SI group. A clinical cardiovascular endpoint included: earliest of nonfatal stroke, significant serial change in ECG, clinical nonfatal MI, CHF, ECG LVH, coronary by-pass surgery, peripheral arterial occlusion, impaired renal function, accelerated hypertension, or CVD death, and

resulted in a hazard ratio of 0.86 [95% CI (0.78, 0.95), adjusted as above] (16). 3.3 Lessons Learned from this ‘‘Landmark Trial’’ Potential reasons for lack of significant findings on the original primary outcomes analyses in MRFIT have been detailed (1,17). The trial design assumed large SI/UC differences in risk factor levels early in the trial, followed by increasing noncompliance; what was observed were small differences, followed by long-term maintenance of those differences. The extent of the UC risk factor changes was unanticipated, and the numbers of UC deaths after 6 years (219 total, 104 CHD) were substantially smaller than expected (442 total, 187 CHD) from the Framingham-based predictions. Thus, overestimated event rates for UC, and an overestimated group difference, led to a study with power of only 60% rather than 88% to detect a significant difference for the primary endpoint of CHD death. The lower than expected event rate for UC was caused in part by less stringent exclusion criteria applied to the Framingham database than those applied to the MRFIT screenees. Also, during the trial, a secular trend was observed in the U.S. of decreasing CHD death rates, perhaps reflective of improving treatments for heart disease and growing efforts to control CVD major risk factors (1,17). 4 LONG-TERM FOLLOW-UP Death dates and causes through 1999 have been determined through searches of Social Security Administration and National Death Index files for all 361,662 men screened, with median follow-up of 24.7 years (3,4). This long-term mortality follow-up in such a large cohort, with demographic information and major CVD risk factor data, is a tremendous resource for medicine and public health, and it has been used extensively. 4.1 Primary Outcome Results at 10.5 Years and at 16 Years Mortality follow-up through 1985 (average follow-up 10.5 years from MRFIT baseline)

MULTIPLE RISK FACTOR INTERVENTION TRIAL (MRFIT)

showed a CHD death rate lower by 10.6% for SI compared with UC [relative difference in proportion dead (pUC -pSI )/pUC = 10.6%, 90% CI (−4.9%, 23.7%)], with the largest reduction for death from acute MI (relative difference 24.3%, P-value = 0.02) (18). Mortality follow-up through 1990 (average follow-up 15.8 years) showed CHD death lower by 11.4% for SI compared with UC [90% CI (−1.9%, 23.0%)], with the largest reduction again for death from acute MI [relative difference 20.4%, 95% CI (3.4%, 34.4%), P-value = 0.02] (19).

•

4.2 Longer-Term Impacts of the MRFIT The results of MRFIT had a substantial impact on the treatment and management of hypertension (20–22), including the use of lower dose diuretics (23), and ultimately led to ALLHAT (the Antihypertensive and Lipid Lowering Treatment to Prevent Heart Attack Trial) (24,25). Epidemiologic results from mortality follow-up of the randomized men showed the importance of white blood cell count (26,27), HDL cholesterol (28), pulmonary function (29,30), and C-reactive protein (31) for cause-specific mortality.

•

5 EPIDEMIOLOGIC FINDINGS FROM LONG-TERM FOLLOW-UP OF 361,662 MRFIT SCREENEES

•

Analyses from long-term mortality follow-up of the 361,662 screened men have been used to address important public health and medical care issues.

•

5

as CHD mortality (32–36). Pulse pressure is not as strong a predictor of mortality as either systolic BP or diastolic BP, systolic BP is a stronger single predictor than diastolic BP of CHD or CVD risk, and both systolic and diastolic BP together predict better than each alone (33,34,37). The extraordinary sample size of this cohort, and its long-term follow-up, made possible for the first time the measurement of CHD and CVD risk for men assessed to be at lower risk at baseline (38). Nonsmokers with favorable levels of serum cholesterol (<200 mg/dL) and blood pressure (≤120/≤80 mmHg), no prior heart attack, and no diabetes (a rare combination) are at substantially lower risk of mortality compared to all others; for these low risk men, CHD and CVD rates are endemic, not epidemic (39). For low risk men, both nonHispanic White and African-American, death rates are lower, irrespective of socioeconomic status (SES), compared with higher risk men (40). Most (87–100%) of those who develop CVD have antecedent cardiovascular risk factors (41,42). History of diabetes was shown to be a powerful predictor of CHD mortality (35), but slightly less powerful than history of heart attack (43). Socioeconomic status (SES) differences between Black and White screenees were strongly predictive of differences in CVD mortality and ESRD (44–47).

• Results from the screened men have had

a substantial impact in particular on guidelines for treatment of high blood cholesterol and BP. Risk of CHD mortality, for example, was shown not to be restricted to those with ‘‘hypertension’’ or those with ‘‘hypercholesterolemia.’’ The relationship of BP with CHD death is strong, continuous, graded, and independent of other major risk factors, as is the relationship of serum cholesterol with CHD death; cigarette smoking amplifies these risks. This is all true for CVD mortality, all-cause mortality, and end stage renal disease (ESRD) as well

6

CONCLUSIONS

Although significant findings were not observed for any of the four pre-specified primary endpoints, significantly lower event rates for SI compared with UC were found for acute MI mortality and for combined cardiovascular morbidity/mortality endpoints. The MRFIT was a large and complex trial that was operationally successful: Recruitment exceeded design goals, and it was completed in only 28 months; randomization resulted in two balanced groups. Loss to follow-up was

6

MULTIPLE RISK FACTOR INTERVENTION TRIAL (MRFIT)

less than expected throughout the trial; 91% of those alive attended their sixth visit; only 30 of 12,866 men had unknown vital status at the end of the study. Thorough ascertainment of dates and classification of causes of death (blinded to group assignment) was achieved. The intervention was largely a success for the SI men: Smoking cessation was larger than anticipated. Diastolic BP reduction was larger than expected. Cholesterol was lowered considerably although not to the target level. Average risk factor levels continued to decline each year, instead of regressing back toward baseline. However, contrary to design expectations, UC risk factor reductions were also considerable, although less than SI, and UC death rates were less than expected, which resulted in a study underpowered for its primary endpoint of CHD death (1). More analyses of the MRFIT data, using continued mortality follow-up, have provided evidence for numerous, substantial public health and medical advances. The MRFIT bibliography includes 250+ published manuscripts and five monographs. The overarching thrust of much of this body of work is the importance of being at low risk and of primary prevention of elevated risk factor levels through improvement of lifestyles. We have described here only a small fraction of the research contributions based on MRFIT data.

REFERENCES 1. Multiple Risk Factor Intervention Trial Research Group, Multiple Risk Factor Intervention Trial: Risk factor changes and mortality results. JAMA 1982; 248: 1465–1477. 2. J. Stamler, for the Multiple Risk Factor Intervention Trial Research Group, Multiple Risk Factor Intervention Trial. In: H. Hofmann (ed.), Primary and Secondary Prevention of Coronary Heart Disease: Results of New Trials. New York: Springer-Verlag, 1985, pp. 8–33. 3. D. Wentworth, J. D. Neaton, and W. Rasmussen, An evaluation of the Social Security Administration MBR File and the National Death Index in the ascertainment of vital status. Am. J. Publ. Health 1983; 73: 1270–1274.

4. L. E. Eberly, J. D. Neaton, A. J. Thomas, and Y. Dai, for the Multiple Risk Factor Intervention Trial Research Group, Multiple-stage screening and mortality in the Multiple Risk Factor Intervention Trial. Clin. Trials 2004; 1: 148–161. 5. J. Ockene, Multiple Risk Factor Intervention Trial (MRFIT): Smoking cessation procedures and cessation and recidivism patterns for a large cohort of MRFIT participants. In: J. Schwart (ed.), Progress in Smoking Cessation. New York: American Cancer Society, 1979, pp. 183–198. 6. G. H. Hughes, N. Hymowitz, J. K. Ockene, N. Simon, and T. M. Vogt, The Multiple Risk Factor Intervention Trial (MRFIT): V. Intervention on smoking. Prev. Med. 1981; 10: 476–500. 7. A. W. Caggiula, G. Christakis, M. Farrand, S. B. Hulley, R. Johnson, N. L. Lasser, J. Stamler, and G. Widdowson, The Multiple Risk Factor Intervention Trial (MRFIT): IV. Intervention on blood lipids. Prev. Med. 1981; 10: 443–475. 8. J. D. Cohen, R. H. Grimm, and W. McFate Smith, The Multiple Risk Factor Intervention Trial (MRFIT): VI. Intervention on blood pressure. Prev. Med. 1981; 10: 501–518. 9. R. Sherwin, C. T. Kaelber, P. Kezdi, M. O. Kjelsberg, and T. H. Emerson (for the MRFIT), The Multiple Risk Factor Intervention Trial (MRFIT): II. The development of the protocol. Prev. Med. 1981; 10: 402–425. 10. Multiple Risk Factor Intervention Trial Group, Statistical design considerations in the NHLI Multiple Risk Factor Intervention Trial (MRFIT). J. Chron. Dis. 1977; 30: 261–275. 11. Multiple Risk Factor Intervention Trial Research Group. The Multiple Risk Factor Intervention Trial: quality control of technical procedures and data acquisition. Control. Clin. Trials 1986; 7: 1S–202S. 12. J. D. Neaton, R. H. Grimm, and J. A. Cutler, Recruitment of participants for the Multiple Risk Factor Intervention Trial (MRFIT). Control. Clin. Trials 1988; 8: 41S–53S. 13. N. L. Lasser, S. Lamb, P. Dischinger, and K. Kirkpatrick, Introduction: Background and organization of quality control in the Multiple Risk Factor Intervention Trial. Control. Clin. Trials 1986; 7: 1S–16S. 14. A. G. DuChene, D. H. Hultgren, J. D. Neaton, P. V. Grambsch, S. K. Broste, B. M. Aus, and W. L. Rasmussen, Forms control and error detection procedures used at the Coordinating

MULTIPLE RISK FACTOR INTERVENTION TRIAL (MRFIT) Center of the Multiple Risk Factor Intervention Trial (MRFIT). Control. Clin. Trials 1986; 7: 34S–45S. 15. Multiple Risk Factor Intervention Trial Research Group, Coronary heart disease death, nonfatal acute myocardial infarction and other clinical outcomes in the Multiple Risk Factor Intervention Trial. Am. J. Cardiol. 1986; 58: 1–13. 16. J. Stamler, J. Shaten, J. Cohen, J. A. Cutler, M. Kjelsberg, L. Kuller, J. D. Neaton, and J. Ockene, for the MRFIT Research Group, Combined nonfatal plus fatal cardiac and cardiovascular events in special intervention (SI) compared to usual care (UC) men during the Multiple Risk Factor Intervention Trial (MRFIT). Canadian J. Cardiol. 1997; 13: 366B, Abstract #1379. 17. A. M. Gotto, The Multiple Risk Factor Intervention Trial (MRFIT). A Return to a Landmark Trial. JAMA 1997; 277: 595–597. 18. Multiple Risk Factor Intervention Trial Group, Mortality rates after 10.5 years for participants in the Multiple Risk Factor Intervention Trial: findings related to a priori hypotheses of the trial. JAMA 1990; 263: 1795–1801.

7

24. The ALLHAT Officers and Coordinators for the ALLHAT Collaborative Research Group, Major cardiovascular events in hypertensive patients randomized to doxazosin vs chlorthalidone: the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT). JAMA 2000; 283: 1967–1975. 25. The ALLHAT Officers and Coordinators for the ALLHAT Collaborative Research Group, Major outcomes in high-risk hypertensive patients randomized to angiotensinconverting enzyme inhibitor or calcium channel blocker vs diuretic. The Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT). JAMA 2002; 288: 2981–2997. 26. R. H. Grimm, J. D. Neaton, and B. Ludwig, The prognostic importance of total white blood cell count for all cause, cardiovascular and cancer mortality. JAMA 1985; 254: 1932–1937. 27. A. N. Phillips, J. D. Neaton, D. G. Cook, R. H. Grimm, and A. G. Shaper, The white blood cell count and risk of lung cancer. Cancer 1992; 69: 681–684.

19. Multiple Risk Factor Intervention Trial Group, Mortality after 16 years for participants randomized to the Multiple Risk Factor Intervention Trial. Circulation 1996; 94: 946–951.

28. D. J. Gordon, J. L. Probstfield, R. J. Garrison, J. D. Neaton, W. P. Castelli, J. D. Knoke, D. R. Jacobs Jr., Bangdiwala, and H. A. Tyroler, High-density lipoprotein cholesterol and cardiovascular disease. Four prospective American studies. Circulation 1989; 79: 8–15.

20. Multiple Risk Factor Intervention Trial Group, Baseline rest electrogardiographic abnormalities, antihypertensive treatment, and mortality in the Multiple Risk Factor Intervention Trial. Am. J. Cardiol. 1985; 55: 1–15.

29. L. H. Kuller, J. Ockene, E. Meilahn, and K. H. Svendsen, Relation of forced expiratory volume in one second (FEV1) to lung cancer mortality in the Multiple Risk Factor Intervention Trial (MRFIT). Am. J. Epidemiol. 1990; 132: 265–274.

21. L. H. Kuller, S. B. Hulley, J. D. Cohen, and J. D. Neaton, Unexpected effects of treating hypertension in men with ECG abnormalities: a review of the evidence. Circulation 1986; 73: 114–123.

30. L. E. Eberly, J. Ockene, R. Sherwin, L. Yang, and L. Kuller, for the Multiple Risk Factor Intervention Trial Group, Pulmonary function as a predictor of lung cancer mortality in continuing cigarette smokers and in quitters. Int. J. Epidemiol. 2003; 32: 592–599.

22. J. D. Cohen, J. D. Neaton, R. J. Prineas, and K. A. Daniels, Diuretics, serum potassium and ventricular arrhythmias in the Multiple Risk Factor Intervention Trial. Am. J. Cardiol. 1987; 60: 548–554. 23. J. D. Neaton, R. H. Grimm, R. J. Prineas, G. Grandits, P. J. Elmer, J. A. Cutler, J. M. Flack, J. A. Schoenberger, R. MacDonald, et al., for the Treatment of Mild Hypertension Study Research Group. Treatment of Mild Hypertension Study (TOMHS): final results. JAMA 1993; 270: 713–724.

31. L. H. Kuller, R. P. Tracy, J. Shaten, and E. N. Meilahn, for the MRFIT Research Group. Relation of C-reactive protein and coronary heart disease in the MRFIT Nested CaseControl Study. Am. J. Epidemiol. 1996; 144: 537–547. 32. J. Stamler, D. Wentworth, and J. D. Neaton, Is the relationship between serum cholesterol and risk of premature death from Coronary Heart Disease continuous and graded? JAMA 1986; 256: 2823–2828.

8

MULTIPLE RISK FACTOR INTERVENTION TRIAL (MRFIT)

33. J. Stamler, J. D. Neaton, and D. N. Wentworth, Blood pressure (systolic and diastolic) and risk of fatal coronary heart disease. Hypertension 1989; 13: 2–12. 34. J. D. Neaton and D. Wentworth, Influence of serum cholesterol, blood pressure, and cigarette smoking on death from coronary heart disease in 316,099 white men age 3557 years: overall findings and differences by age. Arch. Intern. Med. 1992; 152: 56–64. 35. J. Stamler, O. Vaccaro, J. D. Neaton, and D. Wentworth, Diabetes, other risk factors, and 12-year cardiovascular mortality for men screened in the Multiple Risk Factor Intervention Trial. Diabetes Care 1993; 16: 434–444. 36. M. J. Klag, P. K. Whelton, B. L. Randall, J. D. Neaton, F. L. Brancati, C. E. Ford, N. B. Shulman, and J. Stamler, Blood Pressure and End-Stage Renal Disease in Men. N. Engl. J. Med. 1996; 334: 13–18. 37. M. Domanski, M. Pfeffer, J. Neaton, J. Norman, K. Svendsen, R. Grimm, J. Cohen, and J. Stamler, for the MRFIT Research Group, Pulse pressure and cardiovascular mortality: 16-year findings on 342,815 men screened for the Multiple Risk Factor Intervention Trial (MRFIT). JAMA 2002; 287: 2677–2683. 38. J. Stamler, J. D. Neaton, D. Wentworth, J. Shih et al., Lifestyles and life-style related risk factors: their combined impact in producing epidemic cardiovascular disease and the potential for prevention. In: A. M. Gotto, C. Lenfant, R. Paoletti, M. Some (eds.), Multiple Risk Factors in Cardiovascular Disease. Norwell, MA: Kluwer, 1992, pp. 19–25. 39. J. Stamler, R. Stamler, J. D. Neaton, D. N. Wentworth, M. Daviglus, D. Garside, A. Dyer, P. Greenland, and K. Liu, Low risk-factor profile and long-term cardiovascular and noncardiovascular mortality and life expectancy: findings for 5 large cohorts of young adult and middle-aged men and women. JAMA 1999; 282: 2012–2018. 40. Stamler J, Neaton JD, Garside D, and Daviglus ML. Current status: six established major risk factors - and low risk. In: M. Marmot and P. Elliott (eds.), Coronary Heart Disease Epidemiology: From Aetiology to Public Health. 2nd ed. London: Oxford University Press, 2005, pp. 32–70. 41. P. Greenland, M. D. Knoll, J. Stamler, J. D. Neaton, A. R. Dyer, D. B. Garside, and P. W. Wilson, Major risk factors are nearly universal antecedents of clinical coronary heart disease in three large U.S. cohorts. JAMA 2003; 290: 891–897.

42. J. Stamler, Low risk – and the ‘‘no more than 50%’’ myth/dogma. Arch. Intern. Med. 2007; 167: 537–539. 43. O. Vaccaro, L. E. Eberly, J. D. Neaton, L. Yang, G. Riccardi, and J. Stamler, Impact of diabetes and previous myocardial infarction on long-term survival: 18 years mortality follow-up of primary screenees in the Multiple Risk Factor Intervention Trial. Arch. Intern. Med. 2004; 164: 1438–1443. 44. G. Davey Smith, J. D. Neaton, D. N. Wentworth, R. Stamler, and J. Stamler, Socioeconomic differentials in mortality risk among men screened for the Multiple Risk Factor Intervention Trial: Part I - White men. Am. J. Pub. Health 1996; 86: 46–496. 45. G. Davey Smith, D. N. Wentworth, J. D. Neaton, R. Stamler, and J. Stamler, Socioeconomic differentials in mortality risk among men screened for the Multiple Risk Factor Intervention Trial: Part II - Black men. Am. J. Pub. Health 1996; 86: 497–504. 46. G. Davey Smith, J. D. Neaton, D. N. Wentworth, R. Stamler, and J. Stamler, Mortality differences between black and white men in the USA: the contribution of income and other factors among men screened for the MRFIT. Lancet 1998; 351: 934–939. 47. M. J. Klag, P. K. Whelton, B. L. Randall, J. D. Neaton, F. L. Brancati, and J. Stamler, Endstage renal disease in African-American and white men. JAMA 1997; 277: 1293–1298.

FURTHER READING J. Stamler, Established major coronary risk factors: historical overview. In: M. Marmot and P. Elliott (eds.), Coronary Heart Disease Epidemiology: From Aetiology to Public Health. 2nd ed. London: Oxford University Press, 2005, pp. 1–31. J. Stamler, J. D. Neaton, D. B. Garside, and M. L. Daviglus, The major adult cardiovascular diseases: a global historical perspective. In: R. M. Lauer, T. L. Burns, and S. R. Daniels (eds.), Pediatric Prevention of Atherosclerotic Cardiovascular Disease. London: Oxford University Press, 2006, pp. 27–48. J. Stamler, M. Daviglus, D. B. Garside, P. Greenland, L. E. Eberly, L. Yang, and J. D. Neaton, Low risk cardiovascular status: Impact on cardiovascular mortality and longevity. In: R. M. Lauer, T. L. Burns, and S. R. Daniels (eds.), Pediatric Prevention of Atherosclerotic Cardiovascular Disease. London: Oxford University Press, 2006, pp. 49–60.

MULTIPLE RISK FACTOR INTERVENTION TRIAL (MRFIT)

CROSS-REFERENCES Controlled Clinical Trials Data Quality Assurance Disease Trials for Cardiovascular Diseases Diuretics Intervention Design/Nutritional Intervention/Smoking Cessation Intervention Primary Prevention Trials Regression Dilution Bias Selection Bias Survival Study

9

MULTIPLE TESTING IN CLINICAL TRIALS

drugs and increased patient risks. For this reason, regulatory agencies mandate a strict control of the false-positive (Type I error) rate in clinical trials and require that drug developers perform multiple analyses with a proper adjustment for multiplicity. To stress the importance of multiplicity adjustments, the draft guidance document entitled ‘‘Points to consider on multiplicity issues in clinical trials’’ released by the European Committee for Proprietary Medicinal Products on July 26, 2001 states that

ALEXEI DMITRIENKO Eli Lilly and Company Indianapolis Lilly Research Laboratories Indianapolis, Indiana

JASON C. HSU Ohio State University Columbus Department of Statistics Columbus, Ohio

1

INTRODUCTION

‘‘a clinical study that requires no adjustment of the Type I error is one that consists of two treatment groups, that uses a single primary variable, and has a confirmatory statistical strategy that pre-specifies just one single null hypothesis relating to the primary variable. All other situations require attention to the potential effects of multiplicity’’ (1).

Multiplicity problems caused by multiple analyses performed on the same dataset occur frequently in a clinical trial setting. The following are examples of multiple analyses encountered in clinical trials. • Multiple comparisons. Multiple test-

ing is often performed in clinical trials involving several treatment groups. For example, most Phase II trials are designed to assess the efficacy and safety of several doses of an experimental drug compared to a control. • Multiple primary endpoints. Multiplicity can be caused by multiple criteria for assessing the efficacy profile an experimental drug. The multiple criteria are required to accurately characterize various aspects of the expected therapeutic benefits. In some cases, the experimental drug is declared efficacious if it meets at least one of the criteria. In other cases, drugs need to produce significant improvement with respect to all of the endpoints (e.g., new therapies for the treatment of Alzheimer’s disease are required to demonstrate their effects on both cognition and global clinical improvement).

As a result of these regulatory concerns, multiplicity adjustment strategies have received much attention in the clinical trial literature. This article provides a brief review of popular approaches to performing multiple analyses of clinical trial data. It outlines main principles underlying multiple testing procedures and introduces singlestep and stepwise multiple tests widely used in clinical applications. See Hochberg and Tamhane (2), Westfall and Young (3), and Hsu (4) for a comprehensive review of multiple decision theory with clinical trial applications. Throughout the article, H01 , . . . , H0k will denote the k null hypotheses and HA1 , . . . , HAk denote the alternative hypotheses tested in a clinical study. The associated test statistics and P-values will be denoted by T1 , . . . , Tk and p1 , . . . , pk , respectively.

It is commonly recognized that failure to account for multiplicity issues can inflate the probability of an incorrect decision and could lead to regulatory approval of inefficacious

In order to choose an appropriate multiple testing method, it is critical to select the definition of correct and incorrect decisions that reflect the objective of the study.

2

CONCEPTS OF ERROR RATES

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

MULTIPLE TESTING IN CLINICAL TRIALS

2.1 Comparisonwise Error Rate In the simple case when each hypothesis is tested independently, the comparisonwise error rate is controlled at a significance level α (e.g., 0.05 level) if each H0i is tested so that the probability of erroneously rejecting H0i is no more than α. Using the law of large numbers, it can be shown that, in the long run, the proportion of erroneously rejected null hypotheses does not exceed α. However, if the k null hypotheses are true, the probability of rejecting at least one true null hypothesis will be considerably greater than the significance level chosen for each individual hypothesis. Thus, if a correct decision depends on correct inference from all k tests, the probability of an incorrect decision will exceed α. 2.2 Experimentwise Error Rate An early attempt to alleviate this problem and achieve a better control of the probability of an incorrect decision was to consider each experiment as a unit and define the experimentwise error rate. The experimentwise error rate is said to be controlled at α if the probability of rejecting at least one true null hypothesis does not exceed α when the k null hypotheses are simultaneously true. Control of the experimentwise error rate is sometimes referred to as the weak control of the familywise error rate. Note, however, that, in terms of the probability of making an incorrect decision, H01 , . . . , H0k all being true is not always the worst case scenario. Suppose, for example, that H01 , . . . , H0(k−1) are true but H0k is false. Then any multiple testing method for which the probability of incorrectly rejecting at least one null hypothesis when all the null hypotheses are true is no more than α, but the probability of rejecting at least one of H01 , . . . , H0(k−1) , given that they are true and H0k is false is greater than α, protects the experimentwise error rate. It is obvious from this example that preserving the experimentwise error rate does not necessarily guarantee that the probability of an incorrect decision is no greater than α. 2.3 Familywise Error Rate As a result of the described limitation of the experimentwise error rate, clinical

researchers rely on a more stringent method for controlling the probability of an incorrect decision known as the strong control of the familywise error rate (FWER). The FWER is defined as the probability of erroneously rejecting any true null hypothesis in a family regardless of which and how many other null hypotheses are true. This definition is essentially based on the maximum experimentwise error rate for any subset of the k null hypotheses and, for this reason, FWER-controlling tests are sometimes said to preserve the maximum Type I error rate. 2.4 False Discovery Rate Another popular approach to assessing the performance of multiple tests, known as the false discovery rate (FDR), is based on the ratio of the number of erroneously rejected null hypotheses to the total number of rejected null hypotheses (5). To be more precise, the FDR is said to be controlled at α if the expected proportion of incorrectly rejected (true) null hypotheses is no more than α, for example


Number of true H0i rejected E Total number of H0i rejected

 ≤α

FDR-controlling tests are useful in multiplicity problems involving a large number of null hypotheses (e.g., multiplicity problems occurring in genetics) and are becoming increasingly popular in preclinical research. It is important to point out that the FDR is uniformly larger than the FWER and, thus, controlling the FDR may not control the probability of an incorrect decision. In fact, in confirmatory studies, it is often possible to manipulate the design of the clinical trial so that any conclusion desired can be almost surely inferred without inflating the FDR (6). 3 UNION-INTERSECTION TESTING Most commonly, multiple testing problems are formulated as union-intersection (UI) problems (7), meaning that one is interested in testing the global hypothesis, denoted by H0I , which is the intersection of k null hypotheses versus the union of the corresponding alternative hypotheses, denoted by

MULTIPLE TESTING IN CLINICAL TRIALS

HAU . As an illustration, consider a dosefinding study designed to test a low and high dose of an experimental drug (labeled L and H) to placebo (P). The primary endpoint is a continuous variable with larger values indicating improvement. Let µP , µL , and µH denote the mean improvement in the placebo, low dose group, and high dose group, respectively. The individual null hypotheses tested in the trial are HL : µL ≤ µP and HH : µH ≤ µP . In this setting, a UI approach would test H0I : µL ≤ µP and µH ≤ µP versus HAU : µL > µP or µH > µP . According to the UI testing principle, the global hypothesis H0I is tested by examining each of its components individually, rejecting H0I if at least one of the components is rejected. Tests of homogeneity that one learns in elementary statistics courses, such as the F-test, tend to be UI tests. The following is a brief overview of popular methods for constructing UI tests. 3.1 Single-Step Tests Based on Univariate P-Values ˇ ak ´ These tests (e.g., the Bonferroni and Sid tests) are intuitive, easy to explain to non-statisticians and, for this reason, are frequently used in clinical applications. The Bonferroni adjustment for testing H0i amounts to computing an adjusted P-value ˇ ak-adjusted ´ given by kpi . Similarly, the Sid P-value for H0i is equal to 1 − (1 − pi )k . The adjusted P-values are then compared with α and the global hypothesis H0I is rejected if at least one adjusted P-value is no greater than α. Another example of a test based on univariate P-values is the Simes test (8). The adjusted Simes P-value for the global hypothesis H0I is k min(p[1] , p[2] /2, . . . , p[k] /k), where p[1] , . . . , p[k] are ordered P-values (i.e., p[1] ≤ . . . ≤ p[k] ). It is easy to see from this definition that the Simes test is uniformly more powerful than the Bonferroni test in the sense that the former rejects H0I every time the latter does. Although the Simes test has a power advantage over the Bonferroni test, one needs to remember that the Simes test does not always preserve the overall Type I error rate. It is known that the size of this test does not exceed α when p1 , . . . , pk are independent or positively dependent (9). It is important to keep in mind that tests based

3

on univariate P-values ignore the underlying correlation structure and become very conservative when the test statistics are highly correlated or the number of null hypotheses is large (e.g., in clinical trials with multiple outcome variables). 3.2 Parametric Single-Step Tests The power of simple tests based on univariate P-values can be improved considerably when one can model the joint distribution of the test statistics T1 , . . . , Tk . Consider, for example, the problem of comparing k doses of an experimental drug with a control in a one-sided manner. Assuming that T1 , . . . , Tk follow a multivariate normal distribution and larger treatment differences are better, Dunnett (10) derived a multiple test that rejects H0i if Ti ≥ d, where d is the 100(1 − α)% percentile of max(T1 , . . . , Tk ). Dunnett’s method also yields a set of simultaneous one-sided confidence intervals for the true mean treatment differences δ1 , . . . , δk :  δi > δˆi − ds 2/n,

i = 1, . . . , k

where s is the pooled sample standard deviation and n is the common sample size per treatment group. 3.3 Resampling-Based Single-Step Tests A general method for improving the performance of tests based on univariate P-values was proposed by Westfall and Young (3). Note first that the adjusted P-value for H0i is given by P{min(P1 , . . . , Pk ) ≤ pi }. In this equation, P1 , . . . , Pk denote random variables that follow the same distribution as p1 , . . . , pk under the assumption that the global hypothesis H0I is true. The joint distribution of the P-values is unknown and can be estimated using permutation or bootstrap resampling. The advantage of using resampling-based testing procedures is that they account for the empirical correlation structure of the individual P-values and, thus, are more powerful than the Bonferroni and similar tests. Furthermore, unlike the Dunnett test, the resampling-based approach does not rely on distributional assumptions. When carrying out resampling-based tests, it is important

4

MULTIPLE TESTING IN CLINICAL TRIALS

to ensure that the subset pivotality condition is met. This condition guarantees that the resampling-based approach preserves the FWER at the nominal level. The subset pivotality condition is met in most multiple testing problems for which pivotal quantities exist; however, it may not be satisfied in the case of binary variables, for example; see Reference 3 for more details. 4

CLOSED TESTING

A cornerstone of multiple hypotheses testing has been the closed testing principle of Marcus et al. (11). The principle has provided a foundation for a variety of multiple testing methods and has found a large number of applications in multiple testing problems occurring in clinical trials. Examples of such applications include procedures for multiple treatment comparisons and multiple outcome variables (12, 13), testing a dose-response relationship in dose ranging trials (14), and gatekeeping strategies for addressing multiplicity issues developing in clinical trials with multiple primary and secondary endpoints (15, 16). The closed testing principle is based on a hierarchical representation of the multiplicity problem in question. To illustrate, consider the null hypotheses HL and HH from the dose-finding trial example. In order to derive a closed test for this multiple testing problem, construct the closed family of null hypotheses by forming all possible intersections of the null hypotheses. The closed family contains HL , HH , and HL ∩ HH . The next step is to establish implication relationships in the closed family. A hypothesis that contains another hypothesis is said to imply it; for example, HL ∩ HH implies both H L and HH . The closed testing principle states that an FWER-controlling testing procedure can be constructed by testing each hypothesis in the closed family using a suitable level α test. A hypothesis in the closed family is rejected if its associated test and all tests associated with hypotheses implying it are significant. For example, applying the closed testing principle to the dose-finding trial example, statistical inference proceeds as follows:

• If HL ∩ HH is accepted, the closed test

has to accept HL and HH because HL ∩ HH implies HL and HH . • If HL ∩ HH is rejected, but not HL or HH , the inference is at least one of the two alternative hypotheses is true, but which one cannot be specified. • If HL ∩ HH and HH are rejected but HL is accepted, one concludes that HH is false (i.e., µH > µP ). Similarly, if HL ∩ HH and HL are rejected but HH is accepted, the null hypothesis HL is declared to be false (i.e., µL > µP ). • Lastly, if HL ∩ HH , HL and HH are rejected, the inference is that µL > µP and µH > µP . Now, in order to construct a multiple testing procedure, one needs to choose a level α significance test for the individual hypotheses in the closed family. Suppose, for example, that the individual hypotheses are tested using the Bonferroni test. The resulting closed testing procedure is equivalent to the stepwise testing procedure proposed by Holm (17). The Holm procedure relies on a sequentially rejective algorithm for testing the ordered null hypotheses H[01] , . . . , H[0k] corresponding to the ordered P-values p[1] ≤ . . . ≤ p[k] . The procedure first examines the null hypothesis associated with the most significant P-value (i.e., H[01] ). This hypothesis is rejected if p[1] ≤ α/k. Further, H[0i] is rejected if p[j] ≤ α/(k − j + 1) for all j = 1, . . . , i. Otherwise, the remaining null hypotheses H[0i] , . . . , H[0k] are accepted and testing ceases. Note that H[01] is tested at the α/k level and the other null hypotheses are tested at successively higher significance levels. As a result, the Holm procedure rejects at least as many (and possibly more) null hypotheses as the Bonferroni test from which it was derived. This example shows that, by applying the closed testing principle to a single-step test, one can construct a more powerful stepwise test that maintains the FWER at the same level. The same approach can be adopted to construct stepwise testing procedures based on other single-step tests. For example, the popular Hochberg and Hommel testing procedures can be thought of as closed testing

MULTIPLE TESTING IN CLINICAL TRIALS

versions of the Simes test (18, 19). It is worth noting that the Hommel procedure is uniformly more powerful than the Hochberg procedure, and both of the two procedures preserve the FWER at the nominal level only when the Simes test does (i.e., under the assumption of independence or positive dependence). In the parametric case, an application of the closed testing principle to the Dunnett test results in the stepwise Dunnett test defined as follows. Consider again the comparison of k doses of an experimental drug with a control in a one-sided setting. Let T[1] , . . . , T[k] denote the ordered test statistics (T[1] ≤ . . . ≤ T[k] ) and di be the 100(1 − α)% percentile of max(T1 , . . . , Ti ), i = 1, . . . , k. The stepwise Dunnett test begins with the most significant statistic and compares it with dk . If T[k] ≥ dk , the null hypothesis corresponding to T[k] is rejected and the second most significant statistic is examined. Otherwise, the stepwise algorithm terminates and the remaining null hypotheses are accepted. It is easy to show that the derived stepwise test is uniformly more powerful than the single-step Dunnett test. An important limitation of the closed testing principle is that it does not generally provide the statistician with a tool for constructing simultaneous confidence intervals for parameters of interest. For instance, it is not clear how to set up simultaneous confidence bounds for the mean differences between the k dose groups and control group within the closed testing framework. The closed testing principle can also be used in the context of resampling-based multiple tests to set up stepwise testing procedures that account for the underlying correlation structure. 5

PARTITION TESTING

The partitioning principle introduced in References 20 and 21 can be viewed as a natural extension of the principle of closed testing. The advantage of using the partitioning principle is two-fold: Partitioning procedures are sometimes more powerful than procedures derived within the closed testing framework and, unlike closed testing procedures, they are easy to invert in order to set up simultaneous confidence sets for parameters of

5

interest. To introduce the partitioning principle, consider k null hypotheses tested in a clinical trial and assume that H0i states that θ ∈ i , where θ is a multidimensional parameter and i is a subset of the parameter space. Partition the union of 1 , . . . , k into disjoint subsets ∗J , J ⊂ {1, . . . , k}, which can be interpreted as the part of the parameter space in which exactly H0i , i ∈ J are true and the remaining null hypotheses are false. Now define null hypotheses corresponding to the constructed subsets (i.e., HJ∗ : θ ∈ ∗J ) and test them at level α. As these null hypotheses are mutually exclusive, at most one of them is true. Therefore, although no multiplicity adjustment is made, the resulting multiple test controls the FWER at the α level. To illustrate the process of carrying out partitioning tests, consider the null hypotheses HL : µL ≤ µP and HH : µH ≤ µP from the dose-finding trial example. The union of HL and HH is partitioned into three hypotheses: H1∗ : µL ≤ µP

and

µH ≤ µP

H2∗

: µL ≤ µP

and

µH > µP

H3∗

: µL > µP

and

µH ≤ µP

Testing each of the three hypotheses with a level α significance test results in the following decision rule: • If H1∗ is accepted, neither HL nor HH can

be rejected, otherwise infer that µL > µP or µH > µP . • If H1∗ and H2∗ are rejected, one concludes that µL > µP . Likewise, rejecting H1∗ and H3∗ implies that µH > µP . • Finally, if H1∗ , H2∗ and H3∗ are rejected, the inference is that µL > µP and µH > µP . Although these decision rules appears to be similar to the closed testing rules, it is important to point out that the partitioning principle does not deal with the hypotheses in the closed family (i.e., HL , HH and HL ∩ HH ) but rather with hypotheses H1∗ , H2∗ , and H3∗ defined above. As a result of the choice of mutually exclusive null hypotheses, partitioning tests can be inverted to derive a confidence region for

6

MULTIPLE TESTING IN CLINICAL TRIALS

the unknown parameter θ . Recall that the most general method for constructing a confidence set from a significance test is defined as follows. For each parameter point θ0 , test H0 : θ = θ0 using an level-α test and then consider the set of all parameter points θ0 for which H0 : θ = θ0 is accepted. The obtained set is a 100(1 − α) confidence set for the true value of θ . This procedure corresponds to partitioning the parameter space into subsets consisting of a single parameter point and can be used for constructing simultaneous confidence limits associated with various stepwise tests. Consider, for example, confidence limits for the mean treatment differences between k dose groups and a control group (20). If the largest mean difference is not significant (T[k] < dk ), the one-sided limits for the true mean differences δ1 , . . . , δk are given by  δi > δˆi − dk s 2/n,

i = 1, . . . , k

and testing stops. Otherwise, one infers that δ[k] > 0 and examines the second largest difference. At the jth step of the stepwise test, the one-sided limits for δ[1] , . . . , δ[k−j+1] are  δ[i] > δˆ[i] − dk s 2/n,

i = 1, . . . , k − j + 1

if the corresponding test statistic is not significant (T[k−j+1] < dk−j+1 ) and δ[k−j+1] > 0 otherwise. Comparing the resulting testing procedure to the stepwise Dunnett test derived in Section 4 using the closed testing principle, it is easy to see that the partitioning principle extends the closed testing framework by enabling clinical researchers to set up confidence limits for treatment-control differences. The partitioning principle can also be used for constructing confidence sets in a much more general context [e.g., confidence intervals for fixed-sequence testing methods occurring in dose-finding studies and other clinical applications (22)]. REFERENCES 1. European Committee for Propriety Medicinal Products, Points to consider on multiplicity issues in clinical trials. September 19, 2002. 2. Y. Hochberg and A. C. Tamhane, Multiple Comparison Procedures. New York: Wiley, 1987.

3. P. H. Westfall and S. S. Young, ResamplingBased Multiple Testing: Examples and Methods for P-Value Adjustment. New York: Wiley, 1993. 4. J. C. Hsu, Multiple Comparisons: Theory and Methods. London: Chapman & Hall, 1986. 5. Y. Benjamini and Y. Hochberg, Controlling the false discovery rate—a practical and powerful approach to multiple testing. J. Royal Stat. Soc. Series B 1995; 57: 289–300. 6. H. Finner and M. Roter, On the false discovery rate and expected Type I errors. Biometr. J. 2001; 43: 985–1005. 7. S. N. Roy, On a heuristic method for test construction and its use in multivariate analysis. Ann. Stat. 1953; 24: 220–238. 8. R. J. Simes, An improved Bonferroni procedure for multiple tests of significance. Biometrika 1986; 63: 655–660. 9. S. Sarkar and C. K. Chang, Simes’ method for multiple hypothesis testing with positively dependent test statistics. J. Amer. Stat. Assoc. 1997; 92: 1601–1608. 10. C. W. Dunnett, A multiple comparison procedure for comparing several treatments with a control. J. Amer. Stat. Assoc. 1955; 50: 1096–1121. 11. R. Marcus, E. Peritz and K. R. Gabriel, On closed testing procedure with special reference to ordered analysis of variance. Biometrika 1976; 63: 655–660. 12. P. Bauer, Multiple testings in clinical trials. Stat. Med. 1991; 10: 871–890. 13. W. Lehmacher, G. Wassmer and P. Reitmeir, Procedures for two-sample comparisions with multiple endpoints controlling the experimentwise error rate. Biometrics 1991; 47: 511–521. 14. D. M. Rom, R. J. Costello and L. T. Connell, On closed test procedures for dose-response analysis. Stat. Med. 1994; 13: 1583–1596. 15. A. Dmitrienko, W. Offen and P. H. Westfall, Gatekeeping strategies for clinical trials that do not require all primary effects to be significant. Stat. Med. 2003; 22: 2387–2400. 16. P. H. Westfall and A. Krishen, Optimally weighted, fixed sequence, and gate-keeping multiple testing procedures. J. Stat. Plan. Inference 2001; 99: 25–40. 17. S. Holm, A simple sequentially rejective multiple test procedure. Scand. J. Stat. 1979; 6: 65–70. 18. Y. Hochberg, A sharper Bonferroni procedure for multiple significance testing. Biometrika 1988; 75: 800–802.

MULTIPLE TESTING IN CLINICAL TRIALS 19. G. Hommel, A stagewise rejective multiple test procedure based on a modified Bonferroni test. Biometrika 1988; 75: 383–386. 20. G. Stefansson, W-C. Kim and J. C. Hsu, On confidence sets in multiple comparisons. In: S. S. Gupta and J. O. Berger, eds. Statistical Decision Theory and Related Topics IV. New York: Academic Press, 1988, pp. 89–104. 21. H. Finner and K. Strassburger, The partitioning principle: a powerful tool in multiple decision theory. Ann. Stat. 2002; 30: 1194–1213. 22. J. C. Hsu and R. L. Berger, Stepwise confidence intervals without multiplicity adjustment for dose-response and toxicity studies. J. Amer. Stat. Assoc. 1989; 94: 468–482.

FURTHER READING A. Dmitrienko, G. Molenberghs, C. Chuang-Stein, W. Offen. Analysis of Clinical Trials Using SAS: A Practical Guide. Carry, NC (Chapter 2, ‘‘Multiple comparisons and multiple endpoints’’). 2005.

7

MULTISTAGE GENETIC ASSOCIATION STUDIES

Our primary focus is on interactions rather than main effects, which has been the main thrust of most of the literature on multistage designs. In a pharmaceutical trial, there is generally a wealth of prior physiological and biochemical knowledge about the pathway(s) targeted by the agent under study, so the aim of a genetic study will be to characterize the role of particular functional polymorphisms across multiple genes within critical steps in the pathway. The exposure variable (here, treatment) may be studied with a randomized controlled trial, rather than by observational epidemiologic studies with their attendant potential for confounding and other biases. The study design is likely to involve unrelated individuals rather than families, particularly in therapeutic trials where there would be few families with multiple cases eligible to be treated concurrently and even fewer opportunities to assign members of the same family to different treatments.

DUNCAN THOMAS DAVID CONTI Department of Preventive Medicine University of Southern California Los Angeles, CA

We consider the design of genetic association studies within the context of clinical research, where the ultimate question might be to identify and characterize genes that modify the response to pharmacologic agents, either preventive or therapeutic, or other interventions. This includes both existing clinical trials primarily designed to determine treatment effects, expanding them with secondary goals aimed at characterizing the influence of genes, as well as nested challenge or treatment experiments within existing population-based studies, exploiting extensive information (possibly genetic) that may already have been collected. In either case, the questions are similar. Why is it that some people respond favorably to a particular treatment and others do not? Why do some experience a particular adverse effect and others do not? Why is one treatment better for some people, and a different treatment better for someone else? Could something in a person’s genetic makeup explain such differences and help inform a future of personalized medicine? There is, of course, a vast literature on the design of genetic association studies and pharmacogenetic trials, which is summarized in other articles in this volume (see the articles on genetic association analysis and pharmacogenomics) as well as previously in the literature (1–3). Here, we focus on the design of multistage association studies, where statistical sampling can be used in various ways to improve cost-efficiency. General works are available on multistage designs (4–7) and the context of genetic studies (8–10). In particular, we focus on two-stage designs in the pharmacogenetic context. This focus has several implications.

It is also worth considering the various genetic contexts in which such research might be undertaken. One might have in mind a particular candidate gene containing one or more known functional polymorphisms that are known to play a role in the metabolism or other action of the agent under study. Or one might know the function of the gene, but not have identified any (or all) functional polymorphisms, and would like to fully characterize the spectrum of variation within the gene and its potential impact on the outcome under study. This may entail single nucleotide polymorphism (SNP) discovery through resequencing, or extensive genotyping of all known polymorphisms in a sample, before proceeding with typing only a subset of variants in the main study. More generally, one may have some understanding

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

MULTISTAGE GENETIC ASSOCIATION STUDIES

of an entire pathway involving multiple genes and possibly environmental factors affecting their expression or substrates, or even a complex network of interconnected pathways; in either case, one wishes to build a model for the entire system of genes. Whether one is studying a single gene or an entire pathway, it may be helpful to measure certain intermediate biomarkers (gene expression, methylation, metabolomic or proteomic measures) or additional clinical phenotypes. As these may be expensive or difficult to obtain, one might consider using only a subsample for this purpose. If no particular candidate genes are known, one might consider a genomewide search for possible associations, or at least a search for variants within a candidate region (say from a previous linkage scan); in either case, one would be looking for variants that might either be directly causal or in linkage disequilibrium with other causal variants. Each of these contexts provides different design challenges that might be addressed using some form of multistage design. 1 REASONS TO CONSIDER MULTISTAGE DESIGNS The main reason to consider multistage sampling designs is cost. No subsampling can possibly yield a more powerful study than obtaining complete data on all variables on all subjects, unless the cost of doing so would require a smaller overall sample size (1). But some measurements may be more informative or more costly than others, so some trade-off could lead to a more cost-efficient design. Here, two situations must be distinguished, one where the total sample size is fixed (say, by a previously completed clinical trial or population-based sample), and one where it is not. In the former case, one may simply be interested in getting a reasonably precise answer at a reasonable cost, recognizing that additional genetic measurements

or treatment assignments may not improve precision enough to justify the cost of making them on everybody. In this case, one might have an opportunity to conduct a larger main study by saving costs on some measurements through subsampling. A second reason is there may be opportunities to improve the statistical efficiency of the design by using stratified sampling, based on one or more sources of information already collected. These could include treatment arm, outcomes, covariates, genotypes (or surrogates for them), or various combinations of such factors. Depending on the sampling design used, adjustments may be necessary to allow for the sampling scheme in the statistical analysis to obtain unbiased estimates. For example, if one were to overrepresent both cases (adverse outcomes) and active treatment arm subjects, then a na¨ıve analysis would induce a spurious association between treatment and outcome and likely distort the assessment of the modifying effect of genotypes or biomarkers measured on the subsample. Fortunately, allowing for such biased sampling schemes is relatively easy via weighting subjects by the inverse of their sampling probabilities in the analysis (11) or by including the logarithms of these weights as ‘‘offsets’’ (covariates with regression coefficients fixed at unity) in a logistic regression analysis (12). As an extreme example, when investigating a dichotomous outcome within a previously performed randomized control trial, one might only select the ‘‘cases’’ as the subset for further genotyping and then perform the appropriate case-only analysis, yielding an increase in power for detection of gene × treatment interactions (13–17). In many cases, the substudy data only may be sufficient to answer the immediate questions, but some types of more integrative questions might require combining the information from the main study and substudy in some manner. For example, one might use substudy data to build a model for the dependence of biomarkers on genes, treatment, and possibly other factors, and then incorporate these predicted biomarkers into an analysis of the main study in which data on treatments, genotypes, and outcomes (but not biomarkers) were available on everybody (18).

MULTISTAGE GENETIC ASSOCIATION STUDIES

Much of our previous discussion and literature has assumed that the genetic association study is to be conducted as an add-on to a previous clinical trial, with the aim of studying the genetic determinants of response to treatment. An alternative approach might be to add a clinical trial to some preexisting observational study in which the genetic variation has already been characterized (for instance, an etiologic cohort or case-control study), and use the genetics as the basis for a sampling scheme to assess the outcome of some treatment in targeted subgroups. The potential benefit of this lies in being able to balance the genotypic distribution to obtain greater statistical efficiency when testing for gene–treatment interactions. For example, if one were interested in investigating how treatment response varies in relation to a previously measured rare genetic variant, one could overrepresent mutation carriers relative to their original cohort frequencies. However, one may quickly reach the limits for an efficient block design if the number of independent variants under investigation is large (see the article on stratification). Finally, whatever general sampling and analysis scheme is used, it may be possible to predict what the optimal sampling fractions would be for a range of plausible model parameters and costs (19–23). Under some circumstances, such an analysis may reveal that the most efficient design, weighing both statistical efficiency and cost, does not involve any subsampling with differential assessment of variables across samples but rather a complete assessment of all variables on a smaller main study. 2 EXAMPLE 1: CANDIDATE GENE ASSOCIATION STUDY USING TAG SNPs Suppose one has identified a candidate gene that is hypothesized to have an effect on the outcome under study or, in a clinical trial setting, to modify the effect of treatment. If one or more functional polymorphisms have already been identified in this gene and if one were persuaded that the hypothesized effects of the gene could be tested using only these known variants, there would probably be no need for a multistage sampling

3

design. Assuming that the cost for genotyping these known variants is not prohibitive, one would simply genotype these variants in a single sample from the relevant population; this might be, for example, cases and controls for a disease trait, a simple or stratified random sample of the population for a quantitative trait, or possibly the entire sample from a preexisting study such as a clinical trial. The design and analysis of such studies is discussed elsewhere in this volume (see the articles on genetic association analysis and pharmacogenomics). A negative result from such a study could, however, arise if the targeted polymorphism(s) was not the relevant one and would not exclude the possibility that some other variants in the gene might have shown an effect. Complete characterization of a gene—including all rare variations— conventionally entails full sequencing of the entire gene (not just coding regions, but potential regulatory regions, splice-site junctions, and highly conserved intronic regions). Because previous genomic databases could not be relied on to include all rare variants, a large sample would be required to detect them, although there do exist procedures for detecting rare variants in DNA pools (24). If such large samples are required, such a monumental task would be difficult to justify unless one had a strong hypothesis that rare variants were likely to be important (25–28). If, on the other hand, one wished instead to exhaustively characterize only the common variation in the gene, then it would not be necessary to fully sequence every subject, and a staged approach might be considered. The fact that most common variants are in strong linkage disequilibrium (LD) with other nearby variants means that most of them can be effectively predicted by combinations of a modest number of ‘‘tagging SNPs’’ (29–34). In this case, a relatively small sample of individuals might be sufficient to characterize the LD structure of the gene and select this subset of tag SNPs, and only this subset is then tested in the main study. However, in determining the subsample needed, one must also take into account the uncertainty of LD levels and patterns across the genome and the effect this uncertainty may have on characterization of

4

MULTISTAGE GENETIC ASSOCIATION STUDIES

the underlying genetic diversity. Often the SNP selection sample is not part of the main study and is selected to represent the source population (mainly controls, if the trait is rare), so the information on the untagged variants is not used in the analysis of the main association study. Because the International HapMap Project is designed to provide a freely available catalog of the genetic similarities and difference across human populations, this resource serves as a primary example of SNP selection samples and often remains independent of the main study in the final analysis. However, the SNP selection sample could be used to develop a prediction model for each of the untagged variants, which could then be tested for association in the main study (23). Such analyses could be done at the genotype or haplotype level. Suppose, for example, that one wished to test the association of a disease trait Y with a particular untyped SNP polymorphism G using a set T of tagging SNPs. The substudy (in which both G and T are measured) yields an estimate of pα (G|T), where α denotes a vector of LD information to be estimated from the substudy. The main study would then be analyzed using likelihood contributions of the form for each subject: pG (Y|T) = g pβ (Y|G = g)pα (G = g|T) where β is the relative risk parameter of interest. The single-imputation strategy (35) approximates this likelihood by using as the risk factor for each individual in the main study his or her expected allele (or haplotype) ‘‘dosage,’’ E(G|T). Although more complex, the likelihood-based analysis, explicitly incorporating contributions from the substudy subjects, has the advantage of properly allowing for the uncertainty in the estimates of the parameters α in the genotype prediction model, and supports a broader range of analyses, such as the estimation of the location of an unobserved causal variant. A haplotype-based analysis is similar, except that the postulated causal genotype G is replaced by a vector of diplotypes (pairs of haplotypes) H, and the likelihood requires an additional summation over all possible diplotypes that are compatible with the observed unphased tag-SNP genotypes T.

This likelihood can be fitted using either the expectation-maximization (E-M) algorithm (36) or Markov-chain Monte Carlo (MCMC) methods (37). This basic approach could be extended to a multistage design, depending on the extent of ancillary knowledge from genomic databases. For example, one might fully sequence a small sample to search for previously undetected variants (particularly if one were studying a population that was underrepresented in the HapMap or previous resequencing databases); cases with extreme phenotypes are sometimes used for this purpose to identify previously unrecognized variants that are more likely to be causal (38). A somewhat larger sample of controls might then be genotyped at all common variants (from the investigator’s own first sample as well as other databases) to investigate the LD structure and choose tagSNPs, which would then be genotyped in the main study. In a study of gene-treatment interactions, one could optimize a two-stage design by selecting the subsample and stratifying on outcome, treatment, or a correlate of the causal variant (23). For example, one might overrepresent subjects with a positive family history or known carriers of a particular functional polymorphism. 3 EXAMPLE 2: PATHWAY-BASED STUDY INVOLVING BIOMARKERS Suppose one wished to investigate an entire pathway or network of genes, such as those known to be involved in the metabolism of a particular drug or involved in repair of DNA damage induced by ionizing radiation. One could simply genotype all these loci in the entire study population and build an empirical model for the main effects and interactions (gene–gene or gene–treatment, for example) (39) or a physiologically based pharmacokinetic model incorporating these genes as modifiers of the respective metabolic rates (40). Such models might be usefully informed by having additional biomarker data on some of the intermediate steps in the process, such as metabolite concentrations in blood or urine (41). These measurements may be expensive or difficult to collect, however, possibly

MULTISTAGE GENETIC ASSOCIATION STUDIES

requiring multiple samples over time because of the variability in the measurements or biases due to inducing factors, measurement factors (e.g., time of day), or the underlying disease process (making measurements on cases suspect due to the possibility of ‘‘reverse causation’’). One might therefore consider making the biomarker measurements on only a sample, perhaps only for controls. Suppose one is primarily interested in studying the relation of some intermediate metabolite X on outcome Y, where X is postulated to be modified by treatment T and genes G, and the biomarker Z is a flawed surrogate for X. Then one might have measurements of (T,G,Z) on the subsample S and of (T,G,Y) on the main study M. The likelihood is then formed by marginalizing over the unobserved X:
 p(Zi |Xi = x)p(Xi = x|Gi , Ti ) dx iS

×




p(Yj |Xj = x)p(Xj = x|Gj , Tj ) dx

jM

As in Example (1), one might seek to optimize the study design by stratified sampling of S based on information about (T,G,Y) already obtained from the main study. Or, rather than using such a model-dependent likelihood analysis, the technique of Mendelian randomization (42–47) provides an indirect ‘‘nonparametric’’ test of the effect of X on Y by comparing instead the effects of G on both Y and Z separately. Davey Smith and Ibrahim (42) have argued that, under appropriate circumstances, inferences from Mendelian randomization about the effect of an environmentally modifiable intermediate variable can have the same strength of causality as from a randomized controlled trial. 4 EXAMPLE 3: GENOME-WIDE ASSOCIATION STUDY Now, suppose there were no specific genes or pathways with strong prior hypotheses, but one wished instead to scan the entire genome for variants that influenced the trait or response to treatment. This possibility was first seriously suggested a decade ago (48),

5

but it is only very recently that advances in high-volume genotyping technologies have made it possible at a reasonable cost (49–52). Most recent genome-wide association (GWA) studies have adopted some variant of a multistage genotyping strategy first proposed by Sobell et al. (53) and formalized in a series of papers by Satagopan et al. (54–56) and others (57–60). The basic idea of all these approaches is to use relatively inexpensive (per genotype) commercial genotyping technology for dense SNP genotyping (about 300 thousand to 1 million SNPs and increasing rapidly) of a sample of the study population and, on the basis of the observed associations in this sample; one selects a small subset of the most promising SNPs for genotyping in a second sample, using a more expensive (per genotype) customized assay. The two samples are then combined in the final analysis, with critical values at stages 1 and 2 chosen to yield a predetermined level of genome-wide significance (e.g., 0.05 expected false positives across the entire genome) (61). Variants of this general idea include targeted sampling of subjects for the first stage, use of DNA pooling in the first stage followed by individual genotyping 62, 63, adding additional SNPs at the second stage to localize the LD signal (60), or using prior information such as linkage scans or genomic annotation data to prioritize the selection of SNPs for the second stage 64, 65. Needless to say, with hundreds of thousands of markers under consideration, the multiple comparisons problem is serious (see the article on multiple comparisons in this work). A variety of ways of analyzing the data have been proposed, generally using some form of Bonferroni correction or false discovery rate approaches (66–68). Even more daunting are analyses of all possible haplotype associations (69) or all possible gene–gene interactions (70). Given the infancy of this approach, there are many unresolved design and analysis issues, but there are many opportunities to exploit multistage sampling designs in new and creative ways. As in the previous example, of course, one could stratify the sampling of subjects by various combinations of disease, treatment (or other interacting factors), or race. In addition, there are various ways to refine the selection of markers to

6

MULTISTAGE GENETIC ASSOCIATION STUDIES

advance to stage 2, particularly if there are multiple types of tests under consideration (main effects of SNPs, haplotype associations, gene–gene or gene–treatment interactions, etc.) or prior knowledge to be exploited. 5

PERSPECTIVES

Although multistage sampling designs have been well established in statistics for decades and are increasingly used in epidemiology and genetics, this remains an underdeveloped research area, particularly in the pharmacogenetics context. Its utility for exploring pathways appears to be particularly promising, especially in the potential incorporation of pharmacokinetic/pharmacodynamic principles and approaches (71–75). This field is in a rapid state of development, driven in part by the potential for drug development. The interested reader should consult several recent reviews (17, 76–80) for further discussion of these issues. REFERENCES 1. R. C. Elston, R. M. Idury, L. R. Cardon, and J. B. Lichter, The study of candidate genes in drug trials: sample size considerations. Stat Med. 1999; 18: 741–751. 2. L. R. Cardon, R. M. Idury, T. J. Harris, J. S. Witte, and R. C. Elston, Testing drug response in the presence of genetic information: sampling issues for clinical trials. Pharmacogenetics. 2000; 10: 503–510. 3. P. J. Kelly, N. Stallard, and J. C. Whittaker, Statistical design and analysis of pharmacogenetic trials. Stat Med. 2005; 24: 1495–1508. 4. N. Breslow and K. Cain, Logistic regression for two-stage case-control data. Biometrika. 1988; 75: 11–20. 5. N. E. Breslow and N. Chatterjee, Design and analysis of two-phase studies with binary outcome applied to Wilms tumor prognosis. Appl Stat. 1999; 48: 457–468. 6. K. Cain and N. Breslow, Logistic regression analysis and efficient design for twostage studies. Am J Epidemiol. 1988; 128: 1198–1206. 7. J. E. White, A two stage design for the study of the relationship between a rare exposure and a rare disease. Am J Epidemiol. 1982; 115; 119–128.

8. R. C. Elston, D. Y. Lin, and G. Zheng. Multistage sampling for genetic studies. Annu Rev Genomics Hum Genet. 2007; 8: 327–342. 9. K. D. Siegmund, A. S. Whittemore, and D. C. Thomas, Multistage sampling for disease family registries. J Natl Cancer Inst Monogr. 1999; 26: 43–48. 10. A. S. Whittemore and J. Halpern, Multi-stage sampling in genetic epidemiology. Stat Med. 1997; 16: 153–167. 11. D. Horvitz and D. Thompson, A generalization of sampling without replacement from a finite population. J Am Stat Assoc. 1952; 47: 663–685. 12. B. Langholz and L. Goldstein. Conditional logistic analysis of case-control studies with complex sampling. Biostatistics. 2001; 2: 63–84. 13. W. Piegorsch, C. Weinberg, and J. Taylor, Non-hierarchical logistic models and caseonly designs for assessing susceptibility in population-based case-control studies. Stat Med. 1994; 13: 153–162. 14. M. Khoury and W. Flanders, Nontraditional epidemiologic approaches in the analysis of gene-environment interaction: case-control studies with no controls! Am J Epidemiol. 1996; 144: 207–213. 15. S. Greenland, A unified approach to the analysis of case-distribution (case-only) studies. Stat Med. 1999; 18: 1–15. 16. W. J. Gauderman, Sample size requirements for matched case-control studies of geneenvironment interaction. Stat Med. 2002; 21: 35–50. 17. J. Little, L. Sharp, M. J. Khoury, L. Bradley, and M. Gwinn, M. The epidemiologic approach to pharmacogenomics. Am J Pharmacogenomics. 2005; 5: 1–20. 18. D. C. Thomas, Multistage sampling for latent variable models. Lifetime Data Anal. In press. 19. D. Spiegelman and R. Gray, Cost-efficient study designs for binary response data with Gaussian covariate measurement error. Biometrics. 1991; 47: 851–869. 20. D. Spiegelman, R. J. Carroll, and V. Kipnis, Efficient regression calibration for logistic regression in main study/internal validation study designs with an imperfect reference instrument. Stat Med. 2001; 20: 139–160. 21. S. Greenland, Statistical uncertainty due to misclassification: implications for validation substudies. J Clin Epidemiol. 1988; 41: 1167–1174.

MULTISTAGE GENETIC ASSOCIATION STUDIES 22. W. D. Flanders and S. Greenland, Analytic methods for two-stage case-control studies and other stratified designs. Stat Med. 1991; 10: 739–747. 23. D. Thomas, R. Xie, and M. Gebregziabher, Two-Stage sampling designs for gene association studies. Genet Epidemiol. 2004; 27: 401–414. 24. L. T. Amundadottir, P. Sulem, J. Gudmundsson, A. Helgason, A. Baker, A., et al., A common variant associated with prostate cancer in European and African populations. Nat Genet. 2006; 38: 652–658. 25. J. K. Pritchard, Are rare variants responsible for susceptibility to complex diseases? Am J Hum Genet. 2001; 69: 124–137. 26. J. K. Pritchard and N. J. Cox, The allelic architecture of human disease genes: common disease-common variant . . . or not? Hum Mol Genet. 2002; 11: 2417–2423. 27. W. Y. Wang, H. J. Cordell, and J. A. Todd, Association mapping of complex diseases in linked regions: estimation of genetic effects and feasibility of testing rare variants. Genet Epidemiol. 2003; 24: 36–43. 28. N. S. Fearnhead, J. L. Wilding, B. Winney, S. Tonks, S. Bartlett, et al. Multiple rare variants in different genes account for multifactorial inherited susceptibility to colorectal adenomas. Proc Natl Acad Sci USA. 2004; 101: 15992–15997. 29. D. O. Stram, Tag SNP selection for association studies. Genet Epidemiol. 2004; 27: 365–374. 30. D. O. Stram, Software for tag single nucleotide polymorphism selection. Hum Genomics. 2005; 2: 144–151. 31. D. Thompson, D. Stram, D. Goldgar, and J. S. Witte, Haplotype tagging single nucleotide polymorphisms and association studies. Hum Hered. 2003; 56: 48–55. 32. G. C. Johnson, L. Esposito, B. J. Barratt, A. N. Smith, J. Heward, et al. Haplotype tagging for the identification of common disease genes. Nat Genet. 2001; 29: 233–237. 33. Z. Lin and R. B. Altman, Finding haplotype tagging SNPs by use of principal components analysis. Am J Hum Genet. 2004; 75: 850–861. 34. Z. Meng, D. V. Zaykin, C. F. Xu, M. Wagner, and M. G. Ehm, Selection of genetic markers for association analyses, using linkage disequilibrium and haplotypes. Am J Hum Genet. 2003; 73: 115–130. 35. D. O. Stram, C. L. Pearce, P. Bretsky, M. Freedman, J. N. Hirschhorn, et al., Modeling and E-M estimation of haplotype-specific

7

relative risks from genotype data for a casecontrol study of unrelated individuals. Hum Hered. 2003; 55: 179–190. 36. L. Excoffier, G. Laval, and D. Balding, Gametic phase estimation over large genomic regions using an adaptive window approach. Hum Genomics. 2003; 1: 7–19. 37. T. Niu, Z. S. Qin, X. Xu, and J. S. Liu, Bayesian haplotype inference for multiple linked singlenucleotide polymorphisms. Am J Hum Genet. 2002; 70: 157–169. 38. C. A. Haiman, D. O. Stram, M. C. Pike, L. N. Kolonel, N. P. Burtt, et al., A comprehensive haplotype analysis of CYP19 and breast cancer risk: the Multiethnic Cohort. Hum Mol Genet. 2003; 12: 2679–2692. 39. R. J. Hung, P. Brennan, C. Malaveille, S. Porru, F. Donato, et al., Using hierarchical modeling in genetic association studies with multiple markers: application to a case-control study of bladder cancer. Cancer Epidemiol Biomark Prev. 2004; 13: 1013–1021. 40. D. V. Conti, V. Cortessis, J. Molitor, and D. C. Thomas, Bayesian modeling of complex metabolic pathways. Hum Hered. 2003; 56: 83–93. 41. P. Tonolio, P. Boffetta, D. E. K. Shuker, N. Rothman, B. Hulka, and N. Pearce, Application of Biomarkers in Cancer Epidemiology. Lyon, France: IARC Scientific, 1997. 42. G. Davey Smith and S. Ebrahim, ‘‘Mendelian randomization’’: can genetic epidemiology contribute to understanding environmental determinants of disease? Int J Epidemiol. 2003; 32: 1–22. 43. Brennan, P. Commentary: Mendelian randomization and gene-environment interaction. Int J Epidemiol. 2004; 33: 17–21. 44. G. Davey Smith and S. Ebrahim, What can mendelian randomisation tell us about modifiable behavioural and environmental exposures? BMJ. 2005; 330: 1076–1079. 45. G. Davey Smith and S. Ebrahim, Mendelian randomization: prospects, potentials, and limitations. Int J Epidemiol. 2004; 33: 30–42. 46. D. C. Thomas and D. V. Conti, Commentary: the concept of ‘‘mendelian randomization.’’ Int J Epidemiol. 2004; 33: 21–25. 47. J. Little and M. J. Khoury, Mendelian randomization: a new spin or real progress? Lancet. 2003; 362: 390–391. 48. N. Risch and K. Merikangas, The future of genetic studies of complex human diseases. Science. 1996; 273: 1616–1617.

8

MULTISTAGE GENETIC ASSOCIATION STUDIES

49. D. C. Thomas, R. W. Haile, and D. Duggan, Recent developments in genomewide association scans: a workshop summary and review. Am J Hum Genet. 2005; 77: 337–345. 50. D. C. Thomas, Are we ready for genomewide association studies? Cancer Epidemiol Biomarkers Prev. 2006; 15: 595–598. 51. W. Y. S. Wang, B. J. Barratt, D. G. Clayton, and J. A. Todd, Genome-wide association studies: theoretical and practical concerns. Nat Rev Genet. 2005; 6: 109–118. 52. J. N. Hirschhorn and M. J. Daly, Genomewide association studies for common disease and complex traits. Nat Rev Genet. 2005; 6: 95–108. 53. J. L. Sobell, L. L. Heston, and S. S. Sommer, Novel association approach for determining the genetic predisposition to schizophrenia: case-control resource and testing of a candidate gene. Am J Med Genet. 1993; 48: 28–35. 54. J. M. Satagopan, D. A. Verbel, E. S. Venkatraman, K. E. Offit, and C. B. Begg, Two-stage designs for gene-disease association studies. Biometrics. 2002; 58: 163–170. 55. J. M. Satagopan and R. C. Elston, Optimal two-stage genotyping in population-based association studies. Genet Epidemiol. 2003; 25: 149–157. 56. J. M. Satagopan, E. S. Venkatraman, and C. B. Begg, Two-stage designs for gene-disease association studies with sample size constraints. Biometrics. 2004; 60: 589–597. 57. A. Saito and N. Kamatani, Strategies for genome-wide association studies: optimization of study designs by the stepwise focusing method. J Hum Genet. 2002; 47: 360–365. 58. S. K. Service, L. A. Sandkuijl, and N. B. Freimer, Cost-effective designs for linkage disequilibrium mapping of complex traits. Am J Hum Genet. 2003; 72: 1213–1220. 59. H. Wang and D. O. Stram, Optimal twostage genome-wide association designs based on false discovery rate. Comput Stat Data Anal. 2006; 5: 457–465. 60. H. Wang, D. C. Thomas, I. Pe’er, and D. O. Stram, Optimal two-stage genotyping designs for genome-wide association scans. Genet Epidemiol. 2006; 30: 356–368. 61. A. D. Skol, L. J. Scott, G. R. Abecasis, and M. Boehnke, Joint analysis is more efficient than replication-based analysis for two-stage genome-wide association studies. Nat Genet. 2006; 38: 209–213. 62. A. Bansal, D. van den Boom, S. Kammerer, C. Honisch, G. Adam, et al., Association testing by DNA pooling: an effective initial

screen. Proc Natl Acad Sci USA. 2002; 99: 16871–16874. 63. S. Wang, K. K. Kidd, and H. Zhao, On the use of DNA pooling to estimate haplotype frequencies. Genet Epidemiol. 2003; 24: 74–82. 64. K. Roeder, S. A. Bacanu, L. Wasserman, and B. Devlin, Using linkage genome scans to improve power of association in genome scans. Am J Hum Genet. 2006; 78: 243–252. 65. A. S. Whittemore, A Bayesian false discovery rate for multiple testing. Appl Stat. 2007; 34: 1–9. 66. D. Y. Lin, Evaluating statistical significance in two-stage genomewide association studies. Am J Hum Genet. 2006; 78: 505–509. 67. Thomson, G. Significance levels in genome scans. Adv Genet. 2001; 42: 475–486. 68. J. Zhao, E. Boerwinkle, and M. Xiong, An entropy-based statistic for genomewide association studies. Am J Hum Genet. 2005; 77: 27–40. 69. S. Lin, A. Chakravarti, and D. J. Cutler, Exhaustive allelic transmission disequilibrium tests as a new approach to genomewide association studies. Nat Genet. 2004; 36: 1181–1188. 70. J. Marchini, P. Donnelly, and L. R. Cardon, Genome-wide strategies for detecting multiple loci that influence complex diseases. Nat Genet. 2005; 37: 413–417. 71. M. Eichelbaum, M. Ingelman-Sundberg, and W. E. Evans, Pharmacogenomics and individualized drug therapy. Annu Rev Med. 2006; 57: 119–137. 72. T. W. Gant and S. D. Zhang, In pursuit of effective toxicogenomics. Mutat Res. 2005; 575: 4–16. 73. I. Cascorbi, Genetic basis of toxic reactions to drugs and chemicals. Toxicol Lett. 2006; 162: 16–28. 74. R. H. Howland, Personalized drug therapy with pharmacogenetics—Part 1: pharmacokinetics. J Psychosoc Nurs Ment Health Serv. 2006; 44: 13–16. 75. R. H. Howland, Personalized drug therapy with pharmacogenetics—Part 2: pharmacodynamics. J Psychosoc Nurs Ment Health Serv. 2006; 44: 13–16. 76. C. M. Ulrich, K. Robien, and R. Sparks, Pharmacogenetics and folate metabolism—a promising direction. Pharmacogenomics. 2002; 3: 299–313. 77. C. M. Ulrich, K. Robien, and H. L. McLeod, Cancer pharmacogenetics: polymorphisms,

MULTISTAGE GENETIC ASSOCIATION STUDIES pathways and beyond. Nat Rev Cancer. 2003; 3: 912–920. 78. Z. Feng, R. Prentice, and S. Srivastava, Research issues and strategies for genomic and proteomic biomarker discovery and validation: a statistical perspective. Pharmacogenomics. 2004; 5: 709–719. 79. U. Mahlknecht and S. Voelter-Mahlknecht, Pharmacogenomics: questions and concerns. Curr Med Res Opin. 2005; 21: 1041–1048.

9

80. A. C. Need, A. G. Motulsky, and D. B. Goldstein, Priorities and standards in pharmacogenetic research. Nat Genet. 2005; 37: 671–681.

NATIONAL CANCER INSTITUTE (NCI)

of disparities among underserved groups and gaps in quality cancer care, helping to translate research results into better health for groups at high risk for cancer, including cancer survivors and the aging population. As the leader of the National Cancer Program, the NCI provides vision and leadership to the global cancer community, conducting and supporting international research, training, health information dissemination, and other programs. Timely communication of NCI scientific findings help people make better health choices and advise physicians about treatment options that are more targeted and less toxic.

The National Cancer Institute is the world’s largest organization solely dedicated to cancer research. The NCI supports researchers at universities and hospitals across the United States and at NCI Designated Cancer Centers, a network of facilities that not only study cancer in laboratories but also conduct research on the best ways to rapidly bring the fruits of scientific discovery to cancer patients. In the NCI’s own laboratories, almost 5000 principal investigators, from basic scientists to clinical researchers, conduct earliest phase cancer clinical investigations of new agents and drugs. Recent advances in bioinformatics and the related explosion of technology for genomics and proteomics research are dramatically accelerating the rate for processing large amounts of information for cancer screening and diagnosis. The largest collaborative research activity is the Clinical Trials Program for testing interventions for preventing cancer, diagnostic tools, and cancer treatments, allowing access as early as possible to all who can benefit. The NCI supports over 1300 clinical trials a year, assisting more than 200,000 patients. The NCI’s scientists also work collaboratively with extramural researchers to accelerate the development of state-of-the-art techniques and technologies. In addition to direct research funding, the NCI offers U.S. cancer scientists a variety of useful research tools and services, including tissue samples, statistics on cancer incidence and mortality, bioinformatic tools for analyzing data, databases of genetic information, and resources through NCI-supported Cancer Centers, Centers of Research Excellence, and the Mouse Models of Human Cancer Consortium. NCI researchers are also seeking the causes This article was modified from the website of the National Institutes of Health (http://www.nih.gov /about/almanac/organization/NCI.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL CENTER FOR TOXICOLOGICAL RESEARCH (NCTR)

The NCTR’s research program involves basic and applied research specifically designed to define biological mechanisms of action underlying the toxicity of products regulated by the FDA. This research is aimed at understanding critical biological events in the expression of toxicity and at developing methods to improve assessment of human exposure, susceptibility, and risk. The NCTR conducts research through eight divisions. The NCTR research divisions study biochemical and molecular markers of cancer, nutritional modulation of risk and toxicity, developmental toxicity, neurotoxicity, quantitative risk assessment, transgenics, applied and environmental microbiology, and solid-state toxicity. Each division works with the others to support the FDA’s mission to bring safe and efficacious products to the market rapidly and to reduce the risk of adverse health effects from products on the market.

The National Center for Toxicological Research (NCTR) conducts peer-reviewed scientific research and provides expert technical advice and training that enables the U.S. Food and Drug Administration (FDA) to make sound, science-based regulatory decisions. The research is focused toward understanding critical biological events in the expression of toxicity and toward developing methods and incorporating new technologies to improve the assessment of human exposure, susceptibility, and risk through the characterization of present models and the development of new models. The aim of the program is to: • Conduct

peer-reviewed scientific research that provides the basis for FDA to make sound, science-based regulatory decisions, and to promote U.S. health through the FDA’s core activities of premarket review and postmarket surveillance.

• Conduct fundamental and applied re-

search aimed at understanding critical biological events and to determine how people are affected adversely by exposure to products regulated by FDA. • Develop methods to measure human

exposure to products that have been adulterated or to assess the effectiveness and/or the safety of a product. • Provide the scientific findings used by

the FDA product centers for premarket application review and produce safety assurance to the scientific community for the betterment of public health. This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/nctr/overview/mission.htm and http://www.fda.gov/oc/oms/ofm/accounting/cfo /2002/NCTR.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL COOPERATIVE GALLSTONE STUDY

consisting of a standard X-ray of the abdominal region after injestion of iodine that is absorbed and secreted into bile to allow visualization of the gallbladder and its contents on the X-ray. Gallstones formed from pure cholesterol tend to be buoyant and seem to be ‘‘floating’’ on the X-ray. Gallstones containing calcium are termed ‘‘calcified.’’ Gallstones can be benign for a period of years, but they may also cause periodic episodes of pain (biliary colic) caused by bile duct obstruction that occurs when a gallstone has either escaped from the gallbladder or is formed in a bile duct. Obstruction in the cystic duct that connects the gallbladder to the larger common duct leads to debilitating episodes of cholecystitis or gallbladder inflamation that often requires emergency surgery. Gallstones can also promote bacterial infection of the bile in the gallbladder or bile ducts (cholangitis) requiring emergency therapy and when untreated may lead to sepsis. In 1970, the only available treatment for gallstone disease was radical cholecystectomy or surgical removal. Unlike the laparoscopic cholecystectomy practiced today, the radical procedure required an extensive hospital stay and a period of rehabilitation. Several different bile acids are secreted into bile in humans, with the primary bile acid being cholic acid and its derivatives. A small preliminary study showed that feeding chenodoxycholic acid (chenodiol or just cheno, for short) daily via a pill could restore cholesterol solubility in bile. It was hypothesized that prolonged treatment of cholesterol gallstones with chenodiol would desaturate cholesterol in bile and would literally cause the gallstones to dissolve. However, chenodiol, then as now, could not be synthesized from raw chemicals and must be extracted from the bile of animals at slaughter; then it must be purified into crystals, formulated, and then encapsulated. There was no industrial sponsor for the compound. Today chenodiol would be characterized as an ‘‘orphan’’ drug. Thus, the National Cooperative Gallstone Study was organized by the NIAMDD to conduct various studies leading up to a full-scale study of the

John M. Lachin The George Washington University Washington, DC

The National Cooperative Gallstone Study (NCGS) was conducted from 1973 to 1984 with funding from the National Institute of Arthritis, Metabolism and Digestive Diseases (NIAMDD), now the National Institute of Diabetes, Digestive and Kidney Diseases (NIDDK). The ultimate objective was to conduct a Phase III clinical trial of the effectiveness and safety of exogenous bile acid supplementation for treatment of cholesterol gallstones. The design and methods are presented in Reference 1 and the principal results in Reference 2. The function of the gallbladder is to serve as a reservoir for the storage of bile that is secreted by the liver into the gallbladder. In response to a meal, especially a fatty meal, the gallbladder contracts and deposits the bile into the intestine to assist with digestion. Bile consists of bile acids and lipids, among other elements. Hepatic excretion into bile is also one of the principal means by which lipids are excreted. One of the earliest lipidlowering drugs, cholestyramine, is a bile acid sequestrant that promotes clearance of lipids from circulating blood through secretion into bile. Bile is an aqueous medium, and bile acids and phospholipids (lecithin) maintain cholesterol and other lipids in solution. However, if bile becomes super-saturated with lipids, which means that the concentration of bile acids is inadequate to keep the available cholesterol in solution, then the cholesterol begins to precipitate, forming cholesterol gallstones (cholelithiasis). A minority of gallstones may also contain other precipitants, such as calcium, bilirubinate, or other pigments. Today, gallstones are diagnosed by an ultrasound examination. At the time that the NCGS was conducted, gallstones were diagnosed using an oral cholecyctogram (OCG)

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

NATIONAL COOPERATIVE GALLSTONE STUDY

effectiveness and safety of bile acid supplementation with chenodiol for treatment of gallstones. 1

THE NCGS ORGANIZATION

The NCGS was initiated by the NIAMDD through issuance of a Request For Proposals (RFP). Through an open competition, the Cedars-Sinai Medical Center of Los Angeles, CA, was named the Coordinating Center with Leslie Schoenfeld as Director and Jay Marks as Deputy Director (among others), both gastroenterologists, and Gloria Croke as Project Coordinator. Cedars-Sinai contracted with The Biostatistics Center of George Washington University (GWU) to serve as the Biostatistical Unit (BSU) of the Coordinating Center under the codirection of Jerome Cornfield and Larry Shaw. Cedars-Sinai and GWU jointly submitted the winning application. After Cornfield’s death, John Lachin became codirector. Supplemental RFPs issued by CedarsSinai led to the appointment of other central units, including a Central Radiology Unit, Central Serum Laboratory, Central Bile Laboratory, and a Central Morphology and Electron Microscopy Laboratory. 2

2.2 The Hepatic Morphology Study Because of the potential for hepatotoxicity, the FDA mandated that the first 100 subjects treated with chenodiol should undergo liver biopsy before and during treatment for up to two years. Since the NCGS planned to employ a low and a high dose of chenodiol (375 and 750 mg/day, respectively), an initial study of 128 subjects randomly allocated to these doses was conducted in which all subjects had a liver biopsy at baseline and again at 9 and 24 months of treatment. All biopsies were independently assessed using light microscopy by two pathologists at different institutions using a common protocol for the assessment of morphologic abnormalities (5). The study included masked duplicate readings to assess interpathologist and intrapathologist reliability of the detailed morphologic assessments. In addition, biopsies were also read by electron microscopy by one morphologist (6). The study did not include an untreated control group. However, both the light and electron microscopic evaluations showed an increase in the prevalence of lesions over time that were consistent with intrahepatic cholestasis and were unlikely to be a reflection of the natural history of gallstone disease.

STUDIES

2.1 Animal Toxicology

2.3 The Full-Scale Study

Since the objective was to conduct a largescale Phase III study using chenodiol, the U.S. Food and Drug Administration (FDA) required that various animal toxicology studies be conducted. The program of toxicology studies, including studies in monkeys, was organized by the Pharmacology Toxicology Committee appointed by the Coordinating Center. These studies showed that chenodiol was associated with liver enzyme increases, and resulted in hepatic injury in rhesus monkeys (3). Subsequently, however, it was shown that the injury in the monkey was attributed to biotransformation of chenodiol to lithocholic acid and that humans were protected from such injury through sulfation of the lithocholic acid that neutralizes its toxicity (4).

When the morphology (biopsy) study was fully enrolled, the full-scale study was launched (1). A total of 916 subjects were enrolled in 10 clinical centers in the United States and randomly assigned to receive either placebo (n = 305), the low dose of 375 mg/day (n = 360), or the high dose of 750 mg/day (n = 305), double-masked. Subjects had predominantly cholesterol gallstones present on a baseline OCG and were generally in good health. Subjects were to be treated for two years with frequent monitoring of symptoms, frequent laboratory testing including lipids and liver enzymes, periodic follow-up OCGs, and periodic collection of gallbladder bile through duodenal intubation.

NATIONAL COOPERATIVE GALLSTONE STUDY

The results (2) showed a dose-dependent effect on complete confirmed gallstone dissolution, the primary outcome; the cumulative incidence over two years being 13.5% with the high dose, 5.2% with the low dose, and 0.8% with placebo. The cumulative incidence of partial (or complete) dissolution by at least 50% occurred in 40.8%, 23.6%, and 11% in the three groups, respectively. Although statistically significant, these modest effects were of marginal clinical significance. Clinically significant hepatotoxicity occurred in 3% of those treated with the high dose and 0.4% of those treated with the low dose or placebo. Gallstone dissolution with treatment occurred more frequently among women, those with lower body weight, those with small or floating gallstones, and those with higher cholesterol levels at baseline. There were no differences among the treatment groups in the incidence of biliary symptoms (pain, cholangitis, cholecyctitis) or the need for surgery (cholecystectomy). The investigators concluded that highdose chenodiol treatment for up to two years was an ‘‘appropriate therapy for dissolution of gallstones in selected patients who are informed of the risks and benefits’’ (2). The investigators’ evaluation of the study is presented in Reference 7. 2.4 The Recurrence Study Patients who achieved complete dissolution in the main study were then offered the opportunity to receive additional doublemasked therapy with either the low dose or placebo. The cumulative incidence of gallstone recurrence was 27% after up to 3.5 years of treatment with no difference between treatment groups (8). 2.5 Open-Label Treatment The protocol and informed consent stated that subjects randomly assigned to placebo would be offered the opportunity to receive treatment with chenodiol at the conclusion of the study. The final phase consisted of openlabel treatment for up to two years of those subjects originally treated with placebo who opted to receive active therapy.

3

3 MAJOR ISSUES 3.1 Clinical Centers The initial complement of 10 clinical centers was selected after an open competition (9). Early in the study it was clear that two of the clinical centers could not meet the recruitment targets and were promptly replaced. Recruitment, however, proved difficult for all clinical centers, and aggressive steps were taken to achieve the recruitment goals (10). 3.2 The Biospy Study Prior to initiation of the full-scale NCGS, the FDA required that a preliminary Phase II study be conducted with liver biopsies before and after treatment with chenodiol. The NCGS investigators strongly objected to the implementation of this recommendation as part of the main study, in part because a needle liver biopsy was considered to have substantial risk and would be unethical to administer to patients treated with placebo, and in part because it would not be scientifically possible to differentiate the effects of therapy from the natural history of gallstone disease progression in the absence of an untreated (and biopsied) control group. Substantial negotiations among the NIAMDD, the NCGS investigators, and the FDA led to the compromise that the major study could be launched after the enrollment of the uncontrolled biopsy study was completed (9). 3.3 Costs Over the 8-year period from 1973 to 1981 (not including the open-label follow-up), the total cost was $11.2 million. Of this cost, 43% was allocated to the clinical centers, 13% to the Coordinating Center, 19% to the Biostatistical Unit, 10% to the central laboratories and units, 5% to ancillary studies, 4% to animal toxicology studies, 3% to the purchase of drug, and 3% to committee expenses (7). 4 BIOSTATISTICAL CONSIDERATIONS 4.1 Randomization The NCGS randomization sequence was stratified by clinical center and was implemented using a technique (1) developed by Larry Shaw, the BSU co-PI with Cornfield,

4

NATIONAL COOPERATIVE GALLSTONE STUDY

that later became known as ‘‘big-stick’’ randomization (11). This technique is a variation on the biased-coin design (12) but less restrictive in that complete randomization (a coin toss) is employed except where the imbalance between the numbers assigned to each group reaches an unacceptable level. In the NCGS, this level was defined as a deviation from perfect balance assigned to any one group that was greater than one standard deviation under the corresponding binomial (1). For example, for equal allocations to two groups with probability 12 , after n allo√ cations, the standard deviation is 12 n. On the next allocation, one tosses a coin if the number assigned to both groups falls within √ 1 2 (n ± n); otherwise, the next allocation is assigned to whichever group has the smaller number. 4.2 Sample Size Evaluation The sample size for the biopsy study was dictated by the FDA that requested sequential biopsy on 100 subjects. To ensure that close to this number received 24 months of treatment and were biopsied, the number enrolled was increased to 128. Since there was no untreated control group, the power of the study was then described in terms of the change from baseline to follow-up in the proportion of subjects with an abnormality (1). The sample size for the three group main study was based on the test for the difference in proportions achieving gallstone dissolution using pairwise tests for each of the two dose groups versus control (1). The power of the study was also assessed in terms of the ability to detect a treatment group by covariate interaction or heterogeneity of the treatment group effect between two strata. For the comparison of means of K active treatment groups versus a control, the Dunnett (13) procedure is widely used for which the most powerful comparisons are provided √ using the ‘‘square root rule’’ in which K subjects are assigned to the control group for every one subject assigned to a treatment group. For the case of two active treatments, as in the NCGS, this would yield alloca√ tion ratios of 2 : 1 : 1 or 1.41:1:1 for control and the two active treatment groups, respectively. However, it was not known whether

these allocations were also optimal for comparisons of proportions among K active treatment groups versus control. Lachin (14) explored the power of such tests and determined that there was no uniformly optimal allocation ratio for tests for proportions. Rather the optimal ratio depended on the nature of the underlying null and alternative hypotheses, and on average, equal allocations seemed to be robust. 4.3 Outcome Evaluation The primary outcome of the study was the presence or absence of gallstones on an OCG and, if present, the number, total volume, and total surface area. The Central Radiology Unit developed a novel technique for providing these measurements. The gallbladder and gallstones were visualized on a standard flat plate X-ray of the abdomen taken with the patient recumbent. Thus, the actual distance of the gallbladder and stones from the table surface (and film plane) varied as a function of patient girth. To allow for the resulting magnification differential, each clinical center was provided with a half-inch ball bearing that was taped to the patient’s side at the level of the gallbladder. The resulting image of the ball bearing allowed the area of the gallbladder and gallstones on the X-ray to be calibrated to that of the image of the ball bearing (15). A computer program was then written to allow the reader to identify the ‘‘edge’’ of the image of the gallbladder and individual gallstones using a computer monitor, and to then compute areas. Biliary bile acids were measured in a Central Bile Laboratory and other biochemistries measured in a Central Serum Laboratory. 4.4 External Quality Assurance The reproducibility and reliability of all outcome assessments was monitored to assure high-quality data. For the Central Radiology Unit, individual X-rays were randomly selected for masked duplicate re-reading within the unit (2). For the central laboratories, split duplicate specimens were collected in the clinical centers and then shipped to the laboratories for a masked duplicate assessment. The detailed procedures and descriptions of the statistical calculations of reliability are presented in Reference 16.

NATIONAL COOPERATIVE GALLSTONE STUDY

4.5 External Monitoring of the BSU Finally, an external monitoring committee was appointed to site visit the Biostatistical Unit (BSU) periodically, and over a period of years, it reviewed and critiqued virtually all functions of the unit. Details are presented in Reference 17. REFERENCES 1. J. M. Lachin, J. Marks, and L. J. Schoenfield, the Protocol Committee and the NCGS Group. Design and methodological considerations in the National Cooperative Gallstone Study: A multi-center clinical trial. Controlled Clinical Trials, 1981; 2: 177–230. 2. L. J. Schoenfield, J. M. Lachin, the Steering Committee and the NCGS Group. National Cooperative Gallstone Study: A controlled trial of the efficacy and safety of chenodeoxycholic acid for dissolution of gallstones. Ann. Internal Med. 1981; 95: 257–282. 3. R. Okun, L. I. Goldstein, G. A. Van Gelder, EI. Goldenthal, F. X. Wazeter, R. G. Giel, National Cooperative Gallstone Study: nonprimate toxicology of chenodeoxycholic acid. J. Toxicol. Environ. Health, 1982; 9: 727–741. 4. J. W. Marks, S. O. Sue, B. J. Pearlman, G. G. Banorris, P. Varady, J. M. Lachin, and L. J. Schoenfield. Sulfation of lithocholate as a possible modifier of chenodeoxycholic acidinduced elevations of serum transaminase in patients with gallstones. J. Clin. Invest. 1981; 68: 1190–1196. 5. R. L. Fisher, D. W. Anderson, J. L. Boyer, K. Ishak, G. Klatskin, J. M. Lachin, and M. J. Phillips, and the Steering Committee for the NCGS Group. A prospective morphologic evaluation of hepatic toxicity of chenodeoxycholic acid in patients with cholelithiasis: The National Cooperative Gallstone Study. Hepatology 1982; 2: 187–201. 6. M. J. Phillips, R. L. Fisher, D. W. Anderson, S. Lan, J. M. Lachin, J. L. Boyer, and the Steering Committee for the NCGS Group. Ultrastructural evidence of intrahepatic cholestasis before and after chenodeoxycholic acid (CDCA) therapy in patients with cholelithiasis: The National Cooperative Gallstone Study (NCGS). Hepatology 1983; 3: 209–220. 7. L. J. Schoenfield, S. M. Grundy, A. F. Hofmann, J. M. Lachin, J. L. Thistle, and M. P. Tyor, for the NCGS. The National Cooperative Gallstone Study viewed by its investigators. Gastroenterology 1983; 84: 644–648.

5

8. J. W. Marks, S. P. Lan, The Steering Committee, and The National Cooperative Gallstone Study Group. Low dose chenodiol to prevent gallstone recurrence after dissolution therapy. Ann. Intern. Med. 1984; 100: 376–381. 9. J. Marks, G. Croke, N. Gochman, A. F. Hofmann, J. M. Lachin, L. J. Schoenfield, and M. P. Tyor, and the NCGS Group. Major issues in the organization and implementation of the National Cooperative Gallstone Study (NCGS). Controlled Clinical Trials 1984; 5: 1–12. 10. G. Croke. Recruitment for the the National Cooperative Gallstone Study. Clin. Pharm. Ther, 1979; 25: 691–674. 11. J. F. Soares and C. F Wu. Some restricted randomization rules in sequential designs. Communications in Statistics Theory and Methods 1982; 12: 2017–2034. 12. B. Efron. Forcing a sequential experiment to be balanced. Biometrika 1971; 58: 403–417. 13. C. W. Dunnett. A multiple comparison procedure for comparing several treatments with a control. J. Am Stat Assoc. 1955; 50: 1096–1121. 14. J. M. Lachin. Sample size determinations for r x c comparative trials. Biometrics, 1977; 33: 315–324. 15. E. C. Lasser, J. R. Amberg, N. A. Baily, P. Varady, J. M. Lachin, R. Okun, and L. J. Schoenfield. Validation of a computer-assisted method for estimating the number and volume of gallstones visualized by cholecystography. Invest. Radiol 1981; 16: 342–347. 16. R. L. Habig, P. Thomas, K. Lippel, D. Anderson, and J. M. Lachin. Central laboratory quality control in the National Cooperative Gallstone Study. Controlled Clinical Trials 1983: 4: 101–123. 17. P. L. Canner, L. C. Gatewood, C. White, J. M. Lachin, and L. J. Schoenfield. External monitoring of a data coordinating center: Experience of the National Cooperative Gallstone Study. Controlled Clinical Trials 1987: 8: 1–11.

FURTHER READING J. W. Marks, G. G. Bonorris, A Chung, M. J. Coyne, R. Okun, J. M. Lachin, and L. J. Schoenfield. Feasibility of low-dose and intermittent chenodeoxycholic acid therapy of gallstones. Am. J. Digest. Dis. 1977; 22: 856–860. J. J. Albers, S. M. Grundy, P. A. Cleary, D. M. Small, J. M. Lachin, L. J. Schoenfield, and the

6

NATIONAL COOPERATIVE GALLSTONE STUDY

National Cooperative Gallstone Study Group. National Cooperative Gallstone Study. The effect of chenodeoxycholic acid on lipoproteins and apolipoproteins. Gastroenterology, 1982; 82: 638–646. A. F. Hofmann, S. M. Grundy, J. M. Lachin, S. P. Lan, et al. Pretreatment biliary lipid composition in white patients with radiolucent gallstones in the National Cooperative Gallstone Study. Gastroenterology 1982; 83: 738–752. A. F. Hofmann and J. M. Lachin. Biliary bile acid composition and cholesterol saturation. Gastroenterology 1983; 84: 1075–1077. J. M. Lachin, L. J. Schoenfield, and the National Cooperative Gallstone Study Group. Effects of dose relative to body weight in the National Cooperative Gallstone Study: A fixed-dose trial. Controlled Clinical Trials 1983; 4: 125–131. S. M. Grundy, S. Lan, J. Lachin, the Steering Committee and the National Cooperative Gallstone Study Group. The effects of chenodiol on biliary lipids and their association with gallstone dissolution in the National Cooperative Gallstone Study (NCGS). J. Clin. Invest. 1984; 73: 1156–1166. J. L. Thistle, P. A. Cleary, J. M. Lachin, M. P. Tyor, T. Hersh, The Steering Committee, and The National Cooperative Gallstone Study Group. The natural history of cholelithiasis: The National Cooperative Gallstone Study. Ann. Intern. Med. 1984; 101: 171–175.

F. Stellard, P. D. Klein, A. F. Hofmann, and J. M. Lachin. Mass spectrometry identification of biliary bile acids in bile from patients with gallstones before and during treatment with chenodeoxycholic acid. J. Lab. Clin. Med. 1985; 105: 504–513.

CROSS-REFERENCES Biased-coin Randomization Clinical Trial/Study Conduct Multiple Comparisons Orphan Drugs Phase III Trials Placebo-Controlled Trial Quality Assurance Quality Control Reproducibility Sample Size for Comparing Proportions (Superiority and Non-Inferiority)

NATIONAL EYE INSTITUTE (NEI) The U.S. National Eye Institute (NEI) conducts and supports research, training, health information dissemination, and other programs with respect to blinding eye diseases, visual disorders, mechanisms of visual function, preservation of sight, and the special health problems of individuals who are visually impaired or blind. Vision research is supported by the NEI through research grants and training awards made to scientists at more than 250 medical centers, hospitals, universities, and other institutions across the United States and around the world. The NEI also conducts laboratory and patient-oriented research at its own facilities located on the U.S. National Institutes of Health (NIH) campus in Bethesda, Maryland. Another part of the NEI mission is to conduct public and professional education programs that help prevent blindness and reduce visual impairment. To meet these objectives, the NEI has established the National Eye Health Education Program, a partnership of more than 60 professional, civic, and voluntary organizations and government agencies concerned with eye health. The program represents an extension of the NEI’s support of vision research, where results are disseminated to health professionals, patients, and the public.

This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NEI.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

• Coordinates relevant activities with

NATIONAL HEART, LUNG, AND BLOOD INSTITUTE (NHLBI)

other research institutes and all federal health programs in the covered areas, including the causes of stroke. • Conducts educational activities, including development and dissemination of materials for health professionals and the public in the covered areas, with emphasis on prevention. • Maintains continuing relationships with institutions and professional associations, and with international, national, state, and local officials as well as voluntary agencies and organizations working in the covered areas. • Oversees management of the Women’s Health Initiative study.

The National Heart, Lung, and Blood Institute (NHLBI): • Provides leadership for a national pro-

gram in diseases of the heart, blood vessels, lungs, and blood; sleep disorders; and blood resources. • Plans, conducts, fosters, and supports

an integrated and coordinated program of basic research, clinical investigations and trials, observational studies, and demonstration and education projects related to the causes, prevention, diagnosis, and treatment of heart, blood vessel, lung, and blood diseases, and sleep disorders conducted in its own laboratories and by scientific institutions and individuals supported by research grants and contracts. • Plans and directs research in development, trials, and evaluation of interventions and devices related to the prevention of diseases and disorders in the covered areas and the treatment and rehabilitation of patients suffering from such conditions.

1 DIVISION OF CARDIOVASCULAR DISEASES (DCVD) The Division of Cardiovascular Diseases (DCVD) promotes opportunities to translate promising scientific and technological advances from discovery through preclinical studies to networks and multisite clinical trials. It designs, conducts, supports, and oversees research on the causes and prevention and treatment of diseases and disorders such as atherothrombosis, coronary artery disease, myocardial infarction and ischemia, heart failure, arrhythmia, sudden cardiac death, adult and pediatric congenital heart disease, cardiovascular complications of diabetes and obesity, and hypertension.

• Conducts research on clinical use of

blood and all aspects of the management of blood resources. • Supports research training and career

development of new and established researchers in fundamental sciences and clinical disciplines to enable them to conduct basic and clinical research related to heart, blood vessel, lung, and blood diseases; sleep disorders; and blood resources through individual and institutional research training awards and career development awards.

2 DIVISION OF PREVENTION AND POPULATION SCIENCES (DPPS) The Division of Prevention and Population Sciences (DPPS) supports and provides leadership for population-based and clinic-based research on the causes, prevention, and clinical care of cardiovascular, lung, and blood diseases and sleep disorders. Research includes a broad array of epidemiologic studies to describe disease and risk factor patterns in populations and to identify risk factors for

This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NHLBI.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

NATIONAL HEART, LUNG, AND BLOOD INSTITUTE (NHLBI)

disease; clinical trials of interventions to prevent disease; studies of genetic, behavioral, sociocultural, and environmental influences on disease risk and outcomes; and studies of the application of prevention and treatment strategies to determine how to improve clinical care and public health. 3

DIVISION OF LUNG DISEASES (DLD)

The Division of Lung Diseases (DLD) supports basic research, clinical trials, national pulmonary centers, technological development, and application of research findings. Activities focus on understanding the structure and function of the respiratory system, increasing fundamental knowledge of mechanisms associated with pulmonary disorders, and applying new findings to evolving treatment strategies for patients.

NATIONAL HUMAN GENOME RESEARCH INSTITUTE (NHGRI) The National Human Genome Research Institute (NHGRI), which was originally established in 1989 as the National Center for Human Genome Research, became an institute of the U.S. National Institutes of Health (NIH) in 1997. The NHGRI led the NIH’s contribution to the International Human Genome Project, which successfully completed the sequencing of the 3 billion base pairs that make up the human genome in April 2003. The NHGRI mission has evolved over the years to encompass a broad range of studies aimed at understanding the structure and function of the human genome and its role in health and disease. To that end, the NHGRI supports the development of resources and technology that will accelerate genomeresearch and its application to human health as well as the study of the ethical, legal, and social implications of genome research. The NHGRI also supports the training of investigators and the dissemination of genome information to the public and to health professionals. The NHGRI is organized into three main divisions: the Office of the Director, which provides guidance to scientific programs and oversees general operations; the Division of Extramural Research, which supports and administers genomic research; and the Division of Intramural Research, which comprises the in-house genetics research laboratories. Research guidance and guidance related to NHGRI grants comes from the National Advisory Council for Human Genome Research, which meets three times a year. Members include representatives from health and science disciplines, public health, social sciences, and the general public. Portions of the council meetings are open to the public. This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NHGRI.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF ALLERGY AND INFECTIOUS DISEASE (NIAID)

clinical trials of hematopoietic stem cell transplantation for treating severe autoimmune disorders.

The National Institute of Allergy and Infectious Diseases (NIAID) conducts and supports research to study the causes of allergic, immunologic, and infectious diseases, and to develop better means of preventing, diagnosing, and treating these illnesses. The NIAID runs a pediatric allergy clinic at the U.S. National Institutes of Health (NIH) Clinical Center that serves as a focal point for translational research conducted in collaboration with NIAID intramural laboratories and clinical trials of novel therapies.

3

1

4

In collaboration with industry, academia, and other government agencies, the NIAID has established research programs to facilitate drug development, including capacity databases to screen compounds for their potential use as therapeutic agents, facilities to conduct preclinical testing of promising drugs, and clinical trials networks to evaluate the safety and efficacy of drugs and therapeutic strategies in humans.

GENETICS AND TRANSPLANTATION

The NIAID’s basic immunology and genetics research seeks to define the effects of gene expression on immune function and to determine the manner in which the products of gene expression control the immune response to foreign substances such as transplanted organs and cells. Research programs in genetics and transplantation include human leukocyte antigen (HLA) region genetics in immune-mediated diseases, the genomics of transplantation, and clinical trials in organ transplantation. 2

DRUG RESEARCH AND DEVELOPMENT

ANTIMICROBIAL RESISTANCE

The NIAID-supported clinical trials networks with capacity to assess new antimicrobials and vaccines relevant to drug-resistant infections include the Adult AIDS Clinical Trials Groups, the Bacteriology and Mycology Study Group, the Collaborative Antiviral Study Group, and Vaccine and Treatment Evaluation Units. 5 DIVISION OF MICROBIOLOGY AND INFECTIOUS DISEASES The Division of Microbiology and Infectious Diseases (DMID) supports extramural research to control and prevent diseases caused by virtually all human infectious agents except human immunodeficiency virus (HIV). The DMID supports a wide variety of projects spanning the spectrum from basic biomedical research through applied research to clinical trials to test the safety and efficacy of new disease prevention strategies.

IMMUNE-MEDIATED DISEASES

The NIAID conducts and supports basic, preclinical, and clinical research on immunemediated diseases, including asthma and allergic diseases, autoimmune disorders, primary immunodeficiency diseases, and the rejection of transplanted organs, tissues, and cells. Efforts are underway to evaluate the safety and efficacy of tolerance induction strategies for treating immune-mediated diseases as well as to assess the efficacy through This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NIAID.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF ARTHRITIS AND MUSCULOSKELETAL AND SKIN DISEASES (NIAMS) The U.S. National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS) was established in 1986. The mission of NIAMS is to support research into the causes, treatment, and prevention of arthritis and musculoskeletal and skin diseases; the training of basic and clinical scientists to carry out this research; and the dissemination of information on research progress in these diseases. The Institute also conducts and supports basic research on the normal structure and function of joints, muscles, bones, and skin. Basic research involves a wide variety of scientific disciplines, including immunology, genetics, molecular biology, structural biology, biochemistry, physiology, virology, and pharmacology. Clinical research includes rheumatology, orthopaedics, dermatology, metabolic bone diseases, heritable disorders of bone and cartilage, inherited and inflammatory muscle diseases, and sports and rehabilitation medicine. The Institute’s Genetics and Clinical Studies Program supports genetic studies in rheumatic diseases, both in animal models and in humans; clinical trials and complex clinical studies, including epidemiology, outcomes, and prevention of rheumatic and related diseases; and research on Lyme disease and infection-related arthritis.

This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NIAMS.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF BIOMEDICAL IMAGING AND BIOENGINEERING (NIBIB) The mission of the U.S. National Institute of Biomedical Imaging and Bioengineering (NIBIB) is to improve health by leading the development of and accelerating the application of biomedical technologies. The Institute integrates the physical and engineering sciences with the life sciences to advance basic research and medical care: • Supporting research and development

• • • •

•

•

of new biomedical imaging and bioengineering techniques and devices to fundamentally improve the detection, treatment, and prevention of disease. Enhancing existing imaging and bioengineering modalities. Supporting related research in the physical and mathematical sciences. Encouraging research and development in multidisciplinary areas. Supporting studies to assess the effectiveness and outcomes of new biologics, materials, processes, devices, and procedures. Developing technologies for early disease detection and assessment of health status. Developing advanced imaging and engineering techniques for conducting biomedical research at multiple scales.

This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NIBIB.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

• Human growth and development is a

NATIONAL INSTITUTE OF CHILD HEALTH AND HUMAN DEVELOPMENT (NICHD)

lifelong process that has many phases and functions. Much of the research in this area focuses on cellular, molecular, and developmental biology to build understanding of the mechanisms and interactions that guide a single fertilized egg through its development into a multicellular, highly organized adult organism. • Learning about the reproductive health of women and men and educating people about reproductive practices is important to both individuals and societies. Institute-supported basic, clinical, and epidemiologic research in the reproductive sciences seeks to develop knowledge that enables women and men to overcome problems of infertility and to regulate their fertility in ways that are safe, effective, and acceptable for various population groups. Institute-sponsored behavioral and social science research in the population field strives to understand the causes and consequences of reproductive behavior and population change. • Developing medical rehabilitation interventions can improve the health and the well-being of people with disabilities. Research in medical rehabilitation seeks to develop improved techniques and technologies for the rehabilitation of individuals with physical disabilities resulting from diseases, disorders, injuries, or birth defects.

The mission of the National Institute of Child Health and Human Development (NICHD) is to ensure that every person is born healthy and wanted; that women suffer no harmful effects from the reproductive process; and that all children have the chance to fulfill their potential to live healthy and productive lives, free from disease or disability; and to ensure the health, productivity, independence, and well-being of all people through optimal rehabilitation. In pursuit of this mission, the NICHD conducts and supports laboratory research, clinical trials, and epidemiologic studies that explore health processes; examines the impact of disabilities, diseases, and defects on the lives of individuals; and sponsors training programs for scientists, doctors, and researchers to ensure that NICHD research can continue. NICHD research programs incorporate the following concepts: • Events that happen before and through-

out pregnancy as well as during childhood have a great impact on the health and the well-being of adults. The Institute supports and conducts research to advance knowledge of pregnancy, fetal development, and birth for developing strategies that prevent maternal, infant, and childhood mortality and morbidity; to identify and promote the prerequisites of optimal physical, mental, and behavioral growth and development through infancy, childhood, and adolescence; and to contribute to the prevention and amelioration of mental retardation and developmental disabilities.

The Institute also supports research training across all its programs, with the intent of adding to the cadre of trained professionals who are available to conduct research in areas of critical public health concern. In addition, an overarching responsibility of the NICHD is to disseminate information that emanates from Institute research programs to researchers, practitioners, other healthcare professionals, and the public.

This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NICHD.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF DENTAL AND CRANIOFACIAL RESEARCH (NIDCR)

The center has six components: Clinical Trials Program, Dental Practice-Based Research Networks, Epidemiology Research Program, Health Disparities Research Program, Health Promotion and CommunityBased Research Program, and the Basic and Applied Behavioral/Social Science Research Program.

The mission of the National Institute of Dental and Craniofacial Research (NIDCR) is to improve oral, dental, and craniofacial health through research, research training, and the dissemination of health information. We accomplish our mission by:

2 DIVISION OF EXTRAMURAL ACTIVITIES, SCIENTIFIC REVIEW BRANCH

• Performing and supporting basic and

The Scientific Review Branch of the Division of Extramural Activities coordinates the initial scientific peer review of applications for the following mechanisms of support: center research grants, program project grants, small research grants, research conference grants, institutional training grants, shortterm training and fellowship grants, Physician Scientist Awards for Dentists, Dentist Scientist Awards, requests for applications issued by the NIDCR, certain investigatorinitiated clinical trials, cooperative agreements, and all proposals for research and development contracts.

clinical research. • Conducting and funding research training and career development programs to ensure an adequate number of talented, well-prepared, and diverse investigators. • Coordinating and assisting relevant research and research-related activities among all sectors of the research community. • Promoting the timely transfer of knowledge gained from research and its implications for health to the public, health professionals, researchers, and policymakers. 1

CENTER FOR CLINICAL RESEARCH

The Center for Clinical Research (CCR) supports and conducts patient-oriented and population-based research, including clinical trials, practice-based networks, epidemiology, and health-disparity research in all areas of program interest to the NIDCR. Providing statistical support for Institute centers and divisions, the CCR develops and supports programs to foster diversity in the scientific workforce as well as clinical research activities aimed at the health of vulnerable and special needs populations. This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NIDCR.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF DIABETES AND DIGESTIVE AND KIDNEY DISEASES (NIDDK)

of surrogate markers for use in clinical trials for the prevention or treatment of type 2 diabetes, cellular therapies for the treatment of type 2 diabetes, and improving the care of patients with type 2 diabetes. The Type 1 Diabetes Clinical Trials Program supports large, multicenter clinical trials conducted under cooperative agreements or contracts. For example, the Diabetes Prevention Trial Type 1 (DPT-1) was aimed at determining whether it was possible to prevent or delay the onset of type 1 diabetes in individuals who are at immunologic, genetic, and/or metabolic risk. The program also supports future clinical trials that are part of the Type 1 Diabetes TrialNet, which are intervention studies to prevent or slow the progress of type 1 diabetes, and natural history and genetics studies in populations screened for or enrolled in these studies. The program also supports the Epidemiology of Diabetes Interventions and Complications (EDIC) study, an epidemiologic follow-up study of the patients previously enrolled in the Diabetes Control and Complications Trial (DCCT). The Type 2 Diabetes Clinical Trials Program supports large, multicenter clinical trials conducted under cooperative agreements or contracts. For example, the Diabetes Prevention Program (DPP) focuses on testing lifestyle and pharmacologic intervention strategies in individuals at genetic and metabolic risk for developing type 2 diabetes to prevent or delay the onset of this disease. The Gene Therapy and Cystic Fibrosis Centers Program supports three types of centers: Gene Therapy Centers (P30), Cystic Fibrosis Research Centers (P30), and Specialized Centers for Cystic Fibrosis Research (P50). Gene Therapy Centers provide shared resources to a group of investigators to facilitate development of gene therapy techniques and to foster multidisciplinary collaboration in the development of clinical trials for the treatment of cystic fibrosis and other genetic metabolic diseases. Cystic Fibrosis Research Centers (P30) and Specialized Centers for Cystic Fibrosis Research (P50) provide resources and support research on many

The National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK) conducts and supports research on many of the most serious diseases affecting public health. The institute supports much of the clinical research on the diseases of internal medicine and related subspecialty fields as well as many basic science disciplines. The Institute’s Division of Intramural Research encompasses the broad spectrum of metabolic diseases such as diabetes, obesity, inborn errors of metabolism, endocrine disorders, mineral metabolism, digestive and liver diseases, nutrition, urology and renal disease, and hematology. Basic research studies include: biochemistry, biophysics, nutrition, pathology, histochemistry, bioorganic chemistry, physical chemistry, chemical and molecular biology, and pharmacology. The NIDDK extramural research is organized into four divisions: Diabetes, Endocrinology and Metabolic Diseases; Digestive Diseases and Nutrition; Kidney, Urologic and Hematologic Diseases; and Extramural Activities. The Institute supports basic and clinical research through investigator-initiated grants, program project and center grants, and career development and training awards. The Institute also supports research and development projects and large-scale clinical trials through contracts. The Clinical Research in Type 2 Diabetes Program focuses on patient-oriented research (i.e., clinical studies and small clinical trials) related to pharmacologic interventions and/or lifestyle interventions to prevent or treat type 2 diabetes, including studies relevant to new drug development, development This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NIDDK.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

NATIONAL INSTITUTE OF DIABETES AND DIGESTIVE AND KIDNEY DISEASES (NIDDK)

aspects of the pathogenesis and treatment of cystic fibrosis. The Clinical Trials in Digestive Diseases Program supports patient-oriented clinical research focusing on digestive diseases. Small clinical studies (pilot), planning grants, or phase III clinical trials may be appropriate to this program. The small clinical studies focus on research that is innovative and/or potentially of high impact that will lead to full-scale clinical trials. Phase III clinical trials usually are multicenter and involve several hundred participants who are randomized to two or more treatments, one of which is usually a placebo. The aim of the trial is to provide evidence for support of, or a change in, health policy or standard of care. The interventions/treatments may include pharmacologic, nonpharmacologic, and behavioral interventions given for disease prevention, prophylaxis, diagnosis, or therapy. The Clinical Trials in Liver Disease Program supports patient-oriented clinical research in liver diseases to evaluate one or more experimental intervention(s) in comparison with a standard treatment and/or placebo control among comparable groups of patients. Experimental interventions may include pharmacologic, nonpharmacologic, and behavioral interventions given for disease prevention, prophylaxis, diagnosis, or therapy. Either pilot studies or phase III trials may be appropriate. The Obesity Prevention and Treatment Program supports research that focuses on the prevention and treatment of obesity and the overweight condition in humans. Prevention includes primary and secondary approaches to prevent the initial development of obesity through control of inappropriate weight gain and increases in body fat, weight maintenance among those at risk of becoming overweight, and prevention of weight regain once weight loss has been achieved. Treatment includes clinical trials evaluating approaches to lose weight or maintain weight loss, including, but not limited to, behavioral, pharmacologic, and surgical approaches. Look AHEAD: Action for Health in Diabetes is a clinical trial recruiting 5000 obese individuals with type 2 diabetes into an 11.5 year study that investigates the long-term

health consequences of interventions designed to achieve and sustain weight loss. The primary outcome of the trial is cardiovascular events: heart attack, stroke, and cardiovascular death. The Clinical Trials in Nutrition Program supports clinical research on nutrition and eating disorders, focusing on metabolic and/or physiologic mechanisms. Small clinical studies (pilot), planning grants, or phase III clinical trials may be appropriate to this program. The small clinical studies focus on research that is innovative and/or potentially of high impact that will lead to full-scale clinical trials. Phase III clinical trials usually are multicenter and involve several hundred participants who are randomized to two or more treatments, one of which is a placebo.

NATIONAL INSTITUTE OF ENVIRONMENTAL HEALTH SCIENCE (NIEHS) The mission of the U.S. National Institute of Environmental Health Science (NIEHS) is to reduce the burden of human illness and disability by understanding how the environment influences the development and progression of human disease. To have the greatest impact on preventing disease and improving human health, the NIEHS focuses on basic science, disease-oriented research, global environmental health, and multidisciplinary training for researchers: • Funding

extramural research and training via grants and contracts to scientists, environmental health professionals, and other groups worldwide. • Conducting intramural research at the NIEHS facility and in partnership with scientists at universities and hospitals. • Providing toxicological testing and test validation through the National Toxicology Program. • Maintaining outreach and communications programs that provide reliable health information to the public and scientific resources to researchers.

This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NIEHS.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF GENERAL MEDICAL SCIENCES (NIGMS)

Certain NIGMS training programs address areas in which there is a particularly serious need for well-prepared scientists. One of these, the Medical Scientist Training Program, provides investigators who can bridge the gap between basic and clinical research by supporting research training leading to the combined M.D.–Ph.D. degree. Other programs train scientists to conduct research in the rapidly growing field of biotechnology and at the interface between the fields of chemistry and biology. The NIGMS also has a Pharmacology Research Associate Program, in which postdoctoral scientists receive training in the NIH or Food and Drug Administration (FDA) laboratories and clinics. In recent years, the NIGMS has launched initiatives in such cutting-edge areas as structural genomics (the Protein Structure Initiative), pharmacogenetics, collaborative research initiatives (which includes ‘‘glue grants’’), and computational modeling of infectious disease outbreaks. The NIGMS is also participating in the NIH Roadmap for Medical Research, a series of far-reaching initiatives that seek to transform the nation’s biomedical research capabilities and speed the movement of research discoveries from the bench to the bedside. Each year, NIGMS-supported scientists make major advances in understanding fundamental life processes. In the course of answering basic research questions, these investigators also increase our knowledge about the mechanisms and pathways involved in certain diseases. Other grantees develop important new tools and techniques, many of which have medical applications. In recognition of the significance of their work, a number of NIGMS grantees have received the Nobel Prize and other high scientific honors.

The National Institute of General Medical Sciences (NIGMS) was established in 1962, and in fiscal year 2006, its budget was $1.9 billion. The vast majority of this money funds grants to scientists at universities, medical schools, hospitals, and research institutions throughout the country. At any given time, the NIGMS supports over 4400 research grants— about 10% of the grants that are funded by the U.S. National Institutes of Health as a whole—and the NIGMS also supports approximately 25% of the trainees who receive assistance from the NIH. Primarily, the NIGMS supports basic biomedical research that lays the foundation for advances in disease diagnosis, treatment, and prevention. The NIGMS is organized into divisions and a center that support research and research training in basic biomedical science fields. One division has the specific mission of increasing the number of underrepresented minority biomedical and behavioral scientists. The Institute places great emphasis on the support of individual, investigatorinitiated research grants. It funds a limited number of research center grants in selected fields, including structural genomics, trauma and burn research, and the pharmacologic sciences. In addition, NIGMS funds several important resources for basic scientists. The Institute’s training programs help provide the most critical element of good research: well-prepared scientists. The NIGMS research training programs recognize the interdisciplinary nature of biomedical research today and stress approaches to biological problems that cut across disciplinary and departmental lines. Such experience prepares trainees to pursue creative research careers in a wide variety of areas. This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NIGMS.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF MENTAL HEALTH (NIMH)

mental health clinical sciences and other disciplines urgently needed in studies of mental illness and the brain. Another important part of this research is to eliminate the effects of disparities in the availability of and access to high quality mental health services. These disparities, which impinge on the mental health status of all Americans, are felt in particular by many members of ethnic/cultural, minority groups, and by women, children, and elderly people.

Mental disorders occur across the lifespan, from very young childhood into old age, and in the United States, mental disorders collectively account for more than 15% of the overall ‘‘burden of disease’’— a term that encompasses both premature death and disability associated with mental illness. The mission of the National Institute of Mental Health (NIMH) is to reduce the burden of mental illness and behavioral disorders through research on mind, brain, and behavior. Investments made over the past 50 years in basic brain and behavioral science have positioned NIMH to exploit recent advances in neuroscience, molecular genetics, behavioral science and brain imaging; to translate new knowledge about fundamental processes into researchable clinical questions; and to initiate innovative clinical trials of new pharmacologic and psychosocial interventions, with emphasis on testing their effectiveness in the diagnostically complex, diverse group of patients typically encountered in front-line service delivery systems. Investigators funded by NIMH also seek new ways to translate results from basic behavioral science into research relevant to public health, including the epidemiology of mental disorders, prevention and early intervention research, and mental health service research. Diverse scientific disciplines contribute to the design and evaluation of treatments and treatment delivery strategies that are relevant and responsive to the needs of persons with and at risk for mental illness. In this era of opportunity, NIMH is strongly committed to scientific programs to educate and train future mental health researchers, including scientists trained in molecular science, cognitive and affective neuroscience,

1

MECHANISMS OF SUPPORT

NIMH provides leadership at a national level for research on brain, behavior, and mental illness. Under a rigorous and highly competitive process, the institute funds research projects and research center grant awards and contracts to individual investigators in fields related to its areas of interest and to public and private institutions. NIMH also maintains and conducts a diversified program of intramural and collaborative research in its own laboratories and clinical research units at the National Institutes of Health (NIH). NIMH’s informational and educational activities include the dissemination of information and education materials on mental illness to health professionals and the public; professional associations; international, national, state, and local officials; and voluntary organizations working in the areas of mental health and mental illness. 2

AUTISM STAART CENTERS

NIMH supports interdisciplinary research centers through an NIH cooperative agreement in the Studies to Advance Autism Research and Treatment (STAART) Program, in cooperation with the National Institute of Child Health and Human Development (NICHD), the National Institute of Neurological Disorders and Stroke (NINDS), the National Institute on Deafness and Other Communication Disorders (NIDCD), and the National Institute of Environmental Health

This article was modified from the website of the National Institutes of Health (http://www.nih.gov/ about/almanac/organization/NIMH.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

NATIONAL INSTITUTE OF MENTAL HEALTH (NIMH)

Science (NIEHS). By evaluating and treating patients as well as enrolling them in clinical trials, each center helps expand the research base on the causes, diagnosis, early detection, prevention, and treatment of autism. 3 DIVISION ON SERVICES AND INTERVENTION RESEARCH (DSIR) The Division on Services and Intervention Research (DSIR) supports two critical areas of research: intervention research to evaluate the effectiveness of pharmacologic, psychosocial (psychotherapeutic and behavioral), somatic, rehabilitative, and combination interventions on mental and behavior disorders; and mental health services research on organization, delivery (process and receipt of care), related health economics, delivery settings, clinical epidemiology, and the dissemination and implementation of evidencebased interventions into service settings. The division also provides biostatistical analysis and clinical trials operations expertise for research studies, analyzes and evaluates national mental health needs and community research partnership opportunities, and supports research on health disparities. 3.1 DSIR Clinical Trials Operations and Biostatistics Unit The Clinical Trials Operations and Biostatistics Unit serves as the operations focal point for collaborative clinical trials on mental disorders in adults and children. The unit is responsible for overseeing both contractsupported and cooperative agreement–supported multisite clinical trial protocols as well as special projects undertaken by NIMH. In addition, the unit manages overarching matters related to clinical trials operations such as the coordination of the ancillary protocols across the large trials and the implementation of NIMH policy for dissemination of public access datasets. The unit also consults Institute staff and grantees/contractors on biostatistical matters related to appropriateness of study design, determination of power and sample size, and approaches to statistical analysis of data from NIMH-supported clinical trials.

NATIONAL INSTITUTE OF NEUROLOGICAL DISORDERS AND STROKE (NINDS)

hundreds of scientists in training, and provide career awards that offer a range of research experience and support for faculty members at various levels. The purposes and goals of the Extramural Division, Clinical Trials are:

The National Institute of Neurological Disorders and Stroke (NINDS), created by the U.S. Congress in 1950, has occupied a central position in the world of neuroscience for more than 50 years. The mission of NINDS is to reduce the burden of neurologic disease in every age group and segment of society. To accomplish this goal, the Institute supports and conducts research on the healthy and diseased nervous system; fosters the training of investigators in the basic and clinical neurosciences; and seeks better understanding, diagnosis, treatment, and prevention of neurologic disorders. Scientists in the Institute’s laboratories and clinics in Bethesda, Maryland, conduct research in the major areas of neuroscience and on many of the most important and challenging neurologic disorders, and collaborate with scientists in several other NIH institutes. The NINDS vision is:

• To promote the development of clinical

•

• • •

interventions for neurologic disorders and stroke. To stimulate the translation of findings in the laboratory to clinical research and clinical interventions. To ensure measures for protection of human subjects and safety monitoring. To encourage innovation in clinical research methodology. To support the development of neurology clinical researchers with training in biostatistics, epidemiology, and clinical trial methodology.

• To lead the neuroscience community in

shaping the future of research and its relationship to brain diseases. • To build an intramural program that is the model for modern collaborative neuroscience research. • To develop the next generation of basic and clinical neuroscientists through inspiration and resource support. • To seize opportunities to focus our resources to rapidly translate scientific discoveries into prevention, treatment, and cures. The Institute’s extramural program supports thousands of research project grants and research contracts. Institutional training grants and individual fellowships support This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NINDS.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE OF NURSING RESEARCH (NINR)

(NIH) campus, the NINR’s Division of Intramural Research (DIR) focuses on health promotion and symptom management and also provides research training opportunities.

The U.S. National Institute of Nursing Research (NINR) seeks to promote and to improve the health of individuals, families, communities, and populations by supporting and conducting clinical and basic research and research training on health and illness across the lifespan. The NINR research focus encompasses health promotion and disease prevention, quality of life, health disparities, and end-of-life issues. In keeping with the importance of nursing practice in various settings, the NINR seeks to extend nursing science by integrating the biological and behavioral sciences, employing new technologies to research questions, improving research methods, and developing the scientists of the future. The NINR supports basic research on preventing, delaying the onset, and slowing the progression of disease and disability. This includes finding effective approaches to achieving and sustaining a healthy lifestyle, easing the symptoms of illness, improving quality of life for patients and caregivers, eliminating health disparities, and addressing issues at the end of life. The NINR also fosters collaborations with many other disciplines in areas of mutual interest such as long-term care for older people, the special needs of women across the lifespan, genetic testing and counseling, biobehavioral aspects of the prevention and treatment of infectious diseases, and the impact of environmental influences on risk factors for chronic illnesses. In support of finding sound scientific bases for changes in clinical practice, the NINR’s major emphasis is on clinical research, and NINR programs are conducted primarily through grants to investigators across the country. On the National Institutes of Health This article was modified from the website of the National Institutes of Health (http://www.nih.gov/ about/almanac/organization/NINR.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE ON AGING (NIA)

neuroendocrine system, and immune system in aging; and the development of clinical trials and novel interventions to treat these pathologies. The Dementias of Aging Branch supports studies of etiology, pathophysiology, epidemiology, clinical course/natural history, diagnosis and functional assessment, drug discovery, drug development and clinical trials, prevention, behavioral management, and intervention in the cognitive changes associated with the dementias of later life (e.g., mild cognitive impairment, vascular dementia, frontotemporal dementia, Lewy body dementia), especially Alzheimer’s disease.

The U.S. National Institute on Aging (NIA) was established by the U.S. Congress in 1974 to ‘‘conduct and support of biomedical, social, and behavioral research, training, health information dissemination, and other programs with respect to the aging process and diseases and other special problems and needs of the aged.’’ The NIA maintains several branches with varying emphases. The Clinical Trials Branch plans and administers clinical trials on age-related issues that require extensive specialized clinical trials expertise. Examples of current and possible future types of interventions for trials are: • Interventions to prevent or to treat geri-

•

•

•

•

atric syndromes, disability, and complications of comorbidity or polypharmacy. Trials to detect age-related or comorbidity-related differences in responses to interventions against conditions found in middle age and old age. Interventions for problems associated with menopause and other midlife and late-life changes. Interventions that may affect rates of progression of age-related declines in function in early life and midlife. Interventions with protective effects against multiple age-related conditions.

The Integrative Neurobiology Section of the Neurobiology of Aging Branch supports research on the neural mechanisms underlying age-related changes in endocrine functions; neurodegenerative diseases of aging associated with conventional and unconventional infectious agents (e.g., prions); interactions of the central nervous system, This article was modified from the website of the National Institutes of Health (http://www. nih.gov/about/almanac/organization/NIA.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE ON ALCOHOL ABUSE AND ALCOHOLISM (NIAAA) The mission of the National Institute on Alcohol Abuse and Alcoholism (NIAAA) is to provide leadership in the national effort to reduce alcohol-related problems by: • Conducting and supporting research in

a wide range of scientific areas including genetics, neuroscience, and epidemiology to examine the health risks and benefits of alcohol consumption, prevention, and treatment. • Coordinating and collaborating with other research institutes and federal programs on alcohol-related issues. • Collaborating with international, national, state, and local institutions, organizations, agencies, and programs engaged in alcohol-related work. • Translating and disseminating research findings to health-care providers, researchers, policymakers, and the public. The Institute’s efforts to fulfill its mission are guided by the NIAAA vision to support and promote, through research and education, the best science on alcohol and health for the benefit of all by: • Increasing the understanding of normal

and abnormal biological functions and behavior relating to alcohol use. • Improving the diagnosis, prevention, and treatment of alcohol use disorders. • Enhancing quality health care.

This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NIAAA.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTE ON DEAFNESS AND OTHER COMMUNICATION DISORDERS (NIDCD) The U.S. National Institute on Deafness and Other Communication Disorders (NIDCD) conducts and supports research and research training on disorders of hearing and other communication processes, including diseases affecting hearing, balance, smell, taste, voice, speech, and language. The NIDCD sponsors: • Research performed in its own laborato-

ries and clinics. • A program of research grants, individ-

ual and institutional research training awards, career development awards, center grants, conference grants, and contracts to public and private research institutions and organizations. • Cooperation and collaboration with professional, academic, commercial, voluntary, and philanthropic organizations concerned with research and training that is related to deafness and other communication disorders, disease prevention and health promotion, and the special biomedical and behavioral problems associated with people having communication impairments or disorders. • The support of efforts to create devices that substitute for lost and impaired sensory and communication functions. • Ongoing collection and dissemination of information to health professionals, patients, industry, and the public on research findings in these areas.

This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NIDCD.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

• The relationship of drug abuse to the

NATIONAL INSTITUTE ON DRUG ABUSE (NIDA)

acquisition, transmission, and clinical course of human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS), tuberculosis, and other diseases and the development of effective prevention/intervention strategies.

The National Institute on Drug Abuse (NIDA) provides national leadership for research on drug abuse and addiction by supporting a comprehensive research portfolio that focuses on the biological, social, behavioral, and neuroscientific bases of drug abuse on the body and brain as well as its causes, prevention, and treatment. The NIDA also supports research training, career development, public education, and research dissemination efforts. Through its Intramural Research Program as well as grants and contracts to investigators at research institutions around the country and overseas, NIDA supports research and training on:

The Center for Clinical Trials Network (CCTN) supports and leads a network of 17 Regional Research Training Centers (RRTCs) and 240 Community Treatment Programs (CTPs) in a bi-directional effort to bridge the gap between the science of drug treatment and its practice through the study of scientifically based treatments in real world settings. The Clinical Trials Network (CTN) serves as a resource and forum for: • Multisite efficacy and effectiveness tri-

• The neurobiological, behavioral, and

social mechanisms underlying drug abuse and addiction.

•

• The causes and consequences of drug

abuse, including its impact on society and the morbidity and mortality in selected populations (e.g., ethnic minorities, youth, women).

•

• The relationship of drug use to problem

•

behaviors and psychosocial outcomes such as mental illness, unemployment, low socioeconomic status, and violence.

•

•

• Effective prevention and treatment ap-

proaches, including a broad research program designed to develop new treatment medications and behavioral therapies for drug addiction.

als of promising medications and behavioral interventions. Researchers who use the CTN as a platform for studies supported outside of the CCTN. NIDA-supported training using predoctoral and postdoctoral and career awards mechanisms. Secondary analyses of its rich database. Rapid response to emerging public health needs. Systematic transfer of research findings, both positive and negative, to treatment programs, clinicians, and patients.

• The mechanisms of pain and the search

for nonaddictive analgesics. • The relationship of drug abuse to cul-

tural and ethical issues such as health disparities. This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/almanac/organization/NIDA.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTES OF HEALTH (NIH)

alcohol to combating heart disease. In every state across the country, the NIH supports research at hospitals, universities, and medical schools. The NIH is training the current and next generation of researchers to ensure that the capability to advance medical science remains strong. Many of these scientists-in-training will go on to become leading medical researchers and educators at universities; medical, dental, nursing, and pharmacy schools; schools of public health; nonprofit health research foundations; and private medical research laboratories around the country. As a federal agency, the NIH considers many different perspectives in establishing research priorities. A very competitive peer-review system identifies and funds the most promising and highest quality research to address these priorities. This research includes studies that ultimately touch the lives of all people. Currently, with the support of the American people, the NIH annually invests over $28 billion in medical research. More than 83% of the NIH’s funding is awarded through almost 50,000 competitive grants to more than 325,000 researchers at over 3000 universities, medical schools, and other research institutions in every state and around the world. About 10% of the NIH’s budget supports projects conducted by nearly 6000 scientists in its own laboratories, most of which are on the NIH campus in Bethesda, Maryland. The NIH’s own scientists and scientists working with support from the NIH grants and contracts have been responsible for countless medical advances, and more than 100 of these scientists have received Nobel Prizes in recognition of their work.

The National Institutes of Health (NIH), a part of the U.S. Department of Health and Human Services, is the primary federal agency for conducting and supporting medical research. Headquartered in Bethesda, Maryland, the NIH is composed of 27 Institutes and Centers and has more than 18,000 employees on the main campus and at satellite sites across the country. The NIH provides leadership and financial support to researchers in every state and throughout the world. Helping to lead the way toward important medical discoveries that improve people’s health and save lives, NIH scientists investigate ways to prevent disease as well as the causes, treatments, and even cures for common and rare diseases. From the time of its founding in 1887 as the Laboratory of Hygiene at the Marine Hospital in Staten Island, New York, the National Institutes of Health has played an important role in improving health in the United States. Many important health and medical discoveries of the last century resulted from research supported by the NIH. In part because of NIH research, our citizens are living longer and better: life expectancy at birth was only 47 years in 1900; by 2000, it was almost 77 years. The NIH translates research results into interventions and communicates research findings to patients and their families, health-care providers, and the general public. The National Institutes of Health supports and conducts medical research to understand how the human body works and to gain insight into countless diseases and disorders, from rare and unusual diseases to more familiar ones like the common cold. It supports a wide spectrum of research, from learning how the brain becomes addicted to This article was modified from the website of the National Institutes of Health (http://www.nih. gov/about/NIHoverview.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

for sensory function (pupillary response and plantar response) were initially included, but were later removed.

GUSTAVO A. ORTIZ and RALPH L. SACCO MD MS FAAN FAHA, MD

1

Miller School of Medicine University of Miami Miami, Florida

CHARACTERISTICS OF THE SCALE

The NIHSS assesses level of consciousness, gaze, visual fields, facial weakness, motor performance of the extremities, sensory deficit, coordination (ataxia), language (aphasia), speech (dysarthria), and hemi-inattention (neglect) (Tables 1, 2, and 3. Figs. 1 and 2). For all parameters, a value of 0 is normal; so, the higher the score, the worse the neurological deficit. The differences between the levels are not subtle. For example, motor performance for the upper extremity was graded as 0 = normal, 1 = drift (with arms outstretched), 2 = inability to resist gravity for 10 seconds (with arms outstretched), 3 = no effort against gravity, and 4 = no movement. The lowest possible overall NIHSS score is 0 or ‘‘normal,’’ and the highest possible score is 42. The scale was designed to be done quickly and easily at the bedside to provide a rapid and standardized assessment of neurological function in the early periods after a stroke. When first evaluated in 65 patients, it was administered in a mean of 6.6 ± 1.3 minutes and was completed in all patients, regardless of stroke severity (2).

The correlation of NIHSS scores with MRI and clinical findings has been used for assessment of ischemic penumbra, in an effort to identify patients with tissue at risk of infarction for thrombolytic or neuroprotective drugs. Currently, the NIHSS is the most widely used scale for trials related to thrombolytic therapy. Clinical measurement of the severity of cerebral infarction became an important subject of investigation in the late 1980s with the introduction of new therapies for acute stroke. Traditional measures such as mortality or long-term functional status were not as well suited for the assessment of acute stroke therapies in which the immediate effects of stroke had to be quantified. The evaluation tools used to quantify a clinical condition are usually referred to as clinimetric instruments (1). ‘‘Stroke scales’’ are clinimetric instruments used to quantify neurological deficits, functional outcome, global outcome, or health-related quality of life in patients after a stroke. The National Institutes of Health Stroke Scale (NIHSS) is a systematic assessment tool designed to measure the neurological deficits most often seen with acute stroke. It was developed by Thomas Brott, MD and colleagues at the University of Cincinnati (Ohio), and its description was first reported in 1989 (2). The examination format was developed to assess neurologic signs in the distribution of each of the major arteries of the brain. Exam items for the NIHSS were adapted from the Toronto Stroke Scale, the Oxbury Initial Severity Scale (3,4) and the Cincinnati Stroke Scale which graded speech, drift of the affected arm, drift of the affected leg, and grip strength. Two items from the Edinburgh-2 Coma scale were also used to assess mental status (5). Other categories

1.1 Reliability Reliability, an index of the reproducibility and precision of the NIHSS, was first evaluated among twenty-four patients with acute cerebral infarction who were examined twice within a 24-hour interval. Each examination was performed by a neurologist, whereas the other three examination team members (neurology house officer, neurology nurseclinician, and emergency department nurseclinician) observed. Then, each team member independently performed scoring. Inter-rater agreement between these four examiners was high (mean k = 0.69); and intra-rater agreement was also excellent, especially when the rater was a neurologist (k = 0.77) (2). Another study of 20 patients confirmed that the NIH

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

Table 1. The National Institutes of Health Stroke Scale∗ Test 1a. Level of Consciousness (LOC):

1b. LOC Questions:

1c. LOC Commands:

2. Best Gaze:

3. Visual: Visual fields (upper and lower quadrants) 4. Facial Palsy: Ask– or use pantomime to encourage

5. Motor Arm: 5a. Left Arm 5b. Right Arm

6. Motor Leg: 6a. Left Leg 6b. Right Leg

7. Limb Ataxia:

8. Sensory:

Scale Definition 0 = Alert; keenly responsive. 1 = Not alert; however, arousable by minor stimulation to obey, answer, or respond. 2 = Not alert; requires repeated stimulation to attend, or is obtunded and requires strong or painful stimulation to make movements (not stereotyped). 3 = Responds only with reflex motor or autonomic effects or totally unresponsive, flaccid, and are flexic. 0 = Answers both questions correctly. 1 = Answers one question correctly. 2 = Answers neither question correctly. 0 = Performs both tasks correctly. 1 = Performs one task correctly. 2 = Performs neither task correctly. 0 = Normal. 1 = Partial gaze palsy; gaze is abnormal in one or both eyes, but forced deviation or total gaze paresis is not present. 2 = Forced deviation, or total gaze paresis not overcome by the oculocephalic maneuver. 0 = No visual loss. 1 = Partial hemianopia. 2 = Complete hemianopia. 3 = Bilateral hemianopia (blind including cortical blindness). 0 = Normal symmetrical movements. 1 = Minor paralysis (flattened nasolabial fold; asymmetry on smiling). 2 = Partial paralysis (total or near-total paralysis of lower face). 3 = Complete paralysis of one or both sides (absence of facial movement in the upper and lower face). 0 = No drift; limb holds 90 (or 45) degrees for full 10 seconds. 1 = Drift; limb holds 90 (or 45) degrees, but drifts down before full 10 seconds; does not hit bed or other support. 2 = Some effort against gravity; limb cannot get to or maintain (if cued) 90 (or 45) degrees; drifts down to bed, but has some effort against gravity. 3 = No effort against gravity; limb falls. 4 = No movement. UN = Amputation or joint fusion, explain: 0 = No drift; leg holds 30-degree position for full 5 seconds. 1 = Drift; leg falls by the end of the 5-second period but does not hit bed. 2 = Some effort against gravity; leg falls to bed by 5 seconds but has some effort against gravity. 3 = No effort against gravity; leg falls to bed immediately. 4 = No movement. UN = Amputation or joint fusion, explain: 0 = Absent. 1 = Present in one limb. 2 = Present in two limbs. UN = Amputation or joint fusion, explain: 0 = Normal; no sensory loss. 1 = Mild-to-moderate sensory loss; patient feels pinprick is less sharp or is dull on the affected side, or there is a loss of superficial pain with pinprick, but patient is aware of being touched. 2 = Severe to total sensory loss; patient is not aware of being touched in the face, arm, and leg.

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

3

Table 1. (continued) Test

Scale Definition

9. Best Language:

0 = No aphasia; normal. 1 = Mild-to-moderate aphasia; some obvious loss of fluency or facility of comprehension, without significant limitation on ideas expressed or form of expression. Reduction of speech and/or comprehension, however, makes conversation about provided materials difficult or impossible. For example, in conversation about provided materials, examiner can identify picture or naming card content from patient response. 2 = Severe aphasia; all communication is through fragmentary expression; great need for inference, questioning, and guessing by the listener. Range of information that can be exchanged is limited; listener carries burden of communication. Examiner cannot identify materials provided from patient response. 3 = Mute, global aphasia; no usable speech or auditory comprehension. 0 = Normal. 1 = Mild-to-moderate dysarthria; patient slurs at least some words and, at worst, can be understood with some difficulty. 2 = Severe dysarthria; patient speech is so slurred as to be unintelligible in the absence of or out of proportion to any dysphasia, or is mute/anarthric. UN = Intubated or other physical barrier, explain: 0 = No abnormality. 1 = Visual, tactile, auditory, spatial, or personal inattention or extinction to bilateral simultaneous stimulation in one of the sensory modalities. 2 = Profound hemi-inattention or extinction to more than one modality; does not recognize own hand or orients to only one side of space.

10. Dysarthria:

11. Extinction and Inattention (Formerly Neglect):

∗ For all parameters, a value of 0 is normal; so, the higher the score, the worse the neurological deficit. The differences between the levels are not subtle. (Adapted from ‘‘NIH Stroke Scale’’ at www.ninds.nih.gov.)

Table 2. Evaluation of Language: Reading∗ You know how. Down to earth. I got home from work. Near the table in the dining room. They heard him speak on the radio last night. ∗ The patient is asked to read these sentences. (Reproduced from www.ninds.nih.gov.)

Table 3. Evaluation of Dysarthria: the Patient is Asked to Read and Say These Words (Reproduced From www.ninds.nih.gov) MAMA TIP – TOP FIFTY – FIFTY THANKS HUCKLEBERRY BASEBALL PLAYER

4

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

Stroke Scale has substantial or moderate inter-rater reliability for nine of its items (6). The NIHSS was first used in a pilot study of t-PA administered between 0 and 180 minutes from stroke onset (7); and a modified, 13-item form of the scale was later used in the definitive NINDS trial of t-PA (8). In preparation for the latter trial, 162 investigators were trained and certified in the use of the scale by using a two-camera videotape method, an approach that enhanced intra-rater and inter-rater reliability (9). Moderate to excellent agreement was found on most items (unweighted k > 0.60), but facial paresis and ataxia showed poor agreement (unweighted k < 0.40). The scale was also demonstrated to be reliable when performed by non-neurologists and nonphysicians, after training via the videotape method was considered a prerequisite for its use (9–12). 1.2 Validity The validity, or extent to which the NIHSS measured what it purported to measure, was initially assessed by determining reliability and accuracy (13). Accuracy was determined by correlating the NIHSS scores with two other measures of stroke severity: computed tomography (CT) measurement of volume of infarction and clinical outcome at 3 months. 1. Correlation with size of infarction: The Spearmen correlation between the total NIHSS score and the computed tomography lesion at 7 days was 0.74. The initial neurological deficit of the patient as measured by the scale also correlated with the 7–10 day CT lesion volume (correlation coefficient = 0.78) (14). 2. Correlation with clinical outcome: Stroke scale scores at 7 days corresponded to eventual clinical outcome, including placement in the community, with a Spearman correlation of 0.71 (P = 0.0001). The admission stroke scale scores also correlated with the eventual patient functional outcome at 3 months, with a Spearman correlation of 0.53 (P = 0.0001) (2). In 1999, Dr. Lyden et al. with the NINDS tPA Trial investigators validated the scale

as an outcome measure, using data from the NINDS tPA Stroke Trial (15). In this study, it was shown that the internal scale structure remained consistent in treated and placebo groups and when administered serially over time. Also, the correlations between the scale and other clinical outcome scales (Barthel Index, Rankin Scale, and Glasgow Outcome Scale) at 3 months were significant (P < 0.001). The study supported the validity of the scale for use in future treatment trials as an outcome measure. Other studies using the NIHSS have also shown good construct validity. In a post hoc analysis by stroke subtype of 1268 patients enrolled in an acute stroke trial, baseline NIHSS scores strongly predicted outcome at 7 days and at 3 months. An excellent outcome was achieved by almost two thirds of patients with a score of 3 or less at day 7; however, very few patients with baseline scores of more than 15 had excellent outcomes after 3 months (16). Other alternative measures of neurological outcome, such as activities of daily living (ADL) scales, have also been correlated with the NIHSS (17). 2 CORRELATION OF NIHSS SCORES WITH MRI STUDIES MRI scans permitted a more accurate measurement of stroke lesion volume and allowed for a better assessment of the correlation between NIHSS and stroke size. Lovblad et al. in 1997 found a significant correlation between Diffusion Weighted Imaging (DWI) lesion size and both initial (r = 0.53, P = 0.0003) and chronic NIHSS score (r = 0.68, P < 0.0001) in 50 patients with acute middle cerebral artery ischemic stroke (< 24-hour duration) (18). The correlation was also demonstrated in earlier strokes (within 6.5 hours of symptom onset), by comparing DWI or Perfusion Weighted Imaging (PWI) to assess lesion size (19). In eight of nine DWIpositive patients in this series, a strong linear correlation existed between 24-hour NIHSS and DWI volume (r = 0.96, P < 0.001), and PWI correlated better with 24-hour NIHSS than with DWI. The difference was attributed primarily to one patient, who had a substantial perfusion delay in PWI, whereas DWI

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

5

Figure 1. Evaluation of language: Comprehension. The patient is asked to describe what is happening in this picture. (Reproduced from www.ninds.nih.gov).

Figure 2. Evaluation of language: Naming. The patient is asked to name the objects shown in this ‘‘naming sheet.’’ (Reproduced from www.ninds.nih.gov).

showed no abnormality. Clinically, the deficit was substantial (NIHSS = 24). On a subsequent 24-hour MRI scan, a DWI abnormality developed that closely matched her initial perfusion deficit (19). This study led to the

development of the concept of PWI/DWI mismatch. Following the initial report by Dr. Tong, MRI has been used extensively to characterize the ischemic penumbra by identifying regions of brain with reduced blood flow

6

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

(defined by PWI) and regions with irreversible neuronal injury (approximated by DWI). A mismatch between the volume of abnormality on diffusion and perfusion imaging seems to indicate viable penumbral tissue (PWI/DWI mismatch). PWI/DWI mismatch has been shown to correlate with favorable outcomes following late treatment with intravenous rtPA up to 6 hours from symptom onset (20–23). However, PWI is a complex, time-consuming, and not well-standardized technique, with limited availability. Based on the fact that abnormal PWI volume has a higher correlation with the stroke severity (evaluated clinically with the NIHSS) than the DWI volume (19,24), Davalos et al. in 2004 proposed the concept of clinical-DWI mismatch (CDM) (25). CDM defined as NIHSS ≥ 8 and ischemic volume on DWI of ≤ 25 ml was associated with a high probability of infarct growth and early neurological deterioration. Subsequently, it was suggested that CDM may identify patients with tissue at risk of infarction for thrombolytic or neuroprotective drugs. CDM was later shown to predict the presence of PWI/DWI mismatch with a specificity of 93% (95% confidence interval (CI), 62%–99%) and a positive predictive value of 95% (95% CI, 77%–100%) but a sensitivity of only 53% (95% CI, 34%–68%). Efforts are being made to evaluate a computed tomography–NIHSS mismatch, by using the recently applied Alberta Stroke Program Early CT score (ASPECTS), but results are still controversial (26,27). 3

USE OF THE NIHSS

Originally, the NIHSS was developed for use in multicenter acute stroke trials. It proved reliable and valid for the measurement of stroke-related deficits at baseline, as well as for measuring outcome. Currently, the NIHSS is the most widely used scale for trials related to thrombolytic therapy. With the approval of intravenous thrombolysis for the treatment of acute stroke, the use of the NIHSS in the clinical care of patients increased. The ease of administration of the NIHSS, which can be effectively performed by physicians and nonphysicians

after a relatively simple training, led to more widespread acceptance. The NIHSS is used to provide an accurate assessment of the deficits of a patient with stroke during initial presentation. This initial clinical assessment can be easily transmitted to other health-care providers involved in the care of the patient, thus expediting the decision-making process in the emergency setting. The scale is also a sensitive measurement tool for serial monitoring of patients with stroke, quantifying changes in the neurological examination. An increase of 2 points or more on the NIHSS has been suggested to be associated to a relevant worsening of the stroke, although this specific cutoff has not been independently validated (28). In the clinical setting, the NIHSS predicts post-acute care disposition among stroke patients and may facilitate the process of hospital discharge (29,30). In particular, stroke patients with NIHSS scores ≤ 5 can be expected to go home, those with scores > 13 most often go to a nursing facility, and those with intermediate scores most often go to acute rehabilitation (29). 4 STANDARDIZED TRAINING FOR THE USE OF THE NIHSS The American Stroke Association, in conjunction with the American Academy of Neurology (AAN) and the National Institute of Neurological Disorders and Stroke (NINDS), developed a free, CME/CE-certified online training program for healthcare professionals (targeted to emergency physicians, neurologists, nurses, clinical research raters, and medical students) to learn or review how to administer the NIHSS for acute stroke assessment. Available at the American Stroke Association webpage (www. strokeassociation.org) by clicking on the link to Professional Education Center, it requires signing up and creating a profile with username and password before taking the course. Video training is also available on DVD. The course consists of six test groups, A through F. After successfully completing the program, participants can print their CME/CE credit for test groups A and B directly from the website and can also print a certificate of

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

completion for test groups A, B, C, D, E, or F (31). 5

scale are graded numerically, with higher values representing more severe deficits, these are ordinal-level, not interval-level, data (33). The total score is obtained by adding all these individual rankings and may be misleading (34). It has been suggested that clinically it is more useful to think of the NIHSS as a way to quantify serial neurologic findings and measure change within an individual patient over time.

LIMITATIONS OF THE SCALE

Several limitations of the NIHSS are as follows: 1. The inter-rater reliability of the scale is not homogeneous for all the items. It has been shown that the assessments of limb ataxia and facial weakness have a lower agreement between examiners compared with the other items, with calculated k values that were not significantly different from chance. However, the calculated values of k were significantly greater than expected by chance for 11 of 13 items, indicating substantial agreement for 5 items, moderate agreement for 4 items, and fair agreement for 2 items. This rating system compared favorably with other scales (6). 2. The NIHSS has a differential characterization of right versus left hemisphere strokes. Of the 42 possible points on the NIHSS, 7 are directly related to measurement of language function (orientation questions, 2; commands, 2; and aphasia, 3), and only 2 points are for neglect. It has been shown that for a given NIHSS, the total lesion volume for patients with right (usually nondominant for language) hemisphere strokes is statistically larger than the lesion volume for patients with left (usually dominant for language) hemisphere strokes. The difference reflects the differential weighting of the NIHSS with regard to language function, as compared with hemineglect (32). 3. Brainstem infarcts may not be adequately characterized by the NIHSS. Cranial nerves are not fully assessed in the scale; therefore, it is possible that life-threatening brainstem or cerebellar infarctions may result in lower scores that underestimate the clinical severity of the stroke. 4. The total NIHSS score has limited value. Although all items in the

7

6

FUTURE DIRECTIONS

A modified NIHSS (mNIHSS) with 11 items, derived from previous clinimetric studies of the NIHSS, has been proposed. Poorly reproducible or redundant items (level of consciousness, face weakness, ataxia, and dysarthria) were deleted, and the sensory items were collapsed into 2 responses. 10 of the 11 items show excellent reliability, and 1 item shows good reliability (35,36). Other shortened scales are being proposed for prehospital clinical assessment of stroke, but additional studies are needed to assess their value in the screening of stroke by the paramedic services (37). Video telemedicine is being proposed as an option to facilitate cerebrovascular specialty consults to underserved areas. It seems feasible to perform NIHSS remotely using computer-based technology, in an effort to increase the rate of rt-PA administration (38–40). REFERENCES 1. K. Asplund, Clinimetrics in stroke research. Stroke 1987; 18: 528–530. 2. T. Brott, et al., Measurements of acute cerebral infarction: a clinical examination scale. Stroke 1989; 20: 864–870. 3. Oxbury, J.M., R.C. Greenhall, and K.M. Grainger, Predicting the outcome of stroke: acute stage after cerebral infarction. Br. Med. J. 1975; 3: 125–127. 4. R. Cote, et al., The Canadian Neurological Scale: a preliminary study in acute stroke. Stroke 1986; 17: 731–737. 5. K. Sugiura, et al., The Edinburgh-2 coma scale: a new scale for assessing impaired consciousness. Neurosurgery 1983; 12: 411–415.

8

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS)

6. L. B. Goldstein, C. Bertels, and J. N. Davis, Interrater reliability of the NIH stroke scale. Arch. Neurol. 1989; 46: 660–662. 7. E. C. Haley Jr., et al., Urgent therapy for stroke. Part II. Pilot study of tissue plasminogen activator administered 91–180 minutes from onset. Stroke 1992; 23: 641–645. 8. NINDS, Tissue plasminogen activator for acute ischemic stroke. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. N. Engl. J. Med. 1995; 333: 1581–1587. 9. P. Lyden, et al., Improved reliability of the NIH Stroke Scale using video training. NINDS TPA Stroke Study Group. Stroke 1994; 25: 2220–2226. 10. M. A. Albanese, et al., Ensuring reliability of outcome measures in multicenter clinical trials of treatments for acute ischemic stroke. The program developed for the Trial of Org 10172 in Acute Stroke Treatment (TOAST). Stroke 1994; 25: 1746–1751. 11. L. B. Goldstein and G. P. Samsa, Reliability of the National Institutes of Health Stroke Scale. Extension to non-neurologists in the context of a clinical trial. Stroke 1997; 28: 307–310. 12. S. Schmulling, et al., Training as a prerequisite for reliable use of NIH Stroke Scale. Stroke 1998; 29: 1258–1259. 13. R. Cote, et al., Stroke assessment scales: guidelines for development, validation, and reliability assessment. Can. J. Neurol. Sci. 1988; 15:261–265 14. T. Brott, et al., Measurements of acute cerebral infarction: lesion size by computed tomography. Stroke 1989; 20: 871–875. 15. P. Lyden, et al., Underlying structure of the National Institutes of Health Stroke Scale: results of a factor analysis. NINDS tPA Stroke Trial Investigators. Stroke 1999; 30: 2347–2354. 16. H. P. Adams Jr., et al., Baseline NIH Stroke Scale score strongly predicts outcome after stroke: a report of the Trial of Org 10172 in Acute Stroke Treatment (TOAST). Neurology 1999; 53: 126–131. 17. P. W. Duncan, et al., Measurement of motor recovery after stroke. Outcome assessment and sample size requirements. Stroke 1992; 23: 1084–1089. 18. K. O. Lovblad, et al., Ischemic lesion volumes in acute stroke by diffusion-weighted magnetic resonance imaging correlate with clinical outcome. Ann Neurol, 1997; 42: 164–170. 19. D. C. Tong, et al., Correlation of perfusion- and diffusion-weighted MRI with NIHSS score in

acute (< 6.5 hour) ischemic stroke. Neurology 1998; 50: 864–870. 20. G. W. Albers, et al., Magnetic resonance imaging profiles predict clinical response to early reperfusion: the diffusion and perfusion imaging evaluation for understanding stroke evolution (DEFUSE) study. Ann. Neurol. 2006; 60: 508–517. 21. J. Rother, et al., Effect of intravenous thrombolysis on MRI parameters and functional outcome in acute stroke < 6 hours. Stroke 2002; 33: 2438–2445. 22. M. W. Parsons, et al., Diffusion- and perfusionweighted MRI response to thrombolysis in stroke. Ann. Neurol. 2002; 51: 28–37. 23. G. Thomalla, et al., Outcome and symptomatic bleeding complications of intravenous thrombolysis within 6 hours in MRI-selected stroke patients: comparison of a German multicenter study with the pooled data of ATLANTIS, ECASS, and NINDS tPA trials. Stroke 2006; 37: 852–858. 24. T. Neumann-Haefelin, et al., Diffusion- and perfusion-weighted MRI. The DWI/PWI mismatch region in acute stroke. Stroke 1999; 30: 1591–1597. 25. A. Davalos, et al., The clinical-DWI mismatch: a new diagnostic approach to the brain tissue at risk of infarction. Neurology 2004; 62: 2187–2192. 26. H. Tei, S. Uchiyama, and T. Usui, Clinicaldiffusion mismatch defined by NIHSS and ASPECTS in non-lacunar anterior circulation infarction. J Neuro. 2007; 254: 340–346. 27. S. R. Messe, et al., CT-NIHSS mismatch does not correlate with MRI diffusion-perfusion mismatch. Stroke 2007; 38: 2079–2084. 28. B. C. Tilley, et al., Use of a global test for multiple outcomes in stroke trials with application to the National Institute of Neurological Disorders and Stroke t-PA Stroke Trial. Stroke 1996; 27: 2136–2142. 29. D. Schlegel, et al., Utility of the NIH Stroke Scale as a predictor of hospital disposition. Stroke 2003; 34: 134–137. 30. D. J. Schlegel, et al., Prediction of hospital disposition after thrombolysis for acute ischemic stroke using the National Institutes of Health Stroke Scale. Arch. Neurol. 2004; 61: 1061–1064. 31. American Stroke Association, NIH Stroke Scale Training Online, 2007. Available: www.strokeassociation.org. 32. D. Woo, et al., Does the National Institutes of Health Stroke Scale favor left hemisphere

NATIONAL INSTITUTES OF HEALTH STROKE SCALE (NIHSS) strokes? NINDS t-PA Stroke Study Group. Stroke 1999; 30: 2355–2359. 33. A. R. Feinstein, B. R. Josephy, and C. K. Wells, Scientific and clinical problems in indexes of functional disability. Ann. Intern. Med. 1986; 105: 413–420. 34. T. J. Steiner and F. Clifford Rose, Towards a model stroke trial. The single-centre naftidrofuryl study. Neuroepidemiology 1986; 5: 121–147. 35. B. C. Meyer, et al., Modified National Institutes of Health Stroke Scale for use in stroke clinical trials: prospective reliability and validity. Stroke 2002; 33: 1261–1266. 36. P. D. Lyden, et al., A modified National Institutes of Health Stroke Scale for use in stroke clinical trials: preliminary reliability and validity. Stroke 2001; 32: 1310–1317. 37. D. L. Tirschwell, et al., Shortening the NIH Stroke scale for use in the prehospital setting. Stroke 2002; 33: 2801–2806. 38. B. C. Meyer, et al., Prospective reliability of the STRokE DOC wireless/site independent telemedicine system. Neurology 2005; 64: 1058–1060. 39. S. R. Levine and M. Gorman, ‘‘Telestroke’’: the application of telemedicine for stroke. Stroke 1999; 30: 464–469. 40. S. R. Levine, and K. M. McConnochie, Telemedicine for acute stroke: when virtual is as good as reality. Neurology 2007; 69: 819–820.

9

NATIONAL LIBRARY OF MEDICINE (NLM) The U.S. National Library of Medicine (NLM) is the world’s largest research library of the health sciences and serves scientists, health professionals, and the public. The Library has a statutory mandate from Congress to apply its resources broadly to the advancement of medical and health-related sciences. To that end, it collects, organizes, and makes available biomedical information to investigators, educators, practitioners, and the public and carries out programs designed to strengthen existing and develop new medical library services in the United States. It conducts research in health communications, supports medical informatics, and provides information services and sophisticated tools in the areas of molecular biology and toxicology/environmental health. The NLM also creates Web-based services for the general public containing information from the National Institutes of Health (NIH) and other reliable sources.

This article was modified from the website of the National Institutes of Health (http://www.nih.gov/ about/almanac/organization/NLM.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NEW DRUG APPLICATION (NDA) PROCESS For decades, the regulation and control of new drugs in the United States has been based on the New Drug Application (NDA). Since 1938, every new drug has been the subject of an approved NDA before U.S. commercialization. The NDA application is the vehicle through which drug sponsors formally propose that the U.S. Food and Drug Administration (FDA) approve a new pharmaceutical for sale and marketing in the United States. The data gathered during the animal studies and human clinical trials of an Investigational New Drug (IND) become part of the NDA. The goals of the NDA are to provide enough information to permit an FDA reviewer to reach the following key decisions: • Whether the drug is safe and effective

in its proposed use(s), and whether the benefits of the drug outweigh the risks. • Whether the drug’s proposed labeling (the package insert) is appropriate, and what it should contain. • Whether the methods used in manufacturing the drug and the controls used to maintain the drug’s quality are adequate to preserve the drug’s identity, strength, quality, and purity. The documentation required in an NDA is supposed to tell the drug’s whole story, including what happened during the clinical tests, what the ingredients of the drug are, what the results of the animal studies were, how the drug behaves in the body, and how it is manufactured, processed, and packaged.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/regulatory/applications/ nda.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

N-OF-1 RANDOMIZED TRIALS

hypertension, nocturnal leg cramps, attention deficit/hyperactivity disorder, fibromyalgia, Parkinson’s disease, asthma, and chronic obstructive pulmonary disease, among many others.

ANDREW L. AVINS Kaiser-Permanente, Northern California Division of Research and Departments of Medicine and Epidemiology & Biostatistics, University of California, San Francisco

1

GOAL OF N-OF-1 STUDIES

An N-of-1 study may be initiated for several reasons. Most commonly, the primary motivation is to better define the effectiveness of therapy for a symptomatic condition, though the effect of therapy on an asymptomatic endpoint (e.g., blood pressure) may also be investigated with appropriate measurements. N-of-1 studies may be valuable when patients are hesitant to use a therapy and desire more information about their response and when the medical provider and patient disagree about the response to a therapy. Another common use is to understand whether a specific therapy is the cause of an undesirable symptom in a patient. Determining the optimal dose of a medication is a matter that also lends itself well to study by N-of-1 methodology. Because it is often impossible or impractical to conduct typical randomized trials for patients with rare conditions, N-of-1 trials may provide an opportunity for defining optimal therapy for these patients. As will be noted, however, conditions and therapies amenable to N-of-1 trials have specific attributes that render many conditions or treatments unsuitable for study by this design. One of the most valuable situations in which to perform N-of-1 studies is when the clinical response to an intervention is known to be highly variable, there is uncertainty about the patient’s response, and the intervention is expensive or associated with serious adverse effects. In this situation, greater insight into a patient’s true response is particularly helpful in formulating rational treatment decisions. For example, Zucker et al. (8) describe the use of an N-of-1 trial to help fibromyalgia patients make decisions about using the combination of amitriptyline and fluoxetine versus amitriptyline alone; a prior trial had found that only 63% of patients

JOHN NEUHAUS Department of Epidemiology & Biostatistics, University of California, San Francisco

Comparative clinical trials in individual patients, commonly known as ‘‘N-of-1’’ trials, have been used for many years in the field of psychology and have found increasing applications for biomedical problems. Such trials can be structured to provide information generalizable to large groups of patients, but the primary application of N-of-1 trials is to help individual patients make more evidencebased decisions about specific aspects of their medical care (1). In clinical practice, patients frequently undergo ‘‘therapeutic trials’’ in which a health-care provider recommends a course of action that the patient tries and decides whether to continue, modify, or terminate. An N-of-1 study has the same goal as a therapeutic trial but brings a more structured and less biased approach to evaluating a therapeutic response (1–3). Such trials are also known as ‘‘single-patient trials,’’ ‘‘individualized medication effectiveness trials,’’ and ‘‘individual patient trials’’ (4–6). An N-of-1 trial is generally structured as a randomized, double-blind, controlled, multiple-period crossover trial in a single patient, often of a pharmacologic agent (1, 3, 6, 7). In these studies, a patient is assigned to different periods on the active and the comparator treatments, with outcome measurements obtained during each of these periods. At the end of the trial, the patient’s responses during each of the treatment conditions are compared, and the information is used to make a decision about the likelihood that one of the therapeutic options is superior. Such trials have been successfully used in such disparate conditions as osteoarthritis,

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

N-OF-1 RANDOMIZED TRIALS

responded to the combination therapy (and both drugs are associated with a wide range of side effects). 2

REQUIREMENTS

Similar to other crossover studies, N-of-1 trials are possible or practical in only a limited number of settings, as both the condition being treated and the therapy being tested must meet certain conditions in order for the trial to be successful (1, 3, 6, 7, 9, 10). 2.1 Patient’s Motivation Both the adherence to the study protocol and the usefulness of the data obtained are closely related to the motivation of the patient. Hence, there should be a clearly perceived need for conducting the N-of-1 study on the part of both the patient and the health-care provider. There should be uncertainty or disagreement about which choice of therapy (if any) is best and a mutual understanding of the value of conducting an N-of-1 trial to answer the question (2). Given the difficulty and expense of conducting an N-of-1 trial, there should also be a clear understanding of how (and whether) the patient will use the information from the trial to make a decision about therapy. A lack of motivation may seriously undermine the likelihood of success of the trial (see section 5.1). 2.2 Conditions Suitable for Study Because N-of-1 trials are generally structured as multiple-period crossover trials, only certain medical conditions are amenable to study with this methodology. Generally, appropriate illnesses must be chronic or recurring. Conditions whose status changes rapidly are difficult to investigate, though increasing the number of crossover periods may mitigate this problem. Similarly, recurrent illnesses with long asymptomatic intercurrent periods are difficult to study; this problem can sometimes be addressed by increasing the length of the treatment periods, though this approach may reduce the patient’s motivation and adherence to the protocol. Studies of short, self-limited illnesses are unlikely to be informative.

2.3 Therapies Suitable for Study For a successful trial, the study therapy must act quickly or a longer treatment period will be required to fairly evaluate treatment effectiveness. Similarly, it is desirable that once treatment is stopped the patient’s condition quickly returns to its baseline state, or prolonged washout periods will be required. Obviously, a potentially curative treatment is inappropriate for any crossover study. 2.4 Combining Results of Many N-of-1 Trials When many patients with the same diagnosis participate in N-of-1 trials with the same intervention, the results of these studies can be combined to estimate a population treatment effect, much like a formal multiparticipant crossover study. If such a series of N-of-1 trials is contemplated, care should be taken to ensure that they are structured in a way that permits a valid synthesis of the data. These considerations include appropriate definitions of the disease state under study and applicable eligibility criteria, a treatment protocol that provides enough similarity to make a synthesis meaningful, and outcome assessments that are measured with sufficient uniformity to permit combining the data. Using Bayesian methods, the data from an accumulating series of N-of-1 trials can be used to refine the estimates of the likelihood of a response for an individual patient as well as providing a generalizable estimate of effect for a population. Zucker et al. (8, 11) provide a detailed methodology for this technique, including the structure of a hierarchical Bayesian model and an example of this method in a series of N-of-1 studies for testing pharmacologic therapy for fibromyalgia. 3 DESIGN CHOICES AND DETAILS FOR N-OF-1 STUDIES Numerous options exist for the design and implementation of N-of-1 studies. Most variants have been tried successfully, and it is clear that there is no single design that is optimal for all situations; the creativity and insight of the investigator will dictate the best approach among the available

N-OF-1 RANDOMIZED TRIALS

alternatives. Issues that must be addressed include randomization, number and frequency of crossover periods, run-in and washout periods, blinding, outcome assessment, and choice of outcome measure.

3

alternative is to use strictly alternating treatments, with the first treatment chosen at random, though such a design may be more susceptible to unblinding of the treatments. 3.2 Number and Length of Treatment Periods

3.1 Randomization It is not absolutely required that the various treatment periods in an N-of-1 study be randomized, but most authorities agree that randomization is greatly preferable when it can be done. When randomization is employed, there are two major options: an unconstrained randomized design, or a blocked randomized design (2). In the former, the treatment periods are completely randomized without regard to ordering (i.e., without blocking). This method has the advantage of satisfying the underlying assumption of some statistical tests commonly used in single-patient experiments and is easy to implement. However, it suffers from the risk of generating treatment patterns that make causal inference difficult, being more susceptible to period effects. For example, in an N-of-1 study of two therapies with three treatment periods each, an unconstrained design may result in three periods on the first treatment followed by three treatments periods on the second treatment (i.e., AAABBB). Should the second treatment appear superior, one may be left wondering if it is the better therapy or if the patient’s symptoms were beginning to remit spontaneously. In typical crossover studies, this problem is addressed by randomizing participants to different interventions for the initial treatment period, but this option is not available in the N-of-1 design with a single participant. An alternative to the unconstrained randomized design is the blocked randomized design. In this design (using the paradigm of the two-treatment comparison), the treatment periods are randomized in pairs (i.e., blocks of two) so that each treatment is subsequently compared with the other treatment after every two treatment periods, though the specific ordering of the comparison is random. For example, a blocked randomization design might result in treatment ordering of ABBABA, BABABA, or BAABBA. A related

The choice of the number and length of the treatment periods will be a function of the particular characteristics of the illness and therapies under study, the tolerance of the patient for longer trial lengths (and burden of data collection), the need to obtain the clinical information more quickly, the availability of resources for conducting the trial, and the specific outcomes studied. No definitive advice is applicable to all situations. Of course, in general, the longer the trial and the greater the number of crossover periods, the more information will be obtained, and the more stable the resultant estimates of effect will be. 3.3 Run-In and Washout Periods One of the difficulties in all crossover designs is the potential for carryover effects—the possibility that the effect of one treatment has not entirely dissipated when the next treatment is instituted. For example, a crossover comparison of two treatments, A and B, will actually be a comparison of A followed by a comparison of B combined with some effect of A if the effects of treatment A persist into the B-treatment period. Several approaches are available for addressing the carryover effect. A ‘‘run-in’’ period (an initial period off all active treatment, which may be a placebo-assigned period) is often advocated to wash out the effect of any remaining active therapies the patient may have used. A conditiondependent crossover rule may be specified; that is, after treatment A is stopped, the patient’s clinical condition is allowed to return to baseline before starting treatment B. A time-dependent crossover rule is an alternative: a specific amount of time is required to pass before the patient is crossed over to the next therapy. One implementation method for the time-dependent crossover design is to discard data from the first part of each treatment period and analyze only

4

N-OF-1 RANDOMIZED TRIALS

the data from the latter part of each treatment period, timing the data-discard period to an adequate washout period for the prior treatment. For example, Nikles et al. (4) performed a set of N-of-1 studies of medical therapy for osteoarthritis; the treatment periods were 2 weeks long, but the data for only the second week of treatment in each period was used for analysis. Additional advantages to this latter technique are that participants are never off the study drug (which may improve adherence) and using the active treatment for a period of time before true data collection starts allows for some stability of drug levels and drug effects before the clinical response is measured. Another variant is an open-label run-in period to assess the tolerability of the study treatment, ensure that some response is evident, and/or perform an initial dose-finding analysis; this information may help shorten the overall duration of the trial (7, 9). 3.4 Blinding One of the great advantages of a formal Nof-1 study over a typical therapeutic trial is the potential to blind (or ‘‘mask’’) the therapies under study. Successful blinding can help distinguish the placebo effect from the specific treatment effect of a trial agent and potentially provide more meaningful data, depending on the needs of the patient and clinician. Blinding a pharmaceutical N-of-1 trial, however, requires the formulation of a credible placebo (for placebo-controlled trials) or formulation of similar-appearing medications for active-controlled trials. Producing these products is often difficult and can be an insurmountable problem for clinicians who are not supported by an infrastructure to help conduct these studies. Some institutions have created services to support clinicians interested in conducting N-of-1 trials (see section 5.3). 3.5 Choice of Outcome Measures The N-of-1 trial is unique in that the choice of outcome measures is often far less a function of credibility to the research community or the general population of patients than of personal importance to the individual patient

and clinician. For those trials that are conducted to better inform a single patient about the effectiveness or side effects of a particular intervention, it is critical that the outcomes chosen for the study be the ones that will have the greatest impact on the decision to continue, terminate, or change therapy. Therefore, although a large-scale randomized trial of a therapy for osteoarthritis, for example, might choose well-validated measures of pain, stiffness, and physical function, the outcome chosen for an N-of-1 trial of osteoarthritis might be simple ordinal scales of very specific issues such as the degree of resting pain and/or the ability to dance, ride a bicycle, or walk to the grocery store. Alternatively, simple dichotomous choices or ordinal preference scores may also be used (e.g., ‘‘Which did you prefer overall, treatment with A or B?’’ and ‘‘How much better was your preferred treatment than the other one?’’). For example, Pope, et al. (12) conducted a set of N-of-1 trials of a nonsteroidal inflammatory drug for patients with osteoarthritis. Each trial consisted of 2-week treatment pairs; after each pair, the patient was asked to select the preferred study medication. If he or she was unable to do so, up to two additional blinded treatment pairs were attempted. For multiple N-of-1 trials, a more uniform choice of endpoints may be required. However, the number of patients treated with these designs is generally small, and the emphasis may still be on the relevance of the response to the individual patients. In these situations, very specific outcomes or simple choice outcomes may still be chosen, but it is desirable that the scale on which the outcomes are measured be uniform to permit easier aggregation of the data (10). 3.6 Outcome Assessment Like any clinical trial, an unbiased assessment of outcomes is critical. For fully blinded trials, achieving this objective is relatively straightforward. For incompletely blinded trials, an independent outcome assessment method should be employed whenever possible to guard against the biases that may be held by the clinician-investigator. In many trials, multiple assessments of the outcomes over time are often employed

N-OF-1 RANDOMIZED TRIALS

to increase the power of the analysis (e.g., by use of a symptom diary). This additional burden on the patient should be discussed in detail before conducting the trial to enhance adherence. Additional work by highly motivated patients, however, may not be entirely undesirable. For example, in a series of Nof-1 studies of children with attention deficit/ hyperactivity disorder, follow-up interviews with patients and their parents found very positive impressions of the experience of participating in the N-of-1 studies; these included the data collection aspect, which was viewed favorably by many patients as a means of feeling more invested in the study and providing more information (13). 4

STATISTICAL ISSUES

One of the more controversial and underdeveloped aspects of N-of-1 trials is the issue of the statistical design and analysis. Indeed, some investigators have argued that statistical analysis is often superfluous, being irrelevant to the purpose of the trial. Should statistical analysis be desired, data features such as correlation between repeated measures and unequal variances by treatment period must be addressed. 4.1 Should Statistical Analysis of an N-of-1 Study be Conducted at All? It has been suggested that a formal statistical analysis of data from an N-of-1 study need not be undertaken when the goal is to help a patient make a simple therapeutic decision (14). The argument is that the decision is unavoidable. In this context, issues of statistical significance are immaterial: the patient should simply choose the therapy that appears to be most beneficial. Because the decision will be based on evidence gathered in a structured (and, possibly, blinded) fashion, it will be better informed than any alternative, regardless of P-values and confidence intervals. Critical to this argument is the need to present the data in a manner understandable to the patient; graphical methods are often preferred, though variability in interpretation of graphical presentations can prove problematic (15).

5

This line of reasoning is not endorsed by all investigators and several methods of analysis have been proposed, as will be discussed. Even if one accepts that formal statistical testing is not relevant for some clinical applications, there may still be times when it assists decision making. For example, consider a patient at high risk for coronary heart disease who may strongly desire the protective effects of a cholesterol-lowering medication but is concerned about recent gastrointestinal upset that she believes may be caused by the drug. For this patient, there may be a need for a high degree of certainty that any attribution of the symptoms to the drug not be the result of random error, so statistical design and analysis would be helpful for this patient’s decision making. Clearly, if the goal of conducting a series of N-of-1 trials is to generalize beyond the patients in the studies, a formal statistical analysis is required. 4.2 Statistical Analysis Options for the Multiple N-of-1 Study The statistical analysis of N-of-1 studies focuses on estimation of the magnitude and significance of changes in the expected value of the response between periods of treatment change and must accommodate correlation between the repeated measures of the study subject. The repeated responses gathered in an N-of-1 study form a time series, or more accurately an interrupted time series (16), and data analysts could analyze such data using classic time series methods (17). However, the lengths of typical N-of-1 series tend to be shorter than a typical time series, and popular approaches commonly involve linear model methods instead of classic time series approaches. Rochon (18) described the following useful linear model approach to analyze data from an N-of-1 study. Let yit denote the response observed at the tth time point (t = 1, . . . , T) during the i-th treatment period (i = 1, . . . , I). For notational convenience, we assume equal treatment period lengths, but the methods extend easily to settings with unequal period lengths. Let yTi = (yi1 , . . . , yiT) denote the set of T responses gathered in the i-th treatment period and Xi denote a Txp

6

N-OF-1 RANDOMIZED TRIALS

matrix of p explanatory variables. Rochon proposes the linear model, yi = XTi β + ei , where ei = (ei1 , . . . ,eiT ) is a vector of error terms representing unexplained variation in the repeated response. This linear model is quite general and can describe features such as treatment effects, time, and carryover effects. Because the N-of-1 studies gather repeated measures on the same study subject, it is typically unreasonable to assume that the error terms are uncorrelated. Rather, one assumes that the repeated measures are correlated, often with an autoregressive structure. For example, as in Rochon (18), one might assume that the errors follow a first order autoregressive process, eit = ρi ei,t−1 + uit where ρ i is the autoregressive correlation parameter, with |ρ i | < 1 and uit as mutually independent random variables with mean zero and variance σi2 (1 − ρi2 ). Rochon and others found that both the variability and longitudinal correlation of responses from N-of-1 studies seemed to vary with treatment period (18). To address this, useful models allow separate variance and correlation parameters for each of the treatment periods. One typically estimates the model parameters and variability using maximum likelihood methods and can express the estimated magnitude of treatment effects using standard confidence interval methods. Finally, one can test that observed treatment effects are greater than expected by chance using standard hypothesis testing methods such as likelihood ratio tests and associated significance probabilities. Spiegelhalter (14) noted that patients can use such significance probabilities to assess the confidence associated with statements about the superiority of one treatment over another. 5

OTHER ISSUES

5.1 Do N-of-1 Trials Achieve Their Goals? One standard for assessing the success of Nof-1 studies is the extent to which patients accept the results of the study and adhere to the recommended therapy (which may be

no therapy at all). Several authors have measured the proportion of patients who accepted the results of their N-of-1 studies and adopted the therapy recommended by their trials in follow-up. Clearly, large numbers of patients and physicians who participate in N-of-1 studies use the information in their clinical decision making, as evidenced by substantial proportions of patients who ultimately change their choice of therapy (19–22). However, on follow-up, several investigators have found many patients ultimately elect a treatment strategy that is inconsistent with the results of their N-of-1 trial (10, 23, 24). It is unclear why rates of post-trial adherence are not higher. In a study of N-of-1 trials of quinine for the treatment of nocturnal leg cramps, the investigators found that all 10 patients completing the trials elected to use quinine after their studies, despite the fact that the drug was clearly effective only in three. The authors attributed this, in part, to the fact that they did not sufficiently discuss the implications of inconclusive or negative findings with patients prior to initiating the trials (24). Another potential explanation for low rates of adopting the apparently superior therapy after an N-of-1 trial may be that many of these trials may have been instigated by the clinician or investigator, not the patient. It is difficult to glean this aspect of trial initiation from the available publications but, as noted above, it is important that the patient perceive a value to the Nof-1 study or their adherence to its results may be diminished. This problem may also account, in part, for the relatively high rates of withdrawal from many studies (10, 23, 24), though many of these ‘‘withdrawals’’ may simply be early recognition of effectiveness or toxicity, resulting in early termination of the trial. Finally, even when no statistical testing is performed, many patients may intuit that observed differences in the trial were of small degree or the results unstable and act accordingly. Greater understanding of how N-of-1 trial results are understood and used by patients is clearly needed. Despite the inconsistent rates of adopting the superior therapy observed in some studies of N-of-1 trials, many studies attest to high levels of patient satisfaction with

N-OF-1 RANDOMIZED TRIALS

the procedure. Even when the patient’s final decision is not entirely consistent with the information gleaned from the study, most patients report that the trial was useful and provided helpful information as they made their therapeutic decisions (8, 13, 20, 21, 23). Two groups have formally compared patients concurrently randomized to an N-of-1 study versus usual care for therapeutic decision making. Mahon et al. (25) randomized 31 patients with chronic obstructive pulmonary disease with an unclear response to theophylline to take part in an N-of-1 study or usual care. At 6 months, there was less theophylline use among the patients allocated to the N-of-1 trials with no apparent decline in their pulmonary symptoms. In a later study, the same investigators randomized 68 comparable patients using a similar design. Over a 1-year follow-up, the authors found no significant differences between groups in symptoms or theophylline use (26). Finally, Pope et al. (12) randomized 51 patients with osteoarthritis to an N-of-1 study or usual care for deciding whether a non-steroidal anti-inflammatory drug was helpful for their arthritis symptoms. Over the 6-month followup period, the investigators found no differences in use of the drug and apparent response. All studies found that the N-of-1 trials increased total medical-care costs for the duration of the trial period. 5.2 Ethics of N-of-1 Trials All N-of-1 trials, regardless of technique or intent, are intervention studies. As such, the ethics of conducting an N-of-1 trial must be considered carefully. Certainly, a set of multiple N-of-1 trials designed primarily to provide generalizable data is little different from other crossover studies and the full complement of ethical safeguards is likely to be required in this context. Such safeguards include full written informed consent and approval by an institutional review board (IRB). It is less clear that all ethical protections required in typical research studies are required for N-of-1 studies that are designed to help an individual patient make a more informed personal therapeutic decision. Some authors contend that written, informed

7

consent and IRB approval are not necessary, since these studies are simply more structured methods of conducting therapeutic trials, a procedure performed regularly in routine clinical practice (27). 5.3 N-of-1 Trial Support Services Single-patient studies are not commonly performed. Numerous barriers exist to wider application of these methods: unfamiliarity on the part of patients and clinicians, hesitancy to conduct ‘‘studies’’ in the clinical setting, and additional burdens of time and trouble to carry out the study to make ‘‘simple’’ clinical decisions. One of the greatest barriers to wider implementation is the lack of infrastructure for performing these studies: placebos (or indistinguishable active drugs) must be formulated, a randomization scheme must be prepared (to which the clinician and patient are blind), and data must be collected and presented and/or analyzed. Few clinicians have access to these services or expertise (28). Recognizing this difficulty, some academic institutions have established ‘‘N-of-1 Services’’ that support clinicians who desire to use these methods. Investigators at McMaster University and the University of Washington have published their experiences with such services in the 1980s and 1990s (3, 20, 29, 30). Both groups of investigators found the service to be feasible and the results useful to patients and clinicians. A more ambitious effort was organized by Nikles et al. (22) in Australia. These investigators established an N-of-1 trial support service that was available to clinicians throughout the country who desired to carry out a single-patient trial in their practice. The success of this venture was documented in the completion of a large number of trials in conditions such as attention deficit/hyperactivity disorder and osteoarthritis (4, 13, 22). 5.4 Novel Uses of N-of-1 Studies The utility of N-of-1 studies enables a wide variety of applications. In addition to the common use for informing clinical decisions for individual patients, N-of-1 studies have found value in other settings.

8

N-OF-1 RANDOMIZED TRIALS

In one recent randomized, double-blind, parallel-comparison clinical trial, a participant considered withdrawing from the study because he was concerned that the study medication raised his blood pressure. Rather than conduct a simple therapeutic trial of withdrawing the study medication, the investigators offered the participant the option of taking part in an N-of-1 study (using the study medication and known placebo), which he selected. The N-of-1 study showed that, relative to placebo, his assigned study medication had minimal effects on his blood pressure, and he elected to continue in the trial of the study medication (31). It is notable that such N-of-1 studies are relatively simple to perform because there are no problems with the availability of placebo, a research pharmacist, data collection infrastructure, and personnel for presenting and analyzing the data. Guyatt et al. (32) describe another potential use of the multiple N-of-1 method: as a means of gathering critical information about a new drug in development. Such trials can be designed to provide a wide range of information useful for planning larger, fully powered phase III studies. The kinds of information gleaned from such studies include defining the population of patients most likely to benefit from the drug, determining the optimal dose, estimating the rapidity of the drug’s onset of action and loss of treatment effect, identifying the types of outcomes most responsive to the drug, and estimating the magnitude of a potential treatment effect. The investigators provided an example of the use of a tricyclic antidepressant medication for the treatment of fibromyalgia. The same technique has also been used in the initial study of a device for treatment of Tourette’s syndrome (33). 6

CONCLUSIONS

Although N-of-1 trials have been employed in the mental health field for many decades, interest has grown in wider applications of the method in recent years. Because the focus of this type of study is on the individual patient, there are several unique aspects to the design, conduct, and interpretation of the

results of these studies. Serious infrastructure barriers to wider implementation often exist, as do issues of adherence and optimizing the value of the N-of-1 trial for its intended purpose. Analytic issues continue to evolve, and, as clinicians and patients become more familiar with the technique, N-of-1 trials may play a more prominent role in the application of evidence-based, individualized therapeutics. Patient satisfaction with N-of1 trials is high, but the incremental benefit of this methodology over standard practice remains uncertain. REFERENCES 1. G. Guyatt, D. Sackett, D. W. Taylor, J. Chong, R. Roberts, and S. Pugsley, Determining optimal therapy—randomized trials in individual patients. N Engl J Med. 1986; 314: 889–892. 2. R. Jaeschke, D. Cook, and D. L. Sackett, The potential role of single-patient randomized controlled trials (N-of-1 RCTs) in clinical practice. J Am Board Fam Pract. 1992; 5: 227–229. 3. E. B. Larson, N-of-1 clinical trials. A technique for improving medical therapeutics. West J Med. 1990; 152: 52–56. 4. C. J. Nikles, M. Yelland, P. P. Glasziou, and C. Del Mar, Do individualized medication effectiveness tests (N-of-1 trials) change clinical decisions about which drugs to use for osteoarthritis and chronic pain? Am J Ther. 2005; 12: 92–97. 5. P. M. Peloso, Are individual patient trials (Nof-1 trials) in rheumatology worth the extra effort? J Rheumatol. 2004; 31: 8–11. 6. B. Spilker, Single-patient clinical trials. In: Guide to Clinical Trials. Philadelphia: Lippincott-Raven, 1996, pp. 277–282. 7. D. J. Cook, Randomized trials in single subjects: the N of 1 study. Psychopharmacol Bull. 1996; 32: 363–367. 8. D. R. Zucker, R. Ruthazer, C. H. Schmid, J. M. Feuer, P. A. Fischer, et al., Lessons learned combining N-of-1 trials to assess fibromyalgia therapies. J Rheumatol. 2006; 33: 2069–2077. 9. G. Guyatt, D. Sackett, J. Adachi, R. Roberts, J. Chong, D., et al., A clinician’s guide for conducting randomized trials in individual patients. CMAJ. 1988; 139: 497–503. 10. A. C. Wegman, D. A. van der Windt, W. A. Stalman, and T. P. de Vries, Conducting research in individual patients: lessons learnt from two

N-OF-1 RANDOMIZED TRIALS series of N-of-1 trials. BMC Fam Pract. 2006; 7: 54. 11. D. R. Zucker, C. H. Schmid, M. W. McIntosh, R. B. D’Agostino, H. P. Selker, and J. Lau, Combining single patient (N-of-1) trials to estimate population treatment effects and to evaluate individual patient responses to treatment. J Clin Epidemiol. 1997; 50: 401–410. 12. J. E. Pope, M. Prashker, and J. Anderson, The efficacy and cost effectiveness of N of 1 studies with diclofenac compared to standard treatment with nonsteroidal antiinflammatory drugs in osteoarthritis. J Rheumatol. 2004; 31: 140–149. 13. C. J. Nikles, A. M. Clavarino, and C. B. Del Mar, Using N-of-1 trials as a clinical tool to improve prescribing. Br J Gen Pract. 2005; 55: 175–180. 14. D. J. Spiegelhalter, Statistical issues in studies of individual response. Scand J Gastroenterol Suppl. 1988; 147: 40–45. 15. K. J. Ottenbacher and S. R. Hinderer, Evidence-based practice: methods to evaluate individual patient improvement. Am J Phys Med Rehabil. 2001; 80: 786–796. 16. T. D. Cook TD and D. T. Campbell, Quasiexperimentation: Design And Analysis Issues. Boston: MA: Houghton-Mifflin, 1979. 17. P. J. Diggle, Time Series: A Biostatistical Introduction. Oxford: Clarendon Press, 1990. 18. J. Rochon, A statistical model for the ’’N-of-1’’ study. J Clin Epidemiol. 1990; 43: 499–508. 19. G. H. Guyatt and R. Jaeschke, N-of-1 randomized trials—where do we stand? West J Med. 1990; 152: 67–68. 20. E. B. Larson, A. J. Ellsworth, and J. Oas, Randomized clinical trials in single patients during a 2-year period. JAMA. 1993; 270: 2708–2712. 21. L. March, L. Irwig, J. Schwarz, J. Simpson, C. Chock, and P. Brooks, N of 1 trials comparing a non-steroidal anti-inflammatory drug with paracetamol in osteoarthritis. BMJ. 1994; 309: 1041–1046. 22. C. J. Nikles, G. K. Mitchell, C. B. Del Mar, A. Clavarino, and N. McNairn, An N-of-1 trial service in clinical practice: testing the effectiveness of stimulants for attention-deficit/hyperactivity disorder. Pediatrics. 2006; 117: 2040–2046. 23. M. A. Kent, C. S. Camfield, and P. R. Camfield, Double-blind methylphenidate trials: practical, useful, and highly endorsed by families. Arch Pediatr Adolesc Med. 1999; 153: 1292–1296.

9

24. R. Woodfield, F. Goodyear-Smith, and B. Arroll, N-of-1 trials of quinine efficacy in skeletal muscle cramps of the leg. Br J Gen Pract. 2005; 55: 181–185. 25. J. Mahon, A. Laupacis, A. Donner, and T. Wood, Randomised study of N of 1 trials versus standard practice. BMJ. 1996; 312: 1069–1074. 26. J. L. Mahon, A. Laupacis, R. V. Hodder, D. A. McKim, N. A. Paterson, et al., Theophylline for irreversible chronic airflow limitation: a randomized study comparing N of 1 trials to standard practice. Chest. 1999; 115: 38–48. 27. L. Irwig, P. Glasziou, and L. March, Ethics of N-of-1 trials. Lancet. 1995; 345: 469. 28. K. B. Saunders, N of 1 trials. BMJ. 1994; 309: 1584. 29. G. H. Guyatt, J. L. Keller, R. Jaeschke, D. Rosenbloom, J. D. Adachi, and M. T. Newhouse, The N-of-1 randomized controlled trial: clinical usefulness. Our three-year experience. Ann Intern Med. 1990; 112: 293–299. 30. J. L. Keller, G. H. Guyatt, R. S. Roberts, J. D. Adachi, and D. Rosenbloom, An N of 1 service: applying the scientific method in clinical practice. Scand J Gastroenterol Suppl. 1988; 147: 22–29. 31. A. L. Avins, S. Bent, and J. M. Neuhaus, Use of an embedded N-of-1 trial to improve adherence and increase information from a clinical study. Contemp Clin Trials. 2005; 26: 397–401. 32. G. H. Guyatt, A. Heyting, R. Jaeschke, J. Keller, J. D. Adachi, and R. S. Roberts, N of 1 randomized trials for investigating new drugs. Control Clin Trials. 1990; 11: 88–100. 33. J. L. Houeto, C. Karachi, L. Mallet, B. Pillon, J. Yelnik, et al., Tourette’s syndrome and deep brain stimulation. J Neurol Neurosurg Psychiatry. 2005; 76: 992–995.

CROSS-REFERENCES Crossover design Randomization Placebo-controlled trial Washout period Blinding

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

Non-compartmental analysis (NCA) is a standard technique to determine the pharmacokinetics (PK) of a drug. After drug intake, the concentration time profiles (e.g., in plasma or serum) are recorded and used to characterize the absorption, distribution, and elimination of the drug. Less frequently, concentrations in blood, saliva, other body fluids, or amounts excreted unchanged in urine are used instead of or in addition to plasma or serum concentrations. NCA is the most commonly used method of PK data analysis for certain types of clinical studies like bioequivalence, dose linearity, and food effect trials. The common feature of non-compartmental techniques is that no specific compartmental model structure is assumed. The most frequently applied method of NCA is slope height area moment analysis (SHAM analysis) (1,2). For SHAM analysis, the area under the concentration time curve (AUC) is most commonly determined by numerical integration or by curve fitting. Numerical integration is the non-compartmental method of choice for analysis of concentration time data after extravascular input because absorption kinetics are often complex. In comparison to compartmental modeling, numerical integration has the advantage that it does not assume any specific drug input kinetics. For intravenous bolus data, fitting the concentration time curves by a sum of exponentials is the non-compartmental method of choice. This introductory article presents some standard applications of NCA of plasma (or serum) concentration data, as those applications are most commonly used. References to NCA of urinary excretion data and more advanced topics of NCA are provided. This article focuses on exogenous compounds and does not cover endogenous molecules. Our focus is (1) to describe studies and objectives for which NCA is well suited, (2) to provide and discuss assumptions of NCA, and (3) to present a practical guide for performing an NCA by numerical integration and basic approaches to choosing appropriate blood sampling times.

¨ JuRGEN B. BULITTA

Department of Pharmaceutical Sciences, School of Pharmacy and Pharmaceutical Sciences, State University of New York at Buffalo, Buffalo, New York

NICHOLAS H. G. HOLFORD Department of Pharmacology and Clinical Pharmacology, University of Auckland, Auckland, New Zealand

Objectives of NCA are often assessing dose proportionality, showing bioequivalence, characterizing drug disposition, and obtaining initial estimates for pharmacokinetic models. Specific results and applications of NCA are as follows: (1) The area under the concentration time curve (e.g., in plasma or serum) describes the extent of systemic drug exposure; the peak concentration and its timing indicate the rate of drug input (absorption), and (2) NCA provides estimates for clearance, volume of distribution, terminal half-life, mean residence time, and other quantities. Application 1 serves purely descriptive purposes and requires almost no assumptions. Importantly, application 2 does rely on several assumptions that are similar to the assumptions for compartmental modeling. Standard NCA requires frequent (blood) samples to estimate pharmacokinetic parameters reliably. Numerical integration is most commonly used for NCA after extravascular input. Fitting of disposition curves by a sum of exponential functions, for example is the method of choice for intravenous bolus input. NCA of an adequately designed clinical trial can provide robust estimates for extent of drug exposure and rate of absorption and other quantities. NCA estimates for clearance and volume of distribution rely on several assumptions that have to be critically considered for appropriate interpretation of NCA results.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

1 TERMINOLOGY 1.1 Compartment A compartment is a hypothetical volume that is used to describe the apparent homogeneous and well-mixed distribution of a chemical species. ‘‘Kinetically homogeneous’’ assumes that an instantaneous equilibration of a chemical compound (drug or metabolite) is found between all components of the compartment. 1.2 Parameter A parameter is a primary constant of a quantitative model that is estimated from the observed data. For example, clearance (CL) and volume of distribution at steady state (Vss) are the two most important PK parameters. 1.3 Fixed Constant Some models include fixed constants that are known (or assumed) a priori and are not estimated. Examples are stoichiometric coefficients or π . 1.4 Statistic A statistic is a derived quantity that is computed from observed data or estimated model parameters. Examples: The average plasma concentration is a statistic because it is computed from observed concentrations. Another statistic is the AUC. Under certain assumptions, the AUC is F · Dose / CL with F being the fraction of drug that reaches the systemic circulation. The average clearance and its standard deviation are two statistics that can be calculated from individual clearance estimates of several subjects. 1.5 Comment A one-compartment PK model can be defined by any two parameters, for example, by CL and Vss, by CL and t1/2 , or by Vss and t1/2 . NCA estimates CL and t1/2 and derives Vss from other NCA statistics. However, physiologically, the parameterization by CL and Vss is more informative than the other two parameterizations because CL and Vss characterize the physiology of the body and the

physicochemical properties of the drug. Halflife is determined by the primary PK parameters clearance and volume of distribution and should be called a statistic (or secondary parameter). For more complex PK models, Vss is calculated by residence time theory (see below and Reference 3). 2 OBJECTIVES AND FEATURES OF NON-COMPARTMENTAL ANALYSIS Non-compartmental techniques can be applied to PK analyses of studies with a variety of objectives. Some of those objectives are: 1. Characterization of drug disposition. 2. PK analysis of various types of studies, including: • Bioavailability and bioequivalence studies, • Food-effect studies, • PK interaction studies, and • Dose-proportionality studies. 3. Supporting development of pharmaceutical formulations by characterizing drug absorption profiles. 4. Obtaining initial estimates for compartmental modeling. NCA has appealing features, and the majority of studies in literature only report results from NCA. Some specific results from NCA and its features are: 1. NCA can provide robust estimates for important PK parameters like clearance and volume of distribution, if the assumptions of NCA are adequately met. 2. Descriptive measures for the rate of absorption like peak concentration (Cmax) and its timing (Tmax) can be directly derived from the observed data. 3. No need exists to specify the full structure of the compartmental model. 4. NCA and plots of the observed concentration time data may provide valuable insights into the PK characteristics of a drug and can be very helpful for testing model assumptions and for building compartmental models.

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

5. Standard NCA can be learned quickly and is often straightforward to apply. 6. NCA requires minimum decision making by the user, can be highly standardized, and most often yields consistent results between different users. However, it is important to be aware that NCA may be suboptimal or more difficult to apply in situations that need: 1. to analyze sparse data (insufficient to characterize the shape of the profile before the log-linear terminal phase), 2. to derive PK parameters from concentration time profiles of complex dosage regimens, 3. to simulate other-than-the-studied dosage regimens [powerful methods do exist for this task, but they are underused (4)], or 4. to study drugs with mixed-order (‘‘saturable’’) elimination or time-dependent PK. 2.1 Advanced Non-Compartmental Techniques Various advanced non-compartmental techniques have been developed. A detailed presentation of these concepts is beyond the scope of this article. Some situations for which those advanced methods are superior to standard NCA are listed below. Non-compartmental methods to analyze data with saturable elimination (Michaelis– Menten kinetics) were developed by several authors (5–13). Methods for the analysis of metabolite data (14–18), reversible metabolism (19–27), maternal–fetal disposition (28), enterohepatic recirculation (29), nonlinear protein binding (30) or capacity limited tissue-distribution (31), organ clearance models (19,32), target-mediated drug disposition (33,34), and for modeling a link compartment (35) are available. Noncompartmental methods for sparse sampling (36–43) are often applied in preclinical PK and toxicokinetics. Veng-Pedersen presented (4,44) an overview of linear system analysis that is a powerful tool to determine the absorption, distribution, and elimination

3

characteristics and to predict concentrations for other dosage regimens. Although these advanced methods have been developed, they are not as widely used as standard NCA, and some of these methods are not available in standard software packages. 3 COMPARISON OF NON-COMPARTMENTAL AND COMPARTMENTAL MODELS NCA is often called ‘‘model independent,’’ although this phrase is misleading and often misinterpreted (45–47). The common feature of non-compartmental techniques is that they do not assume a specific compartmental model structure. NCA only can be described as (fully) model independent when it is exclusively used for descriptive purposes, for example, for recording the observed peak concentration (Cmax) and its associated time point (Tmax). Importantly, such a descriptive model-independent approach cannot be applied for extrapolation purposes. The AUC from time zero to infinity (AUC0−∞ ) can be interpreted as a descriptive measure of systemic drug exposure (e.g., in plasma) for the studied dosage regimen. This interpretation requires the assumption of a monoexponential decline in concentrations during the terminal phase (or another extrapolation rule). However, the AUC is commonly used to calculate F, for example, in bioavailability and bioequivalence studies. As the AUC is determined by F · dose / clearance, the use of AUC to characterize F assumes that clearance is constant within the observed range of concentrations and over the time course of the data used to calculate the AUC. Therefore, all assumptions required for determination of clearance (see below) are implicitly made, if AUC is used to characterize F. Figure 1 compares the standard noncompartmental model and a compartmental model with two compartments. An important assumption of NCA is that drug is only eliminated from the sampling pool. The two models shown in Fig. 1 differ in the specification of the part of the model from which no observations were drawn (nonaccessible part of the system). In this example,

4

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

Figure 1. Comparison of the standard non-compartmental model and a compartmental model with two compartments.

the models differ only in the drug distribution part. For compartmental modeling, the user has to specify the complete model structure including all compartments, drug inputs, and elimination pathways. In Fig. 1 (panel b), drug distribution is described by one peripheral compartment that is in equilibrium with the central compartment. The ‘‘non-compartmental’’ model does not assume any specific structure of the nonaccessible part of the system. Any number of loops can describe the distribution, and each loop can contain any number of compartments. A more detailed list of the assumptions of standard NCA is shown below.

4 ASSUMPTIONS OF NCA AND ITS REPORTED DESCRIPTIVE STATISTICS NCA parameters like clearance and volume of distribution only can be interpreted as physiological or pharmacological properties of the body and the drug of interest, if the assumptions of NCA are adequately met. NCA relies on a series of assumptions. Violating these assumptions will cause bias in some or all NCA results. This bias has been quantified and summarized by DiStefano and Landaw (45,46) for several examples. Table 1 compares several key assumptions (36–50) between standard NCA and compartmental modeling. One key assumption of standard NCA is linear drug disposition [see Veng-Pedersen (44,50), Gillespie (47), and DiStefano and Landaw (46) for details]. Although advanced methods to account for

nonlinear drug disposition have been developed [see, e.g., Cheng and Jusko (8)], these techniques are seldom applied. As shown in Table 1, the assumptions for a compartmental model with first-order disposition (‘‘linear PK’’) are similar to the assumptions of standard NCA. Compartmental models offer the flexibility of specifying various types of nonlinear disposition. This process is straightforward, if a compartmental model is specified as a set of differential equations. Below is a discussion of the assumptions of NCA and compartmental modeling. 4.1 Assumptions 1 (see Table 1): Routes and Kinetics of Drug Absorption Compartmental modeling uses a parametric function (e.g., first-order or zero-order kinetics) for drug absorption, whereas NCA does not assume a specific time course of drug input. 4.2 Assumptions 2 to 4 (see Table 1): Drug Distribution Standard NCA does not assume a specific model structure for the nonaccessible part of the system (‘‘distribution compartments’’). It only assumes that drug distribution is linear, for example, neither the rate nor the extent of distribution into the peripheral compartment(s) is saturable. The user has to specify the complete structure and kinetics of drug transfer between all distribution compartments for compartmental modeling. Nonlinear drug disposition can be incorporated into compartmental models. Most compartment models assume linear

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

Table 1. Comparison of Assumptions Between Standard NCA and Compartmental Modeling

5

6

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

distribution. In this case, the assumptions for drug distribution are similar for noncompartmental and compartmental methods. 4.3 Assumption 5 (see Table 1): Routes of Drug Elimination Standard NCA assumes that all elimination occurs only from the sampling pool. Most compartmental models also assume that drug is eliminated only from the sampling (central) compartment (no elimination from any peripheral compartment). This assumption seems reasonable because the liver and the kidneys as the two main elimination organs are highly perfused organs and therefore in a rapid equilibrium with plasma (or serum), which is most commonly used for PK analysis. DiStefano and Landaw (45,46) discussed this assumption and the implications of its violation in detail. Other routes of elimination can be specified for compartmental modeling. Nakashima and Benet (51) derived formulas for linear mammillary compartmental models with drug input into any compartment and drug elimination from any compartment. 4.4 Assumption 6 (see Table 1): Kinetics of Drug Elimination Standard NCA assumes that clearance is not saturable. As metabolism and renal tubular secretion are potentially saturable, the results of standard NCA for drugs with saturable elimination need to be interpreted cautiously. This cautious interpretation is especially important when plasma concentrations exceed the Michaelis– Menten constant of the respective saturable process. Several NCA methods that can account for saturable elimination are quoted above. However, those methods are not implemented in software packages like WinNonlinTM Pro or Thermo Kinetica , which makes them unavailable to most users. Compartmental modeling offers a great flexibility for specifying the elimination pathways and the kinetics of the elimination process (e.g., first-order, zero-order, mixedorder, target-mediated, etc.)

4.5 Assumptions 7 and 8 (see Table 1): Sampling Times and Monoexponential Decline For adequately chosen sampling times, the assumption that the last three (or more) observed concentrations characterize a monoexponential decline of concentrations during the terminal phase is often reasonable [see Weiss for details (52)]. Standard NCA requires that the samples be collected during the whole concentration time profile. This time period is usually one dosing interval at steady state or at least three terminal halflives after a single dose (see below for NCA of sparse data). For studies with frequent sampling (12 or more samples per profile) at appropriately chosen time points, the AUC usually characterizes the total drug exposure well. For single-dose studies, it should be ensured that the AUC from time zero until the last observed concentration (AUC0−last ) comprises at least 80% of AUC0−∞ . Blood sampling times should be chosen so that at least two or more observations will be observed before the peak concentration in all subjects. If the peak concentration is at the time of the first observation, then regulatory authorities like the Food and Drug Administration (FDA) (53,54) may require that a study be repeated because the safety of the respective formulation could not be established. In this situation, NCA cannot provide a valid measure for peak concentrations. Blood sampling time schedules can be optimized to select the most informative sampling times for estimation of compartmental model parameters. The latter approach is more efficient as it requires (much) fewer blood samples per patient compared with NCA and is especially powerful for population PK analyses. 4.6 Assumption 9 (see Table 1): Time Invariance of Disposition Parameters This assumption is made very often for both NCA and compartmental modeling. The disposition parameters are assumed to be timeinvariant (constant) within the shortest time interval required to estimate all PK parameters of interest. PK parameters may differ between two dosing intervals because

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

two different sets of PK parameters can be estimated for each dosing interval both by NCA and compartmental methods. Caution is indicated for data analysis of drugs with inducible metabolism or when comedication may affect hepatic metabolism of the drug. In summary, the assumptions of compartmental models with linear PK and of the standard NCA model are similar, if NCA is used to derive PK parameters like CL, Vss, and F (as is most often done). The complete model structure and kinetics of all drug transfer processes must be defined for compartmental models, whereas NCA requires fewer assumptions on the model structure. Both standard compartmental models and the standard NCA model assume linear PK [see Veng-Pedersen (44,50), Gillespie (47), and DiStefano and Landaw (46) for more details]. It is easier to specify nonlinear drug disposition for compartmental models than for NCA. 4.7 Assumptions of Subsequent Descriptive Statistics NCA does not directly make any assumption about the distribution of NCA statistics between different subjects. However, most authors report the average and standard deviation for the distribution of NCA statistics and thereby assume that the average is an appropriate measure for the central tendency and that the standard deviation characterizes the variability adequately. The distribution of Tmax is especially problematic because it is determined primarily by the discrete distribution of the nominal sampling times. It is often not possible to decide whether the between-subject variability follows a normal distribution, log-normal distribution, or another distribution. Therefore, it is often helpful to report the median and representative percentiles (e.g., 5% to 95% percentile and 25% to 75% percentile) to describe the central tendency and variability of data by nonparametric statistics in addition to the average and standard deviation. If only data on a few subjects is available, then the 10% and 90% percentiles or the interquartile range are usually more appropriate than the 5% and 95% percentiles. Reporting only the

7

median and range of the data does not provide an accurate measure for dispersion, especially for studies with a small sample size. 5

CALCULATION FORMULAS FOR NCA

This section presents numerical integration methods to determine the AUC that is most commonly used to analyze plasma (or serum) concentrations after extravascular dosing. Additionally, fitting of concentration time curves by a sum of exponential functions is described. The latter is the non-compartmental method of choice for iv bolus input. The concentration time curves after iv bolus input are called disposition curves because drug disposition comprises drug distribution and elimination but not the absorption process. Both methods are noncompartmental techniques because they do not assume a specific model structure. It is important to note that several compartmental models will result in bi-exponential concentration time profiles. It is appealing that the numerical integration of a standard NCA can be performed with a hand calculator or any spreadsheet program. Nonlinear regression to describe disposition curves can be done by standard software packages like ADAPT, WinNonlinTM Pro, Thermo Kinetica , SAAM II, EXCEL and many others. Performing an NCA with a hand calculator or self-written spreadsheets is good for learning purposes. However, the use of validated PK software for NCA of clinical trials is vital for projects to be submitted to regulatory authorities. Audit trail functionality is an important advantage of validated PK software packages. 5.1 NCA of Plasma or Serum Concentrations by Numerical Integration Numerical integration is the most commonly used method to determine the AUC and other moments of the plasma (or serum) concentration time curve after extravascular input. This method is also often applied for infusion input, if the duration of infusion is nonnegligible. Numerical integration requires a function neither to describe the whole concentration time curve nor to describe the rate of

8

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

drug absorption (e.g., first-order or zero-order input). The Cmax and Tmax are directly recorded from the observed concentration time data. NCA assumes that the concentration during the terminal phase declines monoexponentially. Subsequently, linear regression on semilogarithmic scale is used to determine the slope (−λz ) of the concentration time curve during the terminal phase. Terminal half-life is calculated as t1/2 =

ln(2) λz

A guide for the most appropriate selection of observations for estimation of terminal halflife is described below. Figure 2 illustrates the calculation of terminal half-life for one subject. In this example, the last five observations were used to estimate the terminal slope. The area under the plasma concentration time curve is most often determined by the trapezoidal method. The plasma concentration time profile is usually linearly (or logarithmically) interpolated between two observations. The formulas apply to linear and logarithmic interpolation, and Ci denotes the ith observed concentration: Linear interpolation: Ci + Ci+1 2 Ci + Ci+1 = (ti+1 − ti ) · 2 Logarithmic interpolation: = t · AUCi+1 i

= t · AUCi+1 i

Ci+1 − Ci 
Ci+1 ln Ci

Figure 3 shows the calculation of trapezoids for data after a single oral dose by the linear interpolation rule. The sum of the individual trapezoids yields the AUC from time zero to the last quantifiable concentration: AUC0−last =

n−1 

AUCi+1 (for n observations) i

i=1

Linear interpolation is usually applied, if concentrations are increasing or constant

(Ci+1 ≥ Ci ), and logarithmic interpolation is often used, if concentrations are decreasing (Ci+1 < Ci ). Note that the log-trapezoidal rule is invalid when Ci+1 = Ci and when Ci is zero. Plasma concentration time profiles are curved to the right during an intravenous infusion at a constant rate (see Fig. 4) and also tend to be curved to the right for extravascular administration. Therefore, linear interpolation to calculate the AUC is a reasonable approximation if Ci+1 ≥ Ci . After the end of infusion, drug concentrations often decline mono-, bi-, or triexponentially (see Fig. 4). The number of exponential phases can be assessed visually, when drug concentrations are plotted on log scale versus time. Therefore, assuming an exponential decline (equivalent to a straight line on log-scale) is often a good approximation for the AUC calculation if Ci+1 < Ci . Figure 4 shows that linear interpolation approximates the true plasma concentration time curve slightly better than the logarithmic interpolation during the infusion. After the end of infusion, logarithmic interpolation approximates the true curve better then linear interpolation—as expected. The difference between linear and logarithmic interpolation tends to be small, if blood samples are frequently drawn. Differences are expected to be larger for lessfrequent blood sampling. Linear and logarithmic interpolation have been compared with interpolation by higher-order polynomials and splines (55,56). For PK data, the linear trapezoidal rule for increasing or constant concentrations and the log-trapezoidal rule for decreasing concentrations is usually a good choice (19) because those methods are stable and reasonably accurate even for sparse data. Although higher-order polynomials and spline functions performed better for some datasets, these methods are potentially more sensitive to error in the data and are not available in most standard software packages. The AUC from the last concentration (Clast ) to time infinity (AUClast−∞ ) is extrapolated assuming a log-linear decline of concentrations during the terminal phase: C . AUClast−∞ = λlast z Standard software packages like WinNonlinTM Pro, Thermo Kinetica ,

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

9

Figure 2. Determination of terminal half-life after a single oral dose.

and others provide two options for specifying Clast : One can use the last observed concentration as Clast . Alternativley, the concentration predicted by log linear regression at the time of the last observation may be used as Clast to estimate λz . The current FDA guideline for bioavailability and bioequivalence studies (53) recommends that the last ‘‘measured’’ (observed) concentration should be used. The FDA guideline does not provide a reason why this method is proposed. Theoretically, the last predicted concentration will be more precise than an observed value if the assumptions for estimation of the terminal half-life are met. Use of the last observed concentration may yield a large bias, if this concentration was an ‘‘outlier’’ with a high value, because it would cause a longer half-life and therefore would affect both terms in the equation for AUClast−∞ . The AUC from time zero to infinity (AUC0−∞ ) is calculated as: AUC0−∞ = AUC0−last + AUClast−∞ The residual area is calculated as 1 − AUC0−last / AUC0−∞ . Residual areas above

Figure 3. Calculation of the AUC by the linear trapezoidal rule.

20% are considered too large for reliable estimation of AUC0−∞ and therefore also for CL. The uncertainty in the calculated area under the first moment concentration time curve from time zero to infinity (AUMC0−∞ ) is even larger compared with the uncertainty in AUC0−∞ for larger residual areas. The (apparent) total clearance (CL/F) is calculated from the administered dose and AUC0−∞ as: Dose CL = F AUC0−∞ For intravenous administration, the extent of drug reaching the systemic circulation (F) is 100% by definition, and the ratio of dose and AUC0−∞ yields total clearance (CL). As F is mathematically not identifiable after extravascular administration without an intravenous reference dose, only CL/F can be derived after extravascular administration. The volume of distribution during the terminal phase (Vz) can be calculated from CL and λz . As shown by several authors (57,58), Vz is not an independent parameter that characterizes drug disposition because it depends on, for example, the estimate for

10

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

Figure 4. Interpolation between plasma concentration time points by the linear and logarithmic trapezoidal method for a 2-h infusion (sampling times: 0, 0.5, 2, 2.5, 4, and 6 h).

clearance. Volume of distribution at steady state (Vss) is a better measure for volume of distribution than Vz. The estimate for clearance does not affect the estimate of Vss, if drug is only eliminated from the sampling pool. Vss can always be calculated from data that allow one to calculate Vz (see below). Gobburu and Holford (58) pointed out that the finding of an altered Vz, for example, in a special patient population may lead to the inappropriate conclusion of changing the loading dose in this patient population. Because of the potential misuse of Vz, the use of Vz should be discouraged (58). Statistical moment theory (59) is used to calculate the mean residence time (MRT). The MRT is calculated from AUMC0−∞ and AUC0−∞ . The AUMC is calculated as:

Linear interpolation: ti · Ci + ti+1 · Ci+1 2 Logarithmic interpolation: AUMCi+1 = t · i

= t · AUMCi+1 i

ti+1 · Ci+1 − ti · Ci
 Ci+1 ln Ci

− t2 ·

Ci+1 − Ci 2
Ci+1 ln Ci

The AUMC from time zero to the last observed concentration (AUMC0−last ) and from time zero to infinity (AUMC0−∞ ) are calculated

as: AUMC0−last =

n−1 

AUMCi+1 i

i=1

(for n observations) AUMC0−∞ = AUMC0−last +

Clast · tlast Clast + λz λz 2

For steady-state data (dosing interval: τ ), the AUMC0−∞ can be calculated as (60): AUMC0−∞ = AUMCSS,0−τ + τ · AUCSS,τ −∞ The mean residence time (MRT) is calculated as: AUMC0−∞ MRT = AUC0−∞ [The AUMC0−last and AUC0−last , should not be used for calculation of MRT, because such a calculation yields systematically biased (low) estimates for the MRT.] For an intravenous bolus administration, the MRT is equal to the mean disposition residence time (MDRT). The MDRT is the average time a drug molecule stays in the systemic circulation (44). The MDRT is often called MRTiv (iv indicates iv bolus input). We prefer to write MDRT to refer specifically to drug disposition. If peripheral venous blood is sampled, then the mean transit time of drug molecules from arterial blood to the sampling site needs to be considered for calculation of MDRT, at least for drugs with a short MDRT [see Weiss (3,61) and Chiou (62) for details].

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

Vss is calculated as: Vss = MDRT · CL For high-clearance drugs, calculation of Vss is more complex (3,19,32,63). The MDRT determines the accumulation ratio (RA ), which is defined as the average amount of drug in the body at steady state (after multiple administration) divided by the bioavailable maintenance dose (F · MD). The RA is MDRT divided by the dosing interval (3). Furthermore, MDRT determines the accumulation time and the washout time as described by Weiss (64,65). For noninstantaneous drug input (e.g., extravascular dose or constant rate infusion), the ratio of AUMC0−∞ and AUC0−∞ yields the mean total residence time (MTRT), which is the sum of the mean input time (MIT) and MDRT. For constant rate (intravenous) infusion with a given duration of infusion (Tinf ), the MIT equals Tinf / 2. Therefore, the MDRT can be calculated as: MDRT = MTRT − MIT =

Tinf AUMC0−∞ − AUC0−∞ 2

The MIT after extravascular administration is more difficult to calculate without an intravenous reference. The MIT can be determined as the difference of MTRT and MDRT, when both an extravascular and intravenous dose are given to the same subject on different occasions. To calculate Vss after extravascular administration, the MIT needs to be subtracted from the MTRT (66,67). The MIT can be approximated by Tmax/2 if the input process is close to zero-order [common for drugs in class I of the biopharmaceutical classification system (BCS) because of a zeroorder release of drug from stomach; BCS class I drugs have high solubility and high permeability]. If the process seems to be first-order, then MIT can be approximated by Tmax/(3·ln(2)) because Tmax commonly occurs at around three times the first-order absorption half-life. The extent of bioavailability (F) is typically determined by giving a single oral dose and a single intravenous dose on two different occasions (with an appropriate washout

11

period) to the same subject. From these data, CL, Vss, MDRT, MIT, and F can be calculated by the formulas shown above. The ratio of the AUC after oral and intravenous dosing characterizes F: F=

AUC0−∞,oral AUC0−∞,iv

The MIT values for each subject can be calculated as the difference of MTRT and MDRT. Assuming a first-order absorption (without lag time), MIT is the inverse of the firstorder absorption rate constant (1/ka). The MIT can be correlated with the mean dissolution time determined in vitro to establish an in vitro/in vivo correlation, for example, to compare the release characteristics of various modified release formulations (68,69). 5.2 NCA of Plasma or Serum Concentrations by Curve Fitting For NCA of concentration time curves after iv bolus administration, the results from numerical integration are very sensitive to the timing of samples and to errors in the data (e.g., measurement error or undocumented deviations in the sampling time). For such data, use of analytical functions to fit the disposition curves is the method of choice. For monotonously decreasing concentrations, a sum of exponential functions is often a good choice to describe disposition curves. No a priori reason exists for choosing a sum of exponential functions. Some authors used other functions like gamma curves (52,70–72). Most commonly, disposition curves (concentration CD (t) at time t) are described by a sum of n exponential functions: CD (t) =

n 

Ci · e−λi ·t

i=1

The Ci is the intercept of each exponential phase, and −λi is the associated slope on semilogarithmic scale. The kth moment (MOCk ) of the disposition curve can be calculated as (3): MOCk = k!

n  i=1

λi

Ci (k+1)

for k = 0, 1, 2, 3, . . .

12

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

regression is beyond the scope of this article. See Gabrielsson and Weiner (73) and the manuals of the respective software packages for details. 5.3 NCA with Plasma or Serum Concentrations and Amounts in Urine

Figure 5. Disposition curve after iv bolus administration with three exponential functions.

As AUC is the area under the zeroth-moment concentration time curve and AUMC is the area under the first-moment concentration time curve, this formula yields: AUC0−∞ =

n  Ci , λi

AUMC0−∞ =

i=1

MDRT =

n  Ci i=1

λ2i

,

n  Ci λi 2

AUMC0−∞ i=1 = n  Ci AUC0−∞ λi i=1

Figure 5 shows an example of a triexponential disposition curve. The following parameter values were used for simulation: C1 = 50 mg/L, C2 = 40 mg/L, C3 = 10 mg/L, λ1 = 2.77 h−1 , λ2 = 0.347 h−1 , and λ3 = 0.0578 h−1 . The disposition curve shows three different slopes on a semilogarithmic plot versus time. The contribution of the individual exponential functions is indicated. Fitting the concentration time profiles by a sum of exponential functions may be helpful to interpolate between observed concentrations. This sum of exponentials could be used, for example, as forcing function for a pharmacodynamic model. The parameters of this sum of exponential functions can be estimated by software packages like ADAPT, WinNonlinTM Pro, Thermo Kinetica , SAAM II, and many others. Use of an appropriate error variance model (or of an adequate weighting scheme) is highly recommended. A thorough discussion of nonlinear

Urinary excretion rates can be used instead of plasma concentrations to characterize the PK profile of drugs excreted (primarily) by the kidneys. The majority of studies use plasma (or serum) concentrations to determine the PK. If the excretion of drug into urine is studied in addition to plasma concentrations, then renal clearance (CLR ) and nonrenal clearance/F can be calculated in addition to total clearance. The total amount excreted unchanged in urine from time zero up to the last collected urine sample (Turine ) and the AUC from time zero to Turine are used to calculate CLR :

CLR =

Amount excreted unchanged in urine until time Turine AUC0−Turine in plasma

Importantly, this formula yields renal clearance (and not renal clearance/F) both for intravenous and for extravascular administration because the amount recovered in urine is known. Nonrenal clearance (CLNR ) is calculated as the difference between total (CL) and renal clearance (CLR ) for intravenous administration. Apparent nonrenal clearance is calculated as the difference between CL/F and CLR for extravascular administration. If F is less than 100%, then this difference will not be equal to CLNR after intravenous administration divided by F because the estimate for CLR does not include F. In addition to clearance, MDRT and therefore volume of distribution at steady state can be calculated from urinary excretion data as described by Weiss (3). The urinary excretion rate is calculated as amount of drug excreted unchanged in urine per unit of time. A urinary excretion rate versus time plot is one method to estimate terminal elimination half-life based on urine data [see Rowland and Tozer (74) for details].

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

The calculated half-life from urinary excretion data should be similar to the half-life calculated from plasma data. 5.4 Superposition Methods and Deconvolution Superposition methods assume that the concentration time profiles of each dose can be added if two doses are given at the same or at different times because the concentrations achieved by each dose do not influence each other (47). A linear pharmacokinetic system has this property. This situation allows one to simulate plasma concentration time profiles after multiple doses by adding up concentration time profiles after various single doses at the respective dosing times. Such a nonparametric superposition module is implemented, for example, in WinNonlinTM Pro, and a similar module (on convolution) is available in Thermo Kinetica . In pharmacokinetics, deconvolution is often used to determine the in vivo release profile, for example, of a modified release formulation. The disposition curve after iv bolus administration (impulse response of the system) can be used in combination with the plasma concentration time curve of an extravascular formulation to determine the in vivo release profile. The Wagner–Nelson method (75–78) that is based on a onecompartment model and the Loo–Riegelman method that can account for multiple disposition compartments (79,80) have been applied extensively. These methods subsequently were greatly extended (44,50,73,81–83). A detailed description of those algorithms is beyond the scope of this article. Convolution/deconvolution methods are available in WinNonlinTM Pro and in Thermo Kinetica . 6 GUIDELINES TO PERFORM AN NCA BASED ON NUMERICAL INTEGRATION Before running any PK analysis, it is very helpful to prepare various plots of the observed concentration time data. Typically, the observed individual concentration time data are plotted on linear and semilogarithmic scale (i.e., concentrations on log-scale vs. time on linear scale). A log2 scale is often a good choice to visualize concentration versus

13

time data because half-life can be visualized more easily by such a plot. These plots usually are prepared for all subjects at the same time (‘‘spaghetti plots’’) and for each subject individually. An intraindividual comparison of the observed data for different treatments is often helpful. If different doses were given to the same subject, then plotting dose-normalized concentrations is helpful to assess dose linearity visually according to the superposition principle. Calculation of descriptive statistics in addition to the individual observed data provides useful summary statistics. The average ± standard deviation and the median (25%–75% percentiles) or other representative percentiles are often plotted versus time for each treatment. These observed data plots might already reveal that some assumptions like first-order elimination might not be appropriate. Actual, instead of nominal, protocol sampling times should always be recorded and used for PK analysis. For the very common case that numerical integration by the trapezoidal rule is applied, the two most important user decisions are (1) determination of the terminal phase and (2) choice of the integration rule for the AUC. The choice of the most appropriate AUC calculation method is discussed above. Some guidelines for determining the terminal phase are shown below. 6.1 How to Select Concentration Data Points for Estimation of Terminal Half-Life The following rules provide some practical guidelines for determination of the terminal phase of the concentration time profile: 1. Define a maximum number of data points used to specify the terminal phase. During the terminal phase, drug distribution is in equilibrium. As drug distribution is usually not in equilibrium at Tmax, the terminal phase should not contain Cmax and the data points directly after Cmax. Use of at least three points and not more than five to seven points to define the terminal phase is often appropriate.

14

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

2. Select the ‘‘optimal’’ number of data points according to a statistical criterion for the goodness of fit. Programs like WinNonlinTM Pro, Thermo Kinetica , and others provide an interactive graphical interface that shows the r2 and the r2 -adjusted value (the latter is called G-criterion in Thermo Kinetica ) for the chosen data points. The highest r2 -adjusted (or r2 ) value can be used as a statistical criterion for the best fit (ideal value: 1.0). The r2 adjusted criterion might be preferred to the r2 -criterion because the r2 -adjusted criterion considers the number of data points used to derive λz , whereas the r2 -criterion does not. If the last three, four, and five data points all yield an r2 -value of 0.98, for example, the r2 adjusted criterion is highest for the last five data points. With more data points being used to estimate λz , the probability of this correlation occurring by chance decreases, and, thus, several data points may be preferable to estimate λz . Other criteria for the number of data points may also be appropriate. Proost (84) compared various criteria to estimate λz by simulating profiles for a one-compartment model and concluded that various criteria (including the r2 and r2 -adjusted criterion) had comparable bias and precision for the estimate of λz . Irrespective of the criterion chosen, it seems reasonable to ensure an appropriate selection of data points to estimate the terminal half-life by visual inspection of the observations and the predicted regression line in each subject. 3. If half-life becomes systematically shorter with more data points being used for estimation of λz , then the use of only the last three, four, (or five) data points to define the terminal phase seems reasonable. This situation often occurs for drugs that need to be described by a sum of two, three, or more exponential functions, when the distribution is non-instantaneous.

4. If half-life becomes systematically longer with more data points being used for estimation of λz , then the use of only the last three, four, (or five) data points to define the terminal phase seems reasonable. One possible reason for such an observation would be a mixed-order (Michaelis-Menten) elimination that violates the assumptions of standard NCA (see Table 1). 5. It is reasonable to use at least three data points for estimation of λz . Exception: If the third last point is Cmax, only the last two points should be used to estimate λz or λz should not be estimated in this subject. 6. Exclusion of data points for estimation of λz seems warranted only if a problem with the respective sample has been documented, for example, by the clinical site or by the analytical laboratory. Probably no set of guidelines is applicable to all datasets. This set of rules may need to be revised, if the precision of the bioanalytical assay is low at low concentrations. Terminal half-life sometimes cannot be well determined by NCA for extended-release formulations or for data after multiple dosing, especially if the dosing interval is shorter than about twice the terminal half-life. It may be very helpful to report the median instead of the arithmetic mean to describe the central tendency of terminal half-life, if NCA indicates extremely long half-lives for some subjects. Compartmental modeling may be more powerful in these situations (85). The concentrations during the terminal phase used in linear regression on semilogarithmic scale often span a wide range (sometimes more than a factor of 100 between the highest and lowest concentration). This consideration is important for choosing an adequate weighting scheme. Importantly, unweighted linear regression (uniform weighting) on semilogarithmic scale is approximately equal to assuming a constant coefficient of variation error model. Therefore, unweighted linear regression on semilogarithmic scale is usually an adequate weighting scheme for NCA.

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

6.2 How to Handle Samples Below the Quantification Limit The best way to handle observations reported as being below the limit of quantitation is to ask the chemical analyst to supply the measured values. It may be necessary to tell and convince the analytical team that reporting the measured concentration value for samples ‘‘below the quantification limit’’ (BQL) contributes valuable information. These low concentrations can be adequately weighted when compartmental modeling techniques are applied. No good PK or regulatory reason exists not to use these observations. However, very good statistical theory supports the idea that discarding these values will bias the results of subsequent PK analysis (86–94). Concentrations at the first time points for extravascular administration and during the terminal phase are often reported to be BQL by the analytical laboratory. A thorough discussion of handling BQL samples in a PK data analysis is beyond the scope of this article. Population PK modeling offers more powerful methods for dealing with BQL samples. As a practical guide for NCA, the following procedure can be applied, if the only available information is that the respective samples were reported to be BQL: 1. All BQL samples before the first quantifiable concentration (including the predose sample) are set to zero (see predose, 0.5 and 1 h sample in Fig. 6). This setting will underestimate the AUC of the trapezoids before the first quantifiable concentration slightly, but the bias is usually small. An unreasonable option is to ignore the BQL samples before the first quantifiable concentration and to calculate the first trapezoid from time zero to the first quantifiable concentration because this calculation would yield a much larger overestimation of the AUC compared with the small underestimation described above. 2. BQL samples that were drawn after the first quantifiable concentration and before the last quantifiable concentration are usually ignored. This approach assumes that these samples were in

15

fact lost or never drawn (see 3.5 h sample in Fig. 6). 3. BQL samples after the last quantifiable concentrations are usually all ignored. A less common and potentially suboptimal method for NCA is to set the first BQL sample after the last quantifiable concentration (see 6 h sample in Fig. 6) to a fixed value. Typically, half of the reported quantification limit is used. All subsequent BQL samples are ignored for NCA (see 7 h and 8 h sample in Fig. 6). If the terminal phase is adequately descri bed by the last quantifiable concentrations and if the residual area is below 5%, then ignoring all BQL samples after the last quantifiable concentration seems to be most reasonable. Imputing the first BQL sample at 6 h (see Fig. 6) to be half of the quantification limit (or any other prespecified value) is likely to yield (slightly) biased estimates for terminal half-life in an NCA. Importantly, such an imputed sample should not be used for calculation of residual area. 6.3 NCA for Sparse Concentration Time Data NCA methods for sparse concentration time data have been developed (36–43). These methods are often applied for preclinical PK, toxicokinetic, and animal studies for which usually only sparse data are available. Bailer (36) originally proposed a method that uses concentration time data from destructive sampling (one observation per animal) to estimate the average AUC up to the

Figure 6. Example of a plasma concentration time profile with various samples reported to be below the quantification limit (BQL) that is reported to be 1 mg/L.

16

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

last observation time and to derive the standard error of the average AUC. Assume, for example, that 20 animals are sacrificed at 5 different time points (4 animals at each time point). The average and variance of the concentration at each time point are calculated, and the linear trapezoidal rule is used to derive the average AUC. Bailer provided a formula for the standard error of the average AUC. If various treatments are compared, then the average AUC can be compared statistically between different treatments by use of this standard error. Extensions to the Bailer method were developed (41–43) and were recently implemented into version 5 of WinNonlinTM Pro. Sparse sampling methods for NCA will also become available in the next version of Thermo Kinetica . Subsequently, bootstrap resampling techniques for sparse data (one observation per animal) were developed and evaluated (37–39). The so-called pseudoprofile-based bootstrap (38) creates pseudoprofiles by randomly sampling one concentration from, for example, the four animals at each time point. One concentration is drawn at each of the five time points. These five concentrations represent one pseudoprofile. Many pseudoprofiles are randomly generated, and an NCA is performed for each pseudoprofile. Summary statistics are computed to calculate the between subject variability and standard error of the NCA statistics of interest [see Mager and Goller (38) for details]. 6.4 Reporting the Results of an NCA The PK parameters and statistics to be reported for an NCA depend on the objectives of the analysis and on the types of data. Regulatory guidelines list several PK parameters and statistics to be reported from an NCA (53,54). A detailed report on the design, analysis, and reporting of clinical trials by NCA has been published by the Association for Applied Human Pharmacology (AGAH) (54). Valuable guidelines can also be found on pages 21 to 23 of the FDA guidelines for bioavailability and bioequivalence studies (53). It is often helpful for subsequent analyses to report the average ± SD (or geometric mean and %CV) and median (percentiles) of

the following five NCA statistics: AUC, Cmax, Tmax, t1/2 , and AUMC. Reporting the results on CL and Vss provides insight into the drug disposition. If the average and SD of those five NCA statistics are reported, then the so-called back analysis method (95) can be applied to convert NCA statistics into compartmental model parameters. Alternatively, the individual NCA statistics in each subject can be used instead of average and SD data. One has to decide based on literature data, for example, before applying the back analysis method, if a one- or two-compartment model is likely to be more appropriate for the drug of interest. The back analysis method provides estimates for the mean and variance of the model parameters for a one- or two-compartment model. The resulting PK parameter estimates can be used, for example, to build a population PK model and to run clinical trial simulations. The back analysis method allows one to run such simulations based on NCA results without the individual concentration time data. However, if individual concentration time data are available, population PK analysis is the preferred method of building a population PK model. 6.5 How to Design a Clinical Trial that is to be Analyzed by NCA It is possible to optimize the design of clinical trials (e.g., of bioequivalence studies) that are to be analyzed by NCA. However, those methods rely on compartment models in a population PK analysis and clinical trial simulation and are not covered in this article. Some practical guidelines for selection of appropriate sampling time points are provided below. This article focuses on NCA analysis by numerical integration for the most common case of extravascular dosing. The methods presented here are not as powerful as optimization of the study design by clinical trial simulation but may provide a basic guide. Before planning the sampling time schedule, one needs at least some information on the blood volume required for drug analysis and on the sensitivity of the bioanalytical assay. Some knowledge on the expected average concentration time profile of the drug

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

available is also assumed. The FDA (53) recommends 12 to 18 blood samples (including the predose sample) to be taken per subject and per dose. Cawello (96) and the AGAH working group recommend in total 15 blood samples to be drawn. Five samples (including the predose sample) should be drawn before Tmax. Another five samples are to be drawn between Tmax and one half-life after Tmax, and another five samples should be drawn up to five half-lives after Tmax. If one assumes that the median Tmax is at 1.5 h and that the average terminal halflife is 5 h, then this recommendation would yield, for example, the following sampling times: 0 h (predose), 0.33, 0.67, 1.00, 1.50, 2.00, 2.50, 3.00, 4.00, 6.00, 8.00, 12.0, 16.0, 24.0, and 32.0 h post dose. Cawello (96) recommends that each of the three intervals described above should contain four samples, if only twelve samples can be drawn in total. This recommendation would yield, for example, the following sampling times: 0 h (predose), 0.50, 1.00, 1.50, 2.00, 3.00, 4.00, 6.00, 10.0, 16.0, 24.0, and 32.0 h post dose. Such a sampling schedule may need to be modified, for example, for drugs with a large variability in terminal half-life or in Tmax. If the variability in Tmax is large, then more frequent sampling may be required during the first hour post dose in this example. Overall, this sampling schedule should provide low residual areas and a robust estimate for the terminal half-life. 7

CONCLUSIONS AND PERSPECTIVES

NCA is an important component of the toolbox for PK analyses. It is most applicable for studies with frequent sampling. Numerical integration, for example, by the trapezoidal rule, is most commonly used to analyze data after extravascular dosing. Fitting of disposition curves by a sum of exponential functions is the non-compartmental method of choice for analysis of concentrations after iv bolus dosing. Non-compartmental methods for handling sparse data were developed and are available in standard software packages. Standard NCA is straightforward and can be conveniently applied also by nonprofessional users.

17

It is important to recognize the assumptions and limitations of standard NCA. Almost all applications of NCA require a series of assumptions that are similar to the assumptions required for compartmental modeling. Violation of the assumptions for NCA will result in biased parameter estimates for some or all PK parameters. From a regulatory perspective, NCA is appealing as it involves minimal decision making by the drug sponsor or drug regulator. NCA can provide robust estimates of many PK parameters for studies with frequent sampling. For these studies, NCA and compartmental modeling often complement each other, and NCA results may be very helpful for compartmental model building. Therefore, NCA may be very valuable for PK data analysis of studies with frequent sampling, irrespective of whether the study objectives require subsequent analysis by compartmental modeling. 8

ACKNOWLEDGMENT

We thank one of the reviewers and Dr. William J. Jusko for comments on this ¨ manuscript. Jurgen Bulitta was supported by a postdoctoral fellowship from Johnson & Johnson. REFERENCES 1. O. Caprani, E. Sveinsdottir, and N. Lassen, SHAM, a method for biexponential curve resolution using initial slope, height, area and moment of the experimental decay type curve. J. Theor. Biol. 1975; 52: 299–315. 2. N. Lassen and W. Perl, Tracer Kinetic Methods in Medical Physiology. New York: Raven Press, 1979. 3. M. Weiss, The relevance of residence time theory to pharmacokinetics. Eur. J. Clin. Pharmacol. 1992; 43: 571–579. 4. P. Veng-Pedersen, Noncompartmentallybased pharmacokinetic modeling. Adv. Drug Deliv. Rev. 2001; 48: 265–300. 5. P. Veng-Pedersen, H. Y. Cheng, and W. J. Jusko, Regarding dose-independent pharmacokinetic parameters in nonlinear pharmacokinetics. J. Pharm. Sci. 1991; 80: 608–612.

18

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

6. A. T. Chow and W. J. Jusko, Application of moment analysis to nonlinear drug disposition described by the Michaelis-Menten equation. Pharm. Res. 1987; 4: 59–61. 7. H. Y. Cheng and W. J. Jusko, Mean residence time of drugs in pharmacokinetic systems with linear distribution, linear or nonlinear elimination, and noninstantaneous input. J. Pharm. Sci. 1991; 80: 1005–1006. 8. H. Y. Cheng and W. J. Jusko, Mean residence time concepts for pharmacokinetic systems with nonlinear drug elimination described by the Michaelis-Menten equation. Pharm. Res. 1988; 5: 156–164. 9. H. Cheng, W. R. Gillespie, and W. J. Jusko, Mean residence time concepts for non-linear pharmacokinetic systems. Biopharm. Drug Dispos. 1994; 15: 627–641. 10. H. Cheng, Y. Gong, and W. J. Jusko, A computer program for calculating distribution parameters for drugs behaving nonlinearly that is based on disposition decomposition analysis. J. Pharm. Sci. 1994; 83: 110–112. 11. P. Veng-Pedersen, J. A. Widness, L. M. Pereira, C. Peters, R. L. Schmidt, and L. S. Lowe, Kinetic evaluation of nonlinear drug elimination by a disposition decomposition analysis. Application to the analysis of the nonlinear elimination kinetics of erythropoietin in adult humans. J. Pharm. Sci. 1995; 84: 760–767. 12. M. Weiss, Mean residence time in non-linear systems? Biopharm. Drug Dispos. 1988; 9: 411–412. 13. D. J. Cutler, A comment regarding mean residence times in non-linear systems. Biopharm. Drug Dispos. 1989; 10: 529–530. 14. H. Y. Cheng and W. J. Jusko, An area function method for calculating the apparent elimination rate constant of a metabolite. J. Pharmacokinet. Biopharm. 1989; 17: 125–130. 15. A. T. Chow and W. J Jusko, Michaelis-Menten metabolite formation kinetics: equations relating area under the curve and metabolite recovery to the administered dose. J. Pharm. Sci. 1990; 79: 902–906. 16. M. Weiss, Use of metabolite AUC data in bioavailability studies to discriminate between absorption and first-pass extraction. Clin. Pharmacokinet. 1990; 18: 419–422. 17. M. Weiss, A general model of metabolite kinetics following intravenous and oral administration of the parent drug. Biopharm. Drug Dispos. 1988; 9: 159–176. 18. J. B. Houston, Drug metabolite kinetics. Pharmacol. Ther. 1981; 15: 521–552.

19. W. J. Jusko, Guidelines for collection and analysis of pharmacokinetic data. In: M. E. Burton, L. M. Shaw, J. J. Schentag, and W. E. Evans (eds.), Applied Pharmacokinetics & Pharmacodynamics, 4th ed. Philadelphia, PA: Lippincott Williams and Wilkins, 2005. 20. H. Y. Cheng and W. J. Jusko, Mean residence times and distribution volumes for drugs undergoing linear reversible metabolism and tissue distribution and linear or nonlinear elimination from the central compartments. Pharm. Res. 1991; 8: 508–511. 21. H. Y. Cheng and W. J. Jusko, Mean residence time of drugs showing simultaneous first-order and Michaelis-Menten elimination kinetics. Pharm. Res. 1989; 6: 258–261. 22. H. Y. Cheng and W. J. Jusko, Mean interconversion times and distribution rate parameters for drugs undergoing reversible metabolism. Pharm. Res. 1990; 7: 1003–1010. 23. H. Y. Cheng and W. J. Jusko, Mean residence times of multicompartmental drugs undergoing reversible metabolism. Pharm. Res. 1990; 7: 103–107. 24. H. Y. Cheng and W. J. Jusko, Mean residence time of oral drugs undergoing first-pass and linear reversible metabolism. Pharm. Res. 1993; 10: 8–13. 25. H. Y. Cheng and W. J. Jusko, Pharmacokinetics of reversible metabolic systems. Biopharm. Drug Dispos. 1993; 14: 721–766. 26. S. Hwang, K. C. Kwan, and K. S. Albert, A liner mode of reversible metabolism and its application to bioavailability assessment. J. Pharmacokinet. Biopharm. 1981; 9: 693–709. 27. W. F. Ebling, S. J. Szefler, and W. J. Jusko, Methylprednisolone disposition in rabbits. Analysis, prodrug conversion, reversible metabolism, and comparison with man. Drug Metab. Dispos. 1985; 13: 296–304. 28. M. N. Samtani, M. Schwab, P. W. Nathanielsz, and W. J. Jusko, Area/moment and compartmental modeling of pharmacokinetics during pregnancy: applications to maternal/fetal exposures to corticosteroids in sheep and rats. Pharm. Res. 2004; 21: 2279–2292. 29. M. S. Roberts, B. M. Magnusson, F. J. Burczynski, and M. Weiss, Enterohepatic circulation: physiological, pharmacokinetic and clinical implications. Clin. Pharmacokinet. 2002; 41: 751–790. 30. H. Cheng and W. R. Gillespie, Volumes of distribution and mean residence time of drugs with linear tissue distribution and binding and nonlinear protein binding. J. Pharmacokinet. Biopharm. 1996; 24: 389–402.

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS 31. F. M. Gengo, J. J. Schentag, and W. J. Jusko, Pharmacokinetics of capacity-limited tissue distribution of methicillin in rabbits. J. Pharm. Sci. 1984; 73: 867–873. 32. R. Nagashima, G. Levy, and R. A. O’Reilly, Comparative pharmacokinetics of coumarin anticoagulants. IV. Application of a threecompartmental model to the analysis of the dose-dependent kinetics of bishydroxycoumarin elimination. J. Pharm. Sci. 1968; 57: 1888–1895. 33. D. E. Mager, Target-mediated drug disposition and dynamics. Biochem. Pharmacol. 2006; 72: 1–10. 34. D. E. Mager and W. J. Jusko, General pharmacokinetic model for drugs exhibiting targetmediated drug disposition. J. Pharmacokinet. Pharmacodyn. 2001; 28: 507–532. 35. H. Cheng and W. J. Jusko, Disposition decomposition analysis for pharmacodynamic modeling of the link compartment. Biopharm. Drug Dispos. 1996; 17: 117–124. 36. A. J. Bailer, Testing for the equality of area under the curves when using destructive measurement techniques. J. Pharmacokinet. Biopharm. 1988; 16: 303–309. 37. H. Mager and G. Goller, Analysis of pseudoprofiles in organ pharmacokinetics and toxicokinetics. Stat. Med. 1995; 14: 1009–1024. 38. H. Mager and G. Goller, Resampling methods in sparse sampling situations in preclinical pharmacokinetic studies. J. Pharm. Sci. 1998; 87: 372–378. 39. P. L. Bonate, Coverage and precision of confidence intervals for area under the curve using parametric and non-parametric methods in a toxicokinetic experimental design. Pharm. Res. 1998; 15: 405–410. 40. J. R. Nedelman and E. Gibiansky, The variance of a better AUC estimator for sparse, destructive sampling in toxicokinetics. J. Pharm. Sci. 1996; 85: 884–886. 41. J. R. Nedelman, E. Gibiansky, and D. T. Lau, Applying Bailer’s method for AUC confidence intervals to sparse sampling. Pharm. Res. 1995; 12: 124–128. 42. J. R. Nedelman and X. Jia, An extension of Satterthwaite’s approximation applied to pharmacokinetics. J. Biopharm. Stat. 1998; 8: 317–328. 43. D. J. Holder, Comments on Nedelman and Jia’s extension of Satterthwaite’s approximation applied to pharmacokinetics. J. Biopharm. Stat. 2001; 11: 75–79. 44. P. Veng-Pedersen, Stochastic interpretation of linear pharmacokinetics: a linear system

19

analysis approach. J. Pharm. Sci. 1991; 80: 621–631. 45. J. J. DiStefano, 3rd. Noncompartmental vs. compartmental analysis: some bases for choice. Am. J. Physiol. 1982; 243: R1–6. 46. J. J. DiStefano 3rd, E. M. Landaw, Multiexponential, multicompartmental, and noncompartmental modeling. I. Methodological limitations and physiological interpretations. Am. J. Physiol. 1984; 246: R651–664. 47. W. R. Gillespie, Noncompartmental versus compartmental modelling in clinical pharmacokinetics. Clin. Pharmacokinet. 1991; 20: 253–262. 48. D. J. Cutler, Linear systems analysis in pharmacokinetics. J. Pharmacokinet. Biopharm. 1978; 6: 265–282. 49. J. L. Stephenson, Theory of transport in linear biological systems: I. Fundamental integral equation. Bull. Mathemat. Biophys. 1960; 22: 1–7. 50. P. Veng-Pedersen, Linear and nonlinear system approaches in pharmacokinetics: how much do they have to offer? I. General considerations. J. Pharmacokinet. Biopharm. 1988; 16: 413–472. 51. E. Nakashima and L. Z. Benet, An integrated approach to pharmacokinetic analysis for linear mammillary systems in which input and exit may occur in/from any compartment. J. Pharmacokinet. Biopharm. 1989; 17: 673–686. 52. Weiss M. Generalizations in linear pharmacokinetics using properties of certain classes of residence time distributions. I. Log-convex drug disposition curves. J. Pharmacokinet. Biopharm. 1986; 14: 635–657. 53. Food and Drug Administration (CDER). Guidance for Industry: Bioavailability and Bioequivalence Studies for Orally Administered Drug Products - General Considerations, 2003. 54. Committee for Proprietary and Medicinal Products (CPMP). Note for Guidance on the Investigation of Bioavailability and Bioequivalence. CPMP/EWP/QWP/1401/98, 2001. 55. K. C. Yeh and K. C. Kwan, A comparison of numerical integrating algorithms by trapezoidal, Lagrange, and spline approximation. J. Pharmacokinet. Biopharm. 1978; 6: 79–98. 56. Z. Yu and F. L. Tse, An evaluation of numerical integration algorithms for the estimation of the area under the curve (AUC) in pharmacokinetic studies. Biopharm. Drug Dispos. 1995; 16: 37–58.

20

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS

57. W. J. Jusko and M. Gibaldi, Effects of change in elimination on various parameters of the two-compartment open model. J. Pharm. Sci. 1972; 61: 1270–1273. 58. J. V. Gobburu and N. H. Holford, Vz, the terminal phase volume: time for its terminal phase? J. Biopharm. Stat. 2001; 11: 373–375. 59. K. Yamaoka, T. Nakagawa, and T. Uno, Statistical moments in pharmacokinetics. J. Pharmacokinet. Biopharm. 1978; 6: 547–558. 60. I. L. Smith and J. J. Schentag, Noncompartmental determination of the steady-state volume of distribution during multiple dosing. J. Pharm. Sci. 1984; 73: 281–282. 61. M. Weiss, Definition of pharmacokinetic parameters: influence of the sampling site. J. Pharmacokinet. Biopharm. 1984; 12: 167–175. 62. W. L. Chiou, The phenomenon and rationale of marked dependence of drug concentration on blood sampling site. Implications in pharmacokinetics, pharmacodynamics, toxicology and therapeutics (Part I). Clin. Pharmacokinet. 1989; 17: 175–199. 63. M. Weiss, Nonidentity of the steady-state volumes of distribution of the eliminating and noneliminating system. J. Pharm. Sci. 1991; 80: 908–910. 64. M. Weiss, Model-independent assessment of accumulation kinetics based on moments of drug disposition curves. Eur. J. Clin. Pharmacol. 1984; 27: 355–359. 65. M. Weiss, Washout time versus mean residence time. Pharmazie 1988; 43: 126–127. 66. D. Perrier and M. Mayersohn, Noncompartmental determination of the steady-state volume of distribution for any mode of administration. J. Pharm. Sci. 1982; 71: 372–373. 67. H. Cheng and W. J. Jusko, Noncompartmental determination of the mean residence time and steady-state volume of distribution during multiple dosing. J. Pharm. Sci. 1991; 80: 202–204. 68. D. Brockmeier, H, J. Dengler, and D. Voegele, In vitro-in vivo correlation of dissolution, a time scaling problem? Transformation of in vitro results to the in vivo situation, using theophylline as a practical example. Eur. J. Clin. Pharmacol. 1985; 28: 291–300. 69. D. Brockmeier, In vitro-in vivo correlation, a time scaling problem? Evaluation of mean times. Arzneimittelforschung 1984; 34: 1604–1607. 70. M. E. Wise, Negative power functions of time in pharmacokinetics and their implications. J. Pharmacokinet. Biopharm. 1985; 13:

309–346. 71. K. H. Norwich and S. Siu, Power functions in physiology and pharmacology. J. Theor. Biol. 1982; 95: 387–398. 72. G. T. Tucker, P. R. Jackson, G. C. Storey, and D. W. Holt, Amiodarone disposition: polyexponential, power and gamma functions. Eur. J. Clin. Pharmacol. 1984; 26: 655–656. 73. J. Gabrielsson and D. Weiner, Pharmacokinetic and Pharmacodynamic Data Analysis, Concepts and Applications. 4th ed. Stockholm, Sweden: Swedish Pharmaceutical Press, 2007. 74. M. Rowland and T. N. Tozer, Clinical Pharmacokinetics: Concepts and Applications. Philadelphia, PA: Lippincott Williams & Wilkins, 1995. 75. J. G. Wagner and E. Nelson, Per cent absorbed time plots derived from blood level and/or urinary excretion data. J. Pharm. Sci. 1963; 52: 610–611. 76. J. G. Wagner and E. Nelson, Kinetic analysis of blood levels and urinary excretion in the absorptive phase after single doses of drug. J. Pharm. Sci. 1964; 53: 1392–1403. 77. J. G. Wagner, Modified Wagner-Nelson absorption equations for multiple-dose regimens. J. Pharm. Sci. 1983; 72: 578–579. 78. J. G. Wagner, The Wagner-Nelson method applied to a multicompartment model with zero order input. Biopharm. Drug Dispos. 1983; 4: 359–373. 79. J. C. Loo and S. Riegelman, New method for calculating the intrinsic absorption rate of drugs. J. Pharm. Sci. 1968; 57: 918–928. 80. J. G. Wagner, Pharmacokinetic absorption plots from oral data alone or oral/intravenous data and an exact Loo-Riegelman equation. J. Pharm. Sci. 1983; 72: 838–842. 81. F. N. Madden, K. R. Godfrey, M. J. Chappell, R. Hovorka, and R. A. Bates, A comparison of six deconvolution techniques. J. Pharmacokinet. Biopharm. 1996; 24: 283–299. 82. D. Verotta, Comments on two recent deconvolution methods. J. Pharmacokinet. Biopharm. 1990; 18: 483–489; discussion 489–499. 83. D. P. Vaughan and M. Dennis, Mathematical basis and generalization of the LooRiegelman method for the determination of in vivo drug absorption. J. Pharmacokinet. Biopharm. 1980; 8: 83–98. 84. J. H. Proost, Calculation of half-life - PharmPK Discussion, 2005. Available: http://www.boomer.org/pkin/PK05/ PK2005095.html.

AN INTRODUCTORY GUIDE TO NON-COMPARTMENTAL ANALYSIS 85. A. Sharma, P. H. Slugg, J. L. Hammett, and W. J. Jusko, Estimation of oral bioavailability of a long half-life drug in healthy subjects. Pharm. Res. 1998; 15: 1782–1786. 86. Beal SL. Ways to fit a PK model with some data below the quantification limit. J Pharmacokinet Pharmacodyn 2001; 28: 481–504. 87. V. Duval and M. O. Karlsson, Impact of omission or replacement of data below the limit of quantification on parameter estimates in a two-compartment model. Pharm. Res. 2002; 19: 1835–1840. 88. H. Jacqmin-Gadda, R. Thiebaut, G. Chene, and D. Commenges, Analysis of left-censored longitudinal data with application to viral load in HIV infection. Biostatistics 2000; 1: 355–368. 89. H. S. Lynn, Maximum likelihood inference for left-censored HIV RNA data. Stat. Med. 2001; 20: 33–45. 90. J. P. Hing, S. G. Woolfrey, D. Greenslade, and P. M. Wright, Analysis of toxicokinetic data using NONMEM: impact of quantification limit and replacement strategies for censored data. J. Pharmacokinet. Pharmacodyn. 2001; 28: 465–479. 91. A. Samson, M. Lavielle, and F. Mentre, Extension of the SAEM algorithm to left-censored data in nonlinear mixed-effects model: application to HIV dynamics model. Computat. Stat. Data Anal. 2006; 51: 1562–1574. 92. R. Thiebaut, J. Guedj, H. Jacqmin-Gadda, et al., Estimation of dynamical model parameters taking into account undetectable marker values. BMC Med. Res. Methodol. 2006; 6: 38. 93. J. Asselineau, R. Thiebaut, P. Perez, G. Pinganaud, and G. Chene, Analysis of leftcensored quantitative outcome: example of procalcitonin level. Rev. Epidemiol. Sante Publique 2007; 55: 213–220. 94. S. Hennig, T. H. Waterhouse, S. C. Bell, et al., A d-optimal designed population pharmacokinetic study of oral itraconazole in adult cystic fibrosis patients. Br. J. Clin. Pharmacol. 2007; 63: 438–450. 95. C. Dansirikul, M. Choi, and S. B. Duffull, Estimation of pharmacokinetic parameters from non-compartmental variables using Microsoft Excel. Comput. Biol. Med. 2005; 35: 389–403. 96. W. Cawello, Parameters for Compartment-free Pharmacokinetics - Standardisation of Study Design, Data Analysis and Reporting. Aachen, Germany: Shaker Verlag, 1999.

21

FURTHER READING D. Z. D’Argenio, Advanced Methods of Pharmacokinetic and Pharmacodynamic Systems Analysis. New York: Plenum Press, 1991. D. Foster, NIH course ‘‘Principles of Clinical Pharmacology’’ lecture (including video) on compartmental versus noncompartmental analysis. Available: http://www.cc.nih.gov/training/training. html. M. Weiss, The relevance of residence time theory to pharmacokinetics. Eur. J. Clin. Pharmacol. 1992; 43: 571–579. W. J. Jusko, Guidelines for collection and analysis of pharmacokinetic data. In: M. E. Burton, L. M. Shaw, J. J. Schentag, and W. E. Evans (eds.), Applied Pharmacokinetics & Pharmacodynamics. 4th ed. Philadelphia, PA: Lippincott Williams and Wilkins, 2005.

CROSS-REFERENCES Pharmacokinetic Study Bioavailability Bioequivalence Population Pharmacokinetic Methods Analysis of Variance (ANOVA)

NONCOMPLIANCE Noncompliance with the protocol, Standards of Procedure (SOP), Good Clinical Practice (GCP), and/or applicable regulatory requirement(s) by an investigator/institution, or by member(s) of the sponsor’s staff should lead to prompt action by the sponsor to secure compliance. If the monitoring and/or auditing personnel identify(ies) serious and/or persistent noncompliance on the part of an investigator/institution, the sponsor should terminate the investigator’s/institution’s participation in the trial. When an investigator’s/institution’s participation is terminated because of noncompliance, the sponsor should notify promptly the regulatory authority(ies).

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/guidance/iche6.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

NONINFERIORITY TRIAL

loss in effectiveness from the control may still lead one to conclude that the experimental is effective (i.e., relative to no use of the experimental treatment). However, the term ‘‘noninferior’’ is in some sense misleading. It is, in fact, ‘‘inferior,’’ and what it exactly means is ‘‘not unacceptably inferior.’’ The unacceptable extent of inferiority (e.g., loss of the effect of the active control) would need to be defined. This extent is the so-called noninferiority margin.

H. M. JAMES HUNG Division of Biometrics I, Office of Biostatistics, Office of Translational Sciences, Center for Drug Evaluation and Research U.S. Food and Drug Administration Silver Spring, Maryland

SUE-JANE WANG ROBERT O’NEILL Office of Translational Sciences, Center for Drug Evaluation and Research U.S. Food and Drug Administration Silver Spring, Maryland

1 ESSENTIAL ELEMENTS OF NONINFERIORITY TRIAL DESIGN Literature on noninferiority trials or activecontrolled trials in general is abundant (2–16). Basically, there are two types of noninferiority trial design: with or without a placebo. When the placebo arm is present in the noninferiority trial, the efficacy of the experimental treatment can be evaluated via a direct comparison with the placebo. The comparison of the experimental treatment with the active control elucidates the extent of effectiveness in a relative sense. However, when the placebo is absent, the direct comparison between the experimental treatment and the active control is the only comparison available. It not only elucidates the extent of inferiority or superiority of the experimental treatment over the active control, but it also is expected to serve as a bridge for an indirect assessment of the experimental treatment’s efficacy (relative to a placebo). That is, the indirect inference pertains to the important question of how the experimental treatment would have fared against a placebo had the placebo been in the trial. In the absence of a concurrent placebo, the indirect inference for the experimental treatment’s efficacy entails use of the active control’s effect (relative to a placebo) from the historical placebo-controlled trials. For a simple reason, if the true effect of the experimental relative to the active control and the true effect of the active control versus the placebo are known, one can obtain the effect of the experimental treatment versus the placebo by transitivity.

Traditionally, the effect of an experimental treatment in treating a disease is mostly evaluated in a placebo-controlled clinical trial with the objective of demonstrating that the experimental treatment is more effective than the placebo with or without the standard-of-care medical treatment on the background. This clinical objective drives the formulation of a statistical hypothesis for testing. The trial is designed to provide statistical evidence to reject the null hypothesis that there is no treatment difference in the therapeutic effect and consequently to accept the intended alternative hypothesis that the experimental treatment is more effective than the placebo. This is often referred to as superiority testing. When a placebo can no longer be used ethically in the trial (e.g., with a life-saving drug on the market), a treatment regimen that has been proven to be effective and safe can be selected as a control, often called active or positive control, for an experimental treatment to be compared against (1). In an active-controlled trial, demonstration of superiority over the active control certainly establishes the effectiveness of the experimental treatment. Alternatively, proving the efficacy of the experimental treatment can be based on the so-called noninferiority testing, by which the experimental treatment will be shown to be not much inferior to the active control. Because the effectiveness of the active control has been established, a small amount of

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

NONINFERIORITY TRIAL

Usage of the historical data to obtain the effect of the active control presents several critical issues that directly influence the interpretability of a noninferiority trial. First, the effect of the active control versus a placebo in the historical trials is, at best, only an estimate. Derivation of the historical estimate requires meta-analysis, which is in itself controversial in practice. For instance, can the existing historical trials produce an approximately unbiased estimate of the active control’s effect, given that the negative trials are often not reported (this is the problem of publication bias)? And how does one handle large intertrial variability? Thus, the meta-analysis is often suspected of having a tendency of overestimating a treatment effect. The historical estimates in the individual historical trials are often unstable. Selection of the historical trials relevant to the noninferiority trial setting is a subjective process. Statistical methods for handling between-trial heterogeneity may depend on strong untestable assumptions. For instance, the validity of the resulting interval estimator by some meta-analysis method may be questionable (16–18). Secondly, there is a question of whether the historical estimate deemed valid in the historical trial patient population is applicable to the noninferiority trial setting; this is the question about the so-called ‘‘constancy’’ assumption (19–26). In practice, it is always doubtful that the constancy assumption is satisfied because often there are significant differences between the noninferiority trial and the historical trial settings, such as the differences in patient population, study endpoints, background or concomitant medications, or disease status. Violation of the constancy assumption will induce a bias that is the difference in the effect of the selected active control between the historical trial setting and the noninferiority setting. When the active control is much less effective in the noninferiority trial setting than in the historical trial setting, the bias may substantially increase the risk of falsely concluding noninferiority and the risk of falsely asserting that the experimental treatment is effective. Third, the quality of the noninferiority trial may be problematic because of inevitable

issues of medication nonadherence and noncompliance. The quality affects the ability of the noninferiority trial to distinguish an effective treatment from a less effective or ineffective treatment; this ability is referred as assay sensitivity in the International Conference on Harmonization (ICH) E10 guidelines (19, 20). It is not difficult to understand that if most of the patients in both treatment groups do not take the study drugs they are assigned, then the two treatment groups will be more alike than they should. That is, a noninferiority trial that lacks assay sensitivity may find an inferior treatment to be noninferior to the active control, which will lead to erroneous conclusions about noninferiority and the experimental treatment’s efficacy. The two important determinations as laid out in the ICH E-10 document that deduce the presence of assay sensitivity are (1) historical evidence of sensitivity to drug effects (i.e., that similarly designed trials in the past regularly distinguished effective treatments from less effective or ineffective treatments) and (2) appropriate trial conduct (i.e., that the conduct of the trial did not undermine its ability to distinguish effective treatments from less effective or ineffective treatments). The absence of a placebo arm from the noninferiority trial makes it impossible to verify or even check the assay sensitivity and the constancy assumptions with the trial data. Hence, the extent of the potential bias attributed to violation of these assumptions cannot be quantified in practice. 2 OBJECTIVES OF NONINFERIORITY TRIALS In the context of therapeutic effectiveness, three possible objectives have often been entertained in noninferiority studies. First, noninferiority in effectiveness traditionally means ‘‘not unacceptably inferior’’ in the sense that the effectiveness of the experimental treatment is ‘‘clinically indifferent’’ from that of the active control. Second, as a minimum requirement, the noninferiority trial must be able to assert that the experimental treatment is efficacious; for instance, the experimental treatment would have been

NONINFERIORITY TRIAL

more effective than the placebo had a placebo been in the noninferiority trial. Third, in many cases, it is often stipulated to establish that the experimental treatment preserves at least a certain portion of the effect of the selected active control in the noninferiority trial. On one hand, consideration of effect retention could arise from a part of value assessment of the relative efficacy, such as how much of loss in clinical benefits is acceptable with use of the experimental treatment in lieu of the active control to exchange for a better safety profile. On the other hand, retention of the active control’s effect may be necessary to create a cushion for the noninferiority inference to be valid beyond the many sources of unquantifiable bias. The biases can occur due to the uncertainty in the estimates of the active control’s effect in historical trials, application of such estimates to the noninferiority trial, or a lack of assay sensitivity. The three possible objectives are interrelated, though the term ‘‘noninferiority’’ may arguably be inappropriate when the primary objective is to demonstrate superiority over a putative placebo or to demonstrate the retention of a certain portion of the activecontrol effect. From the design perspective, they have tremendously different influences on the design specifications. For one, the clinical hypothesis and the corresponding statistical hypothesis for these objectives may be quite different. Consequently, statistical analysis, the decision tree, and inferential interpretation can be quite different. In practice, it is a mandatory first step to clearly define the primary objective of the noninferiority trial. The study objective is also a key factor determining the choice of the noninferiority margin. 3

MEASURE OF TREATMENT EFFECT

In addition to the study objective, selection of a measure for quantifying treatment effect is another key determination for defining the noninferiority margin. Let T, C, and P label, respectively, the experimental treatment, the selected active control, and the placebo that are studied or not studied in the noninferiority trial. To simplify the notation, let T, C,

3

and P also denote the targeted parameters of the experimental treatment, active control, and placebo, respectively. For instance, for the continuous response variable, if the effect of interest is expressed in terms of a mean change of a response variable from baseline, then T, C, and P are the mean parameters associated with the experimental treatment, active control, and placebo, respectively. For a binary outcome variable of event or not, T, C, and P may be the probabilities of an event associated with the three respective treatments. For a time-to-event variable, T, C, and P may be the respective hazard functions. The effect of a treatment can be measured on a variety of scales. Difference and ratio are two widely used scales. For a continuous response variable, the effect of the experimental treatment relative to the placebo is often defined on the difference scale, T − P. As an example, if the test drug decreases a mean of 6 mm Hg in sitting diastolic blood pressure after some point of treatment administration and the placebo decreases a mean of 2 mm Hg, then the effect of the experimental treatment is often measured as T − P = 4 mm Hg. On the ratio scale, the effect of the experimental treatment relative to the placebo is often defined as (1 − T/P). For instance, if the probability of death at some point during the study in the patients who receive the experimental treatment is 0.30 (or 30%) and the probability of death associated with the placebo is 0.40 (40%), then the effect of the experimental treatment on the risk scale is often measured by 1 − T/P = 1 − 0.30/0.40 = 25%; that is, the experimental treatment yields a 25% reduction in the mortality risk. On the difference scale, the experimental treatment reduces the probability of death by 10% from 40% to 30%. For a time-to-event parameter, the treatment effect is often quantified using a hazard ratio, if this ratio is approximately constant over time. For notational convenience, let T/P denote a risk ratio that can be a relative risk or an odds ratio for a binary outcome variable and a hazard ratio for a time-to-event variable. For a binary outcome variable or time-toevent variable, statistical inference is often based on the natural logarithm of a risk ratio statistic, rather than the risk ratio itself,

4

NONINFERIORITY TRIAL

because the former is better approximated by a Gaussian (or normal) distribution. On the log risk ratio scale, the treatment effect is defined as a difference of the log parameter, such as log probability of death or log hazard. 4

NONINFERIORITY MARGIN

Determination of a noninferiority margin is probably the most difficult task in designing a noninferiority trial (26–32). For demonstration that the experimental treatment is superior to the selected control, the objective of the trial is to rule out a zero difference between the two treatments and thus reject the null hypothesis of no treatment difference. A superiority margin is selected mostly to size the trial, not for being ruled out; therefore, the superiority margin does not need to be prespecified to define the hypothesis. In contrast, for demonstration of noninferiority, a noninferiority margin usually needs to be specified to define the statistical hypothesis before commencement of the noninferiority trial. The statistical null hypothesis to reject is that the treatment difference is at least as large as the specified margin against the experimental treatment. The noninferiority margin is an acceptable threshold in the value of the targeted parameter of the response variable between the experimental treatment and the selected positive control under comparison. At the Drug Information Association Annual Meeting in 2000, Dr. Robert Temple introduced the idea that this margin should be a statistical margin or a clinical margin, whichever is smaller. The clinical margin is determined on the basis of clinical judgments that are mostly subjective. Selection of the statistical margin depends upon the objective of the noninferiority trial. Suppose that the true probability of death is 30% associated with the selected positive control and 40% with the placebo. Thus, the control reduces 10% in the probability of death. If the noninferiority inference is set out to demonstrate that the experimental treatment is efficacious or effective (i.e., superior to a placebo), then the probability of death with the experimental treatment must be shown to be smaller than 40%; that is, the noninferiority margin for the experimental treatment versus the control can be set

to −10%. If the objective is to show that the experimental treatment preserves at least a certain portion, 50%—say, of the active control’s effect—then the probability of death with the experimental treatment must be shown to be smaller than 35%. Thus, the noninferiority margin for achieving a 50% retention on the difference scale is −5%. For a 75% retention, the noninferiority margin on the difference scale is 2.5% [= (1 − 0.75) × 10%]. These calculations can easily be articulated as follows. To obtain a noninferiority margin for the difference T − C, all that is needed is to examine the P-C axis between the control effect of 10% and the zero control effect. The margin needed for showing a 50% retention of (P − C) is the midpoint 5% between the 10% effect and zero effect. The margin defining a 75% retention of (P − C) is the one-quarter point above zero. Similar arguments can be made for risk ratio. To generate a noninferiority margin for the risk ratio T/C where the control C is the denominator, work on P/C, the risk ratio of placebo versus the control C, so that the two ratios to be compared have the same denominator. In the above example, P/C = 4/3 and the null effect of P versus C is P/C = 1. The noninferiority margin for the risk ratio T/C can then be generated by working on P/C between 4/3 and 1. For 50% retention of the control’s effect on the relative risk, the noninferiority margin is 1.17, which is the midpoint between 4/3 and 1. For 75% retention, the noninferiority margin for T/C is 1.08, which is the one-quarter point above one. In general, mathematical arguments can be made as follows. To retain X% of the control’s effect, we would need the effect of the experimental treatment relative to the placebo, had it been in the trial, to be greater than X times the control’s effect. That is, an X% retention of the control’s effect on the risk ratio amounts to showing (1 − T/P) > X(1 − C/P), equivalently, T/C < X + (1 − X)(P/C),

(1)

which indeed is the weighted average of one and the ratio P/C.

NONINFERIORITY TRIAL

For retention on the log risk ratio scale, the same arguments as those for the difference (P − C) can be made to construct the noninferiority margin for log(T/C) = log(T) − log(C). Thus, for 50% retention on the log risk ratio, the noninferiority margin for log(T/C) is the midpoint between log(4/3) and zero. By inverting log risk ratio to risk ratio, we can obtain that the noninferiority margin for T/C is the geometric mean between P/C = 4/3 and 1 to retain 50% of the control’s effect on the log risk ratio scale. For 75% retention on the log risk ratio scale, the noninferiority margin for T/C is (4/3)1/4 , raising (4/3) to the one-quarter power. For rendering a noninferiority margin on the risk ratio scale, the percent retention on the log risk ratio is convertible to that on the risk ratio and vice versa, given that the effect of active control can be estimated (16). For instance, if the relative risk C/P of the control to the placebo is 0.75, then preservation of 50% of the control effect on the risk ratio scale is equivalent to preservation of 46% of the control effect on the logarithm risk ratio. It is worth mentioning that at any level of retention on the log risk ratio the resulting noninferiority margin is always smaller than the margin for the same level of retention on the risk ratio scale. Thus, preservation of 50% of the control’s effect on the log risk ratio results in a smaller margin than preservation of 50% on the risk ratio. The statistical margin derived using the concept of retention of the control effect cannot always properly characterize clinical indifference. For example, if the selected positive control is highly effective with a relative risk C/P = 0.12, say, for a clinical adverse event (i.e., the control yields a 88% reduction of the risk of having the clinical event), then the statistical margin can be as large as 4.67 for a 50% preservation of the control’s effect and 2.47 for a 80% retention, by setting X to 50% and 80%, respectively, on the right-hand side of inequality (1). With these margins, an experimental treatment that yields at least a one-and-a-half-fold increase in risk relative to the control can still be concluded to be not inferior to the control. Such a large statistical margin cannot possibly be adequate for asserting the clinical indifference that the experimental treatment is as effective as or

5

not much worse than the positive control. For showing clinical indifference, a margin is required to define the clinical indifference. The margin determination discussed above is predicated on the knowledge of the true value of the effect parameter at stake. In practice, the true value is rarely known, and thus the effect of the active control must be estimated from trial data to determine the noninferiority margin. If a placebo is present in a noninferiority trial, the effect of the active control may be better estimated from this trial. In the absence of a concurrent placebo, the estimate will have to come from the external historical trials. In either case, the bias and the variance of the estimate must be properly incorporated in the margin determination. 5 STATISTICAL TESTING FOR NONINFERIORITY As mentioned earlier, the noninferiority margin must be selected and fixed in advance so that the noninferiority hypothesis to test is well defined before designing the trial. For example, if noninferiority is defined as retaining a 50% of the active control’s effect, one conventional approach employs the worst limit of a 95% confidence interval of the historical estimate of the control’s effect (i.e., relative to placebo) as a conservative estimate of the control’s effect in the noninferiority trial and then generates the statistical margin as previously described in section 4. Taking the smaller of the statistical margin and the clinical margin will then determine the noninferiority margin. In most cases, use of a conservative estimate of the active-control effect derived from some kind of meta-analyses of historical trials to define the noninferiority margin is necessary because of statistical uncertainty around the estimate and the unverifiable and yet often doubtful assumptions that must be made in making inferences from the noninferiority study. Once the noninferiority margin is determined, the widely used statistical method for noninferiority testing employs a 95% or higher confidence interval for the experimental treatment versus the selected active control from the noninferiority trial. If this

6

NONINFERIORITY TRIAL

interval rules out the predefined margin, then noninferiority defined by the margin can be concluded. This is in contrast with the superiority testing that depends on the same confidence interval to rule out the null value of no treatment difference. The probability of type I error of falsely concluding noninferiority associated with this confidence interval method is no more than 2.5%, conditional on the estimated noninferiority margin. When a placebo is present in the noninferiority trial, some type of the noninferiority hypothesis, such as percent retention hypothesis, can be tested directly with a more statistically efficient test method (33). The test can be constructed by dividing a relevant sum of the estimate of relative effect of the experimental treatment to the control and the estimate of the control’s effect by the standard error of the sum. Both estimates are derived from the noninferiority trial. For example, the 25% retention test on the log risk ratio scale is constructed by dividing the sum of the estimate of log (T/C) of the noninferiority trial and 75% times the estimate of log(C/P) by the standard error of this sum. A P-value can then be generated from the test. A sufficiently small P-value can indicate that the experimental treatment retains more than 25% of the control’s effect. If the placebo is absent from the noninferiority trial, by the same kind of combination, this test arguably can still be constructed in the same way but the estimate of the active control’s effect has to come from the historical trials. However, this test method is controversial (20–24, 34–39) because it assumes no bias with the historical estimate of the control’s effect when applied to the noninferiority trial—that is, when the constancy assumption holds, which is almost always very much doubtful. There are no data that can verify this assumption. Thus, this test method constructed by incorporating the historical estimate of the active control’s effect is rarely useful in practice. 6 MEDICATION NONADHERENCE AND MISCLASSIFICATION/MEASUREMENT ERROR Interpretation of the results of randomized clinical trials is primarily according to the

intent-to-treat principle, based on which all randomized patients are analyzed as per the treatment to which they are randomized. This analysis is intended to avoid selection bias, which may confound with a valid clinical interpretation. In superiority trials, medication nonadherence generally biases the intent-to-treat analysis toward the null hypothesis of no treatment difference; thus, statistical tests for superiority in intent-totreat analyses tend to be conservative. In noninferiority trials, nonadherence may bias intent-to-treat analyses in either a conservative or nonconservative direction (40–42), and thus it may undermine the clinical interpretability. Misclassification or measurement error also may generate bias. On-treatment or per-protocol analyses include only patients who are adherent to the assigned study treatment and protocol. This analysis is intended to address the question of what the true causal effect of the experimental treatment would have been had all patients adhered to the assigned treatment. In some cases, the on-treatment analyses may apparently be able to account for nonadherence when it is correctly measured, but these analyses require the unverifiable assumption that there is no unmeasured confounding caused by the factors such as selection of the patients for analyses. Hence, nonadherence when related to study outcome can also bias on-treatment analyses in either a conservative or nonconservative direction. Medication nonadherence and misclassification or measurement error may generate bias in both intent-to-treat analyses and on-treatment analyses conservatively or nonconservatively in noninferiority trials. The amount of bias is generally not estimable. Therefore, with serious nonadherence or such errors, most often no valid clinical interpretation of noninferiority trials can be made. Ontreatment or per-protocol analysis is unlikely to be able to rescue the study. 7 TESTING SUPERIORITY AND NONINFERIORITY As already discussed, in an active-control trial, the study goal can be superiority or noninferiority. In order for noninferiority to be

NONINFERIORITY TRIAL

entertained, the noninferiority margin must be fixed and prespecified in advance. The same 95% or higher confidence interval can be used to test both superiority and noninferiority simultaneously or in any sequential order (24–26), with the overall type I error rate associated with testing both objectives of no larger than a two-sided 5% level. The type I error rate for superiority and for noninferiority are each no larger than a two-sided 5% level. However, if the noninferiority margin is not prespecified, this confidence interval approach may still be problematic, particularly when the margin is influenced by the internal noninferiority trial data (43–48). Furthermore, medication noncompliance and misclassification or measurement error may still make the type I error rate for the prespecified noninferiority invalid. Therefore, to achieve noninferiority, the trial design requires the highest quality. To entertain testing for superiority and noninferiority, it is imperative to plan the study for noninferiority testing and the sample size to ensure sufficient power for both superiority and noninferiority, so defined (48). 8

CONCLUSION

It is quite clear that the effect of an experimental treatment should be evaluated, if all possible, by conducting a ‘‘showing superiority’’ trial. Showing noninferiority over a selected active control can be too difficult to provide statistical evidence for assessing the effect of the experimental treatment, particularly when the placebo cannot be used in the trial. Many factors determine the interpretability of a noninferiority trial that does not have a placebo arm. First, a noninferiority margin must be selected and fixed in advance when designing a noninferiority trial. The margin determination depends on the trial objective. Second, the critical assumptions of assay sensitivity and constancy and the judgment of ‘‘clinical indifference’’ also play key roles in the margin determination. Third, statistical uncertainty in the historical estimate of the active control’s effect also needs to be properly incorporated in the margin determination. The historical trials must have assay sensitivity. In contrast with showing superiority, the noninferiority trial must have very

7

high quality in terms of medication adherence in order for the noninferiority trial to have assay sensitivity. Testing for superiority and testing for noninferiority with a prespecified margin can be simultaneously performed; however, from the design perspective, the focus should be on planning for ‘‘showing noninferiority.’’ 8.1.1 Disclaimer. The views presented in this article are not necessarily those of the U.S. Food and Drug Administration.

REFERENCES 1. World Medical Association Declaration of Helsinki. Recommendations guiding physicians in biomedical research involving human subjects. JAMA. 1997; 277: 925–926. 2. W. C. Blackwelder, Proving the null hypothesis in clinical trials. Control Clin Trials. 1982; 3: 345–353. 3. T. R. Fleming, Treatment evaluation in active control studies. Cancer Treat Rep. 1987; 71: 1061–1064. 4. R. Temple, Difficulties in evaluating positive control trials. In: Proceedings of the Biopharmaceutical Section of American Statistical Association. Alexandria, VA: American Statistical Association, 1987, pp. 1–7. 5. G. Pledger and D. B. Hall, control equivalence studies: do they address the efficacy issue? In: K. E. Peace (ed.), Statistical Issues in Drug Research and Development. New York: Marcel Dekker, New York, 1990, pp. 226–238. 6. R. Temple, Problems in interpreting active control equivalence trials. Account Res. 1996; 4: 267–275. 7. J. Rohmel, Therapeutic equivalence investigations: statistical considerations. Stat Med. 1998; 17: 1703–1714. 8. B, Jones, P, Jarvis, J. A. Lewis, and A. F. Ebbutt, Trials to assess equivalence: the importance of rigorous methods. BMJ. 1996; 313: 36–39. 9. A. F. Ebbutt and L. Frith, Practical issues in equivalence trials. Stat Med. 1998; 17: 1691–1701. 10. R, Temple and S. S. Ellenberg, Placebocontrolled trials and active-control trials in the evaluation of new treatments. Part 1: Ethical and scientific issues. Ann Intern Med. 2000; 133: 455–463.

8

NONINFERIORITY TRIAL

11. S. S. Ellenberg and R. Temple, Placebocontrolled trials and active-control trials in the evaluation of new treatments. Part 2: Practical issues and specific cases. Ann Intern Med. 2000; 133: 464–470. 12. T. R. Fleming, Design and interpretation of equivalence trials. Am Heart J. 2000; 139: S171–S176. 13. A. L. Gould, Another view of active-controlled trials. Control Clin Trials. 1991; 12: 474–485. 14. W. C. Blackwelder, Showing a treatment is good because it is not bad: when does ‘‘noninferiority’’ imply effectiveness? Control Clin Trials. 2002; 23: 52–54. 15. R. B. D’Agostino, J. M. Massaro, and L. Sullivan, Non-inferiority trials: design concepts and issues—the encounters of academic consultants in statistics. Stat Med. 2003; 22: 169–186. 16. H. C. Bucher, G. H. Guyatt, L. E. Griffith, and S. D. Walter, The results of direct and indirect treatment comparisons in meta-analysis of randomized controlled trials. J Clin Epidemiol. 1997: 50: 683–691. 17. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E9 Statistical Principles for Clinical Trials. Current Step 4 version, February 5, 1998. Available at: http://www.ich.org/LOB/media/ MEDIA485.pdf 18. D. A. Follmann and M. A. Proschan, Valid inference in random effects meta-analysis. Biometrics. 1999; 55: 732–737. 19. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E10 Choice of Control Group and Related Issues in Clinical Trials. Current Step 4 version, July 20, 2000. Available at: http://www.ich.org/LOB/media/ MEDIA486.pdf 20. S. J. Wang, H. M. Hung, and Y. Tsong, Noninferiority analysis in active controlled clinical trials. In: S. E. Chow (ed.), Encyclopedia of Biopharmaceutical Statistics, 2nd ed. New York: Marcel Dekker, 2003, pp. 674–677. 21. Department of Health and Human Services, Food and Drug Administration [Docket No. 99D-3082]. International Conference on Harmonisation: Choice of control group in clinical trials (E10). Fed Regist. 1999; 64: 51767–51780. 22. V. Hasselblad and D. F. Kong, Statistical

methods for comparison to placebo in activecontrol trials. Drug Inf J. 2001; 35: 435–449. 23. S. J. Wang, H. M. Hung, Y. Tsong, Utility and pitfall of some statistical methods in active controlled clinical trials. Control Clin Trials. 2002; 23: 15–28. 24. H. M. Hung, S. J. Wang, Y, Tsong, J, Lawrence, and R. T. O’Neill, Some fundamental issues with non-inferiority testing in active controlled clinical trials. Stat Med. 2003; 22: 213–225. 25. H. M. Hung, S. J. Wang, and R. O’Neill, A regulatory perspective on choice of margin and statistical inference issue in non-inferiority trials. Biom J. 2005; 47: 28–36. 26. Committee for Medicinal Products for Human Use (CHMP). Guideline on the choice of the non-inferiority margin. Stat Med. 2006; 25: 1628–1638. 27. D. Hauschke, Choice of delta: a special case. Drug Inf J. 2001; 35: 875–879. 28. T. H. Ng, Choice of delta in equivalence testing. Drug Inf J. 2001; 35: 1517–1527. 29. B. Wiens, Choosing an equivalence limit for non-inferiority or equivalence studies. Control Clin Trials. 2002; 23: 2–14. 30. L. L. Laster and M. F. Johnson, Noninferiority trials: ‘‘the at least as good as’’ criterion. Stat Med. 2003; 22: 187–200. 31. L. L. Laster, M. F. Johnson, and M. L. Kotler, Non-inferiority trials: the ‘‘at least as good as’’ criterion with dichotomous data. Stat Med. 2006; 25: 1115–1130. 32. S. C. Chow and J. Shao, On non-inferiority margin and statistical tests in active control trial. Stat Med. 2006; 25: 1101–1113. 33. D. Hauschke and I. Pigeot, Establishing efficacy of a new experimental treatment in the ‘gold standard’ design (with discussions). Biom J. 2005; 47: 782–798. 34. E. B. Holmgren, Establishing equivalence by showing that a prespecified percentage of the effect of the active control over placebo is maintained. J Biopharm Stat. 1999; 9: 651–659. 35. R. Simon, Bayesian design and analysis of active control clinical trials. Biometrics. 1999; 55: 484–487. 36. S. J. Wang and H. M. Hung, Assessment of treatment efficacy in non-inferiority trials. Control Clin Trials. 2003; 24: 147–155. 37. M, Rothmann, N, Li, G, Chen, G. Y. Chi, R. T. Temple, and H. H. Tsou, Non-inferiority methods for mortality trials. Stat Med. 2003; 22: 239–264.

NONINFERIORITY TRIAL 38. S. M. Snapinn, Alternatives for discounting in the analysis of noninferiority trials. J Biopharm Stat. 2004; 14: 263–273. 39. Y, Tsong, S. J. Wang, H. M. Hung, and L. Cui, Statistical issues on objective, design and analysis of non-inferiority active controlled clinical trial. J Biopharm Stat. 2003; 13: 29–42. 40. M. M. Sanchez and X. Chen, Choosing the analysis population in non-inferiority studies: per protocol or intent-to-treat. Stat Med. 2006; 25: 1169–1181. 41. D. Sheng and M. Y. Kim, The effects of non-compliance on intent-to-treat analysis of equivalence trials. Stat Med. 2006; 25: 1183–1199. 42. E. Brittain and D. Lin, A comparison of intentto-treat and per-protocol results in antibiotic non-inferiority trials. Stat Med. 2005; 24: 1–10. 43. T, Morikawa and M. Yoshida, A useful testing strategy in phase III trials: combined test of superiority and test of equivalence. J Biopharm Stat. 1995; 5: 297–306. 44. C. W. Dunnett and A. C. Tamhane, Multiple testing to establish superiority/equivalence of a new treatment compared with kappa standard treatments. Stat Med. 1997; 16: 2489–2506.

9

45. S. J. Wang, H. M. Hung, Y, Tsong, L, Cui, and W. Nuri, Changing the study objective in clinical trials. In: Proceedings of the Biopharmaceutical Section of American Statistical Association. Alexandria, VA: American Statistical Association, 1997, pp. 64–69. 46. P, Bauer and M. Kieser, A unifying approach for confidence intervals and testing of equivalence and difference. Biometrika. 1996; 83: 934–937. 47. H. M. Hung and S. J. Wang, Multiple testing of non-inferiority hypotheses in active controlled trials. J Biopharm Stat. 2004; 14: 327–335. 48. European Agency for the Evaluation of Medicinal Products, Human Medicines Evaluation Unit, Committee for Proprietary Medicinal Products (CPMP). Points to Consider on Switching between Superiority and Non-inferiority. CPMP/EWP/482/99. July 27, 2000. Available at: http://www.emea. europa.eu/pdfs/human/ewp/048299en.pdf

CROSS-REFERENCES Active-Controlled Trial Non-inferiority Margin Non-inferiority Analysis

NONPARAMETRIC METHODS

subset
A, then the random variable V = ni=1 I(Zi ) has a binomial distribution with parameters n and p = Pr(Zi ∈ A). Result 2. Let Z1 , . . . , Zn be a random sample from a continuous distribution with cumulative distribution function (cdf) F(·), and let Ri denote the rank (from least to greatest) of Zi among the n Zs, for i = 1, . . . , n. Then the vector of ranks R = (R1 , . . . , Rn ) has a joint distribution that is uniform over the set of all permutations of the integers (1, . . . , n). Result 3. Let Z be a random variable with a probability distribution that is symmetric about the point θ . Define the indicator function (·) by

DOUGLAS A. WOLFE Ohio State University, Columbus, OH, USA

Many of the earliest statistical procedures proposed and studied rely on the underlying assumption of distributional normality. How well these procedures operate outside the confines of this normality constraint varies from setting to setting. Although there were a few isolated attempts to create statistical procedures that were valid under less restrictive sets of assumptions that did not include normality, such as the early introduction of the essence of the sign test procedure by Arbuthnott (2) in 1710, and the rank correlation procedure considered by Spearman (51) in 1904, it is generally agreed that the systematic development of the field of nonparametric statistical inference traces its roots to the fundamental papers of Friedman (18), Kendall (31), Kendall & Babington Smith (33), Mann & Whitney (38), and Wilcoxon (58). The earliest work in nonparametric statistics concentrated heavily on the development of hypothesis testing that would be valid over large classes of probability distributions— usually the entire class of continuous distributions, but sometimes with the additional assumption of distributional symmetry. Most of this early work was intuitive by nature and based on the principle of ranking to de-emphasize the effect of any possible outliers on the conclusions. Point and interval estimation expanded out of this hypothesis testing framework as a direct result of centering of the test statistics and test inversion, respectively. Most distribution-free test procedures (and associated confidence intervals) are based on one or more of the following three fundamental properties.

(t) = 1,

if t > 0,

= 0,

if t ≤ 0.

Then the random variables —Z − θ — and (Z − θ ) are independent. Statistics based solely on Result 1 are referred to as counting statistics, those based solely on Result 2 are commonly known as ranking statistics, and those based on an appropriate combination of all three results are called signed-rank statistics. Over the years of development in the field, distribution-free procedures have certainly become more sophisticated, both in the problems they address and in their complexity. However, the underlying premise behind almost all such hypothesis tests continues to rest with these three basic results or with modifications thereof. Much of the early work in distributionfree hypothesis tests followed the general approach of mimicking a standard normal theory procedure for a statistical problem by replacing the sample values with some combination of rank or counting statistics. The first nonparametric test statistics looked quite similar in form to their classical normal theory counterparts. However, more recent advances in nonparametric statistics have been less tied to previously

Result 1. Let Z1 , . . . , Zn be a random sample from some probability distribution and let A be a subset of the common domain for the Zs. If I(t) represents the indicator function for this

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

NONPARAMETRIC METHODS

developed normal theory structure and, in fact, there have been a number of settings where nonparametric procedures were the first to be developed, and classical procedures followed a few years later. It is the intent of this article to provide some brief overview of nonparametric statistics. However, the field has grown over the years to such a size that one must rely on standard textbooks in the area for a truly complete picture. The very first such textbooks in nonparametric statistics were the pioneering works of Siegel (49) and Fraser (17), both arriving on the scene in the infancy of the field. Walsh (55–57) published a three-volume handbook covering those nonparametric procedures available at the time. Other texts and reference books have added to the literature of nonparametric statistics over the years, including the applicationsoriented books by Bradley (3), Conover (5), Daniel (10), Gibbons (21), Hollander & Wolfe (27), and Marascuilo & McSweeney (39). The text by Lehmann (36) occupies an intermediate place in the literature. It has a general application orientation, but a considerable amount of the basic underlying theory of some of the procedures is also presented in a substantial appendix. Textbooks dealing primarily with the theory of rank tests and associated point estimators and confidence intervals have been published by Gibˇ ak ´ ´ ´ (23), bons (20), Hajek (22), Hajek & Sid Hettmansperger (25), Noether (44), Pratt & Gibbons (46), and Randles & Wolfe (47). The monograph by Kendall (32) covers the specialized topic of rank correlation methods. These resources vary on the extensiveness of their bibliographies, but it is safe to say that the vast majority of published literature in the field of nonparametric statistics is cited in at least one of these volumes. One of the necessities in the application of distribution-free test procedures and confidence intervals is the availability of the exact null distributions of the associated test statistics. Extensive tables of many of these null distributions are available in some of the applications-oriented texts mentioned previously. In addition, recent software developments have made it a good deal easier both to compute the appropriate test statistics and to obtain the associated P values

for many of these test procedures. Of particular note in this regard are the Minitab and StatXact software packages, for both their rather complete coverage of the basic nonparametric procedures and their ability to circumvent the need for exact null distribution tables by providing the associated exact or approximate P values for many of the test procedures. StatXact also has the option of actually generating the required exact null distributions for some of the better known test statistics, including the appropriate modifications necessary in the case of tied observations. We first turn our attention to brief descriptions of the most commonly used nonparametric procedures in standard statistical settings involving one, two, or more samples, including one- and two-way analysis of variance and correlation. In each case, the emphasis will be on the description of the problem and a particular standard approach to its solution, rather than on attempting to cover the myriad of different nonparametric procedures that are commonly available for the problem. Finally, we will discuss briefly a few nonstandard topics where the development of nonparametric methods has been particularly motivated by the need to analyze medical and health sciences data. Included in these topics will be censored data and survival analysis, as well as proportional hazards models, counting processes, and bootstrap methods.

1 ONE-SAMPLE LOCATION PROBLEM 1.1 Continuity Assumption Only Let Z1 , . . . , Zn be a random sample arising from an underlying probability distribution that is continuous with cdf F(·) and median θ . Here the primary interest is in inference about θ . 1.1.1 Test Procedure. For this setting, we are interested in testing the null hypothesis that θ = θ 0 , where θ 0 is some preset value appropriate for the problem. If no additional assumptions are reasonable about the form of the underlying F, the most commonly

NONPARAMETRIC METHODS

used inference procedures are those associated with the sign statistic B = [number of sample Zs that exceed θ0 ]. The properties of B follow from the basic counting Result 1 with the set A = (θ 0 , ∞). In particular, B has a binomial distribution with number of trials n and success probability p = Pr(Z1 > θ 0 ). When the null hypothesis is true, we have p = 1/2 (since θ 0 is then the median of the underlying distribution) and the null distribution of B does not depend on the form of F. The associated level α sign procedure for testing H0 , vs. the alternative H1 : θ > θ 0 , is to reject H0 if the observed value of B exceeds bα , the upper αth percentile for the null distribution of B, namely, the binomial distribution with parameters n and p = 1/2. The appropriate tests for the other directional alternatives θ < θ 0 and θ = θ 0 rely on the fact that the binomial distribution with n trials and p = 1/2 is symmetric about its mean n/2. 1.1.2 Point Estimation and Confidence Intervals/Bounds. Natural nonparametric confidence intervals and confidence bounds for θ are associated with these sign test procedures through the common process of inverting the appropriate hypothesis tests. These intervals and bounds are based on the ordered sample observations Z(1) ≤ Z(2) ≤ · · · ≤ Z(n) . The 100(1 − α)% confidence interval for θ associated in this manner with the level α two-sided sign test is given by (Z(n+1−bα/2 ) , Z(bα/2 ) ), where bα/2 is again the upper (α/2)th percentile for the binomial distribution with parameters n and p = 1/2. The corresponding 100(1 − α)% lower and upper confidence bounds for θ (obtained by inverting the appropriate one-sided sign tests) are given by Z(n+1−bα ) and Z(bα ) , respectively. The Hodges–Lehmann (26) point estimator of θ associated with the sign test is θ˜ = median {Z1 , . . . , Zn }. 1.2 Continuity and Symmetry Assumption Let Z1 , . . . , Zn be a random sample from an underlying probability distribution that is continuous and symmetric about its median θ . Once again the primary interest is in inference about θ .

3

1.2.1 Test Procedure. We remain interested in testing the null hypothesis that θ = θ 0 . However, the additional symmetry assumption now enables us to provide generally more powerful test procedures. For this setting, the most commonly used inference procedures are those associated with the Wilcoxon signed-rank test statistic (58), T+ =

n 

Ri  i ,

i=1

where  i = 1, 0 as Zi >, < θ 0 , and Ri is the rank of |Zi − θ 0 | among |Z1 − θ 0 |, . . . ,|Zn − θ 0 |. Thus, the Wilcoxon signed-rank statistic corresponds to the sum of the |Z − θ 0 | ranks for those Zs that exceed the hypothesized median value θ 0 . [Since we have a continuous underlying distribution, the probability is zero that there are ties among the absolute values of the (Zi − θ 0 )s. Likewise, the probability is zero that any of the Zi s actually equals θ 0 . However, these events may occur in actual data sets. In such an event, it is standard practice to discard the Zi s that equal θ 0 and reduce n accordingly. Ties among the absolute values of the (Zi − θ 0 )s are generally broken by assigning average ranks to each of the absolute differences within a tied group.] Properties of T + under H0 : θ = θ 0 derive directly from Result 3, which yields the independence of the ranks of the |Zi − θ 0 |s and the  i s, and Result 2, which implies that the ranks of the |Zi − θ 0 |s are uniformly distributed over the set of permutations of the integers (1, . . . , n) under H0 . The associated null distribution of T + does not depend on the form of the underlying F(·) and has been extensively tabled (see, for example, (27) and (59)). The associated level α signed-rank procedure for testing H0 vs. the alternative H1 : θ > θ 0 is to reject H0 if the observed value of T + exceeds tα , the upper αth percentile for the null distribution of T + . The appropriate tests for the other directional alternatives θ < θ 0 and θ = θ 0 rely on the fact that the null distribution of T + is symmetric about its mean n(n + 1)/4. 1.2.2 Point Estimation and Confidence Intervals/Bounds. Once again, natural confi-

4

NONPARAMETRIC METHODS

dence intervals and confidence bounds for θ are associated with these signed-rank procedures through inversion of the appropriate hypothesis tests. These intervals and bounds are based on the ordered values of the M = n(n + 1)/2 Walsh averages of the form W ij = (Zi + Zj )/2, for 1 ≤ i ≤ j ≤ n. Letting W (1) ≤ · · · ≤ W (M) denote these ordered Walsh averages, the 100(1 − α)% confidence interval for θ associated with the level α two-sided signed-rank test is given by(W(M+1−tα/2 ) , W(tα/2 ) ), where once again tα/2 is the upper (α/2)th percentile for the null distribution of T + . The corresponding 100(1 − α)% lower and upper confidence bounds for θ (obtained by inverting the appropriate one-sided signed-rank tests) are given by W(M+1−tα ) and W(tα ) , respectively. The Hodges–Lehmann (26) point estimator of θ associated with the signed-rank test is θˆ = median {Wij , 1 ≤ i ≤ j ≤ n}. We note that both the sign and signedrank inference procedures can be applied to paired replicates data (X i , Y i ), where X i represents a pretreatment measurement on a subject and Y i represents a posttreatment measurement on the same subject, and we collect such paired data from i = 1, . . . , n independent subjects. The appropriate sign or signed-rank procedures are then applied to the post-minus-pre differences Zi = Y i − X i , i = 1, . . . , n.

2

TWO-SAMPLE LOCATION PROBLEM

Let X 1 , . . . , X m and Y 1 , . . . , Y n be independent random samples from the continuous probability distributions with cdfs F(·) and G(·), respectively. We consider here the case where G(y) = F(y − ), with −∞ <  < ∞; that is, the X and Y distributions differ only by a possible location shift , and we are interested in inference about . 2.0.3 Test Procedure. For this setting, the appropriate null hypothesis is that  = 0 , where 0 is some preset value (often zero) of interest for the shift. The most commonly used nonparametric inference procedures for this setting are those associated with the rank sum version of the Wilcoxon–Mann–

Whitney (38,58), W=

n 

Rj ,

j=1

where Rj is the rank of Y j among the combined sample of N = (m + n) observations X 1 , . . . , X m , Y 1 , . . . , Y n . (Once again, ties among the Xs and/or Ys are broken by assigning average ranks to each of the observations within a tied group.) Properties of W under H0 :  = 0 (corresponding to no differences between the X and Y probability distributions) follow directly from the basic ranking Result 2, which implies that the joint ranks of X 1 , . . . , X m , Y 1 , . . . , Y n are uniformly distributed over the set of permutations of the integers (1, . . . , N) under H0 . The associated null distribution of W does not depend on the form of the common (under H0 ) underlying distribution F(·) and has been extensively tabled (see, for example, (27) and (59)). The associated level α rank sum procedure for testing H0 vs. the alternative H1 :  > 0 is to reject H0 if the observed value of W exceeds wα , the upper αth percentile for the null distribution of W. The appropriate tests for the other directional alternatives  < 0 and  = 0 rely on the fact that the null distribution of W is symmetric about its mean n(m + n + 1)/2. 2.0.4 Point Estimation and Confidence Intervals/Bounds. As in the one-sample setting, natural confidence intervals and bounds for  are associated with these rank sum procedures through inversion of the appropriate hypothesis tests. These intervals and bounds are based on the ordered values of the mn differences U ij = Y j − X i , i = 1, . . . , m, j = 1, . . . , n. Letting U (1) ≤ · · · ≤ U (mn) denote these ordered differences, the 100(1 − α)% confidence interval for  associated with the level α two-sided rank sum test is given by U({[n(2m + n + 1) + 2]/2}−wα/2 ) , U(wα/2 − [n(n + 1)/2]) , where once again wα/2 is the upper (α/2)th percentile for the null distribution of W. The corresponding 100(1 − α)% lower and upper confidence bounds for  (obtained by inverting the appropriate one-sided rank sum tests) are given by U({[n(2m+n+1)+2]/2}−wα ) and

NONPARAMETRIC METHODS

U(wα −[n(n+1)/2]) , respectively. The Hodges– Lehmann (26) point estimator of  associˆ = median ated with the rank sum test is  {Uij , i = 1, . . . , m, j = 1, . . . , n}. 3

OTHER TWO-SAMPLE PROBLEMS

The possibility of differences in location between the X and Y distributions is certainly the most common problem of interest in the two-sample setting. However, there are circumstances where differences in scale are of primary concern, as well as situations where it is important to detect differences of any kind between the X and Y distributions. For discussion on nonparametric two-sample procedures designed for scale differences, see Wilcoxon-type scale tests. The development of nonparametric procedures designed to be effective against any differences between the X and Y distributions was initiated by the pioneering work of Kolmogorov (34) and Smirnov (50). These papers have inspired a substantial body of research on such omnibus two-sample procedures. 4 ONE-WAY ANALYSIS OF VARIANCE: k ≥ 3 POPULATIONS This is a direct extension of the two-sample location problem. The data now represent k mutually independent random samples of observations from continuous probability distributions with cdfs F 1 (x) = F(x − τ 1 ), F 2 (x) = F(x − τ 2 ), . . . , F k (x) = F(x − τ k ), where F(·) is the cdf for a continuous population with median θ and τ 1 , . . . , τ k represent the additive effects corresponding to belonging to population 1, . . . , k, respectively. Here, our interest is in possible differences in the population effects τ 1 , . . . , τ k . 4.0.5 Test Procedures. For the one-way analysis of variance setting, we are interested in testing the null hypothesis H0 : [τ 1 = · · · = τ k ], corresponding to no differences in the medians of the k populations. For this setting, the most commonly used test procedures correspond to appropriate extensions of the Mann–Whitney–Wilcoxon joint ranking scheme as specifically directed toward the particular alternative of interest. For testing

5

the null H0 vs. the standard class of general alternatives H1 : (not all τ i s equal), the Kruskal–Wallis (35) test is the most popular procedure. For one-sided ordered alternatives of the form H2 : (τ 1 ≤ τ 2 ≤ · · · ≤ τ k , with at least one strict inequality), the appropriate extension is that proposed independently by Jonckheere (28) and Terpstra (54). Finally, for umbrella alternatives H3 : (τ 1 ≤ τ 2 ≤ · · · ≤ τ q−1 ≤ τ q ≥ τ q+1 ≥ · · · ≥ τ k , with at least one strict inequality), with either the peak of the umbrella, q, known a priori or estimated from the data, the standard test procedures are those proposed by Mack & Wolfe (37).

4.0.6 Multiple Comparisons and Contrast Estimation. After rejection of H0 : (τ 1 = · · · = τ k ) with an appropriate test procedure, one is most often interested in deciding which of the populations are different and then in estimating the magnitudes of these differences. This leads to the use of multiple comparison procedures, based either on pairwise or joint rankings of the observations. With pairwise rankings, where two-sample ranks are used to compare separately the sample data for k each of the pairs of populations, the 2 most commonly used multiple comparison procedures are those considered by Dwass (12), Steel (53), and Critchlow & Fligner (7) for two-sided all-treatment differences, and by Hayter & Stone (24) for one-sided all-treatment differences. The corresponding two-sided all-treatment multiple comparison procedure based on joint rankings, where the sample data from all k populations are ranked jointly, has been studied by Nemenyi (43) and Damico & Wolfe (8), while the joint rankings multiple comparison procedure for one-sided treatments vs. control decisions can be found in (43) and (9). Point estimation of any contrasts in the τ s (that is, any linear

combination β = ki=1 ai τi , with ki=1 ai = 0) is discussed in Spjøtvoll (52). Simultaneous two-sided confidence intervals for all simple contrasts of the form τ j − τ i have been developed by Critchlow & Fligner (7), while the corresponding simultaneous one-sided confidence bounds were studied by Hayter & Stone (24).

6

5

NONPARAMETRIC METHODS

TWO-WAY ANALYSIS OF VARIANCE

We consider here the standard two-way layout setting, where the data consist of one observation on each combination of k treatments and n blocks. The observation in the ith block and jth treatment combination, denoted by X ij , arises from a continuous probability distribution with cdf F(x − β i − τ j ), where F(·) is the cdf for a continuous distribution with median θ , for i = 1, . . . , n; j = 1, . . . , k. Moreover, the nk Xs are assumed to be mutually independent random variables. (This is known as the additive two-way layout model.) Here, our interest is in possible differences among the treatment effects τ 1, . . . , τ k. 5.0.7 Test Procedures. For the two-way layout with one observation per cell, we are interested in testing the null hypothesis H0 : (τ 1 = · · · = τ k ), corresponding to no differences in the k treatment effects. For this setting, the most commonly used procedures correspond to appropriate extensions of the sign test procedure for paired replicates data as specifically directed toward a particular alternative of interest. For testing the null H0 vs. the standard class of general alternatives H1 : (not all τ i s equal), the Friedman (18) test procedure is based on withinblocks ranks of the observations across treatment levels. For ordered alternatives of the form H2 : (τ 1 ≤ τ 2 ≤ · · · ≤ τ k , with at least one strict inequality), the appropriate test based on within-blocks ranks is that given by Page (45). 5.0.8 Multiple Comparisons and Contrast Estimation. After rejection of H0 : (τ 1 = · · · = τ k ) with an appropriate test procedure, one can use either the multiple comparison procedure studied by Nemenyi (43) and McDonald & Thompson (40) to reach the k(k − 1)/2 alltreatments two-sided decisions of the form τ i = τ j vs. τ i = τ j , or the corresponding treatments vs. control multiple comparison procedure due to Nemenyi (43), Wilcoxon & Wilcox (60), and Miller (41) to reach the k − 1 treatments vs. control one-sided decisions of the form τ j > τ control . A method for point estimation of a contrast in the τ s can be found in Doksum (11).

6 INDEPENDENCE Let (X 1 , Y 1 ), . . . , (X n , Y n ) be a random sample from a continuous bivariate probability distribution. The most common distributionfree tests for the independence of the X and Y variables are those considered by Kendall (31) and Spearman (51). The null distribution properties of both of these test procedures are based on the basic Result 2 and the fact that the ranks of the Xs and the separate ranks of the Ys are themselves independent under the independence of X and Y. Approximate 100(1 − α)% confidence intervals and bounds for the Kendall correlation coefficient γ = {2Pr[(Y 2 − Y 1 )(X 2 − X 1 ) > 0] − 1} have been provided by Noether (44), Fligner & Rust (16), and Samara & Randles (48). 7 CENSORED DATA One of the areas where nonparametric methods have played a major role in the analysis of medical and health sciences data in particular has been that of survival analysis of censored lifetime data. We discuss the basic concepts involved in dealing with censored data in the one-sample setting and then provide brief descriptions of the most important nonparametric methods available for other selected settings. There are times in the collection of data that we are prevented from actually observing the values of all of the observations. Such censoring leading to only partial information about the random variables of interest can be a direct result of the statistical design governing our data collection or it can be purely a consequence of additional random mechanisms affecting our data collection process. Considerable attention in the literature has been devoted to three particular types of censoring, which we now describe. The first of these, known as type I censoring, corresponds to a fixed (preset) censoring time, tc , at which the study is to come to an end. In this setting, instead of observing the random variables Z1 , . . . , Zn of interest, we are only able to observe the truncated variables W i = min(Zi , tc ), i = 1, . . . , n. Type I censoring corresponds to medical and health sciences studies

NONPARAMETRIC METHODS

conducted for a fixed period of time after initiation and no entry to the study once begun. A second type of censoring, known as type II censoring, corresponds to collecting survival (lifetime) data until a fixed number, say r < n, of the subjects have failed. Once this has occurred, the study is terminated. In this setting, we only observe the r smallest lifetimes (i.e. the first r order statistics) among Z1 , . . . , Zn . All we know about the remaining n − r unobserved lifetimes is that they are at least as long as the final observed failure. A third type of censoring, called random censoring, is probably the most common and the most complicated type of censoring associated with medical and health sciences data. In this setting, not only are the lifetimes random but the censoring times are also random. In clinical trials, for example, such random censoring could correspond to a study where not all subjects enter the study at the same time, but the study ends at one time, or to subjects leaving a study because they moved from the area or because of serious side-effects leading to discontinuation of the treatment. Probably the earliest nonparametric approach to dealing directly with censored lifetime data was provided by Kaplan & Meier (30) in their development of the product limit estimator for the survival function S(t) = 1 − G(t), −∞ < t < ∞. The first twosample rank procedure designed specifically to test hypotheses with censored data was provided by Gehan (19). He proposed a direct extension of the Mann–Whitney form of the Mann–Whitney–Wilcoxon test statistic that provided a natural way to handle censored values occurring in either the X and/or Y sample data. A generalization of the Gehan two-sample test to the k-sample (k ≥ 3) setting has been provided by Breslow (4). For additional discussion of such rank-based procedures for censored data, the reader is referred to (42). 8 OTHER IMPORTANT NONPARAMETRIC APPROACHES Brief mention must also be made here of three other major initiatives in the development of nonparametric approaches to the

7

analysis of medical and health sciences data. Paramount among such developments is that of the proportional hazards model initially proposed by Cox (6). Seldom has any single paper had such an impact on further research in the field. Kalbfleisch & Prentice (29) provide a nice discussion of the analysis of survival data by the use of the Cox proportional hazards model and extensions thereof. A second important thrust of more recent vintage has been the application of counting process methods in survival analysis. For a good discourse on this important methodology, the reader is referred to (1). Finally, we need to mention the advent of the bootstrap as an important tool in the analysis of medical data. The survey articles (14) and (15) serve very well as introductions to this important topic, and its application to the analysis of censored data is discussed in (13). REFERENCES 1. Andersen, P. K., Borgan, Ø., Gill, R. D. & Keiding, N. (1993). Statistical Models Based on Counting Processes. Springer-Verlag, New York. 2. Arbuthnott, J. (1710). An argument for divine providence, taken from the constant regularity observed in the births of both sexes, Philosophical Transaction of the Royal Society of London 27, 186–190. 3. Bradley, J. V. (1968). Distribution-Free Statistical Tests. Prentice-Hall, Englewood Cliffs. 4. Breslow, N. (1970). A generalized KruskalWallis test for comparing K samples subject to unequal patterns of censorship, Biometrika 57, 579–594. 5. Conover, W. J. (1980). Practical Nonparametric Statistics, 2nd Ed. Wiley, New York. 6. Cox, D. R. (1972). Regression models and life tables (with discussion), Journal of the Royal Statistical Society, Series B 34, 187–220. 7. Critchlow, D. E. & Fligner, M. A. (1991). On distribution-free multiple comparisons in the one-way analysis of variance, Communications in Statistics—Theory and Methods 20, 127–139. 8. Damico, J. A. & Wolfe, D. A. (1987). Extended tables of the exact distribution of a rank statistic for all treatments: multiple comparisons in one-way layout designs, Communications in Statistics—Theory and Methods 16, 2343–2360.

8

NONPARAMETRIC METHODS

9. Damico, J. A. & Wolfe, D. A. (1989). Extended tables of the exact distribution of a rank statistic for treatments versus control multiple comparisons in one-way layout designs, Communications in Statistics—Theory and Methods 18, 3327–3353. 10. Daniel, W. W. (1978). Applied Nonparametric Statistics. Houghton-Mifflin, Boston. 11. Doksum, K. (1967). Robust procedures for some linear models with one observation per cell, Annals of Mathematical Statistics 38, 878–883. 12. Dwass, M. (1960). Some k-sample rank-order tests, in Contributions to Probability and Statistics, I. Olkin, S. G. Ghurye, H. Hoeffding, W. G. Madow & H. B. Mann, eds. Stanford University Press, Stanford, pp. 198–202. 13. Efron, B. (1981). Censored data and the bootstrap, Journal of the American Statistical Association 76, 312–319. 14. Efron, B. (1982). The Jackknife, the Bootstrap, and Other Resampling Plans. Society of Industrial Applications in Mathematics, CBMS-National Science Foundation Monograph, Vol. 38. 15. Efron, B. & Tibshirani, R. (1986). Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy, Statistical Science 1, 54–77. 16. Fligner, M. A. & Rust, S. W. (1983). On the independence problem and Kendall’s tau, Communications in Statistics—Theory and Methods 12, 1597–1607. 17. Fraser, D. A. S. (1957). Nonparametric Methods in Statistics. Wiley, New York. 18. Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance, Journal of the American Statistical Association 32, 675–701. 19. Gehan, E. A. (1965). A generalized Wilcoxon test for comparing arbitrarily singly-censored samples, Biometrika 52, 203–223. 20. Gibbons, J. D. (1971). Nonparametric Statistical Inference. McGraw-Hill, New York. 21. Gibbons, J. D. (1976). Nonparametric Methods for Quantitative Analysis. Holt, Rinehart, and Winston, New York. ´ 22. Hajek, J. (1969). Nonparametric Statistics. Holden Day, San Francisco. ´ ´ Z. (1967). Theory of Rank 23. Hajek, J. & Sidak, Tests. Academic Press, New York. 24. Hayter, A. J. & Stone, G. (1991). Distribution free multiple comparisons for monotonically ordered treatment effects, Australian Journal of Statistics 33, 335–346.

25. Hettmansperger, T. P. (1984). Statistical Inferenc Based on Ranks. Wiley, New York. 26. Hodges, J. L., Jr & Lehmann, E. L. (1963). Estimates of location based on rank tests, Annals of Mathematical Statistics 34, 598–611. 27. Hollander, M. & Wolfe, D. A. (1999). Nonparametric Statistical Methods. 2nd Ed. Wiley, New York. 28. Jonckheere, A. R. (1954). A distribution-free k-sample test against ordered alternatives, Biometrika 41, 133–145. 29. Kalbfleisch, J. D. & Prentice, R. L. (1980). The Statistical Analysis of Failure Time Data. Wiley, New York. 30. Kaplan, E. L. & Meier, P. (1958). Nonparametric estimation from incomplete observations, Journal of the American Statistical Association 53, 457–481. 31. Kendall, M. G. (1938). A new measure of rank correlation, Biometrika 30, 81–93. 32. Kendall, M. G. (1962). Rank Correlation Methods, 3rd Ed. Griffin, London. 33. Kendall, M. G. & Babington Smith, B. (1939). The problem of m rankings, Annals of Mathematical Statistics 10, 275–287. 34. Kolmogorov, A. N. (1933). Sulla determinazione empirica di una legge di distribuzione, Giornale dell’Istituto Italiano degli Attuari 4, 83–91. 35. Kruskal, W. H. & Wallis, W. A. (1952). Use of ranks in one-criterion variance analysis, Journal of the American Statistical Association 47, 583–621. 36. Lehmann, E. L. (1975). Nonparametrics: Statistical Methods Based on Ranks. Holden-Day, San Francisco. 37. Mack, G. A. & Wolfe, D. A. (1981). K-sample rank tests for umbrella alternatives, Journal of the American Statistical Association 76, 175–181. 38. Mann, H. B. & Whitney, D. R. (1947). On a test of whether one of two random variables is stochastically larger than the other, Annals of Mathematical Statistics 18, 50–60. 39. Marascuilo, L. A. & McSweeney, M. (1977). Nonparametric and Distribution-free Methods for the Social Sciences. Wadsworth, Belmont. 40. McDonald, B. J. & Thompson, W. A., Jr (1967). Rank sum multiple comparisons in one- and two-way classifications, Biometrika 54, 487–497. 41. Miller, R. G., Jr (1966). Simultaneous Statistical Inference. McGraw-Hill, New York.

NONPARAMETRIC METHODS ˜ 42. Miller, R. G., Jr, Gong, G. & Munoz, A. (1981). Survival Analysis. Wiley, New York. 43. Nemenyi, P. (1963). Distribution-free multiple comparisons, PhD Thesis. Princeton University. 44. Noether, G. E. (1967). Elements of Nonparametric Statistics. Wiley, New York. 45. Page, E. B. (1963). Ordered hypotheses for multiple treatments: a significance test for linear ranks, Journal of the American Statistical Association 58, 216–230. 46. Pratt, J. W. & Gibbons, J. D. (1981). Concepts of Nonparametric Theory. Springer-Verlag, New York. 47. Randles, R. H. & Wolfe, D. A. (1979). Introduction to the Theory of Nonparametric Statistics. Wiley, New York. 48. Samara, B. & Randles, R. H. (1988). A test for correlation based on Kendall’s tau, Communications in Statistics—Theory and Methods 17, 3191–3205. 49. Siegel, S. (1956). Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill, New York. 50. Smirnov, N. V. (1939). On the estimation of the discrepancy between empirical curves

9

of distribution for two independent samples, Bulletin of Moscow University 2, 3–16 (in Russian). 51. Spearman, C. (1904). The proof and measurement of association between two things, American Journal of Psychology 15, 72–101. 52. Spjøtvoll, E. (1968). A note on robust estimation in analysis of variance, Annals of Mathematical Statistics 39, 1486–1492. 53. Steel, R. G. D. (1960). A rank sum test for comparing all pairs of treatments, Technometrics 2, 197–207. 54. Terpstra, T. J. (1952). The asymptotic normality and consistency of Kendall’s test against trend, when ties are present in one ranking, Indagationes Mathematicae 14, 327–333. 55. Walsh, J. E. (1962). Handbook of Nonparametric Statistics. Van Nostrand, Princeton. 56. Walsh, J. E. (1965). Handbook of Nonparametric Statistics, Vol. II. Van Nostrand, Princeton. 57. Walsh, J. E. (1968). Handbook of Nonparametric Statistics, Vol. III. Van Nostrand, Princeton. 58. Wilcoxon, F. (1945). Individual comparisons by ranking methods, Biometrics 1, 80–83. 59. Wilcoxon, F., Katti, S. K. & Wilcox, R. A. (1973). Critical values and probability levels for the Wilcoxon rank sum test and the Wilcoxon signed rank test, in Selected Tables in Mathematical Statistics, Vol. 1, H. L. Harter & D. B. Owen, eds. American Mathematical Society, pp. 171–259. 60. Wilcoxon, F. & Wilcox, R. A. (1964). Some Rapid Approximate Statistical Procedures, 2nd Ed. American Cyanamid Co., Lederle Laboratories, Pearl River.

NONRANDOMIZED TRIALS

groups must be alike in all important aspects, known or unknown, and differ only in the treatments that they receive. In this way, any differences observed between the groups can be attributed to the treatments, not to other factors such as baseline characteristics. To achieve comparable subject groups, the preferred method is to allocate treatments to subjects using a chance mechanism. Depending on the study design, a subject can be assigned to the treatment groups with the same chance (e.g., 1:1 randomization ratio) or different chance (e.g., 1:2 randomization ratio). Neither the investigator nor the subject knows in advance the treatment to be given before entering a trial. In practice, a randomization schedule is generated by a computer or from a table of random numbers (2). Further details are provided under the topic of randomization in this work. Randomized clinical trials are regarded as the scientific standard for comparing treatments. There are three main reasons why randomized clinical trials are the ideal scientific tool for comparing treatments. First, randomization tends to produce comparable groups. That is, the known or unknown prognostic factors and other characteristics of subjects at the time of randomization will be, on the average, evenly balanced between the treatment groups. Second, randomization eliminates the bias in the allocation of subjects that may potentially arise from either investigators or subjects. The direction of bias may go either way (favor or un-favor of the intervention) and can easily make the results of comparison uninterpretable. The third advantage of randomization is that it guarantees the validity of statistical tests of significance (3). Despite the popularity and wide acceptance of the scientific merits of randomization, some physicians are reluctant to participate in randomized clinical trials. The most frequent objection is the ethical concern with randomization (4–6). Many physicians feel that they must not deprive a patient of a new treatment that they believe to be beneficial, regardless of the validity of the evidence for that claim. Randomization would deprive about one-half of the patients (assuming a

ZHENGQING LI Global Biometric Science Bristol-Myers Squibb Company Wallingford, Connecticut

A clinical trial, as defined by Friedman, Furberg, and DeMets, is ‘‘a prospective study comparing the effect and value of intervention(s) against a control in human subjects’’ (1). Following this definition, a clinical trial must be prospective. Retrospective studies such as case-control studies in which subjects are selected on the basis of presence or absence of an event of interest do not meet this definition and will not be discussed here. Following this definition, a clinical trial must employ one or more intervention techniques and must contain a control group. Without an active intervention, a study is observational because no experiment is being performed. Without a control, there is no comparison group one can use to assess the effect of the intervention. We will be focusing on studies on human beings. Animal (or plant) studies will not be covered in the discussion although they may be studied using similar techniques. This article is a tutorial description of nonrandomized trials under the framework of clinical trial definition. Basic concepts of studies, design features, statistical methods, and applicability of nonrandomized trials will be described and their limitations discussed. In addition, references are provided for readers who are interested in exploring relevant topics further. Readers are also encouraged to read articles under the topics of randomization, stratification, historical control, observational trials, and propensity score covered elsewhere in this work. 1 RANDOMIZED VS. NONRANDOMIZED CLINICAL TRIALS Following the definition of clinical trials already outlined, the fundamental scientific principle underlying the comparison of interventions(s) versus control groups is that these

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

NONRANDOMIZED TRIALS

1:1 randomization ratio) from receiving the new and presumed better treatment. Another reason is that some physicians may feel that the patient–physician relationship is compromised if the physician must explain to the patient that the treatment for their disease would be chosen by a coin toss or computer. In 1976, the National Surgical Adjuvant Project for Breast and Bowel Cancers (NSABP) initiated a clinical trial to compare segmental mastectomy and postoperative radiation, or segmental mastectomy alone, with total mastectomy. Due to the low rates of accrual, a questionnaire was mailed to 94 NSABP principle investigators asking why they were not entering eligible patients in the trial (7). A response rate of 97% was achieved. Physicians who did not enter all eligible patients offered the following explanations: [1] concern that the doctor–patient relationship would be affected by a randomized trial (73%), [2] difficulty with informed consent (38%), [3] dislike of open discussions involving uncertainty (22%), [4] perceived conflict between the roles of scientist and clinician (18%), [5] practical difficulties in following procedures (9%), and [6] feelings of personal responsibility if the treatments were found to be unequal (8%). In addition, not all clinical studies can use randomized controls. For example, in some therapeutic areas, the prevalence of the disease is so rare that a large number of patients cannot be readily found. In such an instance, every qualified patient is precious for study recruitment. If a dramatic treatment effect is expected for the new intervention, the treatment effect may be easily explained based on clinical experience and data available from historical database. In this case, either no control is needed or historical control is sufficient to serve the purpose of the study. A nonrandomized trial is a clinical trial in which qualified subjects are assigned to different treatment groups without involving in the chance mechanism. Subjects may choose which group they want to be in, or may be assigned to the groups by their physician. Depending on how the control group is formed, several types of nonrandomized trials appear commonly in literature. In the next section, we will describe some general features of nonrandomized trials.

2 CONTROL GROUPS IN NONRANDOMIZED TRIALS We are discussing the nonrandomized trials in the context of clinical trial definition with a control group. As mentioned earlier, if the value of the treatment is overwhelmingly beneficial, no comparison may be necessary. However, in this case, one can equally argue that no trial is necessary if one knows the treatment benefit for sure. In practice, the benefit of an active treatment is likely to be of moderate magnitude, requiring care in its evaluation. For these reasons, studies without a control will not be discussed here. We discuss nonrandomized trials after explaining how a control group is formed. For a more general discussion regarding the selection of controls and control groups, readers are encouraged to read the topics of Control and Control Groups covered elsewhere in this work. 2.1 Nonrandomized Concurrent Control In control groups in a nonrandomized concurrent trial, the subjects are treated without the new intervention at approximately the same time as the intervention group is being treated. The patients are allocated to the intervention or control group based on either their physicians’ choice or the patients’ determination. Patients in the control group could be from the same institution or from a different institution. Typically, the control group needs to be selected to match the key characteristics of the intervention group. The main advantage of a nonrandomized concurrent control trial is that it more easily accepted by physicians and patients, especially by those who have objections to randomization. In addition, the data will be collected from subjects who entered the study at approximately the same time. Investigators may feel that data from the same period of time are more comparable in contrast with data collected from studies that were conducted years ago. The major weakness of the nonrandomized concurrent control trial is the potential that the intervention and control groups are not strictly comparable. Although the investigators may match a few known important

NONRANDOMIZED TRIALS

prognostic factors, there is no way to check whether the unknown or unmeasured factors are comparable between the treatment groups. The difficulty increases in a therapeutic area where the prognostic factors affecting the disease are not well characterized. 2.2 Historical Control Historical control studies use comparable subjects from past studies. These include selection of controls from published literature or from previous studies that are documented in medical charts or computer files. The argument for using historical controls is that all patients can receive the new intervention. From the point of view of investigators, a clinician conducting a historical control study has no ethical dilemma that arises potentially from randomization, especially, if he or she is already of the opinion that the new intervention is beneficial. In addition, patients may be more willing to participate in the study if they can be sure of receiving a particular treatment. Other major benefits include these studies’ contribution to medical knowledge and the potential cost savings in sample size and length of study (8, 9). The major criticism of historical control studies is that they are particularly vulnerable to bias. First, patients with more favorable prognoses may be more likely to be selected to receive the new intervention. Because of this, the patients recruited in the study may be substantially different from the target population specified in the protocol, thus making comparability between the new intervention and the historical control groups questionable. As a consequence, the more favorable results with the new intervention may be attributed simply to the fact that more favorable patients receive it. Second, unlike the nonrandomized concurrent control studies in which patients are recruited at approximately the same time, patients included in the historical control may be from studies conducted several years ago. An improvement in outcome of a disease from the new intervention may stem from changes in the patient population and patient management as well as technology change such as technology improvement in diagnosis criteria. For example, because educational and

3

screening programs now encourage people to have their glucose levels checked frequently, many who are asymptotic are identified as having diabetes and are receiving treatment. In the past, only those with symptoms would have chosen to see a physician; as a result, patients classified as diabetics would have comprised a different risk group compared with those currently considered to be diabetics. Third, without randomization, it is impossible to know whether the new intervention and the historical control groups are really comparable. For a therapeutic area where the diagnosis of the disease is clearly established and the prognosis is well known, this may be of less concern if the important prognostic factors are identified and matched through techniques such as stratification and regression. However, for a disease that is not well understood, an imbalance in unknown or unmeasured prognostic factors can easily make the interpretation of results difficult. In addition, historical studies are generally conducted in a nonuniform manner. The inaccuracy and incompleteness of historical studies can add more difficulties to using a historical control. The requirements for a valid historical control, as specified by Pocock (10), include the following: Control group has received a precisely defined treatment in a recent previous study. Criteria for eligibility, work-up, and evaluation of treatment must be the same. Important prognostic features should be known and be the same for both treatment groups. No unexplained indications lead one to expect different results. A further proviso may be added to these requirements (9). If there are differences between treatment groups with respect to these features, then it should be established these are not sufficient to explain any observed differences in outcome between groups. If this further requirement can be met by a study in which important differences in treatment effect had been demonstrated, then it would appear that such results would merit a confirmatory study.

4

NONRANDOMIZED TRIALS

2.3 Baseline Control

3.1 Study Design

In this type of study, the patients’ status over time is compared with their baseline state. Although sometimes these studies are thought to use the patient as his or her own control, they do not in fact have any separate control per se. Instead, changes from baseline are compared with an estimate of what would have happened to the patient in the absence of the treatment with the new intervention. Such estimates are generally made on the basis of general knowledge, without reference to any specific control. Baseline control studies are generally conducted when the effect is dramatic and occurs rapidly after treatment and when the estimate that the investigator intends to compare with is clearly defined. When the case is not so obvious, a specific historical experience should be sought. In addition, this type of design is more appropriate to studies in which the outcome of primary interest is easily established at baseline and can be followed after baseline by laboratory parameters (e.g., blood pressure or glucose level). For a study with patient survival as the primary endpoint, it is impossible to apply this type of study design.

To conduct a nonrandomized trial, a rigorous protocol is needed to stipulate the study’s objectives, inclusion/exclusion criteria, processes for data collection, and statistical methods to be used for data analysis. It is particularly important to specify in advance how the control group will be formed in a nonrandomized trial. For a nonrandomized, concurrent control study, the methods for patient selection and the procedure for treatment assignment need to be specified and communicated in advance between patients and their physicians. For a historical control study, the criteria used for forming the control, such as selection of studies and patients, should be specified in the protocol before conducting the study. Where no single optimal control exists, it may be advisable to study multiple controls, provided that the analytical plan specifies conservatively how each will be used in making inference. In some cases, it may be useful to have a group of independent reviewers reassess endpoints in the control group and in the intervention group in a blinded manner according to common criteria. A priori identification of a reasonable hypothesis and advanced planning for analysis can strengthen the conclusions to be drawn (11). The credibility of results based on post hoc analyses is diminished, and the conclusions are less convincing to researchers and readers. In the planning stage of a clinical trial, one needs to determine the sample size based on the primary hypothesis stated in the study. For the comparison of two treatment groups in a randomized trial, extensive literature exists on sample size determination. The sample size is usually calculated from a simple two-sample Z-test, and most introductory statistical books contain the calculations. (Readers are also encouraged to read the topics pertaining to sample size calculations in this work.) In a nonrandomized trial, some special features in the study design will impact the sample size calculation. For example, in a historical control study, once the control group has been selected, the summary statistics from the control group are known and do not change in hypothetical repetitions of the

3 STATISTICAL METHODS IN DESIGN AND ANALYSES Because randomization is not placed in a nonrandomized trial, the balance between the treatment groups in important prognostic factors, known or unknown, is not protected. The primary challenge in the design and analysis of nonrandomized trials is to address the bias that potentially arises from the treatment incomparability. In this regard, the nonrandomized concurrent control studies and historical control studies face the same fundamental issue. Therefore, we will not distinguish the statistical methods used for these two types of studies. For the baseline control study, the statistical inference underlying the comparison is a one-sample problem. The statistical methods in design and analysis are relatively straightforward. We will focus our attention on the first two types of studies and refer readers to tutorial statistical texts that have good coverage for this topic (1).

NONRANDOMIZED TRIALS

clinical study to be performed. The sample size calculation for a historical control study needs to incorporate this feature. For a binary endpoint, Makuch and Simon (12) provide a sample size formula as well as tables for calculating the sample size required in a historical control study. In general, the sample size required in a historical control study is substantially smaller than that required for a similar randomized trial. When the variability is ignored from the historical control, only 25% of the total number of patients is necessary to undertake a historical control study, compared with a similar randomized trial (8). However, this would imply the unlikely situation that the entire population of historical controls is known exactly. For a nonrandomized trial in which the data from the control group are not available at the design stage, considerations in sample size calculation are different. For example, in a nonrandomized, concurrent control trial, one concern is the impact of the imbalance of important covariates on statistical comparison between the two groups. This imbalance needs to be incorporated into sample size calculation. Lipsitz and Parzen (13) have provided a sample size formula for normal responses based on a regression model. For a nonrandomized study with a 1:1 allocation ratio between the two treatment groups, the number of patients required per treatment group can be calculated as n
= n

2 1 − ρ[Y−E(Y|X), E(Y|X,W)]

1 − r2x|w

where n is the sample size calculated based on 2 the formula for a randomized trial, ρ[Y−E(Y|X), E(Y|X,W)] is the proportion of variation in the response Y jointly explained by treatment indicator X and important covariates W, which is not explained by X alone, and r2x|w is the coefficient of determination obtained by regression X on the other covariates W, which measures the imbalance between the two treatment groups in W. In practice, the two coefficients are typically determined from a pilot study or previous data. If an investigator has no knowledge about these two coefficients, he or she can specify a range of possible values and see how sensitive the estimated sample size is to various parameters.

5

When the two coefficients are equal, the sample size will be the same as the sample size required for a randomized trial. Examples for calculating the sample size have been provided in Lipsitz and Parzen (13). Sample size calculations for failure-time random variables in nonrandomized studies are also available (14). Dixon and Simon (15) have discussed sample size considerations for studies comparing survival curves using historical controls. 3.2 Statistical Analysis As discussed earlier, the control and intervention groups may have large differences on their observed covariates in a nonrandomized trial. These differences can lead to biased estimates of the treatment effects. Statistical methods are available to minimize the bias stemming from the imbalance of these observed covariates. These methods include matching, stratification, and regression adjustment. Matching is a common technique used to select controls who are similar to the treated patients on important covariates that have been identified by the investigator as needing to be controlled. Although the idea of finding matches seems straightforward, it is often difficult in practice to find subjects who match on these covariates, even when there are only a few covariates of interest. A common matching technique is Mahalanobis metric matching (16, 17). Following this method, patients in the intervention group are randomly ordered first. Then, the Mahalanobis distances between the first treated patient and all controls are calculated. The control with the minimum distance is chosen as the match for the treated patient, and both individuals are removed from the pool. This process is repeated until matches are found for all treated patients. One of the drawbacks of the method is that it is difficult to find close matches when there are many covariates included in the model. Stratification is also commonly used to control for systematic differences between the control and intervention groups. Following this technique, patients will be grouped into strata based on the observed covariates that are deemed important by the investigator. Once the strata are defined, the treated

6

NONRANDOMIZED TRIALS

patients and control subjects who are in the same stratum are compared directly. The rationale behind this approach is that subjects in the same stratum are similar in the covariates used to define the strata, and thus are comparable. However, when the number of covariates increases, the number of strata grows exponentially. When the number of strata is large, some strata might contain subjects only from one group, which would make it impossible to estimate the treatment effect in that stratum. Regression adjustment is a technique based on statistical regression models in which the treatment effect is estimated after adjusting the covariates identified by the investigator. The theory behind this approach is that, if there is any bias due to the treatment imbalance on these observed covariates, these covariates should have effects on the outcome variable. By modeling the effects of the covariates and treatment indicator on the outcome variable in the same model, the treatment effect would be estimated on the basis that the subjects in the intervention and control groups hold the same value for these adjusted covariates. Consequently, the bias due to the treatment imbalance on these covariates would be minimized or removed. The selection of regression models will depend on the outcome variables. The most commonly used models include linear regression for continuous outcomes, logistic regression for dichotomous responses, and the Cox regression for time-to-event data. In contrast with the matching and stratifications techniques, the regression adjustment can incorporate multiple covariates. However, Rubin (18) has shown that covariance adjustment may in fact increase the expected squared bias if the covariance matrices in the intervention and control groups are unequal. One common difficulty with both the matching and stratification techniques is the case of multiple covariates. Although the regression technique can accommodate multiple covariates in the model, some concern remains for the issues of overparameterizing and loss of flexibility in including interactions and higher order terms when many parameters are included. One major breakthrough in dimension reduction is the use of propensity scores (19).

The propensity score for a subject is the conditional probability of receiving the intervention rather than the control, given a vector of his or her observed covariates. The propensity score is a balancing score in the sense that the conditional distribution of the covariates, given the propensity score, is the same for the intervention and control subjects. In other words, a group of subjects with the same propensity score are equally likely to have been assigned to the intervention. Within a group of subjects with the same propensity score, some actually received the intervention and some received the control, just as if they had been randomly allocated to whichever treatment they actually received. Therefore, two groups with the same propensity score are expected to be comparable with respect to the observed covariates. The estimate of the propensity score is relatively straightforward. It is estimated by predicting treatment group membership based on the observed covariates—for example, by a multiple logistic regression or discriminate analysis. In the statistical model, the outcome is the event that a subject is in the intervention or control group, and the predictors are these covariates. The clinical outcome of the study is not involved in the modeling. (See D’Agostino [20] for a detailed description and tutorial summary.) Once the propensity score is estimated for each subject, the methods of matching, stratification, or regression adjustment can be used based on one summary score, the propensity score. Examples for implementing the propensity score approach have been described in literature (21). Although the propensity score is a good approach for reducing the covariate dimension, limitations still exist (21). First, propensity score methods can only apply for observed covariates. For unobserved and unmeasured covariates, propensity scores cannot be calculated. Second, to be able to apply the methods to propensity scores, an overlap in propensity scores between the two groups is necessary. If there is insufficient overlap between the two groups, none of the methods (matching, stratification, or regression adjustment) will be very helpful. Third, propensity score methods may not eliminate all bias because of the limitations of propensity score modeling (22), which is a linear combination of covariate. As

NONRANDOMIZED TRIALS

recommended by Braitman and Rosenbaum (23), propensity score methods work better under three conditions: rare events, a large number of subjects in each group, and many covariates influencing the subject selection. A more detailed description for propensity score and its general usage can be found elsewhere in this work.

4

CONCLUSION AND DISCUSSION

Lack of randomization and blinding make a nonrandomized trial vulnerable to bias. Although statistical methods are available to minimize the bias that may arise from the imbalance of observed covariates, it is always difficult, and in many cases impossible, to quantify the bias completely. Nonetheless, careful planning in study design, conduct, and analysis can make a nonrandomized trial more persuasive and potentially less biased. A control group should be chosen for which there is detailed information, and the control subjects should be as similar as possible to the population expected to receive the intervention in the study; the controls should be treated in a similar setting, in a similar manner. To reduce the selection bias, the methods for selecting the controls should be specified in advance. This may not always be feasible in the case of a historical control study because outcomes from the historical control may be in published form. However, efforts should be made to justify the selection of controls on a scientific basis rather than on outcomes. Any statistical techniques used to account for the population differences should be specified before selecting controls and performing the study. As noted earlier, one of the major reasons for conducting a nonrandomized trial is ethics concerns. However, it also is not ethical to carry out studies that have no realistic chance of credibility in showing the efficacy of the new treatment. When should we consider a nonrandomized trial? A few general considerations have been discussed in the International Conference on Harmonisation (ICH) of Technical Requirements for Registration of Pharmaceuticals for Human Use guidelines (24):

7

There is a strong prior belief on the superiority of the new intervention to all available alternatives, and alternative designs appear unacceptable. The disease or condition to be treated has a well-documented, highly predictable course. The study endpoints are objective. The covariates influencing the outcome of the disease are well characterized. The control group closely resembles the intervention group in all known relevant baseline, treatment (other than study drug), and observational variables. Even in these cases, appropriate attention to deign, conduct, and analysis is necessary to help reduce the bias. Nonrandomized trials provide a useful supplement to randomized trials, and nonrandomized methods are useful for exploratory and pilot studies. However, unless the issues of potential biases are fully explored, one needs to be cautious when drawing confirmative conclusions based on a nonrandomized trial. REFERENCES 1. L. M. Friedman, C. Furberg, and D. L. DeMets, Fundamentals of Clinical Trials. 3rd ed. New York: Springer-Verlag, 1998. 2. M. Zelen, The randomization and stratification of patients to clinical trials. J Chronic Dis. 1974; 27: 365–376. 3. D. P. Byar, R. M. Simon, W. T. Friedewald, et al., Randomized clinical trials: perspectives on some recent ideas. N Engl J Med. 1976; 295: 74–80. 4. F. J. Ingelfinger, The randomized clinical trial [editorial]. N Engl J Med. 1972; 287: 100–101. 5. T. C. Chalmers, J. B. Black, and S. Lee, Controlled studies in clinical cancer research. N Engl J Med. 1972; 287: 75–78. 6. L. W. Shaw and T. C. Chalmers, Ethics in cooperative clinical trials. Ann NY Acad Sci. 1970; 169: 487–495. 7. K. M. Taylor, R. G. Margolese, and C. L. Saskolne, Physicians’ reasons for not entering eligible patients in a randomized clinical trial of surgery for breast cancer. N Engl J Med. 1984; 310: 1363–1367.

8

NONRANDOMIZED TRIALS 8. E. A. Gehan and E. J. Freireich EJ, Nonrandomized controls in cancer clinical trials. N Engl J Med. 1974; 290: 198–203. 9. E. A. Gehan, The evaluation of therapies: historical control studies. Stat Med. 1984; 3: 315–324.

10. S. J. Pocock, The combination of randomized and historical controls in clinical trials. J Chronic Dis. 1976; 29: 175–188. 11. J. C. Bailar 3rd, T. A. Louis, P. W. Lavori, and M. Polansky, Studies without internal controls. N Engl J Med. 1984; 311: 156–162. 12. R. W. Makuch and R. W. Simon, Sample size considerations for non-randomized comparative studies. J Chronic Dis. 1980; 33: 175–181. 13. S. R. Lipsitz and M. Parzen, Sample size calculations for non-randomized studies. Statistician. 1995; 44: 81–90. 14. M. V. P. Bernardo, S. R. Lipsitz, D. P. Harrington, and P. J. Catalano, Sample size calculations for failure time random variables in non-randomized studies. Statistician. 2000; 49: 31–40. 15. D. O. Dixon and R. Simon, Sample size considerations for studies comparing survival curves using historical controls. J Clin Epidemiol. 1988; 14: 1209–1213. 16. D. B. Rubin, Bias reduction using Mahalanobis metric matching. Biometrics. 1980; 36: 293–298. 17. R. G. Carpenter, Matching when covariables are normally distributed. Biometrika. 1977; 64: 299–307. 18. D. B. Rubin, Using multivariate matched sampling and regression adjustment to control bias in observational studies. J Am Stat Assoc. 1979; 74: 318–324. 19. P. R. Rosenbaum and D. B. Rubin, The central role of the propensity score in observational studies for causal effects. Biometrika. 1983; 70: 41–55.

20. R. B. D’Agostino, Jr., Propensity score methods for bias reduction in the comparison of a treatment to a non-randomized control group. Statistics Med. 1998; 17: 2265–2281. 21. L. Yue, Practical issues with the application of propensity score analysis to nonrandomized medical device clinical studies. In: 2004 ASA Proceedings. Alexandria, VA: American Statistical Association, 2004, pp. 970–975. 22. D. B. Rubin, Estimating causal effects from large data sets using propensity scores. Ann Intern Med. 1997; 127: 757–763. 23. L. Braitman and P. R. Rosenbaum, Rare outcomes, common treatments: analytical strategies using propensity scores. Ann Intern Med. 2002; 137: 693–696. 24. Center for Biologics Evaluation and Research (CBER), Center for Drug Evaluation and Research (CDER), Food and Drug Administration, U.S. Department of Health and Human Services. Guidance for Industry: E10. Choice of Control Group and Related Issues in Clinical Trials. Rockville, MD: U.S. DHHS, May 2001. Available online at: http://www.fda.gov/cder/guidance/4155fnl. htm. Accessed June 2007.

CROSS-REFERENCES Randomization Historical control Stratification Observational trials Propensity score

OBJECTIVES

reducing the total score of the 17-item Hamilton Depression Rating Scale in subjects who meet criteria for major depressive disorder as defined by the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) (APA 1994).’’ In this protocol, a secondary objective that is confirmatory to the primary could be to evaluate the treatment effect on another efficacy measure. Another secondary objective, complementary to the primary objective, is to assess the safety and tolerability of the study drug versus the placebo. Some protocols use ‘‘exploratory objectives’’ to classify those goals that are stretched beyond what the design can fully evaluate but are somewhat related. Exploratory objectives do not require the same degree of statistical rigor as is required for primary objectives. They are usually used to generate hypotheses for future research.

YILI L. PRITCHETT Abbott Laboratories Abbott Park, Illinois

Objectives describe what clinical researchers intend to achieve in a clinical trial. Objectives vary from study to study; in particular, they are distinctive between studies designed to learn and studies designed to confirm. In the learning phase of clinical drug development, the study objective could be to establish the maximum tolerable dose, to select the most promising treatment agent among a set of candidates, to prove the concept of efficacy, or to estimate the dose-response relationship. On the other hand, in the confirmatory phase, the study objective could be to test a hypothesis that the study drug has superior efficacy than the control, or to demonstrate the acceptable benefit/risk profile of a new molecular entity. Identifying the objectives is the first step in designing a clinical trial. Clearly and properly defined objectives build the foundation for a well-planned clinical trial, since objectives impact the decision for each of the following key elements related to trial design: type of study (e.g., adaptive design, crossover design, or parallel design); sample size, outcome measures, study duration, entry criteria, study monitoring rules, frequency of data collection, and statistical data analysis plan. Objectives should be written clearly in protocols. Objectives can be classified as primary or secondary. Primary objectives are the focus of a study, and data should be collected to first support these objectives. In general, a study is considered successful if the primary objectives are met. A single, well-defined primary objective allows for clear interpretation of the clinical trial results. Secondary objectives can be either confirmatory or complementary to the primary ones. For instance, in a study of an investigational drug for the antidepressant indication, the primary object can be ‘‘To assess the efficacy of study drug (at the dose of x mg) compared with placebo in

FURTHER READING European Medicine Agency, ICH Topic E8, General Considerations for Clinical Trials, March 1998. S. Piantadosi, Clinical Trials, A Methodological Perspective. New York: Wiley, 1997. L. B. Scheiner, Learning versus confirmatory in clinical drug development. Clin. Pharmacol. Therap. 1997; 16(3).

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

OFFICE OF ORPHAN PRODUCTS DEVELOPMENT (OOPD) The U.S. Food and Drug Administration’s Office of Orphan Products Development (OOPD) is dedicated to promoting the development of products that demonstrate promise for the diagnosis and/or treatment of rare diseases or conditions. The OOPD interacts with the medical and research communities, professional organizations, academia, and the pharmaceutical industry as well as rare disease groups. The OOPD administers the major provisions of the 1983 Orphan Drug Act (ODA), which provides incentives for sponsors to develop products for rare diseases. The success of the ODA can be seen in the more than 200 drugs and biological products for rare diseases that have been brought to market since 1983, in contrast to the decade before 1983 which saw fewer than 10 such products come to market. In addition, the OOPD administers the Orphan Products Grants Program, which provides funding for clinical research in rare diseases.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/orphan/) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

OFFICE OF PHARMACOEPIDEMIOLOGY AND STATISTICAL SCIENCE (OPASS) The Office of Pharmacoepidemiology and Statistical Science (OPaSS), which includes the Office of Biostatistics and the Office of Drug Safety, was created as part of a 2002 Center for Drug Evaluation and Research (CDER) reorganization and has about 180 of CDER’s 1700 employees. Staff persons who work in the Office of Biostatistics and the Office of Drug Safety have backgrounds in a variety of disciplines that include medicine, epidemiology, pharmacology, pharmacy, statistics, regulatory science, health science, information technology, as well as administration and support services. OPaSS plays a significant role in the Center’s mission of assuring the availability of safe and effective drugs for the American people by: • Providing leadership, direction, plan-

ning, and policy formulation for CDER’s risk assessment, risk management, and risk communication programs; • Working closely with the staff of CDER’s other ‘‘super’’ offices, the Office of New Drugs and the Office of Pharmaceutical Science, to provide the statistical and computational aspects of drug review evaluation and research.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/Offices/OPaSS/default. htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

OFFICE OF REGULATORY AFFAIRS (ORA)

2

The headquarters of ORA is composed of four offices, each with its own responsibilities, that work together to achieve ORA’s mission.

The U.S Food and Drug Administration’s Office of Regulatory Affairs (ORA) is responsible for the following:

1. The Office of Resource Management (ORM) encompasses four divisions— Management Operations, Information Systems, Human Resource Development, and Planning, Evaluation, and Management—that are responsible for: • Managing bilateral agreements and Memoranda of Understanding (MOUs) with other governments. • Developing field manpower allocations and operational program plans. • Analyzing and evaluating field performance data and overall accomplishments. • Advising the Office of the Associate Commissioner for Regulatory Affairs (ACRA) and the Regional Food and Drug Directors (RFDDs) on all areas of management. • Developing and implementing nationwide information storage and retrieval systems for data originating in the field offices. 2. Office of Regional Operations (ORO) consists of four divisions—Field Science, Federal-State Relations, Import Operations, and Field Investigations— that are responsible for: • Serving as the central point through which the FDA obtains field support services. • Developing, issuing, approving, or clearing proposals and instructions affecting field activities. • Developing and/or recommending to the ACRA policy, programs, and plans for activities with state and local agencies. • Coordinating field consumer affairs and information programs.

• Managing and operating the FDA field

offices. • Coordinating and managing all FDA

field operations. • Providing advice and assistance on reg-

ulations and compliance policy matters that impact policy development, implementation, and long-range goals. • Working with additional federal agen-

cies on issues of compliance and evaluating proposed legal actions. • Directing and conducting criminal inves-

tigative activities in coordination with FDA headquarters units and other federal, state, and local law enforcement agencies.

1

HEADQUARTERS OFFICES

COMPLIANCE

The principal job of ORA is to survey and inspect regulated firms to assess their compliance with public health laws. Compliance strategies include providing information to industry; highlighting areas of significant violations and their impact on public health; prioritizing and targeting high-risk areas; cooperating with state and local public health authorities and regulators; and focusing on covering products imported into the United States through border coverage and foreign inspections. This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/ora/hier/ora overview.html) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

OFFICE OF REGULATORY AFFAIRS (ORA) • Developing and maintaining interna-

tional regulatory policy and activities to ensure the safety, efficacy, and wholesomeness of various imported products. • Providing laboratory support in various highly specialized areas. 3. The Office of Enforcement (OE), which coordinates legal cases and policies within ORA and the Centers, has several Compliance divisions and is responsible for: • Advising the ACRA and other key officials on regulations and compliance matters that have an impact on policy development, implementation, and long-range program goals. • Coordinating, interpreting, and evaluating overall compliance efforts. • Stimulating and awareness of the need for prompt and positive action to ensure compliance by regulated industries. • Evaluating and coordinating proposed legal actions to establish compliance with regulatory policy and enforcement objectives. • Coordinating development of FDAwide bioresearch monitoring activities. • Serving as the focal point of activities relating to the Federal Medical Products Quality Assurance Program. 4. The Office of Criminal Investigations (OCI) focuses on the investigation of criminal activities in the field and is responsible for: • Directing, planning, and developing criminal investigation activities in coordination with other FDA components and with other federal, state, and local law enforcement agencies. • Initiating and conducting criminal investigations under all statutes administered by the FDA. • Providing recommendations to the Office of Chief Counsel on referrals of criminal cases to the Department of Justice, participating in grand jury investigations, and serving as agents of the grand jury.

3 FIELD COMPONENTS The field staff of ORA is organized into five regions, each of which is headed by a Regional Food and Drug Director (RFDD): • The Pacific Region: Alaska, Arizona, Cal-

•

•

•

•

ifornia, Hawaii, Idaho, Montana, Nevada, Oregon, and Washington. The region includes three district offices and two regional labs. The Southwest Region: Arkansas, Colorado, Iowa, Kansas, Missouri, Nebraska, New Mexico, Oklahoma, Texas, Utah, and Wyoming. The region includes three domestic district offices, the Southwest Import District (SWID), and the Arkansas Regional Lab. The Central Region: Delaware, Illinois, Indiana, Kentucky, Maryland, Michigan, Minnesota, Ohio, New Jersey, North Dakota, Pennsylvania, South Dakota, Virginia, West Virginia, and Wisconsin. The region includes seven district offices and the Forensic Chemistry Center. The Southeast Region: Alabama, Florida, Georgia, Louisiana, Mississippi, North Carolina, South Carolina, Tennessee, and the San Juan district (Puerto Rico and the U.S. Virgin Islands). The region includes four district offices and a regional laboratory. The Northeast Region: Connecticut, Maine, Massachusetts, New Hampshire, New York, Rhode Island, and Vermont. The region includes two district offices, a regional lab, and the Winchester Engineering and Analytical Center (WEAC).

ONE-SIDED VERSUS TWO-SIDED TESTS

population variances σ12 and σ22 are essentially known through their consistent estimation by the sample variances s21 and s22 . Then the statistic

The choice between a one-sided test or a twosided test for a univariate hypothesis depends on the objective of statistical analysis prior to its implementation. The underlying issue is whether the alternative∗ against which the (null) hypothesis is to be assessed is one-sided or two-sided. The alternative is often onesided in a clinical trial∗ to determine whether active treatment is better than placebo; a two-sided alternative is usually of interest in a clinical trial to determine which of two active treatments is better. The principal advantage of a one-sided test is greater power∗ for the contradiction of the null hypothesis when the corresponding one-sided alternative applies. Conversely, for alternatives on the opposite side, its lack of sensitivity represents a disadvantage. Thus, if alternatives on both sides of a null hypothesis are considered to be of inferential interest, a two-sided test is necessary. However, where the identification of one direction of alternatives is actually the objective of an investigation, the cost of the broader scope of a two-sided test is the larger sample size it requires to have the same power for this direction as its one-sided counterpart. The benefit provided by the increase in sample size is power for alternatives in the opposite direction. If this purpose for increased sample size is not justifiable on economic, ethical, or other grounds, then a one-sided test for a correspondingly smaller sample size becomes preferable. Thus, both one-sided tests and two-sided tests are useful methods, and the choice between them requires careful judgment. The statistical issues can be clarified further by considering the example of the hypothesis of equality of two population means µ1 and µ2 . The null hypothesis has the specification H0 : µ1 − µ2 = δ = 0.

z = d/σd ,

(2)

where d = (y1 − y2 ) and σd = {(s21 /n1 ) + (s22 /n2 )}1/2 , approximately has the standard normal distribution with expected value 0 and variance 1. A two-sided test for the hypothesis H0 in (1) has the two-sided rejection region
 any observed z such that R2 (αL , αU ) = , z  zαL or z
z1−αU (3) where zαL and z1−αU are the 100αL and 100(1 − αU ) percentiles of the standard normal distribution and (αL + αU ) = α is the specified significance level∗ (or Type I error). For most applications, (3) is symmetric with αL = αU = (α/2), and zαL = zα/2 = −z1−(α/2) = −z1−αU ; this structure is assumed henceforth for two-sided tests of H0 in (1). The one-sided test for assessing H0 in (1) relative to the alternative Hδ : µ1 − µ2 = δ > 0

(4)

of a larger mean for population 1 than population 2 has the one-sided rejection region RU (α) = R2 (0, α) 
any observed z ; = such that z
z1−α

(5)

similarly, if the alternative (4) specified δ < 0, the one-sided rejection region would be RL (α) = R2 (α, 0). Thus, the symmetric twosided test based on R2 (α/2, α/2) is equivalent to simultaneous usage of the two one-sided tests based on RL (α/2) and RU (α/2). The power of the one-sided test (5) with respect to Hδ in (4) is ψU (δ|α) = Pr{RU (α)|Hδ }

(1)

= 1 − {z1−α − (δ/σd )},

Suppose y1 and y2 are sample means based on large sample sizes n1 and n2 (e.g., ni
40) from the two populations; also suppose the

(6)

where ( ) is the cumulative distribution function of the standard normal distribution.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

ONE-SIDED VERSUS TWO-SIDED TESTS

Table 1. Multiplier of One-Sided Test Sample Sizes for Two-Sided Test to Have the Same Power α Power

0.01

0.02

0.05

0.50 0.60 0.70 0.80 0.90

1.23 1.20 1.18 1.16 1.14

1.28 1.25 1.22 1.20 1.17

1.42 1.36 1.31 1.27 1.23

The power of the two-sided test (3) for this situation is ψ2 (δ|α) = Pr{R2 (α/2, α/2)|Hδ } = [1 − {z1−(α/2) − (δ/σd )} +{zα/2 − (δ/σd )}].

(7)

When δ > 0, ψU (δ|α) > ψ2 (δ|α), and the onesided test is more powerful. However, when δ < 0, ψ2 (δ|α) > α/2 > ψU (δ|α), and so the one-sided test’s power is not only much poorer, but is also essentially negligible. Also, in the very rare situations where rejection is indicated, it is for the wrong reason [i.e., H0 is contradicted by large z in RU (α) when actually δ < 0]. When one direction of alternatives such as (4) is of primary interest, the two-sided test, which achieves the same power ψ for specific α and δ as its one-sided counterpart, requires sample sizes that are λ(α, ψ) times larger [where λ(α, ψ)
1]. For usual significance levels 0.01  α  0.05 and power ψ
0.50, the two-sided test multiplier λ(α, ψ) of the one-sided test sample sizes n1 and n2 is given by
λ(α, ψ) =

z1−(α/2) + zψ z1−α + zψ

2 .

(8)

In Table 1, values of λ(α, ψ) are reported for α = 0.01, 0.02, 0.05 and ψ = 0.50, 0.60, 0.70, 0.80, 0.90. For the typical application of power ψ = 0.80 and significance level α = 0.05, the sample size required for a two-sided test is 27% greater than for its one-sided counterpart. Also, the multipliers λ(α, ψ) can be seen to decrease as either α decreases or ψ increases.

Some further insight about one-sided and two-sided tests can be gained from their relationship to confidence intervals∗ . The onesided test based on RU (α) in (5) corresponds to the one-sided lower bound confidence interval δ
d − z1−α σd = dL,α .

(9)

If dL,α > 0, then H0 is contradicted relative to the alternative Hδ in (4); if dL,α  0, then there is not sufficient evidence to support Hδ . In this latter context, δ may be near 0 or less than 0; but the distinction between these interpretations is not an inferential objective of a one-sided confidence interval or hypothesis test. For the purpose of the more refined assessment of whether δ is greater than 0, near 0, or less than 0, a two-sided test is needed; its corresponding confidence interval is dL,α/2  δ  dU,α/2 ,

(10)

where dL,α/2 = {d − z1−(α/2) σd } and dU,α/2 = {d + z1−(α/2) σd }. If dL,α/2 > 0, then H0 is contradicted with respect to δ > 0; if dU,α/2 < 0, then H0 is contradicted with respect to δ < 0; and if dL,α/2  0  dU,α/2 , then H0 is not contradicted and δ is interpreted as being near 0 in the sense of the confidence limits (dL,α/2 , dU,α/2 ). When support for δ > 0 is the objective of an investigation, the cost for the two-sided confidence interval’s or test’s additional capability for distinguishing between δ < 0 or δ near 0 is either reduced power for the same sample size or increased sample size for the same power. A third way to specify one-sided and twosided tests is through one-sided and twosided p-values∗ ; the one-sided p-value for assessing the one-sided alternative Hδ in (4) through z in (2) is pU (z) = 1 − (z);

(11)

if pU (z)  α, then z is interpreted as contradicting H0 on the basis of the small probability  α for repeated sampling under H0 to yield values
z. For symmetric two-sided tests of H0 in (1), the two-sided p-value∗ is p2 (z) = 2{1 − (|z|)};

(12)

ONE-SIDED VERSUS TWO-SIDED TESTS

if p2 (z)  α, then H0 is contradicted. The definition of two-sided p-values for asymmetric situations is more complicated; it involves considerations of extreme outcomes for a test statistic in both directions from H0 . For summary purposes, the rejection region, confidence interval, and p-value specifications of a one-sided test are equivalent in the sense of yielding the same conclusion for H0 ; this statement also applies to symmetric twosided tests. A concern for any one-sided test is the interpretation of values of the test statistic which would have contradicted the hypothesis if a two-sided test were used. From the inferential structure which underlies one-sided tests, such outcomes are judged to be random events compatible with the hypothesis, no matter how extreme they are. However, their nature can be a posteriori described as ‘‘exploratory information supplemental to the defined (one-sided) objective’’ of an investigation. This perspective enables suggestive statements to be made about opposite direction findings; their strengthening to inferential conclusions would require confirmation by one or more additional investigations. Another issue sometimes raised is that one-sided tests seem to make it easier to contradict a hypothesis and thereby to have a weaker interpretation than would have applied to two-sided tests. However, when the null hypothesis H0 is true, the probability of its contradiction is the significance level α regardless of whether a one-sided test or a two-sided test is used. It is easier for the onesided test to contradict H0 when its one-sided alternative applies, but this occurs because the one-sided test is more powerful for such alternatives. Some additional practical comments worthy of attention are as follows: (i) Among the commonly used statistical tests for comparing two population means, z and t-tests lead to one-sided or two-sided tests in a natural manner such as (3) and (5) due to the symmetry about zero of their standardized distributions. Chi-square and F-tests∗ for such comparisons involve squared quantities and so lead to two-sided tests. One-sided counterparts for chi-square and F-test pvalues are usually computed indirectly using

3

p1 = (p2 /2) if the difference is in the same direction as the alternative hypothesis, and p1 = 1 − (p2 /2) if the difference is in the opposite direction, where p1 and p2 are one-sided and two-sided p-values, respectively. (ii) Fisher’s exact test∗ for independence in a 2 × 2 contingency table∗ leads naturally to either a one-sided or two-sided test since the discrete event probabilities for it pertain to one or the other side of the underlying permutation distribution. However, this test is often asymmetric and then a one-sided pvalue (less than 0.5) cannot be doubled to give the corresponding two-sided p-value. (iii) Fisher’s method of combining c independent tests (see Fisher [2] and Folks [3]) is analogous to a one-sided test when its power is directed at a one-sided alternative. For this test, the one-sided p-value is the probability of larger values of QF = −2

c 

log pk

k=1

with respect to the χ 2 distribution with 2c degrees of freedom where the {pk } are one-sided p-values in the direction of the one-sided alternative of interest for the c respective tests. The p-value corresponding to the opposite side is obtained by the same type of computation with {pk } replaced by their complements {1 − pk }. (iv) A practical advantage of one-sided p-values is their descriptive usefulness for summarizing the results of hypothesis tests; such p-values contain more information than their two-sided counterparts because the onesided version identifies the direction of any group difference as well as providing the criterion for evaluating whether the hypothesis of no difference is contradicted. This additional descriptive feature eliminates the need for identifying the direction of difference as would be necessary in summary tables of two-sided p-values. Additional discussion of one-sided and two-sided tests is given in many textbooks dealing with statistical methodology, e.g., see Armitage [1], Hogg and Craig [4], Mendenhall et al. [5]. Also, see HYPOTHESIS TESTING. Acknowledgment

4

ONE-SIDED VERSUS TWO-SIDED TESTS

This research was supported in part by the U. S. Bureau of the Census through Joint Statistical Agreement JSA-84-5. The authors would like to express appreciation to Ann Thomas for editorial assistance.

REFERENCES 1. Armitage, P. (1971). Statistical Methods in Medical Research. Wiley, New York. 2. Fisher, R. A. (1932). Statistical Methods for Research Workers, 4th ed. Oliver and Boyd, Edinburgh, Scotland. 3. Folks, J. L. (1984). Combination of independent tests. In Handbook of Statistics: Nonparametric Methods, Vol. 4, P. R. Krishnaiah and P. K. Sen, eds. North-Holland, Amsterdam, Netherlands, pp. 113–121. 4. Hogg, R. V. and Craig, A. T. (1978). Introduction to Mathematical Statistics, 4th ed. Macmillan, New York. 5. Mendenhall, W., Scheaffer, R. L., and Wackerly, D. D. (1981), Mathematical Statistics with Applications, 2nd ed. Duxbury Press, Boston, Mass. See also CONFIDENCE INTERVALS AND REGIONS; EXPLORATORY DATA ANALYSIS; FISHER’S EXACT TEST; HYPOTHESIS TESTING; INFERENCE, STATISTICAL —I; INFERENCE, STATISTICAL —II; POWER; P-VALUES; and SIGNIFICANCE TESTS, HISTORY AND LOGIC OF.

GARY G. KOCH DENNIS B. GILLINGS

OPEN-LABELED TRIALS

‘‘Concealment’’ refers to whether the identity of the treatment for the next patient to be enrolled is known (1). In a fully blinded study, concealment might be taken for granted because neither the investigator nor the patient (nor all other careers, and trial personnel) are supposed to know the treatment assignment (2,3). However, even in an openlabeled study (or partially blinded study), it should still be possible—and it is certainly highly desirable—that those involved in recruiting patients are not aware of which treatment (new or ‘‘experimental,’’ active control, perhaps placebo, etc.) the next patient to be recruited will receive. If they are so aware, then it is possible (even if not always easy to verify) that they may not recruit a particular patient if they are unhappy with the proposed treatment allocation; in addition, they may delay recruiting that patient until the ‘‘next’’ treatment allocation is deemed more preferable. So, for example, in a placebo-controlled study, an investigator might subconsciously not offer participation to a more severely ill patient if they know that patient will receive placebo; but, conversely, the investigator might offer participation if they know that this patient will receive an active (or presumed active) compound. Other forms of selection bias may also occur— even including preferential participation of patients who will receive placebo, perhaps if for a given patient there is a high expectation of adverse events so that the overall benefit–risk for that patient might not be considered positive. Others have written extensively on this topic of selection bias (4), which includes proposals to measure and correct for it. The potential for selection bias exists even in studies that are planned to be fully blinded, but its obvious potential and, therefore, the scientific concern is heightened in open-labeled studies.

SIMON DAY Roche Products Ltd. Welwyn Garden City, UK

Open-labeled trials are a stark contrast to blinded (or masked) trials. Blinded (or masked) trials are those in which typically the patient and the treating physician (often, even if not always, synonymous with the person who observes the outcome) are unaware of the assigned treatment. Such design aspects of trials are important principally to avoid bias in selection of patients and the measurement and assessment of outcomes. Various methods exist to help keep studies double blind. The types of methods—and their complexity—vary considerably depending on the type of trial and the type of intervention (a broader term than just ‘‘treatment’’) being assessed. Other articles in this volume address such issues. The purpose of the current article is not to defend openlabeled studies but, instead, to explain why they are sometimes necessary and even sometimes preferred to blinded studies. However, even if conceptually an open-labeled study might be necessary (or preferable) to a fully blinded study, major deficiencies need to be recognized and addressed wherever possible in the study design and management. 1

THE IMPORTANCE OF BLINDING

In some sense, this article could be recognized as trying to defend the use of open-labeled studies, whereas other articles in this encyclopedia implicitly criticize them and strongly argue for fully blinded trials. Indeed, the importance of blinding should not be underestimated. Open-labeled trials are generally considered to be of lower scientific merit and particularly susceptible to bias when compared with blinded trials.

1.2 Assessment Bias in Trials 1.1 Selection Bias in Trials: Blinding and Concealment

Outcomes or endpoints in trials should be measured carefully, accurately, and without any differential bias in favor of one or other of the investigational treatments. Some outcomes are easy to measure objectively and

The terms ‘‘blinding’’ and ‘‘concealment’’ are often confused with each other, and sometimes it is assumed that achieving one necessarily achieves the other. This is not the case.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

OPEN-LABELED TRIALS

without any differential bias: The most obvious example being death. However, even this endpoint is not always immune from measurement bias. A primary endpoint of ‘‘death within 2 hours’’ (an endpoint that might be applicable in a study of acute cardiac failure) might be compromised by uncertainties in confirming exact time of death and, hence, whether it has been before or after 2 hours from randomization. This procedure might not introduce any bias in the comparison of treatments in a fully blinded trial, but if the treatment allocation is known, then some bias might ensue. If it were known that a patient had been allocated to receive placebo, then the researcher might have a lower expectation of survival; thus, agreements about time of death being 1 hour 59 minutes or 2 hours 1 minute might be handled differently. Some trials might be designed as openlabeled trials because the intervention group need regular and intensive monitoring (perhaps for dose adjustment or for managing anticipated side effects), and the intensive nature of the monitoring seems a compelling reason not to subject the control patients to the same procedures (or at least at the same frequency). The ethical and practical considerations seem sensible and reasonably easy to justify. Perhaps, in a specific example, patients allocated to a control arm are to be observed for routine follow-up once every 3 months, whereas those patients allocated to the new ‘‘experimental’’ arm might be observed every 6 weeks. Now, in the experimental arm, with patients being observed more often, there is a higher chance that the trial endpoint might be observed sooner or adverse events reported at a higher rate than for patients randomized to the control arm. 1.3 Patient Reporting Bias The most obvious scenario in which bias might be introduced by the patient is with self-reported assessments, which are often referred to as patient-reported outcomes (PROs). Many such examples can be listed, such as pain scales and quality of life. Why might patients report differentially on such scales simply because they know (or believe

they know) the identity of the treatment they have received? We can speculate (and will do so shortly), but it is sufficient to note that empirical investigations have demonstrated differential responses repeatedly to different treatment identities, even when the ‘‘treatments’’ have, in fact, been placebos. de Craen et al. (5) conducted a systematic review of the literature on perceived differential effects based on the color of drugs. They found that red, orange, and yellow tablets were best for stimulant drugs, whereas blue and green were best for sedatives. Kaptchuk et al. (6) compared a sham device (a sham acupuncture needle) with an inert pill in patients with persistent arm pain. Some endpoints showed no evidence of any differential effect (that is not to say they demonstrated there was no differential effect, they simply failed to show any effect); other endpoints did seem to show some differential effect. Ernst and Resch (7) note the important distinction between a true placebo and a ‘‘no-treatment’’ option—the mere distinction highlighting that an observable effect of placebo is not uncommon. Recently, the term ‘‘nocebo’’ has been introduced to describe a placebo that is expected to have harmful effects. So why might patients respond differentially to different colors, shapes, types of placebo? This response is speculation but Sacket (8), for example, presents a whole host of possible sources of biases that may crop up in all aspects of analytical research. Some of those most plausibly likely to influence either how patients report PROs or, indeed, how physiological measurements might be affected include: Apprehension bias. Certain measures (pulse, blood pressure) may alter systematically from their usual levels when the subject is apprehensive, (e.g., blood pressure during medical interviews). Obsequiousness bias. Subjects may systematically alter questionnaire responses in the direction they perceive desired by the investigator. Expectation bias. Observers (or patients) may systematically err in measuring and recording observations so that they concur with prior expectations (8).

OPEN-LABELED TRIALS

1.4 Efficacy and Safety Of course, influences of bias are not restricted only to efficacy measurements or only to safety measurements. Either, or both, could be influenced favorably or unfavorably. Ultimately, the results of any trial—and the decision whether to prescribe a therapy—should be an evaluation of the benefit–risk ratio. A treatment that is perceived as beneficial may have a positively biased assessment of its efficacy and a negatively biased assessment of its safety (that is, ‘‘good all around’’). Conversely, a treatment that is perceived as less beneficial may have both worse efficacy and worse safety reported than its (perceived) better comparison treatment. More confusingly, a treatment that is considered ‘‘strong’’ (perhaps a high dose of a treatment, or even multiple tablets suggestive of a high dose) may have a positive bias in relation to its efficacy but simultaneously a negative bias relating to its safety. And, of course, different patients and different observers all with different expectations may all introduce different degrees (or even directions) of biases. 2 REASONS WHY TRIALS MIGHT HAVE TO BE OPEN-LABEL Achieving the appropriate degree of blinding for a variety of the necessary study staff has been discussed above. Methods include simple ‘‘placebo’’ pills or capsules, ‘‘placebo’’ injections (which may simply be injections of saline solution, for example), sham surgery, and so on. Some situations are more difficult to manage than others are. 2.1 Different Formulations Comparing products that have different pharmaceutical forms can be difficult, but solutions do exist in some cases. Capsules and tablets can sometimes be compared by placing the tablets inside inert capsules (so that patients just believe that they are swallowing a capsule). However, issues of bioavailability of the hidden tablet may need to be addressed, and a bioequivalence study that compares the tablets (swallowed as tablets) and the tablets hidden inside capsules may be necessary. Of course, this procedure raises

3

the issue of how this study could be blinded and, if it can be, then might it be possible (and more efficient) to avoid such a bioequivalence study and simply to carry out the ‘‘real’’ study in a blinded manner. Comparing treatments that are not just different pharmaceutical formulations but that have different routes of administration becomes much more difficult (but see ‘‘doubly dummy’’ below). 2.2 Sham Surgery Comparing surgical with medical interventions (or even comparing with ‘‘no treatment’’) in a blinded way is very challenging— both practically and ethically. The notion of ‘‘sham’’ surgery has often been used but nearly always causes controversy. One example is of investigators who drilled holes into the skulls of patients with Parkinson’s disease to transplant embryonic dopamine neurons (9). Those patients randomized not to receive the intervention still had the holes drilled in their skulls—with all the potential for serious adverse consequences (anesthetics, the drilling procedure, subsequent infection, and so on). The primary outcome was a subjective rating scale of symptoms of Parkinson’s disease so that the maximal level of blinding was considered very important. 2.3 Double Dummy The most common solution to blinding when the treatments are obviously different (either in appearance, route of administration, time of administration, etc.) is to use a technique called ‘‘double dummy.’’ More properly, it might be called ‘‘double placebo’’ because, effectively, two (or possibly more) different placebos are used in the same study. Consider as a simple example a trial to compare the effects of tablet ‘‘A’’ with transdermal patch ‘‘B.’’ Patients randomized to ‘‘A’’ are also given a placebo patch and patients randomized to ‘‘B’’ also take placebo tablets. So every patient receives a tablet and a patch but no patient is unblinded. With more than two treatments, a similar technique can be used, but the burden increases on the patient to take more and more medication (most of it placebo!).

4

OPEN-LABELED TRIALS

2.4 Partially Blinded Studies In trials with more than two arms, when blinding cannot be achieved fully, it may be possible to blind some comparisons between some treatment arms. This technique would seem to be ‘‘better than nothing,’’ although the extent to which the credibility and reliability of the trial can then be assured is difficult to judge. An example of blinding of some treatment comparisons is in the ‘‘TARGET’’ trial (or trials) (10,11). Lumiracoxib was compared with both naproxen and ibuprofen, using a double-dummy approach in two substudies. In substudy one, patients were randomized to lumiracoxib or naproxen; in substudy two, patients were randomized to lumiracoxib or ibuprofen. Each substudy used double-dummy randomization so that, for example, in substudy one, patients did not know whether they were receiving lumiracoxib or naproxen—although they did know they were not receiving ibuprofen. Conversely, in substudy two, patients did not know whether they were receiving lumiracoxib or ibuprofen—but they did know they were not receiving naproxen. So here is an example in which what might have had to be an open-labeled study or a ‘‘triple-dummy’’ study could be made at least partially blinded and practically and logistically manageable.

3 WHEN OPEN-LABEL TRIALS MIGHT BE DESIRABLE In general, open-label studies will be less preferred than double-blind studies. However, it is sometimes argued that in highly pragmatic trials [see, for example, Schwartz et al. (12)] open-labeled treatment is more appropriate. Scientifically, we generally want to know the relative benefit of different pharmaceutical preparations. The relative advantages and disadvantages of the science are of more interest than efficacy (or harm) caused by their physical presentation. Yet it is selfevident that different forms of presentation may be more or less acceptable to patients (which may subjectively influence efficacy); different forms of presentation are also likely to affect patient compliance strongly, which, in turn, will affect both efficacy and safety

(see also the entry on Patient preference trials). So, a balance must be found. Excessive inclusion of placebos (double- or higher-order dummy designs) will affect compliance and adherence to treatment regimens. Seemingly minor issues of taste (or perhaps size of tablet to swallow) may impact on patients’ willingness to take medication so hence on the clinical benefit they might gain. So we need to ask carefully what question we are trying to answer: Is it about the science of the drug (or perhaps surgical procedure) or is it about the drug (or other intervention) ‘‘as it is’’? If we conduct a trial to answer the latter question and show one intervention seems better than another does, then we may not know whether it is the treatment per se that is better, or whether it is the way in which the treatment is presented or given, or whether it is a combination of the two.

4 CONCLUDING COMMENTS This article illustrates the breadth of studies that may fall under the umbrella term of ‘‘open-label’’ and how we might go about minimizing the potential for bias in such studies. Partial blinding can sometimes be a (partial) solution; but it can cause trials (or treatment regimens) to be unlike what would be used in clinical practice. It is very difficult (if not impossible) to evaluate the extent of any bias such procedures might introduce. Finally, we should note that whereas fully blinded studies are typically considered the gold standard, in highly pragmatic trials, it may be the open-label nature of the treatments that is exactly the intervention we wish to study. Blinding in these situations would not be a good thing— the open-label study would be much preferred.

REFERENCES 1. D. G. Altman and K. F. Schulz, Concealing treatment allocation in randomized trials. Br. Med. J. 2001; 323: 446–447. 2. S. Day, Blinding or masking. In: P. Armitage and T. Colton (eds.), Encyclopedia of Biostatistics, 2nd ed., vol. 1. Chichester, UK: John Wiley and Sons, pp. 518–525.

OPEN-LABELED TRIALS 3. S. J. Day and D. J. Altman, Blinding in clinical trials and other studies. Br. Med. J. 2000; 321: 504. 4. V. Berger, Selection Bias and Covariate Imbalances in Randomized Clinical Trials. Chichester, UK: John Wiley and Sons, 2005. 5. A. J. M. de Craen, P. J. Roos, A. L. de Vries, and J. Kleijnen, Effect of colour of drugs: systematic review of perceived effect of drugs and their effectiveness. Br. Med. J. 1996; 313: 1624–1625. 6. T. J. Kaptchuk, W. B. Stason, R. B. Davis, T. R. Legedza, R. N. Schnyer, C. E. Kerr, D. A. Stone, B. H. Nam, I. Kirsch, and R. H. Goldman, Sham device v inert pill: randomized controlled trial of two placebo treatments. Br. Med. J. 2006; 332: 391–397. 7. E. Ernst and K. L. Resch, Concept of true and perceived placebo effects. Br. Med. J. 1995; 311: 551–553. 8. D. L. Sackett, Bias in analytic research. J. Chronic Dis. 1979; 32: 51–63. 9. C. R. Freed, P. E. Greene, R. E. Breeze, W. Tsai, W. DuMouchel, R. Kao, S. Dillon, H. Winfield, S. Culver, J. Q. Trojanowski, D. Eidelberg, and S. Fahn, Transplantation of embryonic dopamine neurons for sever Parkinson’s disease. N. Engl. J. Med. 2001; 344: 710–719. 10. T. J. Schnitzer, G. R. Burmester, E. Mysler, M. C. Hochberg, M. Doherty, E. Ehrsam, X. Gitton, G. Krammer, B. Mellein, P. Matchaba, A. Gimona, and C. J. Hawkey, Comparison of lumiracoxib with naproxen and ibuprofen in the Therapeutic Arthritis Research and Gastrointestinal Event Trial (TARGET), reduction in ulcer complications: randomized controlled trial. Lancet 2004; 364: 665–674.

5

11. M. E. Farkouh, H. Kirshner, R. A. Harrington, S. Ruland, F. W. A. Verheugt, T. J. Schnitzer, G. R. Burmester, E. Mysler, M. C. Hochberg, M. Doherty, E. Ehrsam, X. Gitton, G. Krammer, B. Mellein, A. Gimona, P. Matchaba, C. J. Hawkey, and J. H. Chesebro, Comparison of lumiracoxib with naproxen and ibuprofen in the Therapeutic Arthritis Research and Gastrointestinal Event Trial (TARGET), cardiovascular outcomes: randomized controlled trial. Lancet 2004; 364: 675–684. 12. D. Schwartz, R. Flamant, and J. Lellouch, Clinical Trials (Trans. M. J. R. Healey). London: Academic Press Inc., 1980.

FURTHER READING S. Senn, Statistical Issues in Drug Development. Chichester, UK: John Wiley and Sons, 2007.

CROSS-REFERENCES Active-controlled trial clinical development plan combination trials non-inferiority trial phase III trials postmarketing surveillance preference trials quality of life

OPTIMAL BIOLOGICAL DOSE FOR MOLECULARLY TARGETED THERAPIES

cells, so cytotoxic agents may lead to other organ damage and may eventually lead to shorter overall survival of cancer patients. In contrast, molecularly targeted agents demonstrate tumor growth inhibition but not tumor shrinkage. These agents may offer clinical benefits such as longer overall survival, progression-free survival, and better quality of life. Most molecularly targeted agents are less toxic than conventional cytotoxic agents. Thus, the maximum therapeutic effect may occur at doses well below the MTD. The intensity of dose-toxicity curve may not be predictive of the therapeutic effect. Because dose-escalation is usually guided by toxicity in traditional phase I clinical trials, such designs may be inappropriate for optimizing the use of molecularly targeted drugs. We briefly review the phase I clinical trial designs for cytotoxic agents, and then investigate the designs for molecularly targeted agents.

CHUL AHN Department of Clinical Sciences and Simmons Comprehensive Cancer Center University of Texas Southwestern Medical Center Dallas, Texas

SEUNG-HO KANG Department of Statistics Ewha Woman’s University Seoul, South Korea

YANG XIE Department of Clinical Sciences and Simmons Comprehensive Cancer Center University of Texas Southwestern Medical Center Dallas, Texas

The main purpose of a phase I clinical trial of a cytotoxic chemotherapeutic agent is ordinarily to find the highest dose with an acceptable rate of toxicity, often referred to as the maximum tolerated dose (MTD), of the new agent, which will be used as a recommended dose for experimentation in phase II efficacy studies. This recommended phase II dose is determined under the assumption that the higher the dose, the greater the antitumor activity. Thus, it is assumed that the intensity of dose-toxicity curve is predictive of the therapeutic effect. Over the past decade, a considerable number of studies have been conducted to investigate the statistical properties of phase I clinical trials of cytotoxic anticancer drugs (1–15). The emergence of a growing number of molecularly targeted therapies as anticancer agents challenges the traditional phase I clinical trial paradigm in a variety of ways. The clinical development of cytotoxic agents is based on the assumption that the agents will shrink tumors and the shrinkage of tumors will prolong the progression-free survival and overall survival of cancer patients. However, cytotoxic agents that shrink tumors may kill normal cells in addition to cancer

1 PHASE I DOSE-FINDING DESIGNS FOR CYTOTOXIC AGENTS Phase I cancer clinical trials intend to rapidly identify the MTD of a new agent for further studies. The standard phase I clinical trial identifies the MTD through an algorithm-based dose-finding approach in which dose escalation and de-escalation depend on the number of patients experiencing dose-limiting toxicity (DLT). The standard 3 + 3 algorithm-based dose-finding approach has poor operating characteristics compared with the model-based dose-finding approaches such as the continual reassessment method (CRM) (1, 4). The major criticism of the standard phase I design is that the MTD has no interpretation as an estimate of the dose level that yields a specified toxicity rate. Kang and Ahn (7–9) show that the standard algorithm-based 3 + 3 design cannot provide the accurate estimates of the MTD when the specified target toxicity rate is high. In contrast to the common belief that the standard 3 + 3 design produces 33% toxicity rate at the MTD, Kang and Ahn (7–9)

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

OPTIMAL BIOLOGICAL DOSE FOR MOLECULARLY TARGETED THERAPIES

and He et al. (5) have shown that the expected toxicity rate at the MTD is between 19% and 24%, regardless of target toxicity level. He et al. (5) proposed a model-based approach for the estimation of the MTD that follows a standard 3 + 3 design. They showed that the model-based approach yields a less biased estimate than the standard algorithm-based 3 + 3 design. O’Quigley et al. (12) proposed the CRM, which overcomes the problems of the standard 3 + 3 design by reducing the number of patients treated with possibly ineffective dose levels, and by yielding a dose level with a specified toxicity rate. The CRM design outperforms the standard 3 + 3 design, but it has some difficulties to be implemented in practice because it takes too long to complete the trial. The CRM treats one patient at a time, and clinicians are not comfortable with using the target dose close to the MTD as the starting dose for phase I clinical trials (10). Goodman et al. (4) accommodated these concerns by proposing a modified CRM. Goodman et al. (4) and Ahn (1) showed that the modified CRM reduces the duration of the trial by 50% to 67%, and reduces the toxicity incidence by 20% to 35% from the original CRM. These designs require each patient to be followed for a certain period to observe toxicity. The next cohort of patients is assigned to the following dose level only when full observation period of each cohort has been completed. Muler et al. (16) introduced the timeto-event continual reassessment method to eliminate the need for full observation of each patient before estimating the MTD. This method accounts for the time of the observation period as a proportion of the maximum length of observation. Patients without toxicity are weighted by that proportion, and patients with toxicity receive the full weight. These weights are applied to the likelihood used in the CRM to determine the MTD. There are other phase I designs proposed for cytotoxic agents such as escalation with overdose control (2), random walk rules (3), two-stage design (14), and decision-theoretic approaches (15). In spite of the criticisms on the standard 3 + 3 algorithm-based design, the standard 3 + 3 phase I design is still

widely used in most practical cases. The reason might be that the standard designs do not require elaborate statistical considerations and they have been in use by many investigators over the years. 2 PHASE I DOSE-FINDING DESIGNS FOR MOLECULARLY TARGETED AGENTS The recent emergence of molecularlytargeted therapies has created major challenges in drug development. These newer agents are commonly referred to as having cytostatic effects because many of these agents show antimetastatic or growthinhibitory effects instead of inducing rapid tumor regression (16–30). When many of these agents slow or stop the growth of tumors and the development of metastases, the phase I clinical trial designs proposed for cytotoxic agents may not be effective in identifying the dose level that is clinically suitable for molecularly targeted agents. Because these agents act on highly specific targets that are differentially expressed or activated in cancer cells, they may have a very wide therapeutic ratio. Common toxicity of many cytotoxic drugs is not usually seen in molecularly targeted drugs. In molecularly targeted drugs, efficacy (such as target inhibition, pharmacodynamic effect, and immunologic response) is used as the alternative endpoint for phase I trials to toxicity measurement. The optimal biological dose (OBD) is usually defined as the dose level that gives the highest efficacy. The OBD is also defined as the dose level recommended for phase II clinical trials of molecularly targeted drugs (30). This recommended phase II dose is also defined as the biological modulatory dose (19) or the biologically adequate dose (22). The OBD, the dose recommended for a phase II trial, may occur at doses that are well below the MTD. For example, bevacizumab (Avastin), a monoclonal antibody to vascular endothelial growth factor, was approved for the treatment of metastatic colorectal cancer by the U.S. Food and Drug Administration in February 2004. The MTD of bevacizumab monotherapy is 20 mg/kg due to the toxicity of severe migraine headache in some patients (17). In a randomized phase II trial of bevacizumab with chemotherapy, a 5 mg/kg dose

OPTIMAL BIOLOGICAL DOSE FOR MOLECULARLY TARGETED THERAPIES

yielded a higher response rate, longer median progression-free survival, and longer overall survival in patients with metastatic colorectal carcinoma (18). 2.1 Dynamic De-escalating Designs Most molecularly targeted agents are less toxic than conventional cytotoxic agents, and as a result, the maximum therapeutic effect may occur at doses that are well below the MTD. Dowlati et al. (19) and Emmenegger and Kerbel (20) apply a dynamic deescalating dosing strategy to determine the OBD for the agent, SU5416, which is an oral small-molecule vascular endothelial growth factor receptor-2 inhibitor. The rational for this novel dose de-escalation design is based on the fact that the MTD of SU5416 had been previously determined. The unique feature of this approach is to de-escalate to the OBD (referred to as a biological modulatory dose) based on pharmacodynamic information instead of toxicity. The approach tries to show a pharmacodynamic effect at the MTD. Dose de-escalation will be made to investigateifthe lower dose exhibits the same amount of pharmacodynamic effect as the higher dose. If the lower dose exhibits the same effect, then the lower dose will be chosen as the preferred dose. Dowlati et al. (19) and Emmenegger and Kerbel (20) chose the following pharmacodynamic effect as significant for trial design: (1) a 35% reduction in microvessel density in sequential tumor biopsies, and (2) a 35% reduction in blood flow within tumor as assessed by dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI). The approach of Dowlati et al. to finding the OBD was as follows: Ten patients are enrolled at the MTD. If at least 5 of the 10 patients show the intended pharmacodynamic effect, dose de-escalation will continue until a reduction in pharmacodynamic effect is observed. The dose above the level in which the dose deescalation stops would be considered as the OBD of the agent. The rationale of this design is that, if the lower dose displays similar biological response rates to the MTD, it qualifies as a candidate for the biologically adequate dose. The number of patients at each dose level is greater than that in the standard

3

3 + 3 design. The advantage of this approach over the standard escalating design is that very few patients will receive a drug dose less than the OBD. To be qualified as the biologically adequate dose in this approach, the number of biological responses at the lower dose must not be less than the number of MTD responses minus 2. However, the method is rather ad hoc. The statistical properties of the biologically adequate dose need to be investigated. 2.2 Dose Determination through Simultaneous Investigation of Efficacy and Toxicity Dynamic de-escalating design determines the OBD based on the determination of the MTD (19). That is, the dose level for the MTD should be known in advance to determine the dose level for the OBD. The dose de-escalation is only determined by the response rate in the dynamic de-escalating design. Instead of determining the MTD by toxicity and then de-escalating the dose to identify the OBD by efficacies (such as immunologic response and pharmacodynamic effect), we can determine the OBD by simultaneously investigating efficacy and toxicity from molecularly targeted agents. Suppose that the dose-efficacy curves of the agents are not monotonically increasing and the efficacy rates are at least 30%. The following design is a modified standard 3 + 3 design to accommodate the response rate for the determination of the OBD. This design, just like the standard 3 + 3 design, is based on intuitively reasonable rules instead of formally justifiable statistical rules. Each dosing level enrolls three patients. The design consists of two dose-escalation steps: Step 1 uses the standard dose escalation. If at least one response occurs in the cohort of patients in step 1, dose escalation approach is switched to step 2. Step 2 uses six patients per dose level. The following dose escalation approach is used for the determination of the OBD. Step 1 1. If, in any cohort of three patients, no patient experiences a DLT, the next cohort will be enrolled as follows.

4

OPTIMAL BIOLOGICAL DOSE FOR MOLECULARLY TARGETED THERAPIES

A. If none of them has response, then the next cohort will be enrolled at the next higher dose level. B. If at least one of them has response, then switch to step 2. 2. If one patient experiences a DLT, then three additional patients will be enrolled at the same dose. A. If none of three additional patients experiences a DLT (i.e., a total of 5/6 do not experience a DLT), and none of the extended cohort of six patients experiences a response, the next cohort of three patients will be enrolled at the next higher dose level. B. If none of three additional patients experiences a DLT (i.e., a total of 5/6 do not experience a DLT), and at least one of the extended cohort of six patients experiences a response, then switch to step 2. C. If a DLT occurs in more than one patient among the additional three patients (for a total of ≥2/6), then the MTD is exceeded. 3. If a DLT occurs in two or more patients in a dosing cohort, then the MTD level is exceeded. Step 2 Patients are accrued in cohorts of six including the patients at the current dose level from step 1. That is, if only three patients are recruited at the dose level from step 1, three more patients are accrued at that dose level. 1. If zero or one out of six patients experiences a DLT, A. If no one has a response, then the OBD is exceeded, and the dose escalation is terminated. Three additional patients will be enrolled at the next lower dose level if only three patients are treated at that dose level. (Note that at least one response is observed at the dose level from step 1. However, at the other dose levels in step 2, no response may be observed out of six patients.)

B. If at least one patient has a response, then the dose is escalated in subsequent patients. 2. If at least two patients experience a DLT, then the MTD level is exceeded. Three additional patients will be enrolled at the next lower dose level if only three patients are treated at that dose level. When the MTD level is exceeded, the dose escalation is terminated. Then, the MTD is defined as the next lower dose level. Three more patients will be treated at the next lower dose level if only three patients are treated at that dose. The OBD of a molecular targeted drug is estimated as the dose level at or below the MTD with the highest response rate. Here, the standard 3 + 3 design is modified to accommodate response rates to identify a phase II dose. This modified 3 + 3 design can be generalized to any A + B design. Operating characteristics of this design need to be investigated. 2.3 Individualized Maximum Repeatable Dose (iMRD) Takahashi et al. (21) describe a dose-finding approach to identify an optimal dose, referred to as individualized maximum repeatable dose (iMRD). This design potentially incorporates both escalation and de-escalation steps. A starting dose is half the MTD, and then dose is de-escalated or escalated depending on the toxicity of the agent. The dose is escalated for grade 0 toxicity, maintained at the same dose level for grade 1 toxicity, and de-escalated for toxicity grade ≥2. The modifications are still toxicity guided but allow one to approach the iMRD, which is defined as the dose associated with minimal (grade ≤1) toxicity during chronic administration of the drug. Takahashi et al. (21) suggest that the iMRD is a simple method to identify a patient’s tailored chemotherapy dose and could be the optimal dose for patients with noncurable cancers such as metastatic pancreatic cancer. This approach is appealing because of its easy implementation and the antitumor effects seen. However, this approach does not address the need to find the recommended dose for the phase II study.

OPTIMAL BIOLOGICAL DOSE FOR MOLECULARLY TARGETED THERAPIES

2.4 Proportion Designs and Slope Designs Korn et al. (23) noted that statistical trial designs for identifying the OBD may require more patients than those studied for phase I cytotoxic agents. To address this concern, Hunsberger et al. (22) proposed designs in which the goal is to find a biologically adequate dose under the assumption that the target response is a binary value determined in each patient. They defined an adequate dose as either a dose that yields a specific high response rate or a dose in the plateau, and they developed two types of designs, the proportion design and the slope design, respectively, to incorporate the two definitions of an adequate dose. In the proportion design, proportion [4/6], an adequate dose means a dose that produces a specific high response rate. Three patients are assigned to the first dose. One proceeds to the next higher dose level with a cohort of three patients when ≤1/3 responses are observed. Three more patients are treated at this dose level if ≥2/3 responses are observed. One continues escalation if ≤3/6 responses are observed. The dose level that yields ≥4/6 responses or the maximum dose level tested is considered to be the adequate dose and is recommended for future clinical trials. They also propose the slope design, which is intended to stop escalation if the target response rate seems to remain constant. The escalation design depends on the estimated slope of the regression line, using the response rate at each dose level as the dependent variable and the dose level as the independent variable. The dose with the highest response rate is the recommended dose to be used in subsequent clinical trials. To address the concern that more patients may be required to identify the OBD than those for a phase I cytotoxic agent, Hunsberger et al. (22) investigated the performance of the proportion design and slope designs through simulations. They defined an adequate dose as only a dose in the plateau, and assumed that there is little or no toxicity associated with the molecularly targeted drug being studied. They investigated the performance of the two designs with respect to how often the designs reach a plateau and treat fewer patients at inactive doses. Through limited simulations, the

5

designs were shown to perform adequately with only a few patients treated at each dose level. They suggested immediately switching to a dose-escalation approach based on cytotoxic agents if any DLT is observed. They also suggested the use of aggressive doseescalation steps if the agent is not expected to cause toxicities. The utility of these designs needs to be investigated by prospective evaluation in future phase I clinical trials of molecularly targeted agents. 2.5 Generalized Proportion Designs Kang et al. (24) investigated the statistical properties of the proportion designs that can be used to determine a biologically adequate dose of molecularly targeted agents. They proposed generalized proportion designs that have four parameters; they derived the exact formula for the probability of each dose level that is recommended for phase II clinical trials. They also derived the exact formula for the number of patients needed to complete the trial. Using the exact formulas, they computed the expected number of patients who will complete the trial and computed the expected response rate at the recommended dose for comparison with specific high response rates. In the proportion [4/6] design, Hunsberger et al. (22) considered de-escalation when the starting dose level had achieved ≥4/6 responses. However, such probability is negligible. Furthermore, de-escalation produces very complicated but unimportant terms in exact formulas, so Kang et al. (24) did not consider dose deescalation. Kang et al. (24) generalized the proportion designs as follows: 1. Escalate in cohorts of size A while ≤C/A responses are observed. 2. Treat B more patients at the same dose level if ≥(C + 1)/A responses are observed. 3. Continue escalation as in steps 1 and 2 if ≤D/(A + B) responses are observed. 4. Use the dose level that yields ≥(D + 1)/(A + B) responses as the recommended dose for phase II clinical trials. The proportion [4/6] and [5/6] designs in Hunsberger et al. (22) correspond to the cases

6

OPTIMAL BIOLOGICAL DOSE FOR MOLECULARLY TARGETED THERAPIES

of (A, B, C, D) = (3, 3, 1, 3), (3, 3, 1, 4). To speed up dose escalation, Kang et al. (24) modified the generalized proportion designs by incorporating an accelerated design that uses single-patient cohorts until a response is observed. Accordingly, the modified generalized proportion designs are conducted as follows. One patient is assigned to the first dose level. The design proceeds to the next higher dose level with a single-patient cohort when a response is not observed. If the first response is observed at a dose level k, the accelerated design is converted into the standard proportion designs by assigning (A − 1) more patients so that A patients are assigned to the dose level k. The remaining steps are the same as those in the generalized proportion designs. Kang et al. (24) investigated the statistical properties of the modified generalized proportion design. Specifically, they computed the expected response rate at the recommended dose and the expected number of patients to finish trials for each design, in view of finding out which designs produced specific high response rates such as 60%, 70%, or 80%. 3

DISCUSSION

Drug development currently takes too long and costs too much because it is so unproductive. Most therapeutic drugs were developed with a lack of information on their molecular targets, which can be used to test the therapeutic efficacy (31). DiMasi et al. (32) estimated that the average cost of bringing a new drug from the time of investment to marketing in 2000 was U.S. $802 million. The genetic profile of a patient can improve the diagnosis of the underlying cause of the disease and allow the selection of a specific drug treatment, which will maximize drug efficacy with fewer serious adverse drug reactions (33). Biomarkers are very valuable in the early phases of clinical development for guidance in dosing and for selection of the lead compounds (34). Because the biomarker expression profile will rule out the use of molecular targeted drugs in some patients, this will increase the probability of success of target molecules and reduce the drug development cost. Biomarkers that can be used

to identify eligible patients for clinical trials, measure adverse drug reactions, and quantify drug efficacy are urgently needed to accelerate drug development. For a cytotoxic drug, toxicity is a major endpoint, and the MTD is usually easy to obtain. However, a cytostatic drug usually causes less acute toxicity because most of these agents are target specific. That is, the MTD based on acute toxicity will not be the optimal dose chosen for phase II evaluation of a cytostatic agent. For a cytostatic drug, we need the highest dose that allows chronic administration, which is likely to be different from the traditional acute MTD. Because a cytostatic drug is expected to be used for a prolonged period of time, the determination of the MTD and the OBD based on the first one or two cycles of chemotherapy is likely to be more problematic for the OBD than the MTD. In the early stage of clinical trials of a cytostatic drug, emphasis should be given to describing chronic toxicity (35). There is an increasing need for novel statistical designs for phase I clinical trials of molecularly targeted drugs as there is a growing need to determine a dose that yields optimal biological activity based on target inhibition or response rather than toxicity. The phase I clinical trial designs proposed for cytotoxic agents may not be effective in identifying the dose level that is clinically suitable for molecularly targeted agents. The MTD of molecularly targeted drugs may be higher than the dose level required to achieve the maximum desired biological activity. Determination of the OBD will provide more useful information for further drug development of molecularly targeted drugs. It will be of considerable interest to investigate the performances of dose-finding approaches for molecularly targeted agents. The utility of these designs warrants prospective evaluation in future clinical trials of molecularly targeted drugs. The first-generation, target-based anticancer drugs such as imatinib, trastuzumab, gefitinib are now regarded as established drugs. Combination therapies using a molecularly targeted drug with a conventional cytotoxic agent are frequently being tested (25). Dose-finding approaches for the combinatorial agents need to be developed, and the

OPTIMAL BIOLOGICAL DOSE FOR MOLECULARLY TARGETED THERAPIES

performance of these agents must be thoroughly evaluated. REFERENCES 1. C. Ahn, An evaluation of phase I cancer clinical trial designs. Stat Med. 1998; 17: 1537–1549. 2. J. Babb, A. Rogatko, and S. Zacks, Cancer phase I clinical trials: efficient dose escalation with overdose control. Stat Med. 1998; 17: 1103–1120. 3. S. Durham, N. Floumoy, and W. Rosenberger, A random walk rule for phase I clinical trials. Biometrics. 1997; 53: 745–760. 4. S. Goodman, M. Zahurak, and S. Piantadosi, Some practical improvements in the continual reassessment method for phase I studies. Stat Med. 1995; 14: 1149–1161. 5. W. He, J. Liu, B. Binkowitz, and H. Quan, A model-based approach in the estimation of the maximum tolerated dose in phase I cancer clinical trials. Stat Med. 2006; 25: 2027–2042. 6. A. Ivanova, Escalation, group and A + B designs for dose-finding trials. Stat Med. 2006; 25: 3668–3678. 7. S. Kang and C. Ahn, The expected toxicity rate at the maximum tolerated dose in the standard phase I cancer clinical trial design. Drug Inf J. 2001; 35: 1189–1200. 8. S. Kang and C. Ahn, An investigation of the traditional algorithm-based designs for phase I cancer clinical trials. Drug Inf J. 2002; 36: 865–873. 9. S. Kang and C. Ahn, Phase I cancer clinical trials. In: S. Chow (ed.), Encyclopedia of Biopharmaceutical Statistics. Dekker; 2003; 1–6. DOI:10.1081/E-EBS120022143. 10. E. Korn, D. Midthune, T. Chen, L. Rubinstein, M. Christian, and R. Simon, A comparison of two phase I trial designs. Stat Med. 1994; 13: 1799–1806. 11. Y. Lin and W. Shih, Statistical properties of the traditional algorithm-based designs for phase I cancer clinical trials. Biostatistics. 2001; 2: 203–215. 12. J. O Quigley’ M. Pepe, and M. Fisher, Continual reassessment method: a practical design for phase I clinical trials in cancer. Biometrics. 1990; 46: 33–48. 13. T. Smith, J. Lee, H. Kantarjian, S. Legha, and M. Raber, Design and results of phase I cancer clinical trial: three-year experience at M.D. Anderson cancer center. J Clin Oncol. 1996; 4: 287–295.

7

14. B. Storer, Design and analysis of phase I clinical trials. Biometrics. 1989; 45: 925–937. 15. J. Whitehead, Bayesian decision procedures with application to dose-finding studies. Stat Med. 1997; 11: 201–208. 16. H. Muler, C. J. McGinn, D. Normolle, T. Lawrence, D. Brown, et al., Phase I trial using a time-to-event continual reassessment strategy for dose escalation of cisplatin combined with gemcitabine and radiation therapy in pancreatic cancer. J Clin Oncol. 2004; 22: 238–243. 17. M. A. Cobleigh, V. K. Langmuir, G. W. Sledge, K. D. Miller, L. Haney, et al., A phase I/II dose-escalation trial of bevacizumab in previously treated metastatic breast cancer. Semin Oncol. 2003; 30: 117–124. 18. F. Kabbinavar, H. I. Hurwitz, L. Fehrenbacher, N. J. Meropol, W. F. Novotny, et al., Phase II randomized trial comparing bevacizumab plus fluorouracil (FU)/leucovorin (LV) with FU/LV alone in patients with metastatic colorectal cancer. J Clin Oncol. 2003; 21: 60–65. 19. A. Dowlati, K. Robertson, T. Radivoyevitch, J. Waas, N. Ziats, et al., Novel phase I dose de-escalation design to determine the biological modulatory dose of the antiangiogenic agent SU5416. Clin Cancer Res. 2005; 11: 7938–7944. 20. U. Emmenegger and R. Kerbel, A dynamic dose de-escalating dosing strategy to determine the optimal biological dose for antiangiogenic drugs: commentary on Dowlati et al. Clin Cancer Res. 2005; 11: 7589–7592. 21. Y. Takahashi, M. Mai, N. Sawabu, and K. Nishioka, A pilot study of individualized maximum repeatable dose (iMRD), a new dose finding system, of weekly gemcitabine for patients with metastatic pancreas cancer. Pancreas. 2005; 30: 206–210. 22. S. Hunsberger, L. V. Rubinstein, J. Dancey, and E. Korn, Dose escalation trial designs based on a molecularly targeted endpoint. Stat Med. 2005; 14: 2171–2181. 23. E. Korn, S. G. Arbuck, J. M. Pluda, R. Simon, R. S. Kaplan, and M. C. Christian, Clinical trial designs for cytostatic agents: are new approaches needed? J Clin Oncol. 2001; 19: 265–272. 24. S. Kang, S. Lee, and C. Ahn, An investigation of the proportion designs based on a molecularly targeted endpoint. Drug Inf J. In press. 25. T. Yamanaka, T. Okamoto, Y. Ichinose, S. Oda, and Y. Maehara, Methodological aspects of current problems in target-based anticancer

8

OPTIMAL BIOLOGICAL DOSE FOR MOLECULARLY TARGETED THERAPIES drug development. Int J Clin Oncol. 2006; 11: 167–175.

26. E. Korn, Nontoxicity endpoints in phase I trial designs for targeted, non-cytotoxic agents. J Natl Cancer Inst. 2004; 96: 977–978. 27. W. R. Parulekar and E. A. Eisenhauer, Phase I design for solid tumor studies of targeted, non-cytotoxic agents: theory and practice. J Natl Cancer Inst. 2004; 96: 990–997. 28. Y. Shaked, U. Emmenegger, S. Man, D. Cervi, F. Bertolini, et al., Optimal biologic dose of metronomic chemotherapy regimens is associated with maximum antiangiogenic activity. Blood. 2005; 106: 3058–3061. 29. H. S. Friedman, D. M. Kokkinakis, J. Pluda, A. H. Friedman, I. Cokgor, et al. Phase I trial of O6 -benzlguanine for patients undergoing surgery for malignant glioma. J Clin Oncol. 1998; 16: 3570–3575. 30. E. Deutsch, J. C. Soria, and J. P. Armand, New concepts for phase I trials: evaluating new drugs combined with radiation therapy. Nat Clin Pract Oncol. 2005; 2: 456–465. 31. U. Manne, R. Srivastava, and S. Srivastava, Recent advances in biomarkers for cancer diagnosis and treatment. Drug Discov Today. 2005; 10: 965–976. 32. J. A. DiMasi, R. W. Hansen, and H. G. Grabowski, The price of innovation: new estimates of drug development costs. J Health Econ. 2003; 22: 151–185. 33. C. Ahn, Pharmacogenomics in drug discovery and development. Genomics Inform. 2007; 5: 41–45. 34. R. Frank and R. Hargreaves, Clinical biomarkers in drug discovery and development. Nat Rev Drug Discov. 2003; 2: 566–580. 35. R. Hoekstra, J. Verwij, and F. Eskens, Clinical trial design for target specific anticancer agents. Invest New Drugs. 2003; 21: 243–250.

FURTHER READING E. Fox, G. A. Curt, and F. M. Balis, Clinical trial design for target-based therapy. Oncologist. 2002; 7: 401–409. S. Kummar, M. Gutierrez, J. H. Doroshow, and A. J. Murgo, Drug development in oncology: classical cytotoxics and molecularly targeted agents. Br J Clin Pharmacol. 2006; 62: 15–26. M. Ranson and G. Jayson, Targeted antitumour therapy future perspectives. Br J Cancer. 2005; 92(Suppl 1): S28–S31.

A. Stone, C. Wheeler, and A. Barge, Improving the design of phase II trials of cytostatic anticancer agents. Contemp Clin Trials. 2007; 28: 138–145.

CROSS-REFERENCES Dose-escalation design Maximum tolerated dose Optimal biological dose Phase I trials Cytotoxic drug Cytostatic drug

OPTIMIZING SCHEDULE OF ADMINISTRATION IN PHASE I CLINICAL TRIALS

a preventable disease may occur. In both examples, an MTD based on a single course of treatment may prove to be overly toxic when given over multiple courses. Consider a setting in which conventional dose-finding is done based on one course with a fixed schedule when in fact a safe dose d exists with three courses. If d has substantive antidisease effect with three courses whereas d with only one course does not, then the conventional MTD of one course may lead to the erroneous conclusion in later studies that the agent is ineffective. Similarly, if conventional dose-finding is done with four courses of each dose and it turns out that the lowest dose is excessively toxic, then it may be concluded erroneously that the agent is unsafe at any dose simply because shorter schedules were not examined. Furthermore, it may be the case that two doses will prove to be equally safe if different administration schedules are applied to each. Second, most existing designs require that toxicity be evaluated quickly enough so that each enrolled patient is fully evaluated for DLT before a new patient enters the study. One exception is the TITE-CRM (6), which evaluates long-term toxicity and allows new patients to enroll before all previously enrolled patients have completed observation. However, like other phase I trial designs, the TITE-CRM does not accommodate settings where multiple schedules are studied. Specifically, the TITE-CRM allows the dose to vary across patients while keeping the schedule fixed; our method allows the schedule to vary across patients while keeping the dose fixed. One could consider assessing multiple schedules with the TITE-CRM by treating each schedule as a ‘‘dose’’ and determining the maximum tolerable schedule (MTS) with study-specific modifications as described by Braun, Levine, and Ferrara (7). However, by considering each schedule to be a dose, patients who receive an incomplete schedule essentially have received a partial ‘‘dose.’’ To force this situation into the framework of the TITE-CRM, a patient can only be evaluated up to the point of his or her last fully completed schedule. Furthermore, if there is an additional follow-up

THOMAS M. BRAUN Department of Biostatistics, School of Public Health University of Michigan Ann Arbor, Michigan

PETER F. THALL Department of Biostatistics and Applied Mathematics, University of Texas M. D. Anderson Cancer Center Houston, Texas

In conventional phase I studies, each patient is assigned to a dose of an experimental agent under study, receiving either a single administration or a course consisting of several administrations of the agent. Patients are followed for a relatively short period of time during which a dose-limiting toxicity (DLT) may occur. At the end of the follow-up period, a binary outcome indicating the presence or absence of DLT is recorded for each patient. Generally, the maximum tolerable dose (MTD) is considered the largest dose that is ‘‘safe,’’ that is, that does not present a practical limitation to therapy. Many designs using this approach exist (1–5). These designs have seen widespread use largely because they facilitate adaptive dose-finding methods wherein doses and outcomes of previous patients are used to select doses for new patients. However, these designs have some limitations due to their simplified representation of actual clinical practice. First, in many clinical settings, physicians administer an agent more than once to a patient and monitor the long-term toxicity related to the cumulative effects of the agent. In such settings, the nominal ‘‘dose’’ is actually the dose given per administration, or the total dose given over a fixed number of administrations. For example, multiple cycles of chemotherapy may be administered with the aim to effectively destroy all tumor cells. Similarly, a prophylactic agent may be given repeatedly during the period in which

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

OPTIMIZING SCHEDULE OF ADMINISTRATION IN PHASE I CLINICAL TRIALS

period once each schedule has been completed, the ‘‘doses’’ will overlap, leading to ambiguity as to which dose contributed to a late-onset toxicity. Third, existing phase I trial designs assume that the course of therapy, once assigned to a patient, will not be altered during the study. However, in actual practice, most phase I trials allow modifications to or delays of administration times if a patient has experienced a low-grade (non–dose limiting) toxicity previously during the treatment process. Thus, patients who experience a DLT after one or more treatment modifications will not fit the simplified framework of phase I trial designs unless the DLT is associated with the originally planned treatment schedule. This relies on the implicit assumption that altering a patient’s treatment schedule has no effect on the probability of a DLT. Such an assumption is dubious, given the wellknown schedule dependence of many agents. Nonetheless, our method does assume that if a patient’s treatment is terminated early for reasons unrelated to DLT, that patient is considered to be fully followed without DLT up to the point of their last administration. Thus, we implicitly assume that patient withdrawal is noninformative with regard to the likelihood for DLT. Such an assumption is reasonable, as patients in most phase I trials are at high risk for disease-related mortality, and complete withdrawal of treatment for reasons unrelated to toxicity is extremely rare. Due to the limitations of conventional phase I methods described above, a paradigm shift is needed so that phase I trial designs may better reflect actual clinical practice and thus lead to better decision-making in phase I. Hopefully, this will in turn decrease the false-negative rate in phase II trials so that true treatment advances are more likely to be detected. Because this new paradigm accounts for the actual times when a patient receives the agent, the patient’s time to toxicity will be used as the outcome instead of the usual binary indicator that toxicity has occurred. The hazard of toxicity is modeled as the sum of a sequence of hazards, each associated with one administration of the agent. We determine the MTS that the patient may

receive based on the risk of toxicity occurring within a specified follow-up period that includes the maximum schedule being considered. Patient accrual, data monitoring, and outcome-adaptive decision-making are done continuously throughout the trial under a Bayesian formulation. Each time a new patient is accrued, the most recent data are used to evaluate criteria that define the MTS, which is assigned to the new patient. 1 MOTIVATING EXAMPLE The schedule-finding method was originally motivated by a study was conducted at the University of Michigan involving allogeneic bone marrow transplant (BMT) patients, who are at risk of acute graft-versus-host disease (aGVHD), an immune disease leading to damage to the patient’s skin, liver, and/or gastrointestinal (GI) tract and possible early mortality. Preclinical studies (8) have demonstrated that recombinant human keratinocyte growth factor (KGF) markedly reduces chemotherapy-induced or radiationinduced injury to the mucosal lining of the lower GI tract. Investigators therefore theorized that patients receiving KGF are less susceptible to developing aGVHD in the GI tract if given KGF (9, 10). However, it is unknown how frequently KGF can be given safely without causing an unacceptable rate of grade 4 toxicities, such as rash, edema, and increases in amylase and lipase, both of which are indicators of pancreas dysfunction. An earlier unpublished phase I study in BMT patients determined an MTD of 60 mg/kg of KGF when administered using a 10-day schedule of 3-days-on/4-days-off/3days-on, which we denote by (3+ , 4−, 3+ ). Although one course of KGF using the (3+ , 4−, 3+ ) schedule is considered safe, investigators believed that this schedule would not provide sufficient prophylaxis for aGHVD, which may take up to roughly 100 days after BMT to develop. Consequently, the investigators wished to study the safety of multiple courses of KGF, with 4 days of rest between consecutive courses, and focus on toxicity associated with the entire period of therapy. Thus, two courses would consist of the 24day schedule (3+ , 4−, 3+ , 4−, 3+ , 4−, 3+),

OPTIMIZING SCHEDULE OF ADMINISTRATION IN PHASE I CLINICAL TRIALS

and so on. This study was planned with a maximum enrollment of 30 patients. There are many published schedulefinding studies (11–13), most of which use a crude design enrolling small cohorts of patients on predefined schedules, estimating the DLT probability for each schedule independently of the other schedules, and selecting the schedule with the optimal toxicity profile. The weaknesses of such an approach arise from the lack of an assumed regression model incorporating different schedules, preventing the borrowing of strength among schedules. As a result, a very small number of patients receive each schedule, the estimator of the DLT probability at each schedule is unreliable, and one may not extrapolate the results of the study to schedules that were not examined. Additionally, the absence of a sequentially outcome-adaptive rule for assigning patients to schedules limits the safety of any trial using such a design. Similarly to the gains of model-based methods such as the CRM and its counterparts compared with conventional ‘‘3+ 3’’ dose-finding algorithms, the MTS model and decisionmaking paradigm greatly improve upon existing methods available for schedule-finding studies. 2

DESIGN ISSUES

As mentioned earlier, the MTS method is adaptive in its selection of the schedule assigned to each patient, using Bayesian methods to continually update the MTS. As a result, we need two pieces of information: [1] a parametric model to describe how the risk of toxicity is related to the duration and times of administration under each schedule, and [2] prior distributions that describe plausible values for the parameters in the schedule/toxicity model. For illustrative purposes, assume the setting of the KGF trial. Let t* denote any given time from the start of the trial when one must evaluate the data and make a decision, including what schedule to give a newly accrued patient or whether to terminate the trial early. Let n* denote the number of patients enrolled up to t*. For i = 1, 2, . . . , n*, the most recent data collected on patient i are Di = (si , Yi , δ i ), where si is a vector of administration times, Yi is the amount

3

of follow-up, and δ i is an indicator of whether that patient has suffered toxicity. Thus, si = si,1 , . . . , si,mi lists the administration times since study entry for patient i. Under this general notation, the agent can be administered whenever and as frequently as desired to each patient, and an arbitrary number of different treatment sequences can be studied. This allows some of a patient’s actual administration times to deviate from his or her planned times. However, we assume that the investigators are interested in selecting among a specific set of treatment sequences, s(1) , . . . , s(k) . For example, in the KGF trial, one course of the (3+ , 4−, 3+ ) schedule corresponds to s(1) = (1, 2, 3, 8, 9, 10), two courses correspond to s(2) = (1, 2, 3, 8, 9, 10, 15, 16, 17, 22, 23, 24) = (s(1) , s(1) + 14), and so on. The data collected on all patients up to time t* are entered into a probability model that defines the likelihood of toxicity during the follow-up period of each patient. We now describe the first component of our design: a parametric model describing the risk of toxicity for each schedule. 2.1 Single-Administration Hazard We assume that the hazards for multiple administrations are additive and identical. Therefore, our problem is reduced to stating the hazard of toxicity for a single administration of the agent. Although this hazard can be quite general, we make the simplifying assumption that the hazard of toxicity from a single administration has a finite duration after which it completely vanishes to zero. In fact, this correctly reflects most clinical applications. Under this assumption, one cannot model the hazard with a typical parametric lifetime distribution, such as the gamma or Weibull, unless the distribution is truncated appropriately. As a simple, practical alternative, we assume that the hazard increases linearly to a maximum and decreases linearly thereafter. Specifically, we define the hazard function as  2θ1 u ; 0 ≤ u ≤ θ2   θ2 + θ3 θ2   2θ1 θ2 + θ3 − u h(u|θ ) = ; θ2 < u ≤ θ2 + θ3  θ2 + θ3 θ3    0

; u > θ2 + θ3 or u < 0

(1)

4

OPTIMIZING SCHEDULE OF ADMINISTRATION IN PHASE I CLINICAL TRIALS

Thus, our model uses three parameters (θ 1 , θ 2 , θ 3 ) to fully describe a triangle with base of length θ 2 + θ 3 and area θ 1 , with the height of the triangle occurring at time θ 2 from administration. Figure 1 illustrates this function. Other possible forms for the hazard are described in Braun, Yuan, and Thall (14), which also contains an alternate parameterization for Equation (1). Because θ 1 is the area of a triangle representing the risk of toxicity, it quantifies the cumulative hazard of a single administration. Thus, the cumulative hazard of toxicity for each patient is simply a multiple of θ 1 , where the multiplier is the number of administrations received. Any administration given within θ 2 + θ 3 days of t* would constitute an area less than θ 1 , but the actual area can be derived easily using geometric properties of a triangle. As stated earlier, we assume additive hazards across all administrations,

as illustrated in Figure 2, where the dashed lines represent the hazard of toxicity for each of six administrations given at enrollment and at 1, 2, 7, 8, and 9 days after enrollment. The solid curve is the cumulative hazard of toxicity from all administrations, computed from the sum of the heights of each triangle underneath it. The shaded area displays the cumulative hazard of toxicity 10 days after enrollment. Because the hazard of toxicity for each single administration in Figure 2 lasts a total of 18 days, each administration contributes a fraction of θ 1 to the cumulative hazard 10 days after enrollment. Only once a patient is observed beyond 18 days from enrollment will administrations start to contribute fully to the cumulative hazard. For this model, we now need to identify prior distributions that describe plausible values for the three parameters θ 1 , θ 2 , and θ 3 .

Hazard of Toxicity

2θ1 θ2 + θ3

0

θ2

θ3

Figure 1. Parametric hazard function for a single administration of an agent.

Hazard of Toxicity

Time (in days)

0

2

7

9

18 20 Time (in days)

25 27

Figure 2. Cumulative hazard of toxicity 10 days after enrollment for a patient who received administrations at enrollment and 1, 2, 7, 8, and 9 days after enrollment.

OPTIMIZING SCHEDULE OF ADMINISTRATION IN PHASE I CLINICAL TRIALS

2.2 Developing Prior Distributions The prior distributions of the parameters must be sufficiently uninformative so that they are dominated by the data and the MTS algorithm is able to provide a safe and reliable design. The actual mathematical forms for the prior distributions, although useful, are less important than the numerical hyperparameter values selected for each distribution. For example, because both θ 2 and θ 3 measure spans of time and will take non-zero values, one could assume that these parameters have lognormal prior distributions. Alternatively, one could place an upper bound on the distribution of θ 3 and instead use a generalized beta distribution for the prior of θ 3 , as described by Braun, Yuan, and Thall (14). The influence of the prior on the MTS algorithm may be quantified by the mean and variance hyperparameter values selected for the priors. Appropriate values for the prior mean and variance hyperparameters may be elicited from the investigators in many ways (15, 16), although in general, it is easiest to elicit values on domains with which the investigator is familiar (17). For example, the investigator can supply the expected duration of time after administration when the hazard will reach its maximum; this value can serve as the mean for the prior of θ 2 . Similarly, the expected duration of time required for the hazard to completely vanish or become negligible would serve as the prior mean for θ 3 . The investigator could also supply a range of plausible values for each parameter, and those ranges could be used to develop each parameter’s prior variance (14). The ability of the data to dominate the prior distributions is heavily influenced by the variances of those distributions, and it is essential to carefully evaluate the how changing the values of the variances impacts the MTS algorithm’s performance. This sensitivity analysis may be carried out by simulating the toxicity times of a small number of patients, comparing the prior means with their respective posterior values and evaluating how sensitive the MTS algorithm is to a small amount of data. For example, if the prior reflects the belief that toxicity is unlikely beyond 25 days after administration,

5

simulating a few patients to have toxicities that occur far beyond 25 days allows one to determine whether the prior allows the posterior mean of θ 3 to shift beyond its prior mean and reflect the data appropriately. If not, the prior variances may be calibrated and the exercise repeated until the desired effect is achieved. This exercise demonstrates the important point that the prior variances cannot be made arbitrarily large, as is usually done with Bayesian analyses of large datasets. In any small-scale clinical trial using adaptive methods, very little data are available, especially early in the trial. If there is substantial prior probability mass over too broad a range, this often cannot be overcome by a small amount of data, depending on the particular model, data structure, and decision-making algorithm. In the present setting, unduly large prior variances would severely hinder the algorithm’s ability to assign optimal schedules during the trial, especially early in the trial, and also may degrade the method’s ability to select an optimal MTS at the end. At each evaluation, the likelihood of the data collected on all enrolled patients is combined with the prior distribution to derive a posterior distribution for the parameters in the hazard. Because the posterior cannot be obtained analytically under our assumed model, we compute posterior quantities via Markov chain Monte Carlo (MCMC) methods (18); more details can be found in Braun, Yuan, and Thall (14). 2.3 Maximum Sample Size The maximum sample size proposed for any schedule-finding study should be ascertained during the design stage with an exhaustive simulation study, examining a variety of settings and study design parameters until adequate performance of the MTS algorithm is observed. Specifically, using a range of sample sizes, one should assess: [a] how frequently the algorithm correctly identifies the MTS when it exists, and [b] how frequently the algorithm terminates early when all schedules are overly toxic. Although the largest sample size will always provide the most reliable results, patient resources are typically limited in phase I trials, and it may

6

OPTIMIZING SCHEDULE OF ADMINISTRATION IN PHASE I CLINICAL TRIALS

be that, for example, a decrease of 10 patients leads to a negligible decrease in the values seen for [a] and [b], showing an acceptable trade-off between performance of the MTS algorithm and the duration and cost of the actual study. 3

TRIAL CONDUCT

A maximum of N patients are enrolled in the trial, with each patient assigned a treatment administration sequence upon arrival. The first patient is assigned the shortest sequence. Each patient is followed for a maximum of T days, with treatment terminated if toxicity is observed. From the posterior distribution of the parameters, the cumulative probability of toxicity by time T for each schedule is computed. Given a desired target probability p, which typically is in the range 0.10 to 0.30, depending upon the nature of the DLT, we consider two alternative criteria for choosing each patient’s sequence. The first criterion defines the MTS as that schedule whose average posterior probability of toxicity is closest to the target p. This criterion is analogous to the CRM criterion (2) based upon the posterior mean probability of the more usual binary indicator of toxicity. The second criterion first computes the percentage of the posterior distribution of each schedule that lies above the target p. The MTS is defined as the schedule whose posterior percentage is closest to, but no larger than, a threshold q. The second criterion is similar to the acceptability criteria used by Thall and Russell (19), and Thall and Cook (20) for dose-finding based on both efficacy and toxicity, and it also is similar to the criterion for overdose control proposed by Babb, Rogatko, and Zacks (4). Under either criterion, the current MTS is assigned to the next patient enrolled. If none of the schedules meet the criterion used, we conclude that none of the schedules are safe and the study is terminated. A variety of other stopping rules similar to those used with the CRM (21, 22) can easily be adapted for use with the MTS algorithm. Assuming the trial does not terminate prematurely, the MTS at the end of the trial is defined as the best sequence based on the complete data from all N patients.

To protect patient safety, we impose the additional constraint that escalation can be at most one course longer than the schedules already assigned to previous patients. This restriction on escalation only applies to untried schedules, and we place no restriction on de-escalation of schedule. We could also slow escalation (and prolong the length of the study) by requiring each schedule to be assigned to a cohort of M patients before escalation will be considered. For example, suppose we enroll the first patient on the lowest schedule and 20 days later this patient has not yet experienced a DLT and investigators wish to enroll a second patient. Using our algorithm and cohorts of size M = 1, investigators could enroll the newest patient on either the first or second schedules, depending upon which schedule best satisfies our safety criterion. Once a third patient is eligible, the information from the first two patients would be used to determine if the study should remain at the second schedule, escalate to the third schedule, or de-escalate back to the first schedule. If we instead set M = 2, then a minimum of four enrolled patients would be required before a patient could be assigned to the third schedule. Regardless of the cohort size, the algorithm will eventually identify a neighborhood of the correct MTS and continue to enroll patients in that neighborhood until the study reaches its maximum sample size. Simulation studies of the MTS paradigm demonstrate its excellent operating characteristics (14), provided that the prior has been properly calibrated. The MTS algorithm has a high probability of correctly identifying optimal schedules while maintaining an observed toxicity probability close to the targeted value, although the specific performance is a function of the number of schedules examined and the range of their cumulative toxicity probabilities. A critical issue illustrated by the simulations is that the MTS method is superior to the CRM and any other method that searches for an optimal dose but does not allow schedule to vary. Simply put, if investigators fail to examine the optimal schedule in the study, the study is doomed to fail at its very inception. The MTS method provides a safe, flexible way to examine several schedules to increase the

OPTIMIZING SCHEDULE OF ADMINISTRATION IN PHASE I CLINICAL TRIALS

likelihood of finding the optimal one. In turn, this should increase the number of possibly efficacious agents that reach phase II study.

12.

REFERENCES 1. B. E. Storer, Design and analysis of phase I clinical trials. Biometrics. 1989; 45: 925–937. 2. J. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: a practical design for phase I clinical trials in cancer. Biometrics. 1990; 46: 33–48. 3. S. Goodman, M. Zahurak, and S. Piantadosi, Some practical improvements in the continual reassessment method for phase I studies. Stat Med. 1995; 14: 1149–1161. 4. J. Babb, A. Rogatko, and S. Zacks, Cancer phase I clinical trials: efficient dose escalation with overdose control. Stat Med. 1998; 17: 1103–1120. 5. S. D. Durham, N. Flournoy, and W. F. Rosenberger, A random walk rule for phase I clinical trials. Biometrics. 1997; 53: 745–760. 6. Y. Cheung and R. Chappell, Sequential designs for phase I clinical trials with late-onset toxicities. Biometrics. 2000; 56: 1177–1182. 7. T. M. Braun, J. E. Levine, and J. L. M. Ferrara, Determining a maximum tolerated cumulative dose: Dose reassignment within the TITE-CRM. Control Clin Trials. 2003; 24: 669–681. 8. C. L. Farrell, J. V. Bready, K. L. Rex, J. N. Chen, C. R. DiPalma, et al., Keratinocyte growth factor protects mice from chemotherapy and radiation-induced gastrointestinal injury and mortality. Cancer Res. 1998; 58: 933–939. 9. A. Panoskaltsis-Mortari, D. Lacey, D. Vallera, and B. Blazar, Keratinocyte growth factor administered before conditioning ameliorates graft-versus-host disease after allogeneic bone marrow transplantation in mice. Blood. 1998; 92: 3960–3967. 10. O. I. Krijanovski, G. R. Hill, K. R. Cooke, T. Teshima, J. M. Crawford, et al., Keratinocyte growth factor separates graftversus-leukemia effects from graft-versushost disease. Blood. 1999; 94: 825–831. 11. C. Benz, T. Tillis, E. Tattelman, and E. Cadman, Optimal schedule of methotrexate and

13.

14.

15.

7

5-fluorouracil in human breast cancer. Cancer Res. 1982; 42: 2081–2086. H. Soda, M. Oka, M. Fukada, A. Kinoshita, A. Sakamoto, et al., Optimal schedule for administering granulocyte colony-stimulating factor in chemotherapy-induced neutropenia in non-small-cell lung cancer. Cancer Chemother Pharmacol. 1996; 38: 9–12. J. C. Byrd, T. Murphy, R. S. Howard, M. S. Lucas, A. Goodrich, et al., Rituximab using a thrice weekly dosing schedule in B-cell chronic lymphocytic leukemia and small lymphocytic lymphoma demonstrates clinical activity and acceptable toxicity. J Clin Oncol. 2001; 19: 2153–2164. T. M. Braun, Z. Yuan, and P. F. Thall, Determining a maximum-tolerated schedule of a cytotoxic agent. Biometrics. 2005; 61: 335–343. C. P. Robert, The Bayesian Choice, 2nd ed. New York: Springer, 2001.

16. A. J. Gelman, B. Carlin, H. S. Stern, and D. B. Rubin, Bayesian Data Analysis, 2nd ed. Boca Raton, FL: Chapman and Hall/CRC Press, 2004. 17. R. K. Tsutakawa and H. Y. RK, Bayesian estimation of item response curves. Psychometrika. 1986; 51: 152–267. 18. C. P. Robert and G. Casella, Monte Carlo Statistical Methods. New York: Springer, 1999. 19. P. F. Thall and K. T. Russell, A strategy for dose-finding and safety monitoring based on efficacy and adverse outcomes in phase I/II clinical trials. Biometrics. 1998; 54: 251–264. 20. P. F. Thall and J. D. Cook, Dose-finding based on efficacy-toxicity trade-offs. Biometrics. 2004; 60: 684–693. 21. J. O’Quigley and E. Reiner, A stopping rule for the continual reassessment method. Biometrika. 1998; 85: 741–748. 22. S. Zohar and S. Chevret, The continual reassessment method: comparison of Bayesian stopping rules for dose-ranging studies. Stat Med. 2001; 20: 2827–2843.

CROSS-REFERENCES Phase I trials Bayesian approach Adaptive design Dose-escalation trials

ORPHAN DRUG ACT (ODA) The Orphan Drug Act (ODA), signed into U.S. federal law on January 4, 1983, has helped to bring over 100 orphan drugs and biological products to market. The intent of the Orphan Drug Act was to stimulate the research, development, and approval of products that treat rare diseases. This mission is accomplished through several mechanisms: • Sponsors are granted 7 years of mar-

• •

•

•

keting exclusivity after approval of an orphan drug product. Sponsors are granted tax incentives for clinical research. The U.S. Food and Drug Administration’s Office of Orphan Products Development (OOPD) will assist the drug’s sponsors with study designs. The OOPD permits sponsors to employ open protocols that allow patients to be added to ongoing studies. Sponsors may receive grant funding to defray costs of qualified clinical testing expenses incurred in connection with the development of orphan products.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/orphan.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

ORPHAN DRUGS The term ‘‘orphan drug’’ refers to a product, which can be either a drug or biologic, intended for use in a rare disease or condition that affects fewer than 200,000 Americans. Orphan drugs may be approved or experimental. A drug or biologic becomes an orphan drug when it receives the orphan drug designation from the Office of Orphan Products Development at the U.S. Food and Drug Administration (FDA). The orphan drug designation qualifies the sponsor to receive certain benefits from the federal government in exchange for developing the drug for a rare disease or condition. The drug must then go through the FDA marketing approval process for safety and efficacy like any other drug or biologic. To date, over 1400 drugs and biologics have been designated as orphan drugs, and over 250 have been approved for marketing. The cost of orphan products is determined by the sponsor of the drug and is not controlled by the FDA. The costs of orphan products vary greatly. Generally, health insurance will pay the cost of orphan products that have been approved for marketing. If an orphan product has been approved by the FDA for marketing, it will be available through the normal pharmaceutical supply channels. If the product has not been approved by the FDA, the sponsor (the party studying the drug) may make the product available on a compassionate use basis. For contact information on sponsors of orphan products, contact the Office of Orphan Products Development. Side-effect information for orphan products that have been approved for marketing can be found on the labeling for the product.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/orphan.htm and http://www.fda.gov/orphan/faq/index.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

ORPHAN PRODUCTS GRANT PROGRAM The U.S. Food and Drug Administration’s Office of Orphan Products Development Grant Program encourages the clinical development of products for use in rare diseases or conditions, which are usually defined as affecting fewer than 200,000 people in the United States. The products studied can be drugs, biologics, medical devices, or medical foods. Grant applications are solicited through a Request for Applications (RFA). Presently, only clinical studies qualify for consideration; an applicant may propose one discrete clinical study to facilitate Food and Drug Administration (FDA) approval of the product for a rare disease or condition, and that study may address an unapproved new product or an unapproved new use for a product already on the market. All studies must be conducted under an Investigational New Drug (IND) application or an Investigational Device Exemption (IDE). Medical foods are the only exception to this requirement.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/orphan/grants/info.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

OUTLIERS

other model failure. Heteroscedasticity and nonlinearity, as well as nonnormality, may cause what appear to be isolated outliers.

DOUGLAS M. HAWKINS School of Statistics, University of Minnesota

Different possible origins of outliers call for different resolutions. Where the outliers are caused by imperfect modeling, the model should be refined so that they are accommodated. Where the outliers result from errors of execution, the primary concern is to minimize the damage they do to the analysis. Where the outliers result from mixtures of distributions or other types of contamination, there may be interest in identifying the outliers and (in the mixture case) estimating the characteristics of the mixture components. It is not always clear in applications what caused the outliers, and so ‘‘one size fits all’’ recipes for dealing with outliers are inappropriate. Here, we concentrate on methods of identifying outliers – that is, of deciding whether observations really are implausibly discordant. This has two parts – deciding which of the observations are most discordant from the majority; and measuring their discordancy. The first task – locating the most discordant observations – seems trivial at first glance, but is so only in the simplest case of univariate random samples. Here, it is only the largest and the smallest of the data that could be potentially discordant. Locating the most discordant observations, however, can be very difficult in ‘‘structured’’ data sets such as time series, analysis of variance, multiple regression, and multivariate data. Here, simple diagnostics like regression residuals cannot be relied upon to locate discordant observations. Estimating the parameters of a model without risk of serious damage from outliers is addressed by the techniques of robust estimation, the most familiar example of which may be the box and whisker plot with its outlier identification rules. The objectives of robust estimation and outlier identification

Outliers are an important issue in data analysis. Since a single undetected outlier can destroy an entire analysis, analysts should worry about the origins, relevance, detection, and correct handling of outliers. Intuitively, an outlier is an observation so discordant from the majority of the data as to raise suspicion that it could not plausibly have come from the same statistical mechanism as the rest of the data. Apparent discordancy is what distinguishes an outlier from a contaminant – an observation that did not come from the same mechanism as the rest of the data but was generated in some other way. A contaminant may appear ordinary (and not outlying), while an outlier could be, but is not necessarily, a contaminant. Outliers can arise in several ways. • They may be contaminants generated by

some other statistical mechanism. For example, if the seeds used in a plant growth experiment contain some foreign seed, then the plants produced from the foreign seed will be contaminants and may be outliers. • They may result from procedural errors in data gathering. For example, misreading an instrument will produce a contaminant that may be outlying. It is generally accepted that several percent of even high-quality data are wrong, and some of these errors may be outlying. • The analyst may have a misconception of the true model. For example, if an instrument is thought to produce normally distributed readings, but actually produces Cauchy-distributed readings, then some valid correct readings will be severe outliers relative to the wrongly assumed normal model. • In structured data such as multiple regression or analysis of variance data, they may be a symptom of some

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

OUTLIERS

are logically connected – if the potential outliers are located, then removing them from the sample and fitting the model to the remaining data will neutralize them and so provide robust estimates. Similarly, sturdy estimates of unknown parameters will provide a good picture of the model fitting the majority of the data, thereby helping locate potential outliers. The details of both approaches are less simple than they seem though, and the connection is not enough for either technology to supersede the other. 1

LIKELIHOOD MODELS FOR OUTLIERS

Outliers may be flagged and investigated in varying degrees of formality. For example, making a normal probability plot of residuals from a multiple regression and labeling as outliers any cases that seem visually too far from the line is a method of flagging questionable cases. Being informal though, it suffers from dependence on perceptions of how large a deviation is too large and may be criticized for having no obvious theoretical basis. More formal statistical models for outliers remove the subjective element and are valuable even if used only as benchmarks to assess other less formal models. A general framework that permits the modeling of data sets that might contain outliers is the ‘‘parameter shift’’ model. Each ‘‘good’’ (scalar or vector) observation X i in the sample is modeled as following a specified statistical model with unknown parameter vector θ X i ∼ gi (., θ). There may also be one or more contaminating observations Xi with distribution X i ∼ gi (., θ , αi ), where the parameter α i drives the contaminant’s displacement, and is most conveniently parameterized so that a contaminant with α i = 0 has the same distribution as a ‘‘good’’ value. Commonly, different contaminants are modeled as having different α i , but in some settings, a common α for all is sensible. Contaminants with sufficiently extreme α i values will be outlying and so potentially identifiable; those with less extreme α i values will

be indistinguishable from the good observations. This model is less restrictive than it might seem. An outlier that resulted from a recording error of transposing digits would not plausibly be explained by this parameter shift model, but since a good choice of α i could be fit to the data, this conceptual failing is arguably unimportant. For example, a transposition error of writing 84 for X i instead of 48 is exactly the same as adding α i = 84 − 48 to the value actually generated by the model. This distributional approach to describing outliers is attractive for those who like to work with formal models, as it allows much of the processing of outliers to be formalized and codified. Using say maximum likelihood to estimate the parameters θ and α i automatically gives a robust estimate of θ, and the likelihood gives a formal outlier test through a test of the null hypothesis α i = 0. The likelihood model describes contaminants, not outliers. Contaminants that are not outlying are undetectable; however, since they are undetectable precisely because they behave like ‘‘good’’ observations, their invisibility means that failing to detect them usually does not have serious bad consequences. 2 COMPUTATIONAL ISSUES Fitting the model is less trivial than it might seem as it requires looking at all possible partitionings of the data into the ‘‘good’’ and the ‘‘potentially suspect’’ subsets. In the simplest cases, this can be done by inspection. For a single sample from a normal distribution with mean-shift contamination, it is easy to show that the highest likelihood results when the extreme order statistics of the sample are labeled as ‘‘potentially suspect’’. This means that no other types of partitioning need be studied. In multiple regression, it would be equally intuitive that the cases most likely to be outliers would be those with the largest residuals, or the largest studentized residuals. Here though intuition is misleading: a pair of outliers can so distort the regression line that both have small residuals. This is called ‘‘masking’’. They may also make the residual of a ‘‘good’’ case large – this is termed

OUTLIERS

‘‘swamping’’. Thus, the most discordant observations need not necessarily have large (or even nonzero) residuals. Reliable identification of potentially suspect cases in multiple regression requires the substantial computational exercise of looking at all possible partitioning of the cases into ‘‘good’’ and ‘‘potentially suspect’’ subsets, selecting the partition with the largest likelihood. ‘‘High-breakdown’’ methodologies do indeed guarantee locating even large numbers of outliers, however badly they may be placed, provided enough computation is done. These methodologies are inherently computer-intensive as the guarantee of locating the outliers requires potentially investigating all partitions of the data into an ‘‘inner half’’ and an ‘‘outer half’’. After this exhaustive search, all outliers along with some inliers can be expected to be in the outer half where using some cutoff rule should distinguish the outliers from the more extreme inliers. A much lighter computation is required for ‘‘sequential identification’’ methods. In these, the single observation from the whole sample whose deletion would lead to the greatest increase in likelihood is flagged as potentially suspect, and temporarily removed from the sample. The single observation in the remaining sample whose deletion would lead to the greatest improvement in model fit is then identified as another potentially suspect observation, and is also temporarily removed from the sample. This stripping of observations continues until some stopping rule is reached. Sequential identification methods are of two types. In ‘‘forward selection’’, a discordancy measure is calculated as each new potentially suspect observation is identified and the process stops when it first fails to find an observation sufficiently discordant to be called an outlier. In ‘‘backward elimination’’ a preset number of potentially suspect observations is identified in one pass, and then in a second pass, each of them is tested sequentially to see whether it really is sufficiently discordant to be called an outlier. A case that is not discordant is then reincluded with the ‘‘good’’ observations and the testing of the remaining potentially suspect observations repeated.

3

Both theoretical reasoning and practical experience show that backward elimination is much better than forward selection, which can miss arbitrarily severe outliers. This is because the masking effect may cause the outlier diagnostics of all the outliers to appear modest so that forward selection stops before all outliers have been located. Intuitive though sequential identification is, an even more intuitive approach is onepass identification of all cases whose outlier diagnostics are large. Because of masking and swamping though, this approach is much worse than even forward selection and should not be used at all. This leaves backward elimination as the most reliable approach with low computational requirements. Another ‘‘backward elimination’’ method starts with some small subset of the data that is outlier-free, and then sequentially adds observations that appear not to be discordant, reestimating parameters as each new apparently concordant observation is added. Success with this approach depends on finding a starting subset of cases that is not only outlier-free but also informative enough to correctly distinguish the inliers from the outliers in the not-yet-classified group. Starting with a high-breakdown estimate gives reliable results, but at high computational cost. Methods using full-sample estimates – for example, the cases with the smallest absolute residuals from a full-sample least squares regression fit – may succeed, but come with no guarantees. Even high breakdown methods are not completely bulletproof: see the example in Hawkins and Olive (5) in which even high breakdown regression methods saw nothing surprising about men less than an inch tall, but whose head circumference was some five feet. The second phase, of deciding whether to label a suspect case outlying, may be based on a likelihood ratio or a case diagnostic such as a studentized residual. Two different philosophies on outlier labeling are to use fixed cutoff values; and to use multiple comparison tests. An example of the first approach is to flag as outlying any cases whose regression residual exceeds 2.5 standard deviations regardless of sample size. This rule will delete a fixed percentage of

4

OUTLIERS

the cases in data sets consisting of only good data. A sound multiple comparison method is the Bonferroni approach of testing the externally studentized residual (‘‘outlier t’’, or RSTUDENT) of a regression against the α/n quantile of a Student’s t distribution, where n is the sample size and α a significance level. In this approach, a fixed proportion of good data sets will have one or more observations wrongly labeled outliers. 3

PARTICULAR CASES

The easiest situation is mean-shift outliers from a univariate normal distribution, with inliers distributed as N(µ, σ 2 ), both parameters unknown, contaminated by one or more N(µ + α i , σ 2 ) outliers. The scaled deviations from the mean, (X i − X)/s, where X is the sample mean and s the standard deviation, are effective for both identifying and testing a single outlier. Multiple outliers can be found using sequential identification either by starting with the full sample and stripping one suspect observation at a time, or by starting with the easy-to-find ‘‘inner half’’ of cases and adding concordants. Outliers in the univariate normal can also be modeled by scale contamination, where the outliers are N(µ, σ 2 (1 + α)). This (with a common α) can be thought of as mixing the mean-contamination model’s displacements over a N(0, ασ 2 ) distribution, and is effective for outliers occurring symmetrically to the left and the right of the inliers. Flagging and testing for outliers from a chi-square distribution use the scaled quantities X i /X. Cochran’s test for the largest such ratio is classical, but outlier tests can also be performed on the smallest such ratios. In homoscedastic normal linear modeling (both linear regression and analysis of variance), a general-purpose approach to outlier identification and testing is by variance analysis. Let S0 with ν degrees of freedom denote the residual sum of squares from a fitted model using some set of cases. Write S1 for the residual sum of squares obtained after

deleting a suspect observation. Then, the pseudo-F ratio F=

(ν − 1)(S0 − S1 ) S1

(1)

may be compared with the fractiles (Bonferroni-corrected or fixed) of an F distribution with 1 and ν − 1 degrees of freedom to decide whether the suspect case is or is not within plausible limits. This F ratio is the square of the ‘‘outlier t’’ statistic that software often produces as a case diagnostic. In analysis of variance with replicates, fitting the ANOVA model on the full sample and after removal of individual readings leads to tests for individual outliers. A different model is the ‘‘slippage’’ model in which all the replicate readings in some cell are displaced by the same amount. Reducing the original data to cell means and taking these means as the basic data to which to apply outlier screening methods addresses the slippage model. Multivariate outliers are commonly modeled by the multivariate normal distribution N(µ, ) with unknown mean vector and covariance matrix. The mean contamination model holds that the outliers are N(µ + α i , ), and mixing the outlier displacements over a zero-mean normal distribution leads to the scale-contamination model in which the outliers are N(µ,  + ). With this baseline model, if the sample mean vector and covariance matrix are written as X and S respectively, then the discrepancy of a single case X i can be measured by its squared Mahalanobis distance from the mean Di = (X i − X) S−1 (X i − X). The traditional multivariate outlier statistic, Wilk’s lambda, is a monotonic function of Di . Sequential deletion successively trims the case with the largest Di from the current sample mean vector using the current sample covariance matrix. The likelihood outlier model can handle generalized linear models. If the deviance of any fitted model is S0 and the deviance obtained by deleting some suspect case and refitting is S1 , then the deviance explained by deleting the case is S0 − S1 , and this can be referred to its asymptotic chi-squared distribution to get an outlier test. This framework

OUTLIERS

5

Outlier t After Deletion Number Species

Diplodocus Brachiosaurus Triceratops Human Residual SS

Body Weight

Brain Weight

0

1

2

3

4

11 700 87 000 9400 62

50.0 154.5 70.0 1320.0

− 2.507 − 2.505 − 2.094 1.789 60.988

− 2.507 − 3.644 − 2.835 1.861 48.733

− 3.646 − 3.644 − 6.045 2.098 31.372

− 6.649 − 6.804 − 6.045 3.232 12.117

8.217

covers Poisson and logistic regression and loglinear modeling, among others. The generalized linear model framework is also useful for outliers in contingency tables – for example individual cells whose frequencies violate independence, which holds in the rest of the table. Deleting a particular cell, refitting the model, and computing the change in deviance between the two fits gives an outlier test statistic for that cell. Time series can have two types of outlier – an ‘‘additive outlier’’ displaces a single reading from where it should have been, but does not affect the subsequent observations. ‘‘Innovative outliers’’ displace the whole time series from their point of occurrence onward, and, apart from their occurrence at an unexpected time, can be modeled by the intervention analysis likelihood. 4

EXAMPLE

Rousseeuw and Leroy (7) discuss a data set relating the body weight and brain weight of 25 animals alive today, along with three dinosaurs. Least-squares regression of Y = loge (brain weight) on X = loge (body weight) gives (standard errors in parentheses) Y = 2.555 + 0.496

studentized residual of each of the four suspect cases and residual sum of squares of the fitted regression. The pseudo-F ratios for the successive deletions are 6.29, 13.28, 36.55, and 10.44 with 1 numerator degree of freedom and denominator degrees of freedom 25, 24, 23, and 22. Start with the last of these, 10.44, whose right tail area is 0.004. Multiplying this figure by the remaining sample size, 25, gives a Bonferroni P value of 0.1, which argues against making the fourth deletion. Going to the next F ratio of 36.55 gives a P value of 3.6 × 10−6 , indicating that the deletion of the third dinosaur, and by implication, its two even more extreme companions, is clearly called for. With an initial sample size of 28, the Bonferroni 5% and 1% points for the outlier t statistic would be the two-sided 5/28 = 0.179 and 1/28 = 0.036% points of a t distribution with 25 degrees of freedom, which are 3.50 and 4.13, respectively. None of these four cases is close to significance, illustrating the masking effect. Successively deleting the three dinosaurs makes humans the most outlying species. At this point, humans’ t of 3.232 corresponds to a right tail area of 0.0038, which after Bonferroni adjustment, makes us unremarkable. The regression found after deleting the dinosaurs is

X

(0.413) (0.078).

(2)

Y = 2.150 + 0.752

X

(0.201) (0.046). With 28 data points, a maximum of four deletions (15% of the data) seems reasonable. Starting with the full data set, and sequentially deleting the case with the largest absolute externally studentized residual for four iterations gives the following set of externally

(3)

Evidence of the damaging effect of the dinosaurs is that the slope fitted after their removal is 3.3 standard errors away from the full-sample estimate. The high breakdown regression fit using the least trimmed squares criterion gives

6

OUTLIERS

the model X = 1.88 + 0.776W, close to what we get with ordinary regression after deleting the dinosaurs. 4.1 Further Reading The text by Barnett and Lewis (2) provides an exhaustive coverage of the standard outlier situations and models. Additional useful theoretical material on the robust estimation aspects of outliers may be found in Hampel et al. (3). Rousseeuw and Leroy (7) is a valuable exposition of the high breakdown approach, though readers should be aware that many of the specifics (notably computational procedures and outlier flagging rules) are now obsolete. Better algorithms for high breakdown estimation in the difficult multivariate location/scatter case are given in (6,8), and for the regression case, in (4). Atkinson and Riani (1) provide discussion centered on sequential identification starting from an outlier-free subset of cases. Hawkins and Olive (5) provide theoretical and empirical support for routine use of a variety of multiple regression methods, including sequential outlier identification, in analysis of real data. They also argue that in games against nature, in which a malicious opponent tries to place outliers where you cannot find them in a tolerable amount of time, the opponent will always win, given large enough data sets. In other words, it is impossible to guarantee finding even huge outliers in a large data set if an opponent is allowed to hide them. This argument generalizes to other outlier settings – for example, multivariate location/scatter. REFERENCES 1. Atkinson, A. & Riani, R. (2000) Robust Diagnostic Regression Analysis. Springer, New York. 2. Barnett, V. & Lewis, T. (1994) Outliers in Statistical Data. Wiley, New York. 3. Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J. & Stahel, W. A. (1986) Robust statistics: The Approach Based on Influence Functions. Wiley, New York. 4. Hawkins, D. M. & Olive, D. J. (1999) Improved feasible solution algorithms for high breakdown estimation, Computational Statistics and Data Analysis 30, 1–11.

5. Hawkins, D. M. & Olive, D. J. (2002) Inconsistency of resampling algorithms for high breakdown regression estimators and a new algorithm. (with discussion), Journal of the American Statistical Association 97, 136–148, rejoinder 156–159. 6. Rocke, D. M. & Woodruff, D. L. (2001) Robust cluster analysis and outlier identification, in 2000 Proceedings, American Statistical Association, Arlington, VA, Biometrics Section. 7. Rousseeuw, P. J. & Leroy, A. M. (1987) Robust Regression and Outlier Detection. Wiley, New York. 8. Rousseeuw, P. J. & Van Driessen, K. (1999) A fast algorithm for the minimum covariance determinant estimator, Technometrics 41, 212–223.

= 1, . . . , n, are assumed to be independent variates from a Poisson distribution
n with common mean. The statistic is i=1 [(Yi −
Y)2 /Y], where Y = ni=1 Yi /n, and is called the sample index of dispersion. An important question to ask when a model is suspect is: Will the lack-of-fit affect inference and lead to incorrect conclusions? If the effect is negligible, and if the efforts involved in fitting a more ‘‘exact’’ model are substantial, then the approximate inference obtained under the simpler model may well suffice. In what follows the effect of overdispersion is shown to be nonignorable, and can be quite drastic, so inference under a Poisson or binomial model when overdispersion is present may be very misleading. We focus on the analysis of count and categorical responses, because these are the two areas where overdispersion most commonly arises, with binary responses being an important special case of the latter.

OVERDISPERSION CHARMAINE B. DEAN Simon Fraser University, Burnaby, British Columbia, Canada

The phenomenon which has come to be termed overdispersion arises when the empirical variance in the data exceeds the nominal variance under some presumed model. Overdispersion is often observed in the analysis of discrete data, for example count data analyzed under a Poisson assumption, or data in the form of proportions analyzed under a binomial assumption. Support for overdispersion is most likely first obtained when the ‘‘full’’ model is fitted, and the Pearson or deviance residuals are predominantly too large (4); the corresponding Pearson and deviance goodness-of-fit statistics indicate a poor fit. The Poisson and binomial distributions are derived from simple, but fairly strict assumptions, and it is not surprising that these do not apply generally in practice. Fitting either of these distributions assumes a special mean–variance relationship, since both distributions are fully characterized by a single parameter. This can be contrasted with the analysis of continuous data using a normal assumption. Consider a simple example where a set of univariate observations is modeled by a common N(µ, σ 2 ) distribution. In estimating the fitted distribution, the sample average of the data points defines the location of the normal distribution on the number line (µ), while the sample variance determines the spread of the fitted bell-curve (σ 2 ). The normal distribution is characterized by two parameters, while the Poisson and binomial distributions are completely specified when only the mean, or the probability of success, respectively, is determined; the variance is fixed by the mean. The existence of overdispersion is not a recent observation. Student (27) comments upon this problem, and Fisher (13) discusses a goodness-of-fit statistic for testing the adequacy of the Poisson distribution in the single sample problem, i.e. when the counts Y i , i

1 EFFECT OF OVERDISPERSION ON STANDARD POISSON OR BINOMIAL ANALYSES The effect of overdispersion is determined primarily by how it arises and its degree of incidence. One common way that overdispersion arises in the analysis of proportions is through a failure of the binomial independence assumption. In animal litter studies, for example, responses of animals in a litter are often positively correlated; so, too, in dental studies for responses on individual teeth for a single individual.
Let Y denote a binomial response, Y = m j=1 Yj , where Y j are independent binary variates taking values 0 or 1 with probabilities (1 − p) and p, respectively. Then, E(Y) = mp, and var(Y) = mp(1 − p). If the independence assumption does not hold, and the correlation (Y j , = τ > 0, then E(Y) = mp, and var(Y) = Yk)
var( m j=1 Yj ) = mp(1 − p)[1 + τ (m − 1)], leading to overdispersion with respect to the binomial model. Note that τ < 0 leads to underdispersion, which is rare in practice, but might correspond to competition among the binary variates for a positive response.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

OVERDISPERSION

Another way overdispersion may arise is through a failure of the binomial assumption of constant probability of success from trial to trial. This might occur if the population can be subdivided into naturally occurring subunits, for example colonies, where the probability of a positive response varies over these subunits. Similarly, for count data, failure of the assumptions underlying the use of the Poisson distribution generally leads to overdispersion. In particular, the probability of an event may vary over individuals or over time. For a simple example, suppose the response is the number of days absent due to illness over a period of time, in a situation where the number of episodes of illness, Y, are Poisson (µ) distributed but will likely lead to consecutive days of absence. The distribution of the number of days of absence due to illness will then be overdispersed with respect to the Poisson model. If A represents the number of days absence during a single episode, A and Y being independent, then the total num
ber of days of absence is Yi=1 Ai , which has 
Y  mean and variance, E i=1 Ai = µE(A), and  
Y var i=1 Ai = µE(A){E(A) + [var(A)/E(A)]} >µE(A), if E(A) > 1. In some contexts the Poisson or binomial variation is only a minute part of the overall variability. This is typical, for example, in cancer mapping studies, where the distribution of rates over a region is to be determined. In many cases it is the spatial variation of the cancer mortality rates which is the main component of the dispersion. Overdispersion cannot be ignored. In fact, many statistical packages routinely incorporate overdispersion. The magnitudes and signs of the estimated covariate effects in a loglinear or logistic analysis can be quite similar whether or not overdispersion is properly accounted for, so a researcher may gain no hint of an inappropriate analysis by there being strikingly strange estimated effects. However, the standard errors of the estimated regression parameters will be underestimated; these will reflect only the Poisson or binomial variation. The precision of the resulting estimates will be too high and P values for testing the significance of the included covariates will be correspondingly too low. This will very likely lead to incorrect

inference, unless the overdispersion is almost negligible. 2 TESTING FOR OVERDISPERSION In many situations the presence of overdispersion is clearly indicated by the presence of overly large values of the Pearson or deviance goodness-of-fit statistics, even when the full model is fitted. Formal tests for overdispersion have also been discussed in the literature. Score tests for overdispersion (3,9,10) compare the sample variance with what is expected under the model. For the testing of extra-Poisson variation, the adjusted score test statistic, for testing the null hypothesis H1 :τ = 0 in the model with overdispersed variance function µi + τ µ2i , is n 

TP1 =

{(yi − µˆ i )2 − (1 − hˆ i )µˆ i }

i=1

 2

n 

1/2

.

(1)

µˆ 2i

i=1

In (1) hi is the ith diagonal element of the hat matrix, H, for Poisson regression; H = W1/2 X(X
WX)−1 X
W1/2 , where W = diag(µ1 , . . . , µn ), and X is n × p with ijth entry For loglinear µ−1 i (∂µi /∂βj ), µi = µi (xi ; β). regression, log µi = x
i β, and X is the usual matrix of covariates. Estimates µˆ i and hˆ i ˆ its maxiare obtained by replacing β by β, mum likelihood estimate under the Poisson assumption. The statistic T P1 converges in distribution, as n → ∞, to a standard normal under H1 . For testing the null hypothesis H2 :τ = 0, in the model with variance function (1 + τ )µi , the score test statistic is   n  (yi − µˆ i )2 − (1 − hˆ i )µˆ i TP2 = √ , µˆ i (2n) i=1 (2) which is also asymptotically (n → ∞) distributed as standard normal, under H2 . It is interesting to note that the test statistic for testing H2 when considering µi → ∞ asymptotics, for fixed n, is equivalently 1


= TP2

n  (yi − µˆ i )2 i=1

µˆ i

,

(3)

OVERDISPERSION

which has a limiting distribution of χ 2 (n − p). This is just the Pearson statistic, which is traditionally used to assess correct specification of the mean. The derivation of the tests for extra-Poisson variation assumes that the regression specification is correct. Hence, the Pearson statistic arises as a test of either of two alternative hypotheses: namely, µi = µi (xi , β) is incorrectly specified, or the distribution of the counts has variance form φµi . We would not, however, interpret a significantly large Pearson statistic as indicating overdispersion in a generalized linear model, unless the mean had been reasonably well modeled. Otherwise, the apparent overdispersion could reflect missing covariates, e.g. interaction terms, implying systematic lack of fit, or, the functional form of the mean may be inappropriate. Pregibon (22) develops a test for checking the form of the mean in generalized linear models; for multinomial models, O’Hara Hines et al. (21) develop diagnostic tools for this purpose. Smith & Heitjan (25) develop a test for overdispersion which results when the vector of coefficients in the mean is considered to be random. The score tests T P1 and T P2 arise from a random intercept model, and can therefore be considered a special case of Smith & Heitjan’s test. Dean (8) discusses testing for overdispersion with longitudinal count data. A score test for the adequacy of the binomial model against an alternative with variance form mi pi (1 − pi )[1 + τ (mi − 1)] is 
n

(pˆ i (1 − pˆ i ))−1 [(yi − mi pˆ i )2



  i=1   +pˆ i (yi − mi pˆ i ) − yi (1 − pˆ i )] , (4) TB =  n 1/2
2 mi (mi − 1) i=1

which has a standard normal distribution as n → ∞. This variance form is obtained from the correlated binomial model, discussed in the previous section. Prentice (23) derives this statistic as a score test statistic against beta-binomial model alternatives; Tarone (28) derives it by considering correlated binomial alternatives. As an example in the application of the tests, consider the data, given in (29), from

3

a clinical trial of 59 patients with epilepsy, 31 of whom were randomized to receive the anti-epilepsy drug Progabide and 28 of whom received a placebo. The total seizure count over four follow-up periods is taken here as the response variable. Table 1 shows the results of a Poisson regression analysis of the effect of Progabide on seizure rate which includes two covariates and their interactions, plus terms for a main and interaction treatment effect. The data and this fitted model have been discussed at length in Breslow (4) and his results are given here. The Pearson and deviance goodness-of-fit statistics clearly indicate lack-of-fit of the Poisson model. The score test statistics T P1 and T P2 have observed values 36.51 and 37.56, respectively. The overdispersion here is very substantial.

3

ACCOUNTING FOR OVERDISPERSION

With overdispersion present, the use of the Poisson or binomial maximum likelihood equations for estimating the regression parameters in the mean is still valid. These are the usual generalized linear model (GLM) or quasi-likelihood (QL) estimating equations, and they are unbiased estimating equations regardless of any misspecification of the variance structure. However, the estimated variances of the parameter estimates will be in error, and possibly severely so. If there are alternative ‘‘overdispersed’’ models which are postulated, then certainly one could proceed by maximum likelihood estimation. This will be discussed further below. There are, however, some robust, simple methods for adjusting standard errors to account for overdispersion and these will be considered first. McCullagh & Nelder (20) suggest a simˆ ple adjustment, which is to multiply var(β), obtained from the Poisson or binomial model, by an estimate of the GLM scale factor, φ. This estimate is usually the Pearson or deviance statistic divided by its degrees of freedom. This is appropriate if the overdispersion gives rise to a variance model which is a constant times the nominal variance, e.g. φµ for counts and φmp(1 − p) for proportions.

4

OVERDISPERSION Table 1. Loglinear Poisson Regression Fit to the Epilepsy Data; Case No. 207, with High Leverage, Omitted. Reproduced from Statistica Applicata, Vol. 8, pp. 23–41, by permission of Rocco Curto Editore Coefficient

Value

Std. error

t Statistic

Intercept ln(base count/4) Age/10 ln(base count/4): age/10 Progabide Progabide: ln(base count/4)

3.079 −0.074 −0.511 0.351 −0.610 0.204

0.451 0.201 0.153 0.068 0.191 0.088

6.833 −0.366 −3.332 5.164 −3.197 2.325

Deviance = 408.4; Pearson χ 2

=

456.52; df = 52.

This variance form may also well approximate other, possibly more complicated, variance structures in certain situations; for example, for count data, when var(yi ) = µi + τ µ2i and τ µi do not vary greatly with i. If the sample is large, then an empirical variance estimate can be computed. This is called the ‘‘sandwich’’ variance estimate, cf. Liang & Zeger (17). For loglinear models, for example, the sandwich estimator is  ˆ = var(β)

n 

−1  µˆ i xi x
i

i=1



×

n 

 n   yi − µˆ i 2 i=1

µˆ i

 xi x
i

−1 µˆ i xi x
i

.

random effects models. A simple Poisson random effects model can be derived by considering a model with individual-specific random effects ν i > 0 where, conditional on (ν i , xi ), the distribution of Y i is Poisson with a mean of ν i µi (xi ;β), and the ν i are continuous, independent variates with probability density function p(ν;τ ) depending on a parameter τ . If µi (xi ;β) takes the common form exp(x
i β), then the fixed and random effects are added on the logarithmic scale and the random effects can be construed as representing covariates which are unavailable. The probability function of Y in the mixed model is

(5)



i=1

Unless very large samples are available, the sandwich estimator tends to underestimate the true variance. Resampling techniques, although typically computer-intensive, have become popular for providing estimates of the variance of regression parameters. Bootstrap and jackknife estimates are discussed in (12), and there are methods of approximating these which require less computing effort, e.g. the one-step jackknife estimate. The bootstrap estimates are considered to be quite accurate. Table 2, from (4), compares these estimators in the analysis of the data from (29), mentioned earlier. Notice how much larger these estimates of the standard errors are compared with those obtained from the Poisson model. The treatment effect is no longer significant when overdispersion is taken into account. Model-based methods for incorporating overdispersion lead to mixture models or

∞ 0

(µν)y exp(−µν) p(ν)dν y!

(6)

and the score function for estimating the regression parameters has the intuitively appealing form   n  1 ∂µi = 0. [yi − µi E(νi |yi )] µi ∂βr

(7)

i=1

This equation, with E(ν i —yi ) omitted, is the maximum likelihood equation for Poisson regression [Eq. (8), with σi2 = µi ]. If the distribution of ν is specified, then full maximum likelihood estimation can be performed; if ν is assumed to be gamma, then this leads to a negative binomial distribution for the counts. However, it is more common to adopt the more robust approach of specifying only the first two moments for Y, i.e. µi and σi2 , respectively; the parameters in the mean are then estimated using the quasi-likelihood

OVERDISPERSION

5

Table 2. Overdispersion Adjusted Standard Errors for Table 1 Coefficients. Reproduced from Statistica Applicata, Vol. 8, pp. 23–41, by permission of Rocco Curto Editore Coefficient

Scale Method

Sandwich

One-step Jackknife

True Jackknife

Bootstrap (nb = 5000)

1.263 0.564 0.430 0.190 0.535 0.246

0.711 0.326 0.237 0.104 0.403 0.188

0.792 0.368 0.264 0.117 0.440 0.210

0.792 0.369 0.263 0.117 0.448 0.214

0.870 0.424 0.291 0.137 0.466 0.226

Intercept ln(base count/4) Age/10 ln(base count/4): age/10 Progabide Progabide: ln(base count/4)

estimating equation,   n  (yi − µi ) ∂µi i=1

σi2

∂βr

= 0,

(8)

for count data, together with another estimating equation for the additional parameter τ in σi2 . There are many important reasons for the widespread use of the quasi-likelihood approach. For generalized linear models with a full likelihood, these are the maximum likelihood equations. From the viewpoint of estimating equations, Godambe & Thompson (14) (see also Nelder’s discussion of that paper) derive important optimality properties of the estimators. When σ 2 = µτ , estimation of the regression coefficients is not affected by the value of τ . Estimation is easy with standard software. An important point is that the ˜ the estimate of β, asymptotic variance of β, is independent of the choice of the estimating function for τ , and depends only on the first two moments of the distribution. This is also, asymptotically, a very efficient estimator for a wide range of models. Simulation studies have been conducted to investigate the performance of β˜ in small samples; they support the unbiasedness and efficiency of this estimator. A popular method for estimating τ is pseudo-likelihood. Davidian & Carroll (7) derived the pseudo-likelihood estimating equation as the maximum likelihood equation when residuals are normally distributed. An alternative simple choice is equating the Pearson statistic to its degrees of freedom.

Nonparametric methods of modeling random effects for handling overdispersion have been shown to be useful. Lindsay (18) is a comprehensive source on the topic. He discusses the geometry and theory of mixture models and describes a plethora of applications where mixture models are used, including overdispersion, measurement errors, and latent variable models for cluster analysis. Practical issues for estimation of nonparametric mixing distributions, such as algorithms and computational problems, are discussed at length in (2) and (16). The preceding overdispersion models incorporated a single random effect which was independent over subjects. More general models might include multiplicative random effects (25); large numbers of random effects are common in animal breeding experiments, where it is the prediction of the random effects, representing sire effects, which is the main focus of the study. In studies of the geographic distribution of cancer mortality rates, or disease incidence, the random effects may represent area-specific effects and there may be good reason to suspect that such area effects are not independent, and in some circumstances may be quite similar within small neighborhoods. A general body of theory which synthesizes the incorporation of several random effects, which are not necessarily independent, falls under the heading of generalized linear mixed models. It permits a simple incorporation of overdispersion, as discussed previously, and can also model dependences in outcome variables or random effects, as are typical in repeated measures design. A generalized linear mixed logistic model specifies

6

that

OVERDISPERSION



pi log 1 − pi

 =

x
i β

+

z
i γ ,

(9)

where pi and xi are the probability of a positive response and the vector of covariates, corresponding to the ith proportion, respectively, zi is a vector of covariates, and γ is distributed with a mean of zero, and finite variance matrix. Conditional on γ , the responses are supposed binomially distributed. As an aside, note that the representation above elucidates that apparent overdispersion can be induced by missing covariates, or by outliers. Residual diagnostics are important for identifying the latter. In generalized linear mixed models the random effects are usually assumed to be Gaussian, and maximum likelihood estimation involves q-dimensional integration; here q is the dimension of γ . Alternative simpler approaches for inference have been proposed, using generalizations of moment methods or penalized quasi-likelihood (5). The penalized quasi-likelihood is a Laplace approximation to the integrated likelihood, with some seemingly harmless other approximations added. Breslow & Clayton (5) provide simple algorithms for estimation using an iterative fitting procedure which updates both the parameter values and a modified response variable at each step. They also evaluate the performance of their estimators. It seems that bias corrections are required for small samples, and these are given in (6). Lee & Nelder (15) discuss hierarchical generalized linear models, where the distribution of the random components is not restricted to be normal, and where, like penalized quasi-likelihood, estimation avoids numerical integration. Maximizing what they call the h-likelihood, a posterior density, gives fixed effects estimators which are asymptotically equivalent to those obtained using the corresponding marginal likelihood. Here, ‘‘asymptotically’’ refers to cluster sizes tending to infinity, and the random effects are cluster-specific. This is important to note because many applications with random effects involve several smallsized clusters. In general, their asymptotic arguments require that the total number of

random effects remains fixed, as the overall sample size becomes large. However, they also derive properties of their estimators on a model-by-model basis, and some models require less strict assumptions. When there are many random effects to be estimated, none of the procedures described here will be simple, as can be expected when dealing with complicated mechanisms for incorporating overdispersion. 4 TECHNICAL REFERENCE TEXTS AND SOFTWARE NOTES The general topic of overdispersion is discussed in McCullagh & Nelder [20, Sections 4.5, 5.5, 6.2]. Chapters 9 and 10 of that text are also relevant and discuss quasi-likelihood and joint modeling of mean and dispersion. Diggle et al. (11) discuss random effects models for longitudinal data, and Chapter 5 of Lindsey (19) is devoted to the topic of overdispersion in models for categorical data. Software for incorporating overdispersion includes S-PLUS (26), function glm, SAS ((24), procedures LOGISTIC, MIXED GENMOD, and CATMOD)), and GLIM (1). REFERENCES 1. Baker, R. J. & Nelder, J. A. (1987). GLIM, 3.77. Numerical Algorithms Group, Oxford, England. 2. B¨ohning, D. (1995). A review of reliable maximum likelihood algorithms for semiparametric mixture models, Journal of Statistical Planning and Inference 47, 5–28. 3. Breslow, N. E. (1989). Score tests in overdispersed GLMs, in Workshop on Statistical Modeling, A. Decarli, B. J. Francis, R. Gilchrist & G. U. H. Seeber, eds. Springer-Verlag, New York, pp. 64–74. 4. Breslow, N. E. (1996). Generalized linear models: checking assumptions and strengthening conclusions, Statistica Applicata 8, 23–41. 5. Breslow, N. E. & Clayton, D. G. (1993). Approximate inference in generalized linear mixed models, Journal of the American Statistical Association 88, 9–25. 6. Breslow, N. E. & Lin, X. (1995). Bias correction in generalized linear mixed models with a single component of dispersion, Biometrika 82, 81–91.

OVERDISPERSION 7. Davidian, M. & Carroll, R. J. (1987). Variance function estimation. Journal of the American Statistical Association 82, 1079–1081. 8. Dean, C. B. (1991). Estimating equations for mixed Poisson models, in Estimating Functions, V. P. Godambe, ed. Oxford University Press, Oxford, pp. 35–46. 9. Dean, C. B. (1992). Testing for overdispersion in Poisson and binomial regression models, Journal of the American Statistical Association 87, 451–457. 10. Dean, C. & Lawless, J. F. (1989). Tests for detecting overdispersion in Poisson regression models, Journal of the American Statistical Association 84, 467–472. 11. Diggle, P., Liang, K. Y. & Zeger, S. L. (1994). Analysis of Longitudinal Data. Oxford University Press, New York. 12. Efron, B. & Tibshirani R. J. (1993). An Introduction to the Bootstrap. Chapman & Hall, London. 13. Fisher, R. A. (1950). The significance of deviations from expectation in a Poisson series, Biometrics 6, 17–24. 14. Godambe, V. P. & Thompson, M. E. (1989). An extension of quasi-likelihood estimation (with discussion), Journal of Statistical Planning and Inference 22, 137–152. 15. Lee, Y. & Nelder, J. A. (1996). Hierarchical generalized linear models (with discussion), Journal of the Royal Statistical Society, Series B 58, 619–678. 16. Lesperance, M. L. & Kalbfleisch, J. D. (1992). An algorithm for computing the nonparametric MLE of a mixing distribution, Journal of the American Statistical Association 87, 120–126. 17. Liang, K. Y. & Zeger, S. L. (1986). Longitudinal data analysis using generalized linear models, Biometrika 73, 13–22. 18. Lindsay, B. (1995). Mixture Models: Theory, Geometry and Applications. NSF-CBMS Regional Conference Series in Probability and Statistics, Vol. 5. Institute of Mathematical Statistics, Hayward.

7

19. Lindsey, J. K. (1993). Models for Repeated Measurements. Oxford University Press, New York. 20. McCullagh, P. & Nelder, J. A. (1989). Generalized Linear Models, 2nd Ed. Chapman& Hall, London. 21. O’Hara Hines, R. J., Lawless, J. F. & Carter, E. M. (1992). Diagnostics for a cumulative multinomial generalized linear model, with applications to grouped toxicological mortality data, Journal of the American Statistical Association 87, 1059–1069. 22. Pregibon, D. (1980). Goodness-of-link tests for generalized linear models, Applied Statistics 29, 15–24. 23. Prentice, R. L. (1986). Binary regression using an extended beta-binomial distribution, with discussion of correlation induced by covariate measurement errors, Journal of the American Statistical Association 81, 321–327. 24. SAS Institute Inc. (1990). SAS Release 6. 03 edition. SAS Institute Inc., Cary, NC. 25. Smith, P. J. & Heitjan, D. F. (1993). Testing and adjusting for departures from nominal dispersion in generalized linear models, Applied Statistics 42, 31–41. 26. Statistical Sciences (1995). S-PLUS, Version 3.3, StatSci, a division of Math-Soft, Inc., Seattle. 27. ‘‘Student’’ (1919). An explanation of deviations from Poisson’s law in practice, Biometrika 12, 211–215. 28. Tarone, R. E. (1979). Testing the goodness-offit of the binomial distribution, Biometrika 66, 585–590. 29. Thall, P. F. & Vail, S. C. (1990). Some covariance models for longitudinal count data with overdispersion, Biometrics 46, 657–671.

OVER-THE-COUNTER (OTC) DRUG PRODUCT REVIEW PROCESS

or a request to have a drug approved based on an existing monograph, the OTC division has 180 days to review the data and respond to the submitter. When the submission is reviewed, the drug is categorized through the monograph rulemaking process as follows:

Data regarding over-the-counter (OTC) drug monographs can be submitted to the U.S. Food and Drug Administration’s Center for Drug Evaluation and Research (CDER) OTC program by anyone, including drug companies, health professionals, consumers, or citizen’s groups. If the submission is a request to amend an existing OTC drug monograph or is an opinion regarding an OTC drug monograph, it needs to be submitted in the form of a citizen petition or as correspondence to an established monograph docket. However, if no OTC drug monograph exists, data must be submitted in the format as outlined in 21 Code of Federal Regulations (CFR) Section 10.30. Data are submitted to the Dockets Management Branch, which logs the data and makes copies for the public files. The data are then forwarded to the Division of Over-the-Counter Drug Products for review and action. When the submission is received in the Division of OTC Drug Products, a project manager conducts an initial review to determine the type of drug being referenced and then forwards the submission to the appropriate team for a more detailed review. The team leader determines if the submission will need to be reviewed by other discipline areas in the review divisions, such as chemists, statisticians, or other consultants such as those from other centers or agency offices. When the consultants complete their reviews, their comments are returned to the Division of OTC Drug Products. The submission is then forwarded to a team member for review. If the submission is a comment or opinion on a specific rule or OTC drug monograph, there is no deadline established for the CDER to respond. However, if the submission is a petition or request to amend a monograph,

Category I: generally recognized as safe and effective and not misbranded. Category II: not generally recognized as safe and effective or is misbranded. Category III: insufficient data available to permit classification. This category allows a manufacturer an opportunity to show that the ingredients in a product are safe and effective, or, if they are not, to reformulate or appropriately relabel the product. When the initial review is complete and other consult requests have been received, a feedback letter outlining CDER’s recommendations may be prepared for the submitter. The recommendations will vary depending on the type of data submitted. For example, a response based on a request to amend a monograph may contain explanations approving or disapproving the amendment. If the submitter is not satisfied with the recommendations made by the division, the submitter may request a meeting to discuss any concerns. Advisory Committee meetings are usually held to discuss specific safety or efficacy concerns, or the appropriateness of a switch from prescription to OTC marketing status for a product. Usually the OTC advisory committee meets jointly with the advisory committee having specific expertise in the use of the product. After the Division of OTC Drug Products has completed its review, a feedback letter explaining CDER’s actions or recommendations may be prepared. This letter is forwarded to the submitter, and it remains on file at the Dockets Management Branch. In some cases, the agency responds to a submission by addressing the submitted issues in a rulemaking (i.e., monograph or amendment), in which case a feedback letter is not sent to

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/otc.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

OVER-THE-COUNTER (OTC) DRUG PRODUCT REVIEW PROCESS

the submitter. If the submitter is not satisfied with the recommendations described in the feedback letter, a meeting can be requested with the division to discuss it (e.g., to provide more information or respond to any concerns of the Center). If the submitted data or information supports amending an existing monograph or creating a new monograph, a notice is published in the Federal Register (FR). After the proposal is published in the Federal Register, the public has usually 30 to 90 days to respond to it. This deadline depends on the controversial nature of the notice and can be extended if a request to do so is made, and any concerned party can request an extension. All comments are sent to the Dockets Management Branch and then are forwarded to the Division of Over-the-Counter Drug Products. The comments are reviewed/evaluated by the appropriate team and, if needed, are sent to other discipline areas for further review.

After the public comments have been reviewed, the final monograph is prepared. The final monograph is like a recipe book, which sets final standards that specify ingredients, dosage, indications for use, and certain labeling. The final monograph is sent out for clearance through the appropriate channels, such as the Office of Drug Evaluation (ODE), CDER, Office of General Counsel, Deputy Commissioner for Policy, and Regulations Editorial Staff. If any office does not concur, the submission is returned to the Division of Over-the-Counter Drug Products for revision and is then rerouted to the appropriate sources. After all revisions are made and the final monograph/amendment receives all the appropriate final concurrences, it is published in the Federal Register. All final monographs and amendments that have been published in the Federal Register are then published in the Code of Federal Regulations.

OVER THE COUNTER (OTC) DRUGS

standardized the format for OTC labels (4). Thus, the first step in OTC development is to design the OTC label and optimize its ability to meet its communication objectives (5).

ERIC P. BRASS Harbor-UCLA Center for Clinical Pharmacology Torrance, California

1.1 Identification of Key Messages Although all OTC drugs will share common communication themes such as directions for use, most will also have unique requirements for the safe and effective use of the drug. For example, labels for OTC drugs used to treat heartburn must help consumers with more serious conditions such as gastrointestinal hemorrhage to seek medical attention rather than self-treating with the OTC drug. OTC drug labels may include drug interaction warnings specific for the drug. All the key messages relevant to the safe and effective use of a potential OTC drug must be explicitly identified and incorporated into a prototype label. Explicit recognition and prioritization of these messages facilitates the structuring of study primary and secondary objectives as well as the statistical analysis plans for the studies performed.

Availability of drugs over-the-counter (OTC) allows consumers improved access to safe and effective drugs without the burden of obtaining a physician’s prescription (1–3). Many drugs have moved from prescriptiononly to OTC status increasing consumers’ options for managing health problems. However, safe and effective use of OTC drugs places substantial responsibilities on the consumer. Consumers must be able to selfdiagnose their condition as matching the indication for a specific OTC product; assess whether their personal health history represents a contraindication to using the OTC drug or requires the input of a healthcare professional; understand the directions for using the drug (including dose, frequency, and duration of therapy); and self-manage the course of treatment based on their response to the drug. In the absence of a healthcare professional, the OTC drug label must effectively provide the information required by the consumer to accomplish these tasks. When considering a switch of a prescription drug to OTC status, the safety and efficacy of the drug has typically been established in the original clinical development program. Thus, the OTC development program is focused specifically on the ability of the consumer to use the drug as intended and as guided by the OTC label. Novel clinical research designs have been developed, and continue to evolve, to support OTC development. 1

1.2 Assessing Label Comprehension OTC labels are used by consumers with diverse backgrounds. Thus, the ability of consumers to understand the proposed label syntax must be evaluated formally. This evaluation is conducted using potential consumers recruited through varied means. Demographic information is collected to allow for the influence of personal characteristics or medical history on label comprehension to be defined. Consumers are provided a prototype label and then are asked specific questions designed to assess their understanding of the key communication messages. Items can be in the form of multiple-choice questions or short-answer formats. Questions are designed either to probe simple recognition of the correct message or to probe the consumer’s ability to make a decision in a described clinical scenario based on correct use of the key message being studied. As emphasis is placed on the ability of the

LABEL COMPREHENSION STUDIES

As noted, the OTC label plays a central role in the safe and effective use of OTC drugs. The label must effectively communicate to the typical consumer the key messages required to use the drug. In the United States, the Food and Drug Administration (FDA) has

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

OVER THE COUNTER (OTC) DRUGS

label to communicate the key messages effectively, consumers may refer to the label while answering the questions, and no time limit is usually given. Use of multiple-choice questions necessitates use of distractors (incorrect responses) that are credible to avoid cueing of the correct response. Use of options that might be viewed as correct in any scenario (for example, ‘‘ask your doctor’’) are problematic in assessing the true effectiveness of the label and should be avoided. Scoring of short-answer responses should be done using objective, prespecified criteria based on inclusion of key components in the response. It may be useful to assess an individual communication objective using more than one question format to gain confidence in the reliability of the responses. The label development process should involve iterative studies with improvements in the label based on the results of each round of testing. Thus, each step need not be powered to yield definitive results but must only be adequate to inform internal decision making. During these early steps head-tohead comparison of different label strategies may be particularly useful. The composition of the study population in label development studies is important. For example, poor literacy skills remain prevalent in the United States (6,7). Comprehension of medication labels may pose a serious challenge for consumers with low literacy (3). Thus, assessment of the proposed label’s comprehension by a low-literacy cohort is an important component of the development program. Analysis of label comprehension testing should include specific analysis of the low-literacy cohort (for example, lower than eighth-grade reading level) as a prespecified secondary analysis. To ensure adequate statistical power for this analysis, recruitment requirements may specify a total number of subjects and the number of subjects that must meet low-literacy criteria. For specific drugs, other subpopulations may be of particular interest. For example, if a drug has a risk of an adverse drug–drug interaction, then the OTC label should contraindicate the use of the interacting drug. This message must be communicated to the

patients using the interacting drug of interest and not necessarily to the general consumer population. Thus, study of consumers currently using the drug of interest may allow focused development of the interaction contraindication labeling. Understanding whether patients recognize the brand name versus generic name of the drug, some lay term for the drug class (for example ‘‘blood thinner’’ for warfarin), or identification of the population using the drug of interest (for example, patients with HIV infection) can be ascertained by such focused studies. Once iterative testing has preliminarily identified optimal label strategies for all messages, a pivotal label comprehension study should be preformed. This study should use the full proposed final OTC label in a general consumer population with sufficient representation from low-literacy consumers to permit generalized conclusions. Although the entire label and all messages should be evaluated, particular attention should be given to the key messages unique to the new proposed OTC drug and those with the greatest potential impact on the safe and effective use of the drug. The assessment of these messages can be prespecified as primary endpoints, and the statistical analysis plan for the study can prespecify criteria for adequate comprehension. The potential importance of this prespecification of adequate comprehension has been emphasized in discussions by FDA’s Nonprescription Drug Advisory Committee (8). Although criteria for meeting a comprehension standard can be based on the study’s point estimate for correct responses, this action risks a false positive conclusion based on an underpowered study. Thus, consideration can be given to defining a positive result on the basis of the lower bound of the 95% confidence interval exceeding a predefined threshold, or similar criteria, to ensure that the conclusion is robust (Fig. 1). The threshold or performance standard for an adequate level of comprehension can be specified based on the importance of the message to the safe and effective use of the drug. For most messages, ensuring that 75% of the population understood the message may be adequate. However, for critical safety messages in

OVER THE COUNTER (OTC) DRUGS

3

Figure 1. Prespecification of adequate response rates. The percentages of correct respondents to a hypothetical label comprehension question are plotted. In situation A, the number of correct respondents is simply plotted, but interpretation is limited as no threshold for a positive result was prespecified and the variability in the point estimate is unknown. In situation B, the point estimate for the percentage of correct respondents exceeds a prespecified threshold (dotted line), but the lower bound of the 95% confidence interval crosses this threshold. Thus, one might interpret the point estimate with caution. In situation C, the point estimate for the percentage of correct respondents exceeds a prespecified threshold (dotted line), and the lower bound of the 95% confidence interval does not cross this threshold. This result can be interpreted as supporting the hypothesis that the label message was understood by the study population.

which lack of comprehension may result in personal harm, a higher (for example, 90%) standard may be required. 2 BEHAVIORAL STUDIES IN OTC DEVELOPMENT The label development program will yield a proposed label that effectively communicates all key messages. However, this communication does not mean that consumers will follow the guidance provided by the label. Consumers may discount the importance of certain messages, augment the information provided by the label with contradictory information obtained from other sources, or apply personal value judgments to override an understood label message. Thus, more clinical research is required to define consumers’ actual behaviors if they are provided access to the proposed OTC drug.

2.1 Self-Selection Studies The first decision point in an OTC setting is the consumers’ assessment that the drug is appropriate for their personal use, followed by their purchase of the drug. This action reflects the consumer’s selection decision. Self-selection studies focus specifically on this decision. The studies are designed to allow consumers to make their selection decision in as naturalistic a setting as possible to yield data relevant to the actual OTC marketplace. Cueing of important information by study personnel must be minimized and poses challenges to all aspects of trial conduct including the obtaining of informed consent. Consumers are recruited to self-selection studies on the basis of their representation of the OTC consumers likely to evaluate the product if it were available. Thus, recruitment is usually based on the presence of the

4

OVER THE COUNTER (OTC) DRUGS

drug’s indication. For example, recruitment materials might solicit participants suffering from recurring heartburn or interested in lowering their cholesterol. Interested consumers then visit a study site where they can examine the proposed OTC drug in its proposed packaging, including the previously tested and optimized OTC label. Consumers then are asked if they feel that the drug is ‘‘right for them’’ or similar question, which requires an individual decision based on the participant’s actual medical history and the information on the label. After responding to the self-selection question, participants may be offered the opportunity to purchase the drug at a price similar to what would be anticipated in the OTC market if the drug were approved for OTC sales. This purchase decision requires a second level of analysis by the consumer. In a self-selection study, only after making the purchase decision is the consumer informed that he or she cannot actually purchase the product, and the formal study is completed. Comprehensive demographic information is collected after the participants’ selection and purchase decisions have been made to avoid cueing important elements of the consumer’s medical history. The accuracy of the self-selection decision is usually the primary endpoint in selfselection studies, with the purchase decision as a secondary endpoint. However, evaluation of the correctness of the self-selection decision may be complex. For example, if an OTC label has five elements that apply to an individual study participant (for example, age, concomitant medications, comorbid disease, etc.) then a fully correct decision would require that the participant meet each of the five criteria. If each decision by the consumer were independent and the consumer could achieve a 90% correct rate for each, then all five would be correct only 59% of the time. If all five messages are equivalently important clinically, then requiring correctness for all five elements in this example would be reasonable, but this is rarely the case. Thus, consideration of the two self-selection decisions around each key label communication message allows identification of important coprimary or secondary endpoints in selfselection studies.

It is the clinical implications of ‘‘correctness’’ of self-selection decisions that pose substantial challenges to the interpretation of these studies. For example, when study participants decide to ignore a label age restriction by 1 year because they know they have the drug’s indication and feel that 1 year age difference cannot be important, it is unlikely to pose a health risk. In contrast, participants that inappropriately select to use the drug despite a drug interaction risk may expose themselves to substantial harm. In the context of the trial, both are ‘‘wrong,’’ but their import is clearly different. In this context, it is often useful after the self-selection and purchase decisions to ask consumers who made incorrect self-selection decisions why they made those decisions. Responses to these questions might yield information that mitigates some incorrect decisions or might suggest unanticipated misunderstandings that require strengthening of label elements. The analysis of a self-selection study is guided by the concepts introduced above for the pivotal label comprehension study. Analysis is focused on the cohort that selected to use the drug, as an inappropriate decision to not use the drug when the consumer may have benefited does not represent an individual or public health risk. The primary endpoint is the percentage of self-selectors who are label compliant, with a prespecified definition of a positive trial. Correct self-selection based on individual critical label components and the correctness of the purchase decision represent important secondary endpoints. The ability of low-literacy consumers to make self-selection decisions is of particular importance. The performance of a lowliteracy subpopulation is an appropriate secondary endpoint for self-selection studies. Other populations may be of special interest. For example, a self-selection study might be done exclusively in a cohort currently using a contraindicated medication to ensure that the label drug interaction warning effectively prevents self-selection by these consumers. Reliance on a general population sample would be unlikely to yield sufficient numbers of such patients to allow a meaningful conclusion relative to what might be an important health consideration.

OVER THE COUNTER (OTC) DRUGS

2.2 Actual Use Studies Actual use studies extend the assessments made in self-selection studies to fully define consumer behaviors if consumers were provided access to the drug in the absence of a learned intermediary. The scope of the study extends beyond the selection decision and allows consumers to purchase and use the product, purchase additional supplies as desired, and self-manage the full course of treatment. The latter component is particularly important as consumers must selftriage based on the evolution of their own medical condition and response to the drug and must discontinue treatment if adverse events develop (a ‘‘deselection’’ decision). Thus, the actual use study can uniquely probe hypotheses relevant to safe use of OTC drugs (Table 1). Initial recruitment to an actual use study is similar to that for a self-selection study as described above. The study center may emphasize naturalistic elements such as shelf displays to facilitate both study participant comfort and opportunities for repurchase of drug, if incorporated into the protocol. As study participants actually will have the opportunity to use the study medication, the need to assess consumer behaviors must be balanced by the need to protect participant safety. Thus, exclusion criteria are kept to a minimum. However, consumer comorbid conditions, concomitant medications, or other conditions such as pregnancy that represent a health risk if the consumer used the drug must exclude the patient from actually taking the drug after their purchase decision. To allow assessment of unbiased selection and purchase decisions, screening for exclusion criteria should be done after the purchase decision but before the opportunity for actual product use.

5

Similarly, data collection during the treatment period should be minimal to avoid cueing of desired behaviors. Use of study diaries, follow-up phone calls, or formal study visits may reinforce the importance of correct behaviors and mask the consumers’ true inclinations if the drug were used in an actual unsupervised setting. Thus, a natural tension occurs between collecting comprehensive data and the use of the data collected in the actual use studies. One compromise is to collect as much data as possible at the conclusion of the study when consumer behavior can no longer be affected. However, this method means that participant recall may be a major factor. Objective data, such as repurchase activity, may provide more useful data on duration and amount of drug used. However, collecting some data during the trial is often necessary, and it is then important that the data collection methods be as unobtrusive as possible and minimize cueing of behaviors of interest. Primary endpoints for actual use studies should be prespecified based on the behaviors of most importance to the safe and effective use of the proposed OTC drug. As discussed for label comprehension studies, prespecification of thresholds for acceptable behavior rates enhance the interpretation of the trial results. 3

SUMMARY AND CONCLUSIONS

Clinical trials used to support prescriptionto-OTC switch development programs face unique challenges in defining how consumers would use the drug if guided by the drug’s OTC label and in the absence of a healthcare professional. Trial endpoints are often less clear than those in a traditional efficacy study, as they focus on levels of consumer

Table 1. Examples of Hypotheses that can be Assessed in Actual Use Studies Consumers with contraindications to use of the drug will not purchase and use the drug. Consumers will discontinue use of the drug after the duration of treatment specified on the label. Consumers will not exceed the dose specified on the OTC label. Consumers will discontinue use of the drug if they develop symptoms consistent with an adverse drug effect as directed by the label. Consumers will self-triage to a healthcare professional if their medical condition worsens as defined by the OTC label.

6

OVER THE COUNTER (OTC) DRUGS

understanding and their behaviors based on proposed labeling. Nonetheless, rigorous trial design and conduct allows for these trials to inform and facilitate clinical and regulatory decision making. What combination of label comprehension, self-selection, and actual use studies are required for a specific OTC development program will vary widely. However, the goal of the development program remains to allow a data-driven, affirmative assessment that the proposed drug can be used safely and effectively without healthcare professional supervision in the OTC marketplace.

improved labelling and consumer education, Drugs Aging. 2004; 21(8): 485–98. 4. Food and Drug Administration. (2000). Labeling OTC Human Drug Products Using a Column Format. (online). Available: http:// www.fda.gov/cder/guidance/3594FNL.PDF.

The author is a consultant to GSK Consumer Healthcare, Johnson & Johnson–Merck, McNeil Consumer Pharmaceuticals, and Novartis Consumer Health.

5. E. P. Brass and M. Weintraub, Label development and the label comprehension study for over-the-counter drugs, Clin. Pharmacol. Ther. 2003; 74(5): 406–412. 6. M. Kutner et al., Literacy in Everyday Life: Results from the 2003 National Assessment of Adult Literacy. Washington, DC: National Center for Education Statistics, April 2007. 7. J. A. Gazmararian et al., Health literacy among Medicare enrollees in a managed care organization, JAMA. 1999; 281(6): 545–551. 8. Food and Drug Administration. (2006). Nonprescription Drugs Advisory Committee Meeting, September 26, 2006. [cited 2007 April 11] (online). Available: http://www.fda .gov/ohrms/dockets/ac/06/transcripts/20064230t.pdf.

REFERENCES

CROSS-REFERENCES

1. E. P. Brass, Changing the status of drugs from prescription to over-the-counter availability, N. Engl. J. Med. 2001; 345(11): 810–816. 2. R. P. Juhl, Prescription to over-the-counter switch: a regulatory perspective, Clin. Ther. 1998; 20 (Suppl. C): C111–C1117. 3. C. L. Roumie and M. R. Griffin, Over-thecounter analgesics in older adults: a call for

Over-the-Counter (OTC) Drugs – Case Studies

4

DISCLOSURES

Confidence Interval Statistical Analysis Plan Informed Consent Power

Overview of Anti-Infective Drug Development The identification and development of antibacterials, which are synthetic compounds, and antibiotics, which are semisynthetic derivatives of natural products, is a nonstandard, somewhat empirical process. The effort is based on practices developed in the industrial setting, and it generally requires 7 to 10 years to identify, optimize, develop, and launch a drug (Milne, 2003). The success rate is relatively low, with costs in excess of $800 million per marketed anti-infective (DiMasi et al., 2003). In general, the development of antibacterials is more rapid than for other drug classes, due to the characteristics of the preclinical and clinical tests used in their discovery. The discovery and development of antiinfectives can be divided into at least eight steps: exploratory phase, early phase, lead optimization, candidate selection, preclinical profiling, clinical development, regulatory filing, and approval and launch (Table 13A.1.1). While these eight steps apply to drug discovery and development in any therapeutic area, an antibacterial discovery program has some unusual characteristics compared to corresponding programs for other drug classes. Antibacterial drug discovery is often conducted by simultaneously screening for inhibitors of multiple, independent bacterial targets. This differs from other therapeutic area targets in which only one or two sites are targeted by a single compound. Indeed, there may be over 200 targets for antibiotics, with inhibition of any one having the potential to yield important therapeutic benefits. For this reason, antibacterial research and development may involve overlapping processes of optimizing multiple lead inhibitor series against multiple, individual targets. For example, one project may involve screening against a cell wall target, such as MurA, while at the same time the project team is optimizing a lead identified months earlier in another screen. In fact, genomics has yielded so many targets that a project team may be working on a dozen targets simultaneously, with only one or two new compounds ultimately being tested in vivo. In contrast, a eukaryotic target for a disease state may have just a few mechanistic approaches for pharmacological intervention.

UNIT 13A.1

EXPLORATORY PHASE For antibacterials, the exploratory phase of discovery lasts approximately 3 months. It encompasses the selection and validation of the target(s), and the design of a critical path for implementation to ensure the technical feasibility of the approach. All therapeutic areas are similar in their exploratory timeline and process, with the possible exception of target validation. Validation in microorganisms is usually accomplished in a matter of months by using any of several genetic tools, such as knock-out animals or conditional lethal genetic strategies. One characteristic of the validation process for antibiotics concerns the uniqueness of the genomes of different pathogens, which is a key to bacterial target validation. That is, for an antibiotic to have broad-spectrum effects, each of the various pathogens must produce the same target with sufficient structural similarity so as to respond to the same chemical agent. The discovery of an effective antibacterial hinges on the successful identification of a compound that inhibits an essential gene product of one or more bacterial pathogens (Dougherty et al., 2002). At least two approaches may be taken to accomplish this task. The first is to identify compounds with wholecell antimicrobial activity, after which the site of action is determined. The second approach is to use in vitro screens to identify inhibitors for a selected bacterial target and then demonstrate its antimicrobial activity (see UNIT 13A.2). In terms of drug discovery, an antimicrobial target is a bacterial gene product essential for survival of the organism (Haney et al., 2002). When a target is found in just a single pathogen, the relevant antimicrobial agent is referred to as being narrow-spectrum. If a target is essential for a group of pathogens, such as all Gram-positive or Gram-negative organisms, an effective antibiotic is referred to as a limited- or directed-spectrum agent. Drugs that inhibit a gene product that is essential for all pathogens are termed broadspectrum agents. Targets may be identified and/or selected by analyses of DNA sequences from bacterial pathogens by genomics/ bioinformatics. The essentiality of a target is assessed using established protocols involving

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

Anti-Infectives

13A.1.1 Supplement 31

Table 13A.1.1 Phases of the R&D Process of Antibacterial Drug Discovery

Phase

Time

Description and unique characteristics

Exploratory phase

3 months

Project team selects multiple targets from over 250 essential genes, in contrast to a single or few eukaryotic targets for chronic diseases. Genomics may be used to select the target by multiple genome analyses to assure conservation of the target and broad-spectrum inhibition. The bacterial genome is remarkably small and simple (∼3-9 Mb) in contrast to the human genome with its multiple complex regulatory networks.

Early phase

3-6 months

High-throughput screening (HTS) encompasses both activity against the bacterial target(s) and the ability to access the target (permeate the bacterium), versus eukaryotic targets, which are generally accessible

Lead optimization

12-18 months

The antimicrobial agent must be optimized with respect to a multitude of variables against multiple pathogens. This adds a level of optimization not encountered with eukaryotic targets.

Candidate selection

1-3 months

Prokaryotic and eukaryotic systems have the same decision points here in that they are safety- and efficacy-driven. A distinguishing feature of antimicrobial research is that preclinical animal models of efficacy reliably predict clinical outcomes.

Preclinical profiling

3-6 months

Same as target candidate drug profiling; safety-driven

Clinical development

4-6 years

Antimicrobial development distinguishes itself from eukaryotic targets in that there are multiple pathogens to be covered for each indication; therefore clinical trials must include at least 10 cases of each individual pathogen to attain a labeled indication.

Regulatory submission

6-12 months

Similar between prokaryotes and eukaryotes, but antimicrobials dealing with acute, unmet medical needs due to resistance may more easily receive Priority Review.

Approval and launch 0-3 months

Overview of Antimicrobial Drug Development

Similar between prokaryotes and eukaryotes.

genomics and bioinformatics (Arigoni et al., 1998; Judson and Mekalanos, 2000; Chalker and Lunsford, 2002; Thanassi et al., 2002; Pucci et al., 2003; Lindsay, 2004). The process for the selection of the target(s) for antibacterial drug development is described in UNIT 13A.2. Alternatively, targets can be identified by reverse chemical genomics. With this approach, a target-specific inhibitor is used to identify the actual target in the intact bacterium, as opposed to an in vitro screen against an isolated target, after which the antibacterial activity is assessed. A compound exhibiting the desired activity is then tracked back to its target by any of a number of procedures (DeVito et al., 2002).

High-throughput screening (HTS; UNIT 9.4) is used to identify chemical hits which, once confirmed and characterized, progress to lead status and, ultimately, following optimization, to candidate status. The identification of hits/leads is often a difficult and tedious process requiring the testing of between 100,000 and 1 million compounds. The goal is to identify a bona fide, target-based hit/lead suitable for medicinal chemistry-based optimization, or druggable chemotype (Beyer et al., 2004; Look et al., 2004; Pratt et al., 2004). A druggable chemotype is a compound that possesses a structure that can likely be modified to become a drug candidate (Lipinski et al., 1997). For

13A.1.2 Supplement 31

Current Protocols in Pharmacology

antimicrobials in particular, natural product screening is often used to identify a chemotype that is then chemically modified before attaining drug candidate status (Silver and Bostian, 1990; Bleicher et al., 2003). Automated systems are used to analyze large numbers of compounds in miniaturized biological assays. Among the equipment utilized in these systems are dilution and mixing devices, delivery robotics, and detection systems.

ity, and pharmacokinetics, including the Cmax , T1/2 , clearance rate, and area under the curve. Sometimes additional tests must be performed in order for a hit to advance to lead status, such as profiling against a panel of unrelated eukaryotic targets to assure that there are no, or minimal, untoward off-target activities in the human host, or even to the more detailed preliminary evaluation of taste for an oral compound.

EARLY PHASE

LEAD OPTIMIZATION

This phase of antibacterial discovery research, which generally consumes 3 to 6 months, includes the establishment of highthroughput screens and biological assays to support a medicinal chemistry program from lead optimization through candidate selection. The early phase also includes the HTS identification of hits (compounds that are active in the initial screen), confirmation of hits in the screening assay, and determination of the concentration of the compound that inhibits the targeted activity by 50% (the IC50 value). Successful completion of this phase leads to declaration of a lead. A lead is a compound initially identified as a hit in the primary screening effort, which is confirmed using secondary assays. To be considered a lead, the compound must permeate the target bacteria and demonstrate a minimum inhibitory concentration (MIC) of 4 to 16 µg/ml. It must also display other physical, chemical, and biological properties that lend themselves to chemical development into a clinical candidate (Lipinski et al., 1997). Some of the secondary assays used to promote a hit to a lead include susceptibility of additional strains of pathogenic bacteria, susceptibility of resistant clinical isolates and of strains with defined resistances, determination of minimum bactericidal concentrations (MBC), kill curves (UNIT 13A.3), pattern of labeling of macromolecular synthesis (DNA, RNA, protein, cell walls, and lipids), specificity or selectivity against other targets, and determination of cytotoxicity in mammalian cells. An advantage for antibacterial drug discovery is that all in vitro target interventions can be verified using in vivo preclinical animal models with certainty that the results will predict efficacy in humans. This early-phase effort may include synthesis of a limited number of analogs to assess the effects of chemically modifying the hit. Chemical modifications are usually necessary to optimize solubility, in vitro microbiological activity, the spectrum of inhibitory activ-

The lead optimization process generally requires ∼12 to 18 months to complete, and employs a fully integrated team of scientists conducting a battery of biological assays to support a structure-activity relationship (SAR) program. For lead optimization, the entire complement of biological assays is usually employed, including in vitro microbiological assays and in vivo assays for predicting efficacy. The latter entail the use of one or more animal species, along with assays to measure solubility, pharmacokinetic properties, and ancillary pharmacology, e.g., metabolism, and a study of possible side effects. Goals must be established beforehand to determine whether the results obtained with a particular compound indicate the lead has been optimized (Table 13A.1.2). It is important to establish specific, quantitative objectives for a discovery program, with both minimum and optimum criteria. These criteria are established on the basis of information contained in the published literature and obtained from personal experience (Table 13A.1.2). Quantitative objectives should be established for data that are to be obtained from microbiological, biochemical, pharmacological, and toxicological tests. Under the headings below are some general criteria that must be displayed for a potential antimicrobial to be considered an optimized lead.

Target Activity Criterion: Target inhibition with an IC50 value ≤10 µM, and an optimum IC50 <1 µM. In general, prokaryotic targets are less sensitive to inhibition than eukaryotic targets, with IC50 values for the former ranging from high nanomolar to low micromolar, whereas IC50 values are picomolar to nanomolar for most eukaryotic target inhibitors. Criterion: The target(s) should be confirmed as essential, such that inhibition leads to death of the microbe.

Anti-Infectives

13A.1.3 Current Protocols in Pharmacology

Supplement 31

Table 13A.1.2 Criteria for a Lead Versus a Candidate

Criteria

Lead minimum value

Candidate optimum value

Target inhibition (IC50 )

≤10 µM

≤1 µM

In vitro susceptibility (MIC90 )

4 to 16 µg/ml

≤1 µg/ml

In vivo efficacy (ED50 or PD50 )

≤20 mg/kg

≤5-10 mg/kg

Solubility

≥100 µg/ml

≥2-4 mg/ml

−8

Resistance emergencea

≤1 × 10

≤1 × 10−10

Pharmacokinetics properties enabling dosing

tid

Once or twice a day

Off-target activityb

>10-fold

>100-fold

Therapeutic index (selectivity over dose-limiting toxicity)

≥10

≥100

a Frequency of selection of a drug-resistant mutant from a bacterial population of between 109 and 1012 cells. b Off-target activity is a measurement of selectivity of inhibition of a microbial target compared to homologous or

nonhomologous host protein/targets.

Whole-Cell Bacteria Activity Criterion: Minimum inhibitory concentration (MIC) of 4 to 16 µg/ml, with an optimal value being ≤1 µg/ml (UNIT 13A.3). The MIC, which is determined using standard procedures developed by the National Committee for Clinical Laboratory Standards (NCCLS, 1997), is a broth- or agar-based growthinhibitory assay used to quantify the intrinsic activity of an antibacterial agent on intact bacterium. The MIC takes into account the intrinsic target activity, the ability of the compound to penetrate into the organism, the tendency for the compound to be transported out of the cell (efflux), accessibility of the compound to the target, the stability of the compound with regard to microbial resistance factors, chemical stability, and the stoichiometry between target and test agent.

In Vivo Efficacy in Preclinical Animal Species

Overview of Antimicrobial Drug Development

Criterion: ED50 or PD50 <20 mg/kg, with the optimum value being ≤5-10 mg/kg. Assessment of in vivo activity is the standard procedure for measuring the potential of a compound to prevent (prophylaxis) or treat human infections. In vivo activity is measured in laboratory animals by examining the ability of a compound to protect against death due to infection (Bergeron, 1978; Acred, 1986; Fernandez et al., 1997; Xuan et al., 2001). Such preclinical tests must utilize a quantitative, reproducible model of the target infection. The effectiveness of the test agent may be examined in any of a number of species,

including mouse, rat, gerbil, and dog (UNITS 13A.4 & 13A.5). The overall activity of a test agent in these assays is sometimes referred to as its potency, taking into account all of the factors related to its in vitro activity and its pharmacokinetics. The in vivo activity of a test agent depends on its specificity and selectivity, and its ability to be delivered in vivo in a pharmacologically active form.

Pharmacokinetics Criterion: It must be demonstrated that the test agent has systemic properties that allow for once to three times per day dosing for hospital-based products, to once or twice per day dosing for oral administration on an outpatient basis. Pharmacokinetic measurements taken to assess these properties include oral bioavailability, time to maximum blood concentration (Tmax ), peak plasma concentration of drug (Cmax ), time for the concentration of drug to decrease from the maximum in blood/serum by 50% (T 12 ), area under the curve (which measures the drug concentration in blood over the dosing interval or over a 24 hr period), clearance rate (CL, either over the dosing interval or a 24 hr period), and volume of distribution (Vd ), which represents the relative distribution of the agent between the blood and the tissues or organs.

Therapeutic Index (TI)

Criterion: A minimum TI >10, with an optimum ≥100. The therapeutic index is the ratio of the minimum dose producing adverse effects to the dose required for efficacy.

13A.1.4 Supplement 31

Current Protocols in Pharmacology

Opinions vary as to the minimum TI acceptable for antibacterials/antibiotics. Given existing agents, a TI of ∼100 is generally thought to represent a safety margin acceptable for human testing (Hann and Oprea, 2004).

CANDIDATE SELECTION This last step in the process of selecting a clinical candidate usually requires 1 to 3 months of advanced medicinal chemistry support to identify and fully characterize a compound (Table 13A.1.2) that displays the required characteristics of an antibacterial agent based on in vitro and in vivo assays, and a safety profile appropriate for testing in humans. Included in the candidate-selection phase are more extensive ancillary pharmacology tests, acute toxicity testing, and gross toxicological testing. Although each pharmaceutical firm may use different criteria for declaring a clinical candidate, there is general agreement about the minimum standards required with regard to efficacy of a new anti-infective. In general, efficacy must be demonstrated in one or more laboratory animal species, usually using different models reflective of different infections anticipated in humans (UNIT 13A.4 & 13A.5). The effective dose, which represents the dose needed to protect 50% of the animals or to reduce an infection by two log units of bacterial growth (ED99 ), should be <20 mg/kg. Single-digit ED50 or ED99 mg/kg values indicate excellent in vivo potency for an antibiotic.

PRECLINICAL PROFILING This phase generally requires 3 to 6 months, including the time needed for scaling up chemical synthesis to produce sufficient compound for extensive preclinical toxicity testing. In this case, one batch of compound for good manufacturing procedures/good laboratory practices (GMP/GLP) testing is used to ensure consistent, reproducible data. During this time, a project team makes the final preparations for Phase I clinical studies by scaling compound synthesis to the level necessary for completion of both Phase I and the first part of Phase II studies. A formal safety assessment study on the same batch of material is performed using two different animal species to minimize the chance of injury to a study participant. These preclinical safety studies are usually conducted for a period of time in excess of that anticipated for the clinical studies and at doses higher than those to be used in humans.

CLINICAL DEVELOPMENT This frequently takes 4 to 6 years or more for antibacterials, depending on the number of clinical indications sought, the patient-safety database required, the medical need prioritization by the Food and Drug Administration (FDA), the difficulty in recruiting patients, and the complexity of the clinical trials. The three stages of antibacterial clinical development are described under the following headers.

Phase I This first human-experience trial measures safety and pharmacokinetics of single and multiple doses of the drug candidate in a small number of healthy human volunteers. This testing is often conducted outside the United States because it is easier to obtain permission for human studies abroad. A Phase I study of a potential antibiotic usually takes 4 to 6 months to complete.

Phase II The clinical efficacy of the antibacterial candidate is assessed in a Phase II study, which normally requires 12 to 18 months to complete. Phase II may include “pivotal studies,” which are experiments that must succeed to demonstrate proof-of-concept that this compound and drug class are effective in humans. Failure of pivotal studies usually leads to termination of development. For example, a pivotal study for an antibacterial candidate intended for community use in the treatment of respiratory tract infections (RTIs) may include a clinical protocol that assesses efficacy in eradicating the predominant RTI bacterial pathogens. To demonstrate the efficacy required for FDA licensing for communityacquired pneumonia (CAP), the pivotal study is designed to demonstrate both microbiological and clinical (symptomatic) cure. This includes demonstrating that the agent eradicates the pathogen as measured microbiologically and relieves symptoms observed in those with the most frequently occurring RTI pathogens (Streptococcus pneumoniae, Haemophilus influenzae, Moraxella catarrhalis), or any of the atypical pathogens (Mycoplasma pneumoniae, Chlamydia pneumoniae, Legionella pneumophila). Depending on the design of the clinical trial, approximately 100 to 300 patients may be enrolled to ensure that statistically significant differences between the treatment and placebo groups are reliably detected.

Anti-Infectives

13A.1.5 Current Protocols in Pharmacology

Supplement 31

Phase III This phase of clinical testing usually involves 2000 to 3000 patient volunteers and requires 3 years or more to complete, depending on the indications sought and the complexity of the trials. In Phase III studies, large numbers of patients are treated with the candidate drug, which is directly tested against one or more comparator drugs to assess its efficacy and safety in an unbiased, often blinded, infected population selected on the basis of FDA-approved criteria. The trial subjects are randomly assigned to one of three groups: placebo controls, comparator agent, or candidate drug. For example, expanding on a Phase II pivotal study for community-acquired pneumonia, a sponsoring company may examine efficacy and adverse events in a 400- to 700patient Phase III study for CAP. The sizes of the treatment groups are carefully calculated to ensure statistical validity. Phase III studies also involve extensive monitoring of adverse events (AEs) from which the FDA draws the specific wording to describe limitations in the labeling if the agent should be approved for use. For antibacterials, AEs typically encountered range from nausea, diarrhea, headache, and taste perversion to myelosuppression and cardiac and central nervous system side effects.

REGULATORY FILING Preparation of the documents necessary for seeking marketing approval by the FDA generally occurs during the final 6 to 12 months of clinical testing, usually overlapping with Phase III. The final report represents the New Drug Application (NDA). The FDA review of this application must conclude within 12 months of the filing date, or 6 months if it is granted a priority review because the agent addresses a critical unmet need, such as treatment of methicillin-resistant Staphylococcus aureus. The NDA contains all preclinical and clinical data collected during the research and development process, including detailed case study reports of all patients and extensive statistical analyses of outcomes and safety. The NDA also contains a requested label for approval, listing the proposed indications, method(s) of delivery, dosage(s), length of treatment, and the results relating to efficacy, pharmacology, and toxicology. Overview of Antimicrobial Drug Development

APPROVAL AND LAUNCH The research and development of a new antibacterial ends with FDA approval, or disap-

proval, of the request to sell the drug. In some cases, the NDA is awarded an Approvable Letter providing conditional permission to license the drug for human use provided certain criteria are met. This process always requires additional review by the FDA. The last stage of the approval process, which generally lasts for approximately 3 months, entails review and negotiation with the FDA about the content of the drug label if the application is approved. Unique to antimicrobial drug labels is specification of both the indication for treatment and the specific pathogens within the individual indication. Thus, an approved label may indicate that the pathogen is covered in one disease indication but not in another.

STRATEGIC PLANNING While, to a large extent, the drug-discovery process is customized for each project, certain principles apply to all (McDevitt and Rosenberg, 2001; Mills, 2003; Nwaka and Ridley, 2003; Rich, 2004). Some of these principles are discussed below.

Know the Goal by Defining the Endpoint All scientists engaged in discovery and development of antibacterials seek to identify a novel, patentable agent that is effective against a relevant spectrum of pathogens (Barrett and Barrett, 2003; Haney et al., 2002). For a new chemical entity to be considered a drug candidate, it must display the desired antimicrobial activity in vitro and in the clinic, and it must address an unmet medical need. For example, for an agent to be positioned as a community, oral RTI drug, it should have antimicrobial activity, both in vitro and in the clinic, against S. pneumoniae, H. influenzae, Moraxella catarrhalis, and against several socalled atypical pathogens (i.e., M. pneumoniae, C. pneumoniae, L. pneumophila). Given the current agents on the market, medical need would require activity against penicillinresistant S. pneumoniae or macrolide-resistant H. influenzae. Likewise, a drug intended for use in intensive care should display a broad spectrum of antimicrobial activity in vitro, and clinical activity against Gram-positive pathogens, MRSA, vancomycin-resistant enterococci, and/or Gram-negative pathogens such as Pseudomonas aeruginosa, Klebsiella pneumoniae, and Acinetobacter baumannii. In addition, incremental advances that may improve treatment outcomes, such as dosing interval, length of treatment, or better safety

13A.1.6 Supplement 31

Current Protocols in Pharmacology

or tolerability, may also be viewed as meeting a medical need. Moreover, the commercial viability of a product is directly related to its ability to address an unmet medical need.

Quality of the Target The target chosen must meet well defined criteria to maximize the opportunity for identifying and developing a drug. The criteria and process for target selection are described in detail in UNIT 13A.2. The term “quality” is a subjective evaluation that encompasses, includes, and pertains to the microbiological, genetic, biochemical, molecular, pharmacological, and toxicological characteristics of the target.

Patent Position An unencumbered patent position is particularly important for antibacterial drug discovery because a significant number of DNA sequence–based patents have been issued. These patents may restrict the use of a DNA sequence in discovery efforts and in technologybased platforms, greatly hindering the discovery process.

Ability to Assess Inhibitor/Target Activity Success in identifying new antimicrobial agents requires the ability to assay the target and identify inhibitors. The tools necessary to attain these objectives are described elsewhere in this chapter. Given the long-term financial impact of the decision, the selection of a lead should be based on the totality of the data. Thus, for example, while the off-target activity of a compound is a variable in prioritizing compounds for SAR studies (Milne, 2003), absence of the target gene in a vital bacterial pathogen is reason alone to terminate further work on these compounds. Another fatal flaw is an anomalous SAR that does not permit the engineering of desired improvements into the molecule without adversely affecting efficacy or toxicity.

SUMMARY Most industrial scientists would agree that a challenging aspect of drug-discovery research is the number of variables that must be considered when moving chemical leads towards clinical candidate status. To this end, it is necessary to balance intrinsic inhibitory activity against the target with the following: (1) a molecular structure that allows the drug

to access the target, (2) appropriate pharmacokinetic qualities across the desired dosing interval, (3) selectivity over the eukaryotic homolog, and (4) safety. The difficulties associated with coordinating these properties often account for the low success rate associated with drug discovery. The merger of data from all biological, chemical, and physiochemical laboratory measurements with medicinal chemistry allows for the development of a chemical series SAR which, in turn, yields improvements in the characteristics of the lead compound (Milne, 2003). For example, the analyses of IC50 data for a series of related analogs are compared to the maintenance, improvement, or loss of MICs, to characterize the effect of structural modifications of the molecule. Likewise, tracking the ability of a compound to penetrate into the target organism—which is usually measured in genetically defined strains by MIC determinations, while maintaining the same IC50 values against the target—makes it possible to compare the MIC values to measurements of toxicity, such as inhibition of cell growth or passive lytic potential. In the end, the measurement of success of an antibacterial drug discovery program is the identification of compounds that are eventually marketed. Although it may be possible to achieve scientific prominence in one aspect of the drug discovery and development process, success of a program is ultimately judged by how well an organization is able to execute all aspects of the program, as measured by the number of new drugs developed.

LITERATURE CITED Acred, P. 1986. The Selbie or thigh lesion test. In Experimental Models in Antimicrobial Chemotherapy (O. Zak and M.A. Sande, eds) pp. 109-121. Academic Press, London. Arigoni, F., Talabot, F., Peitsch, M., Edgerton, M.D., Meldrum, E., Allet, E., Fish R., Jamotte, T., Curchod, M.L., and Loferer, H. 1998. A genome-based approach for the identification of essential bacterial genes. Nat. Biotechnol. 3:483-489. Barrett, C.T. and Barrett, J.F. 2003. Antibacterials: Are the new entries enough to deal with the emerging resistance problems? Curr. Opin. Biotechnol. 14:1-6. Bergeron, M.G. 1978. A review of models for the therapy of experimental infections. Scand. J. Infect. Dis. 14:189-206. Beyer, D., Kroll, H.P., Endermann, R., Schiffer, G., Siegel, S., Bauser, M., Pohlmann, J., Brands, M., Ziegelbauer, K., Haebich, D., Eymann, C., and Brotz-Oesterhelt, H. 2004. New class of bacterial phenylalanyl-tRNA synthetase inhibitors

Anti-Infectives

13A.1.7 Current Protocols in Pharmacology

Supplement 31

with high potency and broad-spectrum activity. Antimicrob. Agents Chemother. 48:525-532. Bleicher, K.H., Bohm, H.J., Muller, K., and Alanine, A.I. 2003. Hit and lead generation: Beyond high-throughput screening. Nat. Rev. Drug Discov. 2:369-378. Chalker, A.F. and Lunsford, R.D. 2002. Rational identification of new antibacterial drug targets that are essential for viability using a genomicsbased approach. Pharmacol. Ther. 95:1-20. DeVito, J.A., Mills, J.A., Liu, V.G., Agarwal, A., Sizemore, C.F., Yao, Z., Stoughton, D.M., Cappiello, M.G., Barbosa, M.D.F.S., Foster, L.A., and Pompliano, D.L. 2002. An array of targetspecific screening strains for antibacterial discovery. Nat. Biotechnol. 20:478-483. DiMasi, J.A., Hansen, R.W., and Grabowski, H.G. 2003. The price of innovation: New estimates of drug development costs. J. Health Econ. 22:151185. Dougherty, T.J., Barrett, J.F., and Pucci, M.J. 2002. Microbial genomics and novel antibiotic discovery: New technology to search for new drugs. Curr. Pharm. Des. 8:1119-1135. Fernandez, J., Barrett, J.F., Licata, L., Amaratunga, D., and Frosco, M. 1999. Comparison of efficacies of oral levofloxacin and oral ciprofloxacin in a rabbit model of a staphylococcal abscess. Antimicrob. Agents Chemother. 43:667-671.

Mills, S.D. 2003. The role of genomics in antimicrobial discovery. J. Antimicrob. Chemother. 51:749-752. Milne, G.M. 2003. Pharmaceutical productivity: The imperative for new paradigms. Annu. Rep. Med. Chem. 38:383-397. NCCLS (National Committee for Clinical Laboratory Standards). 1997. Methods for dilution antimicrobial susceptibility tests for bacteria that grow aerobically. Approved standard M7A4. National Committee for Clinical Laboratory Standards, Villanova, Pa. Nwaka, S. and Ridley, R.J. 2003. Virtual discovery and development for neglected diseases through public-private partnerships. Nat. Rev. Drug Discov. 2:919-928. Pucci, M.J., Barrett, J.F., and Dougherty, T.J. 2003. Bacterial “Genes-To-Screens” in the postgenomic era. In Pathogen Genomics: Impact on Human Health (K. J. Shaw, ed.). Humana Press, Totowa, N.J. Pratt, S.D., David, C.A., Black-Schaefer, C., Dandliker, P.J., Xuei, X.L., Warrior, X., Burns, D.J., Zhong, P., Cao, Z.S., Saiki, A.Y.C., Lerner, C.G., Chovan, L.E., Soni, N.B., Nilius, A.M., Wagenaar, F.L., Merta, P.J., Traphagen, L.M., and Beutel, B.A. 2004. A strategy for discovery of novel broad-spectrum antibacterials using a high-throughput Streptococcus pneumoniae transcription/translation screen. J. Biomol. Screen. 9:3-11.

Haney, S.A., Alkane, L.E., Dunman, P.M., Murphy, E., and Projan, S.J. 2002. Genomics in antiinfective drug discovery – getting to endgame. Curr. Pharm. Des. 8:1099-1118.

Rich, A. 2004. The excitement of discovery. Annu. Rev. Biochem. 73:1-37.

Judson, N. and Mekalanos, J.J. 2000. Transposonbased approaches to identify essential bacterial genes. Nat. Biotechnol. 18:740-745.

Silver, L. and Bostian, K. 1990. Screening of natural products for antimicrobial agents. Eur. J. Clin. Microbiol. Infect. Dis. 9:455-461.

Lindsay, M.A. 2004. Target discovery. Nat. Rev. Drug Discov. 2:831-837.

Thanassi, J.A., Hartman-Neumann, S.L., Dougherty, T.J., Dougherty, B.A. and Pucci, M.J. 2002. Identification of 113 conserved essential genes using a high-throughput gene disruption system in Streptococcus pneumoniae. Nucl. Acids Res. 30:3152-3162.

Lipinski, C.A., Lombardo, F., Dominy, B.W. and Feeney, P.J. 1997. Experimental and computational approaches to estimate solubility and permeability in drug discovery and development settings. Adv. Drug Deliv. Rev. 23:3-25. Look, G.C., Vacin, C., Dias, T.M., Ho, S., Tran, T.H., Wiesner, C., Fang, F., Marra, A., Westamacott, D., Hromockyj, A.E., Murpy, M.M., and Schullel, J.R. 2004. The discovery of biaryl acids and amides exhibiting antibacterial activity against Gram-positive bacteria. Bioorg. Med. Chem. Lett. 14:1423-1426. McDevitt, D. and Rosenberg, M. 2001. Exploiting genomics to discover new antibiotics. Trends Microbiol. 9:611-617.

Xuan, D., Zhong, M., Mattoes, H., Bui, K.-Q., McNabb, J., Nicolau, D., Quintilliani, R., and Nightingale, C.H. 2001. Streptococcus pneumoniae response to repeated moxifloxacin or levofloxacin exposure in a rabbit tissue cage model. Antimicrob. Agents Chemother. 45:794-799.

Contributed by John F. Barrett Merck Research Laboratories Rahway, New Jersey

Overview of Antimicrobial Drug Development

13A.1.8 Supplement 31

Current Protocols in Pharmacology

Overview of Safety Pharmacology Safety pharmacology, a relatively recent concept, is situated between classical toxicology and pharmacology (Fossa, 1994). Toxicology examines chemical-induced changes as a result of drug action, usually after repeated treatment at supratherapeutic doses, whereas safety pharmacology entails the search for changes that occur after acute administration of a chemical agent at doses approximating those to be employed clinically. A first attempt at defining safety pharmacology was contained in the Japanese Guidelines (Japanese Ministry of Health and Welfare, 1995), which provides considerable details on the types of pharmacology studies considered essential before an agent is administered to humans. More recently, the European Agency for the Evaluation of Medicinal Products proposed a new set of guidelines contained within the International Conference on Harmonization (ICH) safety guidelines (http://www.ich. org/cache/compo/276-254-1.html). ICH S7A deals with core battery studies, whereas ICH S7B is concerned specifically with assessment of QT interval prolongation, an effect that can lead to cardiac arrhythmias (torsades de pointe) and death. The ICH S7A guidelines came into effect in Europe in June, 2001 and have since been adopted in both the United States (US Food and Drug Administration, 2001) and Japan. The ICH S7B guidelines have also been adopted in the United States (US Food and Drug Administration, 2005). Described in this unit are a number of important issues relating to safety pharmacology as it is currently defined by regulatory agencies.

TERMINOLOGY A source of confusion about safety pharmacology arises from the various terms used to categorize the kinds of studies covered by this discipline. Besides safety pharmacology, other terms used to describe such studies include general pharmacology, ancillary pharmacology, secondary pharmacology, high-dose pharmacology, and regulatory pharmacology. Moreover, experiments performed to define the mechanisms of adverse effects observed during toxicological studies are often considered part of safety pharmacology. In its strictest sense, the word safety implies the absence of untoward effects that might endanger the health of a patient. Thus, the

UNIT 10.1

term safety pharmacology could be applied to all pharmacological studies undertaken to ensure the absence of adverse effects when the drug candidate is administered in a manner, and over a dose range, that is clinically relevant. Only studies predictive of risk should be part of a safety pharmacology assessment. The primary aim is to demonstrate that, at doses thought to be appropriate for obtaining the therapeutic benefit, there are no other effects that could be considered risk factors. A further aim would be to determine the maximum dose that could be administered before encountering adverse events. Such studies should also be useful for establishing a bridge between therapeutic doses and those to be used in toxicological studies, and for determining the maximum doses that can be administered and still ensure safety during Phase I human studies. While the term high-dose pharmacology would appear to be relevant in this regard, it is too restrictive in not including the notion that the drug candidate might have adverse effects on other systems even at therapeutic doses. In contrast, the terms general or ancillary pharmacology encompass all studies undertaken to characterize the probable therapeutic responses to the new substance. The aim of general pharmacology studies is to determine the selectivity of the drug candidate for the intended indication. For example, a novel anticancer compound would not usually be expected to display psychotropic activity, although there might be a therapeutic advantage if the drug had antidepressant or anxiolytic effects. Studies aimed at exploring such issues would come under general pharmacology since proof of safety is not their main purpose.

SAFETY PHARMACOLOGY VERSUS TOXICOLOGY Traditionally, safety pharmacology studies were conducted in a toxicological context. Toxicology differs from pharmacology in that toxicologists investigate mainly structure whereas pharmacologists investigate mainly function (Sullivan and Kinter, 1995). Indeed, there is an entire area of study linking morphological with functional changes and in defining their relationship to adverse events. For example, vomiting, diarrhea, or a change in body weight are easily detected during the clinical evaluations performed as part of

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

Safety Pharmacology/ Toxicology

10.1.1 Supplement 32

toxicological studies. Other potentially adverse responses, such changes in cardiac rhythmicity, might not be noted unless specific pharmacological studies are undertaken. Furthermore, changes in physiological function may occur in the absence of changes in organ structure and frequently occur at doses lower than those necessary to induce a structural change; and not all structural modifications are clearly associated with a detectable change in function. Although structural and functional changes are sometimes clearly related, it is not always possible to link them in terms of cause and effect. Thus, safety pharmacology and classical toxicology are complementary, with both providing information important for determining the safety of a new substance. While regulatory authorities may place more emphasis on formal toxicology studies, clinical pharmacologists may find safety pharmacology data more reassuring when designing clinical trials (Sullivan and Kinter, 1995).

COMPARISON OF JAPANESE AND ICH GUIDELINES

Overview of Safety Pharmacology

The Japanese Guidelines have their origins in a notification issued by the Japanese Ministry of Health and Welfare in September 1967 concerning “basic policies for approval to manufacture drugs.” The 1995 version of the Japanese Guidelines (Japanese Ministry of Health and Welfare, 1995) does not specifically mention safety pharmacology, describing instead general pharmacology as having the following aims: 1. to assess the overall profile of “general pharmacological effects” as compared to “principal pharmacological effects,” 2. to “obtain useful information on potential adverse effects,” and 3. to evaluate “effects of drugs on physiological functions not necessarily detectable in toxicological studies.” ICH S7A specifically mentions safety pharmacology and defines it as “those studies that investigate the potential undesirable pharmacodynamic effects of a substance on physiological functions in relation to exposure in the therapeutic dose-range and above.” The ICH definition has the considerable advantage in restricting safety pharmacology to the assessment of risk, thereby eliminating a wide range of studies that could be included if the definition were more general. Reducing the number of studies reduces drug development costs, which is an advantage to both the drug developer and the consumer.

The Japanese guidelines differ from ICH S7A in making more specific recommendations concerning the tests to be employed. Safety pharmacology studies are divided into Category A and Category B. Category A includes essential evaluations, whereas Category B studies are those to be carried out “when necessary.” For central nervous system (CNS) evaluations, Category A includes general behavioral observations, measures of spontaneous motor activity, general anesthetic effects (including assessment of potential synergism/antagonism with general anesthetics), proconvulsant effects, analgesia, and effects on body temperature. Category B includes effects on the electroencephalogram (EEG), the spinal reflex, conditioned avoidance response, and locomotor coordination. As far as the other body systems, the Japanese Category A includes studying effects on the cardiovascular and respiratory systems (e.g., respiration, blood pressure, blood flow, heart rate and electrocardiogram or ECG) the digestive system (including intestinal transit and gastric emptying) water and electrolyte metabolism (e.g., urinary volume, urinary concentrations of sodium, potassium and chloride ions) and “other important pharmacological effects.” The three vital systems the European guidelines (i.e., ICH guidelines) include in core battery studies are the CNS, cardiovascular (CV) system, and the respiratory system. Core battery CNS studies include motor activity, behavioral changes, coordination, sensory/motor reflex responses, and body temperature, with the remark that “the CNS should be assessed appropriately.” A similar remark is made about appropriate CV assessment, with specific mentions of blood pressure, heart rate and ECG, together with a suggestion that “in vivo, in vitro and/or ex vivo evaluations, including methods for repolarization and conductance abnormalities, should also be considered.” Comparison of the Japanese Guidelines and ICH S7A suggests a clear intent expressed in the ICH guidelines to free safety pharmacology from the constraints of a cookbook approach. On the other hand, their vagueness does not provide a clear idea of what evaluations could or should be performed. This is most apparent in the recommendations for follow-up studies. For the CNS, the ICH indicates that follow-up studies should include “behavioral pharmacology, learning and memory, ligand-specific binding, neurochemistry, visual, auditory and/or electrophysiology examinations, etc.” The subject of drug dependence/abuse, although a major concern for

10.1.2 Supplement 32

Current Protocols in Pharmacology

many substances with CNS effects, receives only a one-word mention in the section on “Other Organ Systems.” The ICH guidelines are much more explicit for the CV and respiratory systems. Indeed, all of ICH S7B is devoted to an analysis and recommendations for arrhythmogenic risk. ICH S7A specifically mentions renal/urinary, autonomic and gastrointestinal systems, although, surprisingly, no mention is made of nausea, despite the fact that it is a common adverse event.

CNS SAFETY PHARMACOLOGY ICH S7A guidelines recommend that core battery CNS studies include measures of drug-induced signs of CNS dysfunction as well as measures of spontaneous locomotion and motor coordination. Three other measurements, originally recommended by the Japanese Guidelines Category A, but dropped from the ICH S7A core battery, are the convulsive threshold, interaction with hypnotics, and effects on pain threshold (Porsolt et al., 2005). In spite of their exclusion from ICH S7A, such measures are useful in a core battery of CNS safety pharmacology procedures. Decreases in the convulsive threshold are an important component in the assessment of CNS safety. Several substances, including antipsychotics, such as clozapine, do not induce frank convulsions at any dose but clearly decrease the convulsive threshold. Even anticonvulsive activity, which in itself is not a risk factor, could be a predictor of cognition-impairing effects. Several anticonvulsants, such as benzodiazepines and glutamic acid NMDA receptor antagonists, are known to impair cognition. Thus, anticonvulsant activity could represent a useful first screen for potential cognitionimpairing effects. Likewise, sleep-inducing or sleep-attenuating activity could be unmasked by a barbiturate interaction procedure. While benzodiazepines, for example, do not by themselves induce sleep, their sleep-enhancing activity can be readily detected by studying their interaction with barbiturates. The same is true for psychostimulants, which may or may not induce signs of excitation in a primary observation procedure, but clearly block barbiturateinduced sleep. Finally, a drug-induced increase in pain sensitivity can be readily assessed using a simple nociception procedure (e.g., the hotplate; UNIT 5.7). Whereas analgesic activity is not in itself a risk factor, the presence of analgesic

activity could be a useful predictor of abuse liability.

CARDIOVASCULAR SAFETY PHARMACOLOGY ICH S7B guidelines, which deal exclusively with the evaluation of proarrhythmic risk, recommend that core battery cardiovascular studies include measures of drug-induced effects on arterial blood pressure, heart rate, and ECG, together with the suggestion that “in vivo, in vitro and/or ex vivo evaluations, including methods for repolarization and conductance abnormalities, should also be considered.” Recommended methodologies include measurements of ionic currents in isolated animal or human cardiac myocytes, and in cultured cell lines or heterologous expression systems with cloned human channels. Other procedures include measurement of action potential (AP) parameters in isolated cardiac preparations or analysis of specific electrophysiologic parameters indicative of AP duration in anesthetized animals, and measurement of ECG in conscious or anesthetized animals. While it is clear that proarrhythmic effects represent a major cardiovascular danger for new substances, the excessive focus on this risk by ICH S7B, together with the avalanche of reports dealing with methodological issues, have taken attention away from the many other types of drug-induced cardiovascular risk. A comprehensive systemic, cardiac, pulmonary, and renal hemodynamic evaluation in a large animal species like the dog is essential for an adequate evaluation of cardiovascular risk. Other risk factors, such as drug-induced depression of myocardial contractility or pulmonary hypertension, are critically important, even in the absence of other cardiac electrical disorders. In contrast to the telemetered conscious dog, the number of cardiovascular parameters that can be studied in an acute hemodynamic study in the anesthetized animal yields a wealth of information regarding the mechanisms responsible for effects on the cardiovascular system (Lacroix and Provost, 2000). Parameters that can be studied in the anesthetized animal include aortic blood pressure, heart rate, cardiac output, stroke volume, pulmonary artery blood pressure, peripheral vascular resistance, renal blood flow, and ECG. The hERG channel assay is now considered the model of choice for cardiac proarrhythmic risk assessment. Whereas hERG channel assays, using binding techniques and automated

Safety Pharmacology/ Toxicology

10.1.3 Current Protocols in Pharmacology

Supplement 32

Overview of Safety Pharmacology

technology, would seem appropriate as a first screen for QT prolongation in the early stages of safety evaluation, the use of the hERG channel patch clamp technique (UNIT 10.8) is recommended for the core battery of cardiovascular studies. Although more time-consuming, the patch clamp technique is an indicator of function, as opposed to receptor affinity, and lends itself more readily to compliance with good laboratory practice (GLP; see http://www.oecd.org), which is mandatory for ICH S7A-recommended core battery studies. On the other hand, the hERG channel assay cannot constitute a stand-alone in vitro test for evaluating proarrhythmic risk. Calcium agonists, for example, are known to lengthen the AP duration and favor the occurrence of early afterdepolarizations and/or delayed afterdepolarizations, either of which can lead to torsades de pointe. Cardiac risk related to this calciumdependent mechanism cannot be detected by the hERG channel assay. Furthermore, the hERG channel assay can frequently overestimate the cardiac risk for a new substance because partial inhibition of the potassium ion channel conductance (IKr ) may not result in AP prolongation in a Purkinje fiber preparation because of counteracting effects on other cardiac ion channels. For this reason the Purkinje fiber assay (UNIT 11.3) constitutes a necessary adjunct for investigating the effects of a test substance on the different parameters of the AP. Whichever in vitro assay is used, it cannot fully mimic the in vivo situation. All in vitro data must be considered in the context of plasma protein binding (UNIT 7.5), pharmacokinetic parameters (UNIT 7.1), and anticipated plasma concentrations of the test substance; thus, in vivo analyses in conscious animals monitored by telemetry remain an essential component in the assessment of proarrhythmic risk. Nonetheless, neither does telemetry in conscious animals constitute a stand-alone technique, since it provides little information regarding the mechanism responsible for any observed effect. Orthostatic hypotension is another common cardiovascular risk not covered by the ICH S7A core battery. As this constitutes a major adverse effect associated with many different classes of drugs, it is important to determine whether it is a property of a new chemical entity intended for human use. A simple animal model for orthostatic hypertension is the tilting test in the anesthetized rat (Hashimoto et al., 1999). With this assay, orthostatic hypotension is exacerbated by prazocine and

β-adrenoceptor antagonists. Inclusion of such a test in a cardiovascular core battery would not constitute a major expense but would provide a more complete assessment of cardiovascular risk.

RESPIRATORY SAFETY PHARMACOLOGY Drugs can cause changes in pulmonary function (UNIT 10.9) by direct actions on the respiratory system or as a consequence of central, metabolic (alterations in acid-base balance) or vascular (pulmonary hypertension) effects. ICH S7A guidelines include respiration as a vital function that must be assessed during safety evaluations. Whole-body plethysmography is a method of choice for examining whether a test substance affects airway function. Whole-body plethysmography is performed either in the unrestrained rat or guinea pig following oral or intravenous administration of the test agent. Modern systems allow a number of ventilatory parameters to be measured, including inspiratory and expiratory times, peak inspiratory and expiratory flows, tidal volume, respiratory rate, relaxation time, and pause and enhanced pause. This allows for the differentiation between effects on respiratory control and the mechanical properties of the lung (Murphy, 2002). As a general screen, the whole-body test is preferable to the head-out method because the animals can be studied over longer periods of time since they have freedom of movement. A weakness of whole-body plethysmography is that it is not sensitive to the respiratory-depressant effects of some drugs, such as barbiturates and opioids. Attempts to increase the sensitivity of this assay system by placing the animals in a CO2 -enriched environment (Van den Hoogen and Colpaert, 1986; Gengo et al., 2003) appear promising. Although the anesthetizeddog preparation does not lend itself to the evaluation of spontaneous lung function, it is well suited for evaluating the risk of pulmonary hypertension (as mentioned above). For this reason the anesthetized dog constitutes an important component in a comprehensive respiratory safety evaluation.

TIMING OF SAFETY PHARMACOLOGY STUDIES AND GOOD LABORATORY PRACTICE One of the issues not clearly addressed in either the Japanese or ICH guidelines is the timing of safety pharmacology studies. Whereas the Japanese guidelines imply that safety data are required for marketing

10.1.4 Supplement 32

Current Protocols in Pharmacology

approval, ICH S7A clearly states they are needed prior to initiating Phase 1 human trials. Both the Japanese and ICH guidelines require that Category A or core battery studies be performed according to Good Laboratory Practice (GLP). Exceeding these regulatory requirements, many pharmaceutical firms engage in safety pharmacology studies early in the drug discovery process, even at the very beginning of in vivo experiments (Sullivan and Kinter, 1995). A primary observation procedure, such as the Irwin test (UNIT 10.10), is frequently the first assessment in living animals for determining acute toxicity, the active dose-range, and the principal effects on behavior and physiological function. Substances are also frequently screened very early in the discovery process for potential proarrhythmic risk using hERG procedures (UNIT 10.8). Such early safety screening is rarely conducted according to GLP, and therefore falls outside the requirements of the regulatory agencies. Nonetheless, information gained from such tests is vital in directing the discovery program and the selection of clinical candidates. Safety studies are performed differently early in the discovery process than at pre-Phase 1. An early-stage Irwin test tends to use fewer animals, the mouse rather than the rat, and with doses selected sequentially on the basis of effects observed with previous doses, to determine the lethal dose as rapidly as possible. With later-stage Irwin tests, the dose range is fixed in advance, beginning with the lowest dose approximating the therapeutic dose, followed by multiples of 10 or 100, up to but not including a lethal dose. Similarly, early stage hERG tests employing a binding assay or a patch clamp procedure might examine just a single high concentration of several substances within the same chemical series rather than a range of concentrations, as would be the case in a later hERG evaluation. Indeed the experimental aims of the early and late tests are different. In early-stage work the objective is to detect the presence of risk as a guide in the selection of substances for development. Later-stage analysis is performed to confirm the absence of risk in the relevant dose-range for the selected compound.

possible and to minimize their suffering and discomfort. Since the goal of safety pharmacology is to assess the risk of side effects, the possibility of causing suffering in the experimental animals is higher than in other areas of pharmacology. The investigator must remain sensitive to these issues, not only in planning and designing the protocols, but also while performing the experiments. For example, procedures for terminating the experiment in the event of well-defined events, such as pain or death, must be established. Ethical issues must be considered within the context of the aims of the experiments, which, ultimately, are to minimize human risk. While reducing risk to humans is of paramount importance, it is still possible to devise scientifically valid experiments using a reduced number of laboratory animals. For example, it is now accepted that the traditional LD50 acute toxicity test, which requires the use of a large number of animals, yields only limited information. Considerably more information, requiring the use of fewer animals, is obtained with the Irwin procedure (UNIT 10.10).

STATISTICAL EVALUATION Since identification of risk is the chief aim of a safety pharmacology test, it is essential that positive results not be overlooked. In other words, the generation of false negatives should be kept to a minimum (Porsolt et al., 2005). False positives, or the erroneous detections of possible risk, although bothersome, are less serious and can usually be corrected with supplementary testing. Thus, the risk of false negatives should be decreased as much as possible, even if there is an increase in the risk of false positives. A test substance found not to have significant safety risks based on preclinical studies, even after the use of oversensitive statistics, is more likely to be truly devoid of risk. As a consequence, safety pharmacology, in contrast to efficacy pharmacology, should employ statistical procedures possessing maximal sensitivity for detecting possible effects on a dose-by-dose basis, at the acknowledged risk of increasing the number of false positives.

CONCLUSIONS ETHICAL AND ANIMAL WELFARE ISSUES As with all procedures involving living animals, there are important ethical considerations in the choice of methods. The guiding principles are to use as few animals as

Thanks in part to ICH S7A, safety pharmacology can now be considered an independent discipline situated between traditional toxicology and efficacy/discovery pharmacology. Safety pharmacology is, however, a pharmacological rather than a toxicological

Safety Pharmacology/ Toxicology

10.1.5 Current Protocols in Pharmacology

Supplement 32

discipline since it concerns the study of drug actions on physiological function rather than on physical/anatomical structure. Although using identical methods, safety pharmacology differs from efficacy pharmacology in that the former evaluates the potentially adverse effects of test substances on normal function, whereas the latter is aimed at establishing therapeutic potential. Both provide information critical for drug discovery and development.

LITERATURE CITED Fossa, A.A. 1994. Inaugural conference on general pharmacology/safety pharmacology. Drug Dev. Res. 32:205-272. Gengo, P.J., Pettit, H.O., O’Neill, S.J., Su, Y.F., McNutt, R., and Chang, K.J. 2003. DPI-3290 [(+)-3-((α-R)-α-((2S,5R)-4-Allyl2,5-dimethyl-1-piperazinyl)-3-hydroxybenzyl)N-(3-fluorophenyl)-N-methylbenzamide]. II. A mixed opioid agonist with potent antinociceptive activity and limited effects on respiratory function. J Pharmacol. Exp. Ther. 307: 1227-1233. Hashimoto, Y., Ohashi, R., Minami, K., and Narita, H. 1999. Comparative study of TA-606, a novel angiotensin II receptor antagonist, with losartan in terms of species difference and orthostatic hypotension. Jpn. J. Pharmacol. 81:63-72. Japanese Ministry of Health and Welfare. 1995. Japanese guidelines for nonclinical studies of drugs manual. Pharmaceutical Affairs Bureau, Japanese Ministry of Health and Welfare, Yakugi Nippo, Japan. Lacroix, P. and Provost, D. 2000. Basic safety pharmacology: The cardiovascular system. Therapie 55:63-69. Murphy, D.J. 2002. Assessment of respiratory function in safety pharmacology. Fundam. Clin. Pharmacol., 16:183-196.

Sullivan, A.T. and Kinter, L.B. 1995. Status of safety pharmacology in the pharmaceutical industry. Drug Dev. Res. 35:166-172. US Food and Drug Administration. 2001. ICH guidance for industry: S7A safety pharmacology studies for human pharmaceuticals. US Food and Drug Administration, Rockville, Md. US Food and Drug Administration. 2005. ICH guidance for industry: S7B nonclinical evaluation of the potential for delayed ventricular repolarization (QT interval prolongation) by human pharmaceuticals. US Food and Drug Administration, Rockville, Md. Van den Hoogen, R.H. and Colpaert, F.C. 1986. Respiratory effects of morphine in awake unrestrained rats. J. Pharmacol. Exp. Ther. 237:252-259.

Internet Resources http://www.fda.gov/cber/gdlns/ichs7a071201.htm The FDA’s S7A safety guidelines for pharmacology studies of human pharmaceuticals. http://www.fda.gov/cber/gdlns/ichqt.htm The FDA’s S7B safety guidelines for pharmacology studies of human pharmaceuticals. http://www.ich.org/cache/compo/276-254-1.html Contains links to both the ICH S7A (Safety Pharmacology Studies for Human Pharmaceuticals) and S7B (The Nonclinical Evaluation of the Potential for Delayed Ventricular Repolarization (QT Interval Prolongation) By Human Pharmaceuticals) safety guidelines. http://www.oecd.org The OECD Website contains comprehensive information about GLPs.

Contributed by Roger D. Porsolt Porsolt & Partners Pharmacology Boulogne-Billancourt, France

Porsolt, R.D., Picard, S., and Lacroix, P. 2005. International safety pharmacology guidelines (ICH S7A and S7B): Where do we go from here? Drug Dev. Res. 64:83-89.

Overview of Safety Pharmacology

10.1.6 Supplement 32

Current Protocols in Pharmacology

PAIRED T TEST

data, it is natural to form the differences, and perform the analysis on the differences. An interesting recent account is by Senn & Richardson (5) who discuss the study which W.S. Gosset (Student) used in his paper. Moses (3) gives a summary of the theoretical aspects of this test. We assume that there are n pairs of observations, and that each pair is independent of the other pairs. Denote the paired observations as xi and yi , and their difference as di for i = 1, . . . , n. The analysis assumes that the observations are normally distributed with means µx and µy . The random variable D = X − Y is then normally distributed with mean µd = µx − µy and variance σd2 . The null hypothesis is H0 : µd = 0. A Student t statistic can be computed as

HENRY HSU and PETER A. LACHENBRUCH Food and Drug Administration, Rockville, MD, USA

A basic concept in study design is that extraneous sources of variation should be controlled, so that the comparison is made among groups which are alike except for the intervention. Techniques which may be used include stratification and analysis of covariance. Data which are naturally paired form useful strata and often facilitate a more precise comparison than one in a set of unrelated subjects. This leads to the paired t test. The cost of pairing lies in the effort needed to determine the values of the matching variables (which is low when there are natural pairs such as litter mates or pre- and post-intervention measures), and the reduction of degrees of freedom (df) from 2(n − 1) to n − 1. The loss of degrees of freedom is almost never a problem when n is greater than 15. The statistic is based on the difference of the members of the pair, so the variance of the difference is 2σ 2 (1 − ρ), where ρ is the correlation coefficient, and σ 2 is the variance of a single observation. If ρ = 0, then the variance is the same as that of the difference between unpaired observations (2σ 2 ). In this case, the loss is that of half of the degrees of freedom. When the pairing is effective, ρ > 0, and the difference will have a smaller variance than if the observations were unpaired. The tradeoff is almost always in favor of pairing. The paired t test is used to compare mean differences when the observations have been obtained in pairs, and are thus dependent. Examples include observations made on weight before and after an intervention in a subject, serum cholesterol of two members of a family, and blood concentration of a toxin in litter mates. Subjects are often matched on characteristics such as age, gender, race, and study site (in multicenter studies). In each of these examples, the two observations are made on the same response and will be correlated. Accounting for this is necessary to analyze the study properly. For paired

t=

d √ . (sd / n)

This statistic is then compared to Student’s t distribution with n − 1 df. This analysis is equivalent to a randomized blocks analysis of variance in which the pairs correspond to blocks. If the variances in X and Y are equal, the variance of D is 2σ 2 (1 − ρ), as noted above. If ρ < 0, the variance is increased. This would not usually be the case when the matching process is effective. If the variances are unequal, the variance of D is σ12 + σ22 − 2ρσ1 σ2 . The relative efficacy of a paired t-test is 1/(1 − ρ) as compared with a twoindependent-sample t test in a parallel comparison. For example, if ρ = 1/3, the relative efficacy is 1.5. This means that 100 paired observations would have the same power to detect differences as an unpaired study of 150 observations per group. 0.1 Example We give here, in Table 1, data used by Student (Gosset) as cited in Senn & Richardson (5). The data were on the amount of sleep gained under two soporic drugs. The means given here agree with those of Senn & Richardson, but the standard deviations are slightly larger. The values they

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

PAIRED T TEST

2

Table 1. Patient

Dextro

Laevo

1 2 3 4 5 6 7 8 9 10 Mean Standard deviation

0.7 −1.6 −0.2 −1.2 −0.1 3.4 3.7 0.8 0.0 2.0 0.75 1.79

1.9 0.8 1.1 0.1 −0.1 4.4 5.5 1.6 4.6 3.4 2.33 2.00

report were those given by Student, who used the divisor n in computing the variance. Using the unpaired t-test we obtain a t = 1.86 with 18 df and a P value of 0.0792. By pairing the observations, √ as we should, the result is t = 1.58/(0.39/ 10) = 4.06 with 9 df and a P value of 0.0028. It is better to report the mean of the differences and its standard deviation rather than showing only the t statistic or the P value (or worse, NS for ‘‘not significant’’, or some number of asterisks!). 1

ROBUSTNESS

The robustness properties correspond to those of the one-sample t test. The effect of nonnormality is fairly small if n is at least 30, since the distribution of d will be close to normal in that case. If one difference (or a few) appear to be quite large (i.e. outliers) the results can be affected. Outliers can be considered a form of nonnormality. They affect the variance of the observations, and can also affect the skewness of the distributions. The P values reported from an analysis are often given as P < 0.05 or P < 0.01. Since the P value depends on the behavior of the distribution in its tails, nonnormality generally means that statements such as P < 0.001 are rarely accurate (the quoted P value for the example thus should be regarded as P < 0.01). Lack of independence among the pairs (which might arise if multiple members of a litter or a family were included in a study,

i.e. clustering) can seriously affect the level of the test. If the correlation between any pair of differences is γ , the variance of the differences is σ 2 (1 + 2γ ). Thus, the estimated variance is biased. If γ > 0, the denominator of the t statistic is too small, and the significance levels are incorrect. If the correlation holds only among certain pairs (e.g. independent clusters would lead to a block diagonal covariance matrix), the analysis is more complex, but the estimated variance is still biased. Lack of common variance in X and Y does not formally affect the analysis. However, unless the primary interest is in the difference between the observations, the lack of common variance indicates that X and Y do not have the same distribution, although they might have the same mean. If the variance differs over the pairs, heteroscedasticity concerns arise. Rosner (4) has suggested a random effects model which accounts for this. Missing values can create problems. Usually, only one member of the pair is missing. If the missing value is related to the mean value within the pair, the missingness is not random, and the t test is affected. For further discussion of these points, see Miller (2) or Madansky (1). Several alternatives exist to the paired t test. These are useful if the distribution is not normal and there is concern that this may affect the performance of the test. The sign test uses the number of positive (or negative) signs as a binomial variable with probability parameter 1/2 under H0 , and, for large samples, computes the standard normal deviate, z, to test H0 . The asymptotic relative efficiency (ARE) of this test is 0.637 when the differences are normal. The signed-rank test ranks the absolute values of the differences and sums the ranks corresponding to the positive (or negative) signs. The ARE of this test is 0.955. The normal scores test replaces the ranks of the differences by their expected values under normality and computes a t test on these. Its ARE is 1.0. For observations from nonnormal distributions, the efficiencies of the nonparametric procedures may be higher than indicated here and the t test can be very inefficient.

PAIRED T TEST

REFERENCES 1. Madansky, A. (1988). Prescriptions for Working Statisticians. Springer-Verlag, New York. 2. Miller, R. G. (1985). Beyond Anova. Wiley, New York. 3. Moses, L. (1985). Matched pairs t-tests, in Encyclopedia of Statistical Sciences, Vol. 5, S. Kotz

3

& N. L. Johnson, eds. Wiley, New York, pp. 289–203. 4. Rosner, B. (1982). A generalization of the paired t-test, Applied Statistics 31, 9–13. 5. Senn, S. & Richardson, W. (1994). The first t-test, Statistics in Medicine 13, 785–803.

PARALLEL TRACK A mechanism to permit wider availability of experimental agents is the ‘‘parallel track’’ policy (Federal Register of May 21, 1990), which was developed by the U.S. Public Health Service in response to acquired immunodeficiency syndrome (AIDS). Under this policy, patients with AIDS whose condition prevents them from participating in controlled clinical trials can receive investigational drugs shown in preliminary studies to be promising.

This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cder/handbook/parallel.htm) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

PARTIALLY BALANCED DESIGNS

ASSOCIATION SCHEMES Suppose that there are t treatments. There are three equivalent ways of defining an association scheme on the treatments: as a partition of the pairs of treatments, as a set of matrices, and as a colored graph. According to the most common definition, an association scheme with s associate classes is a partition of the unordered pairs of treatments into s associate classes. If u and υ are treatments and {u, υ} is in the ith associate class, then u and υ are called ith associates. The partition must have the property that there are numbers pkij , for i, j, k = 1, 2, . . . , s, such that if u and υ are any kth associates the number of treatments that are ith associates of u and jth associates of υ is pkij . (These numbers are not powers, so some authors write them as pijk .) It follows that each treatment has ni ith associates, where

This article should be read in conjunction with BLOCKS, BALANCED INCOMPLETE; BLOCKS, RANDOMIZED COMPLETE; GENERAL BALANCE; GROUP-DIVISIBLE DESIGNS; and INCOMPLETE BLOCK DESIGNS. Balanced incomplete block designs (BIBDs) have many desirable qualities. They are easy to analyze; many families of them are easily constructed; the loss of information on treatment estimates (in the sense of increase in variance) due to blocking is as small as it can be for the given block size; and it is easy to compare them with other incomplete block designs, because usually the BIBDs are superior in every respect. However, for many combinations of block size, number of treatments and number of replications, there is no balanced incomplete block design. Bose and Nair [5] introduced partially balanced incomplete block designs in 1939 for use in such situations, hoping to retain many of the desirable properties of BIBDs. They succeeded in their aim to a certain extent: some of the new class of designs are very good, while others are undoubtedly useless for practical experimentation. Unfortunately, any discussion of partial balance must necessarily be more technical than that of BIBDs, because there is an extra layer of complication. Moreover, many of the ideas have been developed and clarified by pure mathematicians, without ever being reexpressed in statistical language. Thus some sections of this article are unavoidably technical. It is hoped that the division into sections with titles will enable the nonmathematical reader to learn something from this article. During the development of the subject there have been minor modifications to the definition of partial balance: the most generally accepted definition, which we use here, is from a review paper in 1963 by Bose [3]. The key idea is the combinatorial concept of association scheme, to which the first part of this article is devoted. The designs themselves are introduced in the second part of the article; the third part discusses generalizations of partial balance and related ideas.

ni = pki1 + pki2 + · · · + pkis for all k
= i. It is convenient to say that each treatment is its own zeroth associate. The preceding property still holds, with j

p0ij = pi0 = pi0j = 0, p0ii = ni ;

if

i
= j;

pii0 = pi0i = 1.

An association scheme may be represented by a set of t × t matrices. For i = 0, . . . , s let Ai be the t × t matrix whose (u, υ) entry is 1 if u and υ are ith associates and 0 otherwise. The matrices A0 , A1 , . . . , As are called association matrices, and satisfy the following conditions: (a) (b) (c) (d) (e)

Each element of Ai is 0 or 1. Ai is symmetric. A0 = I, the identity matrix.
s i=0 Ai = J, the all-ones matrix.
Ai Aj = k pkij Ak .

Conversely, any set of square matrices satisfying conditions (a)–(e) defines an association scheme. An association scheme may also be represented by a colored graph. There is one

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

PARTIALLY BALANCED DESIGNS

vertex for each treatment, and there are s colors. The edge uυ is colored with the ith color if u and υ are ith associates. Now condition (e) may be interpreted as, e.g.: If uυ is a green edge then the number of green-redblue triangles containing uυ is independent of u and υ. If s = 2, no information is lost by erasing the edges of the second color to obtain an ordinary graph. A graph obtained in this way is called a strongly regular graph [4]. Much literature on association schemes with two associate classes is to be found under this heading, with no explicit mention of association schemes. See also GRAPH THEORY. The simplest association scheme has one associate class, and all treatments are first associates. Denote this scheme by B(t), where t is the number of treatments. While not very interesting in its own right, B(t) can be used to generate other association schemes. Examples of Association Schemes with Two Associate Classes

Group-Divisible Association Scheme. Suppose that t = mn. Partition the treatments into m sets of size n. The sets are traditionally called groups in this context, even though there is no connection with the technical meaning of group used later in this article. Two treatments are first associates if they are in the same group; second associates, otherwise. Call this scheme GD (m, n). Triangular Association Scheme. Suppose that t = n(n − 1)/2. Put the treatments into the lower left and upper right triangles of an n × n square array in such a way that the array in symmetric about its main diagonal, which is left empty. Two treatments are first associates if they are in the same row or column; second associates, otherwise. Call this scheme T(n). Then T(4) is GD (3, 2) with the classes interchanged. The square

Table 1. * A B C D

A * E F G

B E * H I

C F H * J

D G I J *

Table 2. 0 1 2 3 4

0

1

2

3

4

0 1 2 2 1

1 0 1 2 2

2 1 0 1 2

2 2 1 0 1

1 2 2 1 0

array for T(5) is shown in Table 1. The complement of the corresponding strongly regular graph is in Fig. 1, known as a Petersen graph. In an alternative description of T(n), the t treatments are identified with the unordered pairs from a set of size n. Then treatments are first associates if, as pairs, they have an element in common. The association scheme T(n) arises naturally in diallele cross experiments, where the treatments are all individuals (plants or animals) arising from crossing n genotypes. If there are no self-crosses, and if the gender of the parental lines is immaterial,   then there are (n 2) treatments and the first associates of any given treatment are those having one parental genotype in common with it. Latin Square Association Schemes. Suppose that t = n2 , and arrange the treatments in an n × n square array. Two treatments are first associates in the association scheme L2 (n) if they are in the same row or column. Let  be any n × n Latin square∗ . Then  defines an association scheme of type L3 (n): two treatments are first associates if they are in the same row or column or letter of . Similarly, if 1 , . . . , r−2 are mutually orthogonal Latin squares (3
r
n + 1), we may define an association scheme of type Lr (n) by declaring any two treatments in the same row, column, or letter of any of 1 , . . . , r−2 , to be first associates; any other pair, to be second associates. When r = n + 1, this scheme degenerates into B(t). Cyclic Association Schemes. Identify the treatments with the integers modulo t. Suppose that D is a set of nonzero integers modulo t with the properties that if d ∈ D, then t − d ∈ D; and that the differences d − e modulo t

PARTIALLY BALANCED DESIGNS

3

Figure 1.

with d, e in D include each element of D exactly N times and each other nonzero integer modulo t exactly M times, for some fixed numbers N and M. In the cyclic association scheme C (t, D) two treatments u and υ are first associates if u − υ (modulo t) is in D; second associates, otherwise. For example, if t = 5 we may take D = {1, 4}. Then N = 0 and M = 1. The association scheme C (5, D) is shown in Table 2. Here the (u, υ) entry of the table is i if u and υ are ith associates. Combining Association Schemes For n = 1, 2, let Cn be an association scheme with sn associate classes for a set Tn of tn treatments. There are two simple ways of combining these two schemes to obtain an association scheme for t1 t2 treatments. In both cases, we identify the new treatments with the set T1 × T2 of ordered pairs (u1 , u2 ) with u1 ∈ T1 and u2 ∈ T2 (see also NESTING AND CROSSING IN DESIGN). We may nest C2 in C1 to obtain the association scheme C1 /C2 with s1 + s2 associate classes. Treatments (u1 , u2 ) and (υ1 , υ2 ) are ith associates if u1 and υ1 are ith associates in C1 (for 1
i
s1 ) or if u1 = υ1 and u2 and υ2 are (i − s1 )th associates in C2 (for s1 + 1
i
s2 ). For example, B(m) / B(n) is GD (m, n).

We may cross C1 and C2 to obtain the association scheme C1 × C2 with s1 s2 + s1 + s2 associate classes. If the zeroth associate class is included in C1 , C2 and C1 × C2 , then the associate classes of C1 × C2 may be labeled by ij, for 0
i
s1 and 0
j
s2 . Treatments (u1 , u2 ) and (υ1 , υ2 ) are ijth associates in C1 × C2 if u1 and υ1 are ith associates in C1 and if u2 and υ2 are jth associates in C2 . The crossed scheme B(m) × B(n) is called a rectangular association scheme R (m, n), because its treatments may be arranged in an m × n array: two treatments are 01-th or 10-th associates if they are in the same row or the same column, respectively, and 11-th associates otherwise. Factorial Association Schemes There is a natural way of defining an association scheme on treatments with a factorial∗ structure. For example, suppose that treatments are all combinations of the t1 levels of factor F1 with the t2 levels of factor F2 . If F2 is nested in F1 , then the association scheme B(t1 )/B(t2 ) [i.e., GD (t1 , t2 )] corresponds naturally to the breakdown of treatment effects into the two factorial effects: F1 and F2 -within-F1 . If there is no nesting, then the association scheme B(t1 ) × B(t2 ) [i.e., R(t1 , t2 )] corresponds to the three factorial effects: main effects∗ F1 and F2 , and interaction∗ F1 F2 .

4

PARTIALLY BALANCED DESIGNS

Table 3. Operators Experienced Sow 1 1 2 2 3 3 4 4

Novices

Site

1

2

3

4

5

6

7

8

1 2 1 2 1 2 1 2

0 3 6 9 6 9 6 9

1 4 7 10 7 10 7 10

1 4 7 10 7 10 7 10

1 4 7 10 7 10 7 10

2 5 8 11 8 11 8 11

2 5 8 11 8 11 8 11

2 5 8 11 8 11 8 11

2 5 8 11 8 11 8 11

More generally, if there are n treatment factors F1 , F2 , . . . , Fn with t1 , t2 , . . . , tn levels, respectively, a factorial association scheme may be built up from B(t1 ), B(t2 ), . . . , B(tn ) by repeated use of crossing and / or nesting. Roy [49] described repeated nesting, and Hinkelmann [19], repeated crossing, but the two operations may be combined. One of the simplest crossed-and-nested schemes is (B(t1 ) × B(t2 ))/B(t3 ), which is also called a generalized right angular scheme. A somewhat more general construction, depending only on the nesting relations among the factors, is given by Speed and Bailey [54]. Example. An experiment was conducted to find out how the ultrasonic fat probe measurement of a sow was affected by the particular sow, by the site on the sow where the measurement was taken, and by the operator who did the measuring. Each of eight operators measured each of four sows at each of two sites on the body (technically known as P2 and H). The operators consisted of two batches of four: those in the first batch were experienced users of the small portable measuring instruments being used, while those in the second were novices. The 64 treatments (all combinations of sow, site, and operator), therefore, had a 4 × 2 × (2/4) factorial structure. The corresponding association scheme is shown in Table 3. Here the 64 treatments are shown in an 8 × 8 array: the entry for treatment u is i if u is an ith associate of the treatment in the top left-hand corner. An unstructured set of treatments may have a dummy factorial structure imposed on it to help with the construction of a design or

the subsequent analysis. Any corresponding factorial association scheme is called pseudofactorial. Other Association Schemes We have space to mention only a few of the many other families of association schemes. The d-dimensional triangular asso  and s= ciation scheme T(d, n) has t=n d min(d, n − d). The treatments are identified with subsets of size d from a set of size n. Two treatments are ith associates if, as subsets, they have d − i elements in common. Thus T(2, n) = T(n). These schemes are also called Johnson schemes. Conceptually similar is the d-dimensional lattice association scheme, also called the Hamming association scheme H (d, n), which has t = nd and s = d. The treatments are identified with ordered d-tuples of the numbers 1, 2, . . . , n, and two treatments are ith associates if their coordinates are the same in d − i positions. The Latin square∗ association scheme Lr (n) also has generalizations. The scheme Lr (n) differs from Lr (n) in that it has r + 1 associate classes. Treatments are first, second, third, . . . , rth associates if they are in the same row, column, letter of 1 , . . ., letter of r−2 . Any way of amalgamating associate classes in Lr (n) produces another association scheme (this is not true for association schemes in general): in particular, amalgamation of the first r classes produces Lr (n). The cyclic scheme C (t, D) is formed by amalgamating classes of the general cyclic association scheme C(t), which has (t − 1)/2 associate classes if t is odd, t/2 if t is even. Treatments u and υ are ith associates if

PARTIALLY BALANCED DESIGNS

u − υ = ±i modulo t. Both C (t, D) and C(t) may be further generalized by replacing the integers modulo t by any Abelian group (in the algebraic sense of ‘‘group’’). There are also many other individual association schemes that do not fit into any general families. Algebra Conditions (a)–(e) for an association scheme have algebraic consequences that are important for partially balanced designs. Let A be the set of linear combinations of the matrices A0 , A1 , . . . , As , which is called the Bose–Mesner algebra of the association scheme. Every matrix M in A is ‘‘patterned’’ according to the association scheme; that is, the entry M uυ depends only on the associate class containing {u, υ}. If M and N are in A, then so are M + N, MN and αM, for every real number α; moreover, MN = NM. It follows that the matrices in A are simultaneously diagonalizable; i.e., the t-dimensional vector space corresponding to the treatments is a direct sum of spaces W0 , W1 , . . . , Ws , each of which is an eigenspace of every matrix in A. Hence if M is invertible and M is in A, then M−1 is in A. Usually the eigenspaces W0 , W1 , . . . , Ws are easy to calculate. We may always take W0 to be the one-dimensional space spanned by the vector (1, 1, . . . , 1). At worst, the problem of finding W0 , W1 , . . . , Ws can be reduced to a similar problem for (s + 1) × (s + 1) matrices. See Bose and Mesner [6], Cameron and van Lint [9], Delsarte [16], and Dembowski [17] for further details. The trivial association scheme B(t) has two eigenspaces, W0 and W1 . The space W1 consists of all contrasts, i.e., all vectors whose entries sum to zero. For the group-divisible association scheme GD (m, n) the spaces W1 and W2 consist of all between-groups contrasts∗ and all withingroups contrasts, respectively. That is, a vector is in W1 if its entries are constant on each group and sum to zero overall; a vector is in W2 if its entries sum to zero on each group. The rectangular association scheme has four eigenspaces: W1 and W2 consist of betweenrows contrasts and between-columns contrasts respectively, while W3 contains all vectors that are orthogonal to W0 , W1 and

5

W2 . In general, the eigenspaces of every factorial association scheme are the spaces corresponding to main effects and interactions in the factorial sense. If t is odd, every eigenspace (except W0 ) of the cyclic association scheme C(t) has dimension 2. If the treatments are written in the order 0, 1, 2, . . . , t − 1, a basis for Wi consists of (1, cos(2π i/t), cos(4π i/t), . . . , cos(2π (t − 1)i/t)); (0, sin(2π i/t), sin(4π i/t), . . . , sin(2π (t − 1)i/t)), for each i with 1
i
(t − 1)/2. If t is even, there is an additional eigenspace Wt/2 spanned by (1, −1, 1, . . . , −1). For every association scheme, there are real numbers eij (0
i, j
s) such that every vector in Wi is an eigenvector of Aj with eigenvalue eij . The array [eij ] is sometimes called the character table of the association scheme. Table 4 shows the character tables of some association schemes. Denote by Si the matrix representing orthogonal projection onto Wi . The eigenprojections are symmetric, idempotent, and mutually orthogonal; i.e., Si = Si , S2i = Si , and Si Sj = 0 if i
= j. Moreover, i Si = I. For every j, Aj =



eij Si .

i

The (s + 1) × (s + 1) matrix [eij ] is invertible, with inverse [fij ], say. Then Si =



fji Aj

j

for each i. (See refs. 6, 9, and 16 for more details.) PARTIALLY BALANCED INCOMPLETE BLOCK DESIGNS Let  be an incomplete block design∗ for t treatments, each replicated r times, in b blocks each of size k. Suppose that no treatment occurs more than once in any block: such a design is said to be binary. For

6

PARTIALLY BALANCED DESIGNS

Table 4.

a

Association Scheme B(t)

GD(m, n)

B(m) × B(n)

C (5)

a The

Eigenspace Wi

Dimension of Wi

W0 W1 = contrasts

1 t−1

W0 W1 = between groups W2 = within groups

1 m−1 m(n − 1)

W0 W1 = between rows W2 = between columns W3

1 m−1 n−1 (m − 1)(n − 1)

W0 W1 W2

1 2 2

Associate Classes Aj A0 1 1 A0 1 1 1 A00 1 1 1 1 A0 1 1 1

A1 t−1 −1 A1 m−1 m−1 −1 A01 m−1 m−1 −1 −1 A1 2 0.618 −1.618

A2 m(n − 1) −m 0 A10 n−1 −1 n−1 −1 A2 2 −1.618 0.618

A11 (m − 1)(n − 1) −(m − 1) −(n − 1) 1

entry in row Wi and column Aj is the character eij .

treatments u and υ, denote by λuυ the number of blocks in which u and υ both occur, that is, the concurrence of u and υ. The design  is said to be partially balanced with respect to an association scheme C if λuυ depends only on the associate class of C that contains {u, υ}. Usually we abbreviate this and simply say that  is a partially balanced (incomplete block) design, or PBIBD. A design may be partially balanced with respect to more than one association scheme; usually, the simplest possible association scheme is assumed. The balanced incomplete block design∗ (BIBD) is just a special case of a PBIBD, for a BIBD with t treatments is partially balanced with respect to the association scheme B(t). A less trivial example is shown in Table 5. Here t = 12, r = 3, b = 4, and k = 9; treatments are denoted by capital letters and blocks are columns. This design is partially balanced with respect to GD (4, 3): the ‘‘groups’’ of treatments are {A, B, C}, {D, E, F}, {G, H, I}, and {J, K, L}. Partially balanced designs often inherit the name of the appropriate association scheme. Thus a design that is partially balanced with respect to a group-divisible association scheme is called a group-divisible design∗ . Partial balance with respect to a factorial association scheme is known as factorial balance. Although PBIBDs were

Table 5. A B C D E F G H I

A B C D E F J K L

A B C G H I J K L

D E F G H I J K L

introduced only in 1939, the importance of factorially balanced designs had been recognized previously by Yates [56]. Construction and Catalogs There are two elementary methods of constructing a PBIBD for a given association scheme. Each method has a block B (u, i) for each treatment u and a fixed i with 1
i
s. In the first construction B (u, i) consists of all ith associates of u; in the second, u is also in B (u, i). If 1 is partially balanced with respect to an association scheme C for t1 treatments, we may obtain a new design  for t1 t2 treatments by replacing each treatment of 1 by t2 new treatments. Then  is partially balanced with respect to C/B(t2 ). In particular, if 1 is a BIBD, then  is group divisible. The design in Table 5 was constructed by this method with t1 = 4 and t2 = 3. For further constructions of

PARTIALLY BALANCED DESIGNS Table 6. 123

124

125

134

135

A B E

A C F

A D G

B C H

B D I

145

234

235

245

345

C D J

E F H

E G I

F G J

H I J

group divisible designs, see GROUP-DIVISIBLE DESIGNS. The lattice designs∗ for n2 treatments are partially balanced with respect to Latin square∗ association schemes Lr (n) for various r: for a simple lattice, r = 2; for a triple lattice, r = 3. The scheme L2 (n) is also the two-dimensional Hamming scheme H (2, n). Designs partially balanced with respect to H (d, n) for d = 3, 4, 5 are sometimes called cubic, quartic, or quintic, respectively. A simple construction of such a PBIBD gives a block B (i, j) for 1
i
d and 1
j
n: the treatments in B (i, j) are all those d-tuples whose ith coordinate is equal to j. The method of triads gives a PBIBD for the triangular association scheme T(n). There   n are blocks of size 3, one for each subset 3 of size 3 of the original set of size n. The block corresponding to the set {α, β, γ } contains the treatments corresponding to the pairs {α, β}, {α, γ }, and {β, γ }. The triad design for T (5) is shown in  Table  6. A PBIBD with   m n blocks of size may be constructed 2 m in a similar manner. A simpler PBIBD for T(n) has n blocks of size n − 1: the blocks are the columns of the square array that defines the association scheme. Most factorial designs in common use have factorial balance. Some constructions may be found in CONFOUNDING and FACTORIAL EXPERIMENTS. Cyclic Designs∗ . These designs are partially balanced with respect to cyclic association schemes. An initial block B0 is chosen. A second block B1 is generated by adding 1 (modulo t) to every element of B0 . Further blocks B2 , B3 , . . . are generated from B1 , B2 , . . . in the same way. The process stops either with Bt−1 or with Bu−1 , where Bu is the

7

first block identical to B0 . The cyclic design for five treatments in blocks of size 3 with initial block {1, 2, 4} is shown in Table 7. It is also possible to have more than one initial block. Essentially the same method gives PBIBDs with association scheme based on any Abelian group. Although there are many fascinating methods of constructing PBIBDs, the practitioner who needs a single PBIBD should consult a catalog. Clatworthy [14] gives an extensive catalog of PBIBDs with two associate classes. John et al. [26] list cyclic designs with high efficiency factors (see below). Randomization The randomization∗ of a PBIBD is in two parts. First, the blocks of the design should be allocated randomly to the actual blocks. Second, in each block independently the treatments prescribed for that block should be randomly allocated to the plots. (If the design is resolvable∗ , and the replicates are intended to match features of the experimental material or its management, the blocks should be randomized only within replicates.) There is no need for any randomization of treatments to the letters or numbers used in the design. Any such randomization would be positively harmful if the association scheme has been chosen with reference to the specific treatments, as, e.g., in a factorial design. If a series of similar experiments is to be conducted at different sites, any randomization of treatments at individual sites will hamper, or even prevent, the subsequent analysis across all sites. The first two stages of randomization should, of course, be done, independently, at all sites. Efficiency Factors The standard analysis of an incomplete block design∗ uses a generalized inverse∗ of the t × t matrix
= I − r−1 k−1 , where  is the matrix whose diagonal elements are equal to r, and whose (u, υ)
element is equal to λuυ if u
= υ. Thus
= i θi Ai , where θ0 = 1 − k−1 Table 7. 1 2 4

2 3 0

3 4 1

4 0 2

0 1 3

8

PARTIALLY BALANCED DESIGNS

and θi = −λi /rk otherwise, where λi is the common value of the concurrence of pairs of treatments which are ith
associates. Hence

= j Ej Sj , where Ej = i eji θi and the eji are the characters defined in the subsection ‘‘Algebra’’ of the section ‘‘Association Schemes’’. A generalized inverse  of
is
−1 E S , the sum being restricted to those j j j j for which Ej
= 0. Moreover,
 has the same ‘‘pattern’’ as
, for  = i φi Ai , where
φi = j fij E−1 j . Denote by τu the effect of treatment
u. If x is any contrast vector then x · τ = u xu τ , is a linear combination of the treatment effects, and the variance of the intrablock∗ estimate of x · τ is xx σ 2 /r, where σ 2 is the intrablock variance (see INCOMPLETE BLOCK DESIGNS). Thus, if x is a vector in Wi , then the variance of x · τ is (x · x)σ 2 /(rEi ) and hence the efficiency factor (see BLOCKS, BALANCED INCOMPLETE) for x · τ is Ei . The Ei are called the canonical efficiency factors of the design. They lie between 0 and 1, and E0 is zero. If no other canonical efficiency factor is zero the design is said to be connected: in this case, there is an intrablock estimate of x · τ for every contrast x. The efficiency factor for such an x · τ may be calculated in terms of the Ei , and it lies between the extreme values of the Ei with i
= 0. In particular, the efficiency factor for τu − τυ is 1/(φ0 − φi ) if u and υ are ith associates. For example, consider the design given at the top of Table 8. It is a groupdivisible design with t = b = 4, r = k = 2, λ1 = 0, λ2 = 1. The groups are {A, C} and {B, D}. The contrast eigenspaces are the between-groups eigenspace W1 , with basis x(1) = (1, −1, 1, −1), and the within-groups eigenspace W2 , with basis consisting of x(2) = (1, 0, −1, 0) and x(3) = (0, 1, 0, −1). The canonical efficiency factors are given by     Ei = ei0 1 − 12 − ei2 14 . The character table [eij ] is the second part of Table 4. Thus ⎡

⎤ 1 1 2 [eij ] = ⎣1 1 −2⎦ 1 −1 0

and we have E0 = 1 ×

1 2

−2×

1 4

= 0,

E1 = 1 ×

1 2

+2×

1 4

= 1,

E2 = 1 ×

1 2

+0×

1 4

= 12 .

Moreover, [fij ] = [eij ]−1 = ⎡ ⎤ 1 1 2 1⎣ 1 1 −2⎦ 4 1 −1 0 and so 1 −1 5 φ0 = 14 E−1 1 + 2 E2 = 4 , 1 −1 3 φ1 = 14 E−1 1 − 2 E2 = − 4 , 1 φ2 = − 14 E−1 1 = −4.

Thus we have ⎡
=

1 4

=

1 4

2 ⎢−1 ⎢ ⎣ 0 −1 ⎡ 5 ⎢−1 ⎢ ⎣−3 −1

⎤ −1 0 −1 2 −1 0⎥ ⎥, −1 2 −1⎦ 0 −1 2 ⎤ −1 −3 −1 5 −1 −3⎥ ⎥, −1 5 −1⎦ −3 −1 5

and direct calculation shows that 
=
 = I − 14 J, so that  is a generalized inverse for
. Thus the variance–covariance matrix for the intrablock estimates of treatment effects is σ 2 /r. Furthermore, x(1) = x(1) , and so the variance of the intrablock estimate of x(1) · τ is x(1) · x(1) σ 2 /r, just as it would be in a complete block design with the same variance, so the efficiency factor for this contrast is 1. On the other hand, x(2) = 2x(2) and x(3) = 2x(3) so the variance of the intrablock estimates of the within-groups contrasts x(2) · τ and x(3) · τ is 2x(2) · x(2) σ 2 /r, which is twice the variance that would be achieved from a complete block design (assuming that the larger blocks could be found with the same intrablock variance), and so the efficiency factor for within-groups contrasts is 12 . In fact, x(2) is the contrast vector for τA − τC , and the efficiency factor 12 is equal to { 54 − (− 34 )}−1 = (φ0 − φ1 )−1 ,

PARTIALLY BALANCED DESIGNS

9

Table 8. Block Treatment Yield y

1 A 4

1 B 2

2 B 6

2 C −11

3 C −7

3 D 5

4 D 10

4 A 1

x(1) D x(1) B x(1) P

1 0 1

−1 0 −1

−1 0 −1

1 0 1

1 0 1

−1 0 −1

−1 0 −1

1 0 1

x(2) D x(2) B x(2) P

1

0

−1 − 12 − 12

−1 − 12 − 12

0 − 21

1

1 2 − 12

0 − 21

0

1 2 1 2

1 2

1 2 − 12

1 2 1 2

x(3) D x(3) B x(3) P

0

1

1

0

1 2 − 12

1 2 1 2

1 2 1 2

1 2 − 12

0 − 12

−1 − 21 − 21

−1 − 12 − 12

0 − 21

1 2

in agreement with theory, since A and C are first associates. In the same way, if z = (1, −1, 0, 0), then the efficiency factor for τA − τB is

1 2

1 2

other contrasts are obtained as linear combinations of these. The contribution of x to the intrablock sum of squares is (xP · y)2 /(xP · xP ) = (xP · y)2 /(rEi (x · x)).



z · z/zz = 2/3 = 1/{ 54 − (− 41 )} = 1/(φ0 − φ2 ), and A and B are second associates. Analysis The algebraic structure of a PBIBD gives an alternative method of performing the calculations required to estimate means and variances that does not explicitly use matrix inverses. Choose an orthogonal basis for each eigenspace Wi . If x is a basis vector for Wi , calculate the following rt-vectors: xD , which has entry xu on each plot that receives treatment u; xB , in which each entry of xD is replaced by the mean of the entries in the same block; xP = xD − xB . Let y be the rt-vector of yields. The intrablock estimate of x · τ is (xP · y)/(rEi ). Such estimates are found for each of the chosen basis vectors x. Intrablock estimates of any

The residual intrablock sum of squares is obtained by subtraction. Interblock∗ estimates and sums of squares are obtained similarly, using xB in place of xP and 1 − Ei in place of Ei . Sometimes it is desirable to combine the inter- and intrablock information. (See Brown and Cohen [7], Nelder [35], Sprott [55], and Yates [57,58].) We demonstrate the calculations on some fictitious data on eight plots using the design given at the top of Table 8. Although this design is so small that it is unlikely to be of much practical use and the calculations are not difficult to perform by other means, it serves to demonstrate the method, which is no more difficult for larger designs. We may use the basis vectors x(1) , x(2) , x(3) given in the preceding section, where we calculated the canonical efficiency factors to be E0 = 0, E1 = 1, E2 = 12 . The calculations of effects and sums of squares are shown in Table 9, and the analysis of variance∗ in Table 10. Pros and Cons of Partially Balanced Designs The advantage of partially balanced designs is the great simplification that may be achieved in the calculations needed to compare potential designs for an experiment and to analyze the results of the experiment. In

10

PARTIALLY BALANCED DESIGNS Table 9.

xP · y rEi Intrablock estimate Intrablock sum of squares xB · y r(1 − Ei ) Interblock estimate Interblock sum of squares

x(1)

x(2)

−36 2 −18 162 0 0 — 0

11 1 11 60.5 12 1 12 72

x(3) −3 1 −3 4.5 −4 1 −4 8

Table 10. Stratum Blocks

Plots

Source

d.f.

SS

W2 Residual Total

2

80

W1 W2 Residual Total

1 2

162 65

1 4

32 259

complete block designs are usually assessed by the values of their canonical efficiency factors, all of which should be as large as possible. Some contrasts are more important to the experimenter than others, so it is relatively more important for the efficiency factors for these contrasts to be large. Many catalogs of designs give a single summary value for the canonical efficiency factors, such as their harmonic mean∗ , geometric mean∗ , minimum, or maximum. Like the canonical efficiency factors themselves, these summaries can in general be calculated only by diagonalizing a t × t matrix for each design. However, for any given association scheme, the eigenspaces Wi , eigenprojections Si , and characters eij may be calculated by diagonalizing a few s × s matrices, and the canonical efficiency factors for every PBIBD for this association scheme may be calculated directly from the characters and the concurrences, with no further matrix calculations. Thus it is extremely easy to compare the canonical efficiency factors of two of these PBIBDs, and the comparison is particularly relevant because the efficiency factors apply to the same eigenspaces.

1 3

0.5 80.5

The story is the same for the analysis of PBIBD experiments. In general, each incomplete block design requires the inversion of a t × t matrix for the analysis of experiments with that design. We have shown that PBIBD experiments may be analyzed without using this inverse. Even if the standard method of analysis is preferred, the inverse can be calculated directly from the canonical efficiency factors and the matrix [fij ], which is obtained by inverting the (s + 1) × (s + 1) matrix [eij ]. Moreover, this single matrix inversion is all that is needed for all PBIBDs with a given association scheme. In the days before electronic computers, these advantages of PBIBDs were overwhelming. Even today, routine use of PBIBDs permits considerable saving of time and effort and more meaningful comparison of different designs than is generally possible. When the association scheme makes sense in terms of the treatment structure, as with factorial designs, the case for using PBIBDs is still very strong. If the treatment structure does not, or cannot, bear any sensible relationship to an association scheme, the efficiency factors of most interest to the experimenter will not be the canoncial efficiency factors of any PBIBD,

PARTIALLY BALANCED DESIGNS

so the ease of the preceding calculations does not seem very helpful (see Pearce [42]). Moreover, for an arbitrary treatment structure, the best designs (according to some efficiency criterion) may not be PBIBDs. Thus other designs should certainly be examined if the necessary computing facilities are available. However, PBIBDs still have a role, and if one is found with a very small range of nonzero canonical efficiency factors, then the efficiency factors that interest the experimenter will also be in that range, and the design is probably close to optimal according to many optimality criteria. GENERALIZATIONS OF PBIBDs AND RELATED IDEAS Generalizations of Association Schemes Various authors have tried to extend the definition of PBIBD by weakening some of the conditions (a)–(e) for an association scheme without losing too much of the algebra. Shah [52] proposed weakening condition (e) to (e)

For all i and j, there are numbers
such that Ai Aj + Aj Ai = k qkij Ak .

qkij

This condition is sufficient to ensure that every invertible matrix in A still has its inverse in A, and that every diagonalizable matrix in A has a generalized inverse in A, and so any one inverse can be calculated from s linear equations. Thus Shah’s designs retain the property that the variance of the estimate of the elementary contrast τu − τυ depends only on what type of associates u and υ are. However, the property of simultaneous eigenspaces Wi for every PBIBD with a given association scheme is lost, so the calculation of inverses must be done afresh for each design. Moreover, there is no easy method of calculating efficiency factors. As an example, Shah gave the ‘‘generalized association scheme’’ in Table 11a. (This is obtained in the same way as a cyclic association scheme, but using the non-Abelian group of order 6.) The same convention is used as in Table 2. Parts b and c of Table 11 show incomplete block designs that are ‘‘partially balanced’’ with

11

respect to this generalized association scheme. (Design b was given by Shah [52] and also discussed by Preece [43]. Design c is formed from its initial block by a construction akin to the cyclic construction.) The concurrence matrices of both designs have W0 and W1 as eigenspaces, where W1 is spanned by (1, 1, 1, −1, −1, −1). However, the remaining eigenspaces for design b are √ √ W2 , spanned by (2, −1, −1, − 3, 0, 3) √ √ and (0, − 3, 3, −1, 2, 1), and W3 , √ √ spanned by (2, −1, −1, 3, 0, − 3) and √ √ (0, 3, − 3, −1, 2, −1). The remaining eigenspaces for design c are V2 , spanned by (2, −1, −1, 1, −2, 1) and (0, 1, −1, 1, 0, −1), and V3 , spanned by (2, −1, −1, −1, 2, −1) and (0, 1, −1, −1, 0, 1). Nair [32] suggested retaining condition (e) while weakening condition (b) by allowing nonsymmetric association relations, with the proviso (b) If Ai is an association matrix, then so is Ai . Then it remains true that A is an algebra in the sense of containing the sum and product of any two of its elements, and the scalar multiples of any one of its elements; moreover, A still contains generalized inverses of its symmetric elements. Thus the Nair schemes have the good properties of the Shah schemes. However, for a Nair scheme, the algebra A may or may not be commutative in the sense that MN = NM for all M and N in A. All the examples given in Nair’s paper did have commutative algebras, and Nair wrongly concluded that this is always the case. A counterexample is provided by Table 11d, which is a Nair association scheme with respect to which the designs in Tables 11b and 11c are partially balanced. If A is commutative, then the Nair scheme has all the good properties of genuine association schemes. However, since concurrence and covariance are symmetric functions of their two arguments, one may as well replace Ai by Ai + Ai if Ai is nonsymmetric, and obtain a genuine association scheme, because the commutativity of A ensures that condition (e) will still hold. If A is not commutative then the Nair scheme has the same disadvantages as the general Shah schemes. In fact, most Shah schemes arise by taking a

12

PARTIALLY BALANCED DESIGNS

Table 11. (a) Association Scheme (Shah Type)

A B C D E F

A

B

C

D

E

F

0 1 1 2 3 4

1 0 1 4 2 3

1 1 0 3 4 2

2 4 3 0 1 1

3 2 4 1 0 1

4 3 2 1 1 0

C F

A E

B F

D F A

E D B

F E C

(b) Shah’s Design A D

A D

B E

B E

C F

C D

(c) Other Design A B D

B C E

C A F

(d) Association Scheme (Nair Type)

A B C D E F

A

B

C

D

E

F

0 5 1 2 3 4

1 0 5 4 2 3

5 1 0 3 4 2

2 4 3 0 1 5

3 2 4 5 0 1

4 3 2 1 5 0

Nair scheme and fusing nonsymmetric associate classes, as in the preceding example. Another possibility is to allow selfassociates to form more than one class: thus condition (c) becomes (c) The nonzero elements of Ai are either all on, or all off, the diagonal. This was first suggested implicitly in the statistical literature when Pearce [38] introduced supplemented balance as the judicious addition of a control treatment to a BIBD. There are two classes of self-associates: the control, and all other treatments. There are also two other associate classes: control with new treatment, and new treatment with another new treatment. More general designs satisfying (c) rather than (c) were given by Rao [48] and Nigam [36]. For these designs A cannot be commutative, because the association matrices for self-associates do not

commute with J, and so the useful property of common eigenspaces is lost. If the association relations must be symmetric, then condition (e) is satisfied while (e) is not, whereas if the pairs (control, new) and (new, control) are allocated to different classes then (b) and (e) are satisfied. In either case, A contains a generalized inverse of each of its symmetric elements, and so the pattern of variances of elementary contrasts matches the pattern of concurrences. Bearing in mind the importance of condition (e) for properties of A, Higman [18] defined a generalization of association schemes called coherent configurations. These satisfy conditions (a), (b) , (c) , (d), and (e). If they also satisfy (c), they are called homogeneous. Homogeneous coherent configurations arise naturally in the context of transitive permutation groups, and so they are intimately related to recent ideas of

PARTIALLY BALANCED DESIGNS

13

Sinha [53], who defined an incomplete block design to be simple if there is a group G of permutations of the treatments with the properties that (i) if the rows and columns of the concurrence matrix  are both permuted by the same element of G, then the matrix is unchanged as a whole; (ii) given any two treatments, there is a permutation in G that carries one into the other. Such designs are indeed simple in the sense that once their structure has been analyzed with respect to a single treatment, everything is known about their structure. This fact is often used implicitly by other authors, and the idea of its simplicity seems worth pursuing. As a whole, coherent configurations are of considerable mathematical interest and have many potential applications to the design of statistical experiments, but they cannot be discussed in any further detail here. Approximations to Balance∗ in Terms of Concurrence In a review paper on types of ‘‘balance’’ in experimental design, Pearce [39] defined an incomplete block design to be partially balanced if there are real numbers θ0 , θ1 , . . . , θs , n1 , . . . , ns such that every diagonal element of the matrix
is equal to θ0 and every row of
contains exactly ni elements equal to θi . Thus he effectively removed condition (e) (and even (e) ) from the definition of partial balance. Although Pearce [42] has argued strongly that condition (e) is entirely abstract and artificial and has no place in practical experimentation, without condition (e) all the algebraic theory collapses, and general designs that are partially balanced in Pearce’s sense have none of the good properties we have described. It is unfortunate that Pearce retained the term partial balance, because his paper was influential and caused other authors to use the term in his way, which may account for a certain amount of confusion about the meaning of the term. Sinha [53] called this property the extended Latin Square property (ELSP) and attempted to relate ELSP, partial balance, and simplicity. Unfortunately,

Figure 2. The regular graph for the design in Table 12.

some of the specific results in his paper are incorrect. Jarrett [24] used the term sconcurrence design for Pearce’s PBIBD and clarified the situation somewhat. However, he insisted that θi
= θj unless i = j. Although s-concurrence designs have an obvious attraction, in general,
−1 does not have the same pattern as
, and little can be predicted about the efficiency factors. For a 2-concurrence design, the matrices A1 and A2 define a regular graph and its complement. If balance is impossible, the nearest approximation seems, intuitively, to be a 2concurrence design whose concurences differ by 1. Such designs were called regular graph designs∗ (RGDs) by John and Mitchell [25]. An example is given in Table 12, with one of its corresponding regular graphs in Fig. 2. (Thus a regular graph design is partially balanced if and only if its graph is strongly regular.) Intuition is not quite correct, however, and some RGDs are poor in terms of efficiency factors. Mitchell and John [31] gave a catalog of RGDs that are E-optimal in the sense of having their smallest efficiency factor as large as possible, and they conjectured [25] that, for each fixed value of t, b, and k, if there exists any RGD then there exist RGDs that are A-optimal, D-optimal, and E-optimal respectively. Cheng [10–12] and Jacroux [21] have shown that this is true in several special cases; the optimal designs are often simultaneously RGDs and PBIBDs.

14

PARTIALLY BALANCED DESIGNS Table 12. E A F B G E H F I H

B C F G E

C D G H F

Cheng and Wu [13] allowed for the case where the number of plots is not divisible by t and defined a nearly balanced incomplete block design (NBIBD) to have treatment replications differing by at most 1, and, for all treatments u, v, and w, concurrences λuv and λuw differing by at most 1. This concept is so far (algebraically) from partial balance that there seems little hope of any good general theory about NBIBDs. Nevertheless, Cheng and Wu were able to show that the best NBIBDs do have high efficiency factors (see also NEARLY BALANCED DESIGNS). Approximations to Balance in Terms of Variance or Efficiency In general, if one allows the incomplete block design  to have unequal replication, unequal block sizes, or more than one occurrence of a treatment per block, then the information matrix r
must be replaced by the matrix L = R2 − NK−1 N , where R is the t × t diag√ onal matrix whose (u, u) entry is ru if treatment u is replicated ru times; K is the b × b diagonal matrix whose (c, c) entry is the size of block c; and N is the t × b matrix whose (u, c) entry is the number of times that treatment u occurs in block c. Various authors have extended the definition of partial balance to cover designs that have one or more of the following conditions relaxed: equireplicate, equal block sizes, binary. Unequal block sizes make very little difference to the theory discussed so far; while for nonbinary designs equal replication is no longer equivalent to L having equal diagonal elements. Mehta et al. [30] defined  to be partially balanced if it has equal replication, equal block sizes, and L is in the algebra A of an association scheme; while Kageyama [29] relaxed the complementary conditions and defined  to be partially balanced if it is binary and L is in the algebra A of an association scheme. Note that in both cases the diagonal elements of L are all the same.

D A H E G

A B C G I

B C D H I

C D A E I

D A B F I

The supplemented balance designs described earlier are similar in spirit, except that L belongs to the algebra A of an inhomogeneous coherent configuration, whose classes of selfassociates consist of all treatments with the same replication. For all these extended definitions, L has a generalized inverse M in A. Since the variance of the estimate x · τ is xMx , the variance of the elementary contrast τu − τv depends only on the associate class containing the pair (u, v). Thus it would be correct to call these designs partially variancebalanced. Nigam’s [36] designs have only two variances: he called his designs nearly balanced if the two variances are within 10% of each other, but this is not related to NBIBDs. However, if treatments are not equally replicated then treatment contrasts would not have equal variances even in an unblocked design, and it may be unreasonable to expect them to have equal variances in the incomplete block design. The efficiency factor takes account of the unequal replication by comparing the variance in the blocked and unblocked designs, and so it is a more useful measure of information loss than variance. James and Wilkinson [22] showed that the efficiency factors may be obtained from the matrix R−1 LR−1 . This matrix is symmetric, so it has a complete eigenvector basis, and its eigenvalues are the canonical efficiency factors: if there are m nonzero eigenvalues, then James and Wilkinson define  to have mth order balance. As Houtman and Speed [20] pointed out, this is just a special case of the general balance∗ introduced by Nelder [34]. Following a similar, but independent, line ´ of thought, Calinski [8] showed that the canonical efficiency factors are the eigenvalues of R−2 L. Since R−2 L and R−1 LR−1 are similar (R−2 L = R−1 (R−1 LR−1 )R) they have the same eigenvalues, and the eigenspaces of one can be obtained easily from those of the other. In particular, R−2 L also has a complete

PARTIALLY BALANCED DESIGNS

´ eigenvector basis. However, Calinski did not realize this, and Puri and Nigam [44] defined  to be partially efficiency balanced if R−2 L has a complete eigenvector basis. Since all incomplete block designs are partially efficiency balanced, this definition seems unnecessary. A stream of papers followed the ´ Calinski/Puri/Nigam school (e.g., Puri and Nigam [46]) and another stream followed the Nelder/James/Wilkinson school (e.g., Jarrett [23]); there has been little cross-reference between the two. However, Pal [37] has recently pointed out that all designs are partially efficiency balanced, and it is to be hoped that these separate strands will be unified and simplified. More Complicated Block Structures The definition of partial balance can be extended to more complicated block structures obtained from several block factors by crossing and nesting∗ (e.g., the simple orthogonal block structures of Nelder [33]). The treatments must form a PBIBD with each system of blocks separately, ignoring all the other block systems, and the association scheme must be the same throughout. The results described in this article extend to these structures with no difficulty. See Houtman and Speed [20] for details. It is also possible to use partially balanced designs when time is a block factor, or there is some other definite order on the experimental units. Blaisdell and Raghavarao [2] have used partial balance in changeover∗ designs. Literature There is an enormous statistical and mathematical literature on association schemes and partial balance, and it is impossible to cite here all who have made important contributions. The statistical reader with some mathematical background will find comprehensive accounts of partial balance in the textbooks by John [27,28] and Raghavarao [47]; the references in Bose [3], Clatworthy [14], Pearce [42], and Preece [43] are also useful sources of further reading. A more detailed account of the practical role of the eigenspaces in some particular cases is in Corsten [15]. A full account of the

15

role of the eigenspaces is given by Houtman and Speed [20], James [22], and Nelder [34], while papers such as refs. 40 and 41 concentrate on just the eigenvalues (i.e., the canonical efficiency factors). References 44 and 46 explain the eigenspace analysis more specifically in the context presented in this article. For a mathematical treatment of association schemes and strongly regular graphs, see refs. 1, 9, 16, 17, 50, and 51. REFERENCES 1. Biggs, N. L. and White, A. T. (1979). ‘‘Permutation Groups and Combinatorial Structures.’’ Lond. Math. Soc. Lect. Notes Ser., 33, Cambridge University Press, Cambridge, England. (Strictly for mathematicians; includes a section on strongly regular graphs.) 2. Blaisdell, E. A. and Raghavarao, D. (1980). J. R. Statist. Soc. B, 42, 334–338. (PBIBDs as changeover designs.) 3. Bose, R. C. (1963). Sankhya, ¯ 25, 109–136. (A review article on partial balance.) 4. Bose, R. C. (1963). Pacific J. Math., 13, 389–419. (Introduction of strongly regular graphs.) 5. Bose, R. C. and Nair, K. R. (1939). Sankhya, ¯ 4, 337–372. (Introduction of partial balance.) 6. Bose, R. C. and Mesner, D. M. (1959). Ann. Math. Statist., 30, 21–38. (The major step in the algebra of partial balance.) 7. Brown, L. D. and Cohen, A. (1974). Ann. Statist., 2, 963–976. ´ 8. Calinski, T. (1971). Biometrics, 27, 275–292. 9. Cameron, P. J. and van Lint, J. H. (1980). ‘‘Graphs, Codes and Designs.’’ Lond. Math. Soc. Lect. Notes Ser., 43, Cambridge University Press, Cambridge. (Strictly for mathematicians; includes very clear sections on strongly regular graphs and on association schemes.) 10. Cheng, C. -S. (1978). Commun. Statist., A7, 1327–1339. 11. Cheng, C. -S. (1980). J. R. Statist. Soc. B, 42, 199–204. 12. Cheng, C. -S. (1981). Ann. Inst. Statist. Math. Tokyo, 33, 155–164. 13. Cheng, C. -S. and Wu, C. -F. (1981). Biometrika, 68, 493–500. 14. Clatworthy, W. H. (1973). ‘‘Tables of TwoAssociate Class Partially Balanced Designs.’’ Natl. Bur. Stand. (U.S.) Appl. Math. Ser. 63

16

PARTIALLY BALANCED DESIGNS (Washington, DC). (A large catalog; very useful.)

30. Mehta, S. K., Agarwal, S. K., and Nigam, A. K. (1975). Sankhya, ¯ 37, 211–219.

15. Corsten, L. C. A. (1976). In Essays in Probability and Statistics, S. Ikeda, ed. Shinko Tsusho, Tokyo, 125–154.

31. Mitchell, T. and John, J. A. (1976). ‘‘Optimal Incomplete Block Designs.’’ Oak Ridge Nat. Lab. Rep. No. ORNL/CSD-8. Oak Ridge, TN. (A catalog of regular graph designs.)

16. Delsarte, P. (1973). ‘‘An Algebratic Approach to the Association Schemes of Coding Theory.’’ Thesis, Universit´e Catholique de Louvain (appeared as Philips Research Reports Supplement, 1973, No. 10). (For mathematicians, a splendid account of association schemes and their relation to other mathematical objects.) 17. Dembowski, P. (1968). Finite Geometries. Springer-Verlag, Berlin. (Strictly for mathematicians; includes an appendix on association schemes and partial designs.) 18. Higman, D. G. (1975). Geom. Dedicata, 4, 1–32. 19. Hinkelmann, K. (1964). Ann. Math. Statist., 35, 681–695. 20. Houtman, A. M. and Speed, T. P. (1983). Ann. Statist., 11, 1069–1085. (Contains the natural extension of PBIBDs to more complicated block structures; also a useful bibliography on the use of efficiency factors and eigenspaces in analysis.) 21. Jacroux, M. (1980). J. R. Statist. Soc. B, 42, 205–209. 22. James, A. T. and Wilkinson, G. N. (1971). Biometrika, 58, 279–294. (A lucid account of the use of efficiency factors and eigenspaces in analysis—for those who are familiar with higher-dimensional geometry and algebra.) 23. Jarrett, R. G. (1977). Biometrika, 64, 67–72. 24. Jarrett, R. G. (1983). J. R. Statist. Soc. B, 45, 1–10. (A clear separation of designs with equal concurrence patterns from PBIBDs.) 25. John, J. A. and Mitchell, T. (1977). J. R. Statist. Soc. B, 39, 39–43. (Introduction of regular graph designs.) 26. John, J. A., Wolock, F. W., and David, H. A. (1972). ‘‘Cyclic Designs.’’ Natl. Bur. Stand. (U.S.) Appl. Math. Ser. 62 (Washington DC). (A catalog of designs with high efficiency factors.) 27. John, P. W. M. (1971). Statistical Design and Analysis of Experiments. Macmillan, New York. (A textbook for statisticians, with a comprehensive section on PBIBDs.)

32. Nair, C. R. (1964). J. Amer. Statist. Ass., 59, 817–833. 33. Nelder, J. A. (1965). Proc. R. Soc. Lond. A, 283, 147–162. 34. Nelder, J. A. (1965). Proc. R. Soc. Lond. A, 283, 163–178. 35. Nelder, J. A. (1968). J. R. Statist. Soc. B, 30, 303–311. 36. Nigam, A. K. (1976). Sankhya, ¯ 38, 195–198. 37. Pal, S. (1980). Calcutta Statist. Ass. Bull., 29, 185–190. 38. Pearce, S. C. (1960). Biometrika, 47, 263–271. (Introduction of supplemented balance.) 39. Pearce, S. C. (1963). J. R. Statist. Soc. A, 126, 353–377. 40. Pearce, S. C. (1968). Biometrika, 55, 251–253. 41. Pearce, S. C. (1970). Biometrika, 57, 339–346. 42. Pearce, S. C. (1983). The Agricultural Field Experiment. Wiley, Chichester, UK. (This text, full of good advice on practical experimentation, discusses the place of various kinds of theoretical balance in the practical context.) 43. Preece, D. A. (1982). Utilitas Math., 21C, 85–186. (Survey of different meanings of ‘‘balance’’ in experimental design; extensive bibliography.) 44. Puri, P. D. and Nigam, A. K. (1977). Commun. Statist., A6, 753–771. 45. Puri, P. D. and Nigam, A. K. (1977). Commun. Statist., A6, 1171–1179. (Relates efficiency balance, pairwise balance, and variance balance.) 46. Puri, P. D. and Nigam, A. K. (1982). Commun. Statist. Theory Meth., 11, 2817–2830. 47. Raghavarao, D. (1971). Constructions and Combinatorial Problems in Design of Experiments. Wiley, New York. (This remains the best reference work on construction of incomplete block designs; it requires some mathematical background.) 48. Rao, M. B. (1966). J. Indian Statist. Ass., 4, 1–9.

Block

49. Roy, P. M. (1953). Science and Culture, 19, 210–211.

29. Kageyama, S. (1974). Hiroshima Math. J., 4, 527–618. (Includes a useful review of different types of association schemes.)

50. Seidel, J. J. (1979). In ‘‘Surveys in Combinatorics,’’ B. Bollobas, ed. Lond. Math Soc. Lect. Notes Ser., 38. Cambridge University Press,

28. John, P. W. M. (1980). Incomplete Designs. Marcel Dekker, New York.

PARTIALLY BALANCED DESIGNS Cambridge, England, pp. 157–180. (Survey of strongly regular graphs.) 51. Seidel, J. J. (1983). In Proceedings of the Symposium on Graph Theory, Prague, 1982 (to appear). 52. Shah, B. V. (1959). Ann. Math. Statist., 30, 1041–1050. 53. Sinha, B. K. (1982). J. Statist. Plan. Infer., 6, 165–172. 54. Speed, T. P. and Bailey, R. A. (1982). In Algebraic Structures and Applications, P. Schultz, C. E. Praeger, and R. P. Sullivan, eds. Marcel Dekker, New York, pp. 55–74. (Gives the eigenprojection matrices for a class of association schemes.) 55. Sprott, D. A. (1956). Ann. Math. Statist., 27, 633–641. 56. Yates, F. (1935). Suppl. J. R. Statist. Soc., 2, 181–247. 57. Yates, F. (1939). Ann. Eugen. (Lond.) 9, 136–156. 58. Yates, F. (1940). Ann. Eugen. (Lond.) 10, 317–325. See also ANALYSIS OF VARIANCE; BLOCKS, BALANCED INCOMPLETE; BLOCKS, RANDOMIZED COMPLETE; COMBINATORICS; CONFOUNDING; CYCLIC DESIGNS; DESIGN OF EXPERIMENTS; FACTORIAL EXPERIMENTS; GENERAL BALANCE; GRAPH THEORY; GRAECO-LATIN SQUARES; GROUP-DIVISIBLE DESIGNS; INCOMPLETE BLOCK DESIGNS; INTERBLOCK INFORMATION; INTERACTION; INTRABLOCK INFORMATION; LATIN SQUARES, LATIN CUBES, LATIN RECTANGLES; LATTICE DESIGNS; MAIN EFFECTS; NEARLY BALANCED DESIGNS; NESTING AND CROSSING IN DESIGN; OPTIMAL DESIGN OF EXPERIMENTS; RECOVERY OF INTERBLOCK INFORMATION; and REGULAR GRAPH DESIGNS.

ROSEMARY A. BAILEY

17

‘‘PATIENTS’’ VS. ‘‘SUBJECTS’’

The delineation of the differences between physicians and patients and researchers and subjects is in no way meant to be critical of researchers or their goals. Rather it is important for all parties to the research to understand properly their roles obligations, and expectations. For example, it is important for subjects to understand that being a research subject is not the same as being a patient, and that being in a clinical study is not the same as receiving standard therapy. It is just as important for investigators to recognize these differences. In the absence of this recognition, both researchers and subjects can suffer from what is called ‘‘the therapeutic misconception’’ in which both researchers and subjects mistakenly conflate research with therapy.

LEONARD H. GLANTZ Boston University School of Public Health, Boston, Massachusetts

1 CONFUSION BETWEEN ‘‘PATIENTS’’ AND ‘‘SUBJECTS’’ In discussions and reports of clinical trials, the terms ‘‘patients’’ and ‘‘subjects’’ are often used interchangeably. This conceptual error affects both physicians/researchers and patients/subjects. Patients go to physicians for the purpose of diagnosis and treatment. Both patients and physicians have identical goals—the cure or alleviation of the condition for which the patient sought the physician’s help. It is generally recognized that physicians have a ‘‘fiduciary obligation’’ to patients that requires them to act in their patients’ best interests. It is not the case with researchers. Subjects do not seek out researchers, but researchers seek out subjects. The primary goal of researchers is not to benefit subjects but to increase knowledge. Subjects are a means to this end rather than the end in themselves as is the case in the physician–patient relationship. Whether the subject actually benefits from the clinical trial is not of primary concern since the goal of increasing knowledge will be achieved regardless of the outcome for a single subject. Because of these considerations, it cannot be said that a ‘‘fiduciary relationship’’ exists between the subject and the researcher. It is these differences in these two very different relationships that provide the basis for why researchers with subjects are much more highly regulated than physicians with patients. Since the researcher and subject interests do not coincide, and the researchers are in a position of power, it is important that a system be in place to protect the rights and welfare of the human subjects. Thus, specific ethical codes, laws, and regulations have been implemented that provide special standards for conducting research on subjects that go beyond the standards for treating patients.

2 USE OF RANDOMIZATION AND PLACEBOS A concrete example of how subjects and patients differ is provided by examining the process of randomization and the use of placebos. Randomization involves placing subjects in different arms of a research project through the use of chance allocations. No professional judgment is made as to which arm the subject will be placed in. It is immediately obvious that this approach is quite different from the way medicine is practiced in which patients are not randomly assigned to treatments but are provided the treatment that the physician believes will best alleviate or cure or the patient’s condition. Similarly, placebos are not regularly used in the practice of medicine. When patients come for treatment, they are not randomly assigned to a treatment or no-treatment arm and then ‘‘fooled’’ into believing that they may have received effective treatment when they have not. Although the use of placebos is standard in randomized clinical trials and is a useful tool in establishing the safety and efficacy of drugs, its use is designed to increase knowledge, a researcher’s goal, not to provide treatment, a physician’s goal. Nevertheless, the use of placebos is not uncontroversial in the context of research. Although placebo controlled trials are not

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

‘‘PATIENTS’’ VS. ‘‘SUBJECTS’’

particularly controversial when it comes to relatively minor well-controlled medical conditions, ethical considerations prohibit the use of placebos in certain circumstances. For example, if a person has a bacterial infection that requires the use of antibiotics, and a new antibiotic was being tested, it would be unethical to withhold the effective treatment solely for the purpose of testing the new unknown treatment. In other words, although researchers are not the subjects’ physicians, they still have an obligation not to injure others, an obligation that extends to all individuals from which researchers are not exempt. 3 WHEN THE PHYSICIAN IS ALSO THE RESEARCHER One of the more difficult conceptual issues that develops in distinguishing between ‘‘patients’’ and ‘‘subjects,’’ and therefore between ‘‘physicians’’ and ‘‘researchers,’’ is when the researcher is also the subject’s physician. In such an instance, the problem of role confusion can become quite stark. It is important for both the physician and the patient to recognize when their roles shift to that of researcher and subject. Indeed, best practice may require that whenever possible physicians not become researchers who use their patients as subjects, but that another medical professional should play the role of researcher. This practice also assures that the research subjects always have available physicians who have their best interests in mind while they are going through the research process. In circumstances where this situation is not practical, it is essential for the physician and for the patient to be explicitly aware of the change in their relationship when the physician becomes the researcher. REFERENCES 1. R. C. Fox and J. P. Swazey, The Courage to Fail: A Social View or Organ Transplants and Dialysis. Chicago, IL: University of Chicago Press, 1974. 2. H. Jonas, Philosophical reflections on experimenting with human subjects. In: P. A.

Freund (ed.), Experimentation with Human Subjects. New York: George Braziller, 1969. 3. J. Katz, Human experimentation and human rights. Saint Louis University Law J. 1993; 38(1):7–55. 4. National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, The Belmont Report. 1978. DHEW Publication No. (OS) 78-0012. 5. T. Parsons, Research with human subjects and the ‘‘professional complex.’’ In: P. A. Freund (ed.), Experimentation with Human Subjects. New York: George Braziller, 1969, pp. 116–151. 6. P. Ramsey, The patient as person. New Haven, CT: Yale University Press, 1970.

PERMUTATION TESTS IN CLINICAL TRIALS

4. Within each group, the error terms all have the same variance. (On occasion it is also assumed that the variance is the same in the two groups, although this assumption can be avoided.)

DAVID M. ZUCKER Hebrew University of Jerusalem Department of Statistics

Obviously, the more complex the statistical analysis, the more assumptions are involved. In a thorough statistical analysis, efforts are made to check the validity of the assumptions and to examine the sensitivity of the results to departures from the assumptions. In routine analyses, however, these checks are often skipped. Moreover, and more critically, the checks are not foolproof, and even after the checks, some unquantifiable element of uncertainty remains. In an experiment in which treatments have been assigned to subjects by randomization, however, it is possible to apply a statistical method for testing treatment effect that does not require any statistical assumptions about the data beyond those inherently satisfied due to the randomization itself. This method is called a permutation test (or randomization test). The ability thus afforded to compute a P-value for testing treatment effect without relying on uncertain statistical assumptions is a major strength of the randomized trial.

1 RANDOMIZATION INFERENCE—INTRODUCTION Randomization—allocation of study treatments to subjects in a random fashion—is a fundamental pillar of the modern controlled clinical trial. Randomization bolsters the internal validity of a trial in three major respects: 1. It prevents possible investigator bias (which may otherwise exist even unintentionally) in the allocation of subjects to treatments. 2. It generates study groups that, on average, are balanced with respect to both known and unknown factors. 3. It provides a probability structure whereby the study results may be evaluated statistically without exogenous statistical modeling assumptions. The purpose of this article is to elaborate on the last of these three points. To begin, we note that many, probably most, statistical analyses in empirical research, especially in nonexperimental studies, involve some statistical modeling assumptions. For example, to take a simple case, the classical two-sample t-test for comparing two groups makes the following assumptions:

2 PERMUTATION TESTS—HOW THEY WORK The classic exposition of the randomized experiment in general, and the permutation test in particular, was given by R. A. Fisher (1). Rosenberger and Lachin (2) give an up-to-date comprehensive exposition. This article will describe how a permutation test works in the simple context of comparing two groups, a treatment group and a control group, but the idea applies more generally to multi-arm trials. The classical formulation of the permutation test is as a test of the null hypothesis that the treatment has no effect whatsoever on the subject’s response; that is, each subject would exhibit exactly the same response whether given the study treatment or the control regimen. This null hypothesis is referred to by some authors as the ‘‘strong’’

1. Each observation in each group is equal to the sum of a fixed group-specific population mean value plus a mean zero random error term. 2. All random error terms, both within and between groups, are statistically independent. 3. The random error terms have a normal distribution.

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

PERMUTATION TESTS IN CLINICAL TRIALS

null hypothesis. Some remarks will be made later in this article on an alternative form of null hypothesis and the performance of permutation tests in that context. For now, however, we will stick with the null hypothesis that each subject would exhibit exactly the same response irrespective of the regimen given. We will illustrate how the test works in the context of an example presented in Reference 3. We first must review the concept of a P-value in general. Given a statistic aimed at comparing the two groups—the difference between the means, for example—the P-value is defined as the probability that the statistic would be equal to or more extreme than the value actually observed if the null hypothesis were in fact true. A small P-value means that the observed value of the statistic is so extreme that it is unlikely to have arisen if the null hypothesis were true, and is thus an (indirect) indicator that the null hypothesis is false. It is conventional to say that there is ‘‘statistically significant’’ evidence of a treatment difference if the two-sided P-value (i.e., considering extremes in both directions) is less than 0.05. With this background, we may now turn to the example. Suppose eight subjects were randomly assigned to either treatment or control on an equal basis (four per group). By ‘‘randomly assigned,’’ we mean that the researchers chose the actual allocation at random from among the 70 possible ways of dividing eight subjects into two groups of four subjects each, with each possible allocation having an equal 1/70 chance of being employed. Suppose further that the final results with respect to some response variable were follows:

Table 1. Illustrative Experimental Data Subject

Group

A B C D E F G H

Control Control Control Control Treatment Treatment Treatment Treatment

Response 18 13 3 17 9 16 17 17

The groups are to be compared by examining the difference in sample means between the treatment and control groups. The observed sample means are 14.75 for treatment and 12.75 for control, so that the observed mean difference is 2.00. We wish to evaluate the statistical significance of this result. We argue as follows. As noted, there are 70 possible ways of dividing the eight subjects into two groups of four subjects each. Under the null hypothesis that the treatment has no effect whatsoever on the response, the responses of all eight subjects would be the same no matter what the allocation was. The only thing that differs from allocation to allocation is who received which regimen. By way of analogy, imagine a deck of eight cards, one corresponding to each subject, with the subject’s eventual end of study response written on the face of the card. At the beginning, all cards are face down, corresponding to the fact that at the beginning we do not know the response values. We shuffle the cards well, and then split the deck into two packs of four cards each, one pack corresponding to treatment and the other to control. We wait some period of time, during which the numbers on all cards remain the same, corresponding to the fact that the treatment has no effect. Finally, we are allowed to turn the cards face up and see the response values. We can then can compute the mean difference between the groups. We wish to determine the probability of obtaining a mean difference as extreme or more so than that observed through mere ‘‘luck of the draw’’ in splitting the deck into the two packs. Given the final observed response values—the numbers on each of the eight cards—we may enumerate easily all the possible realizations that could have eventuated in this experiment with the given subjects under the null hypothesis. The list of realizations consists simply of the list of the 70 possible ways of dividing the eight subjects into the two groups, with the response value associated with each subject being in every case the value observed in fact in the actual experiment—because, again, under the null hypothesis, the treatment does not affect the response at all. Each of these possible realizations has an equal chance of 1/70 of having arisen, because the allocation was chosen at

PERMUTATION TESTS IN CLINICAL TRIALS

3

Table 2. List of Possible Realizations of the Experiment Under the Null Hypothesis Case

Subjects On Treatment

Observations On Treatment

Observations On Control

Mean Treatment

Response Control

Mean Diff

EFGH DEFG DFGH .. . ABCD

9, 16, 17, 17 17, 9, 16, 17 17, 16, 17, 17 .. . 18, 13, 3, 17

18, 13, 3, 17 18, 13, 3, 17 18, 13, 3, 16 .. . 9, 16, 17, 17

14.75 14.75 16.75 .. . 12.75

12.75 12.75 10.75 .. . 14.75

2.00 2.00 6.00 .. . −2.00

1 2 3 .. . 70

Table 3. Null Hypothesis Distribution of the Mean Difference Mean Diff Probability

−7 1/70

−6.5 3/70

−6 1/70

−5 3/70

−4.5 4/70

−4 3/70

Mean Diff Probability

−3 3/70

−2.5 4/70

−2 3/70

−1 3/70

−0.5 4/70

0 6/70

Mean Diff Probability

0.5 4/70

1 3/70

2 3/70

2.5 4/70

3 3/70

4 3/70

Mean Diff Probability

4.5 4/70

5 3/70

6 1/70

6.5 3/70

7 1/70

random. For each possible realization (case) listed, we may compute the mean difference between treatment and control that arises under that realization. We obtain a list of the form shown in Table 2. Keeping in mind that each of these cases has a 1/70 chance of having occurred, we may obtain from this list the null hypothesis probability distribution— called the ‘‘permutation distribution’’— for the mean difference. Table 3 displays this null distribution: the table provides a list of the various possible values of the mean difference, along with the probability of obtaining each value. The mean difference observed in the actual experiment was 2.00. We can compute the probability of observing a difference of this magnitude or greater under the null hypothesis, by pure chance, from the probability distribution in Table 3. For a two-sided test we work with the absolute value of the difference. The probability is found to be 50/70, which equals 0.71. Thus, the permutation test two-sided P-value for testing the null hypothesis in this experiment is P = 0.71. Note that in the calculation above, no assumptions whatsoever were made about the behavior of the response value. The calculation depended only on the following basic facts.

1. The treatment allocation followed was chosen at random, on an equally likely basis, from the 70 possible allocations of eight subjects to two groups of four subjects each. 2. The null hypothesis says that the treatment does not affect the response values in any way. 3. In light of Fact 2, we can—after the fact, given the observed data— construct a list of all possible realizations of the experiment with the given subjects under the null hypothesis, with each realization having the same 1/70 chance of having eventuated. 4. Given the list of realizations, we can, for each possible value of the observed mean difference, calculate the probability of observing that value with the given subjects under the null hypothesis. In the case of 2 × 2 table analysis with a dichotomous response variable, the permutation test is known as the Fisher exact test and is available in many standard statistical routines such as SAS PROC FREQ (SAS Institute Inc., Cary, NC). It is possible to adapt the permutation test method to more complex statistical models.

4

PERMUTATION TESTS IN CLINICAL TRIALS

See References 2 and 4 for a discussion on how this adaptation is carried out. In an experiment with a extremely small number of subjects, such as in the above example, the range of possible P-values that can arise is limited, and so the capacity to test the null hypothesis is limited. But with a moderate-to-large sample size, the range of possible P-values is typically large. Similarly, in an experiment with a very small number of subjects, it is possible to list all possible outcomes and compute the P-value in a direct fashion. But as the sample size increases, the calculation quickly becomes formidable. For example, in an experiment with 5 subjects per group, there are 252 possible allocations; with 6 subjects per group, there are 924 possible allocations; with 10 subjects per group, there are 184,756 possible allocations; and with 20 subjects per group, there are about 138 billion possible allocations. With modern computing power and special algorithms, however, it is possible to perform permutation test calculations easily for experiments with 20–30 subjects or more per group. A well-known software package for such calculations is the StatXact package (Cytel Inc., Cambridge, MA). The calculations can also be performed in SAS PROC NPAR1WAY (SAS Institute Inc., Cary, NC), using the SCORES=DATA and EXACT options. At a certain point, however, the calculations become unmanageable, and a large sample approximation is employed. Section (3) discusses this approximation. 3 NORMAL APPROXIMATION TO PERMUTATION TESTS Generally speaking, for a sample size sufficiently large, the permutation distribution of a statistic for testing the null hypothesis can be well approximated by a normal distribution. This result is a generalized version of the familiar classical central limit theorem of probability, which says that the distribution of the average of a large number of independent observations tends to a normal bell-curve distribution. The setup here is somewhat different from that of the classical theorem, but the nature of the final result is essentially the same. The normal limit result

for permutation tests was proven mathematically around 40–50 years ago (5–8). This limit theorem provides justification for the application of classical statistical tests, such as the two-sample t-test described in Section 1, to randomized experiments without relying on the classical assumptions underlying the tests. The classical test simply serves as an approximation—with theoretical backing—to the exact permutation test. Essentially, when the sample size is large, the two tests are approximately equivalent (9). Often, as in the two-sample t-test, the t-distribution is used as an approximating distribution rather than the normal distribution, which improves the approximation to some extent by matching the first two moments (10–12). In practice, for a continuous response variable such as blood pressure, with 30–40 subjects per group the normal-theory test is quite adequate unless the distribution of the response variable is very peculiar. For 2 × 2 table analysis for a dichotomous response variable, the standard recommendation is to use the normal-theory chi-square test (the continuity correction usually improves the approximation) if the expected cell size, given the table margins, is five or more in all four cells, and otherwise to fall back to the Fisher exact test. Thus, in general, it is common practice to use the normal-theory tests unless the study involves a small sample size or small number of events. Again, the normal-theory tests are justified as approximations to a permutation test. When the sample size or number of events is small, the proper course is to use an exact permutation test. 4 ANALYZE AS YOU RANDOMIZE For a permutation-based analysis (either an exact permutation test or a corresponding normal approximation) to be valid without added assumptions, as described above, the form of analysis must match the randomization scheme. This fact is known as the ‘‘analyze as you randomize’’ principle. Thus, if a matched-pairs design has been used, with randomization carried out within each pair, then the pairing must be accounted for in the

PERMUTATION TESTS IN CLINICAL TRIALS

analysis. Similarly, if stratified randomization has been carried out, then the analysis must account for the stratification. Failure to include matching or stratification factors in the analysis can lead to inaccurate P-values. The level of error tends to be small in trials with a continuous endpoint and a large sample size, but it can be more substantial in very small trials or dichotomous endpoint trials with a low event rate. Another case calling for emphasis on the need to ‘‘analyze as you randomize’’ is the cluster randomization design (or group randomization design). Here the unit of randomization is some aggregate of individuals. This design is common in community-based and school-based trials, which often involve aggregate-level interventions. In cluster randomization trials, the unit of analysis must be the cluster rather than the individual. This is necessary to preserve the validity of the permutation-based analysis, as indicated above, and also to take proper account of between-cluster variation and thereby avoid serious type I error inflation (13, 14). To provide statistically rigorous results, a cluster randomization trial must include an adequate number of clusters. Many cluster randomization studies involve a very small number of clusters per arm, such as two or four. In such a study, it is almost impossible for permutation-based analysis to yield a statistically significant result. A normaltheory procedure such as the t-test has no justification in this case. With a trial of this small size, the normality assumption cannot be effectively checked, and the central limit theorem argument presented in the preceding section to justify the use of a normal-theory analysis as an approximation to a permutation-based analysis does not apply. A trial of such a small size may be useful as a pilot study, but it cannot yield statistically definitive conclusions. By contrast, studies such as CATCH (15, 16) (96 schools) and COMMIT (17, 18) (11 matched pairs of communities) included enough units for meaningful statistical analysis. In COMMIT, because of the relatively small number of units, an exact permutation test was used rather than a normal-theory test. In References 16 and 19, methods are described that allow individual-level explanatory variables

5

to be accounted for while maintaining the cluster as the primary unit of analysis. An article by Donner and Klar in this encyclopedia provides further discussion on statistical analysis of group-randomization trials. 5 INTERPRETATION OF PERMUTATION ANALYSIS RESULTS When a permutation-based analysis yields a statistically significant result, this is evidence against the null hypothesis that the treatment has no effect whatsoever on any subject’s response. If the result is in the positive direction, the inference to be made is that there are at least some individuals for whom the treatment regimen is better than the control regimen. It cannot necessarily be inferred that treatment is better than control for all individuals or even that treatment is better than control on an ‘‘average’’ basis; the only truly definitive statement that can be made is that some individuals do better with treatment than with control. This conclusion is admittedly one of limited scope, but it is nonetheless meaningful and is achieved with a high degree of certainty, in that it does not rely on any outside assumptions. Note, however, that along with the conclusion that the treatment is better than control for some people comes the proviso that treatment might be the same or worse than control for other people, so that ‘‘on average’’ there might be no difference between the two regimens. Usually clinical trial investigators like to go further and try to infer that treatment is better than control on some ‘‘average’’ basis. It must first be understood how this objective can be formulated from a formal statistical standpoint. In a typical clinical trial, patients are recruited on a volunteer, ‘‘catch-as-catch can’’ basis, and the set of patients entering the trial does not represent a formal random statistical sample from any particular defined population. Accordingly, we must suppose the trial patients behave like a random sample from some hypothetical superpopulation. This supposition has some plausibility in some circumstances, but it must be realized that it is essentially a statistical modeling assumption.

6

PERMUTATION TESTS IN CLINICAL TRIALS

Given that we are willing to make the above-described supposition, in trials with large sample sizes, the classical normaltheory procedures generally will provide valid tests of the ‘‘population-level’’ null hypothesis that the superpopulation mean response is the same for treatment as for control, against the alternative that it is better on treatment (or worse, or different). There remains the question of what happens in trials with small sample sizes, and in particular how permutation tests perform in relation to the ‘‘population-level’’ null hypothesis. This question has been investigated in References (20–22). Overall, these investigations have shown that permutation tests generally can have inflated type I error relative to the ‘‘population-level’’ null hypothesis, but if the number of subjects (or units, in a cluster randomized trial) is the same in the two experimental arms, then the type I error level is typically close to the desired level. If the ‘‘population-level’’ null hypothesis has been rejected with results in the positive direction, the inference to be made is that treatment is superior to control on an ‘‘average’’ basis for individuals similar to those who participated in the trial. Generalization of the results to other types of individuals requires careful judgment. 6

SUMMARY

The permutation test is a method for analyzing randomized trials through which the null hypothesis that the treatment has no effect whatsoever on the response may be assessed statistically without statistical distribution assumptions beyond those arising from the randomization process itself. With a sufficiently large sample size, the permutation test can be approximated satisfactorily by a classical normal-theory test. This result provides clear justification for application of normal-theory tests in trials with moderateto-large sample sizes. In very small trials, it is preferable to perform an exact permutation test.

REFERENCES 1. R. A. Fisher, The Design of Experiments. Edinburgh: Oliver and Boyd, 1935. (8th Ed. New York: Hafner, 1966). 2. W. F. Rosenberger and J. M. Lachin, Randomization in Clinical Trials: Theory and Practice. New York: Wiley, 2002. 3. O. Kempthorne, The Design and Analysis of Experiments. New York: Wiley, 1952. 4. M. H. Gail, W. Y. Tan, and S. Piantadosi, Tests for no treatment effect in randomized clinical trials. Biometrika 1988; 75: 57–64. 5. J. Hajek, Some extensions of the WaldWolfowitz-Noether theorem. Ann. Math. Stat. 1961; 32: 506–523. 6. W. Hoeffding, A combinatorial central limit theorem. Ann. Math. Stat. 1951; 22: 558–566. 7. G. E. Noether, On a theorem of Wald and Wolfowitz. Ann. Math. Stat. 1949; 20: 455–458. 8. A. Wald and J. Wolfowitz, Statistical tests based on permutations. Ann. Math. Stat. 1944; 15: 357–372. 9. W. Hoeffding, The large-sample power of tests based on permutations of observations. Ann. Math. Stat. 1952; 23: 169–192. 10. E. J. G. Pitman, Significance tests which may be applied to samples from any populations. J. R. Stat. Soc. 1937; 4 (suppl.): 119–130. 11. E. J. G. Pitman, Significance tests which may be applied to samples from any populations. II. The correlation coefficient test. J. R. Stat. Soc. 1937; 4 (suppl.): 225–232. 12. E. J. G. Pitman, Significance tests which may be applied to samples from any populations. III. The analysis of variance test. The analysis of variance test. Biometrika, 1937; 29: 322–335. 13. G. V. Glass and J. C. Stanley, Statistical Methods in Education and Psychology. Englewood Cliffs, NJ: Prentice-Hall, 1970. 14. D. M. Zucker, An analysis of variance pitfall: the fixed effects analysis in a nested design. Educ. Psychol. Measure. 1990; 50: 731–738. 15. R. V. Luepker, C. L. Perry, S. M. McKinlay, P. R. Nader, G. S. Parcel, E. J. Stone, L. S. Webber, J. P. Elder, H. A. Feldman, C. C. Johnson, S. H. Kelder, and M. Wu, Outcomes of a field trial to improve children’s dietary patterns and physical activity: The Child and Adolescent Trial for Cardiovascular Health (CATCH). JAMA 1996; 275: 768–776.

PERMUTATION TESTS IN CLINICAL TRIALS 16. D. M. Zucker, E. Lakatos, L. S. Webber, D. M. Murray, S. M. McKinlay, H. A. Feldman, S. H. Kelder, and P. R. Nader, Statistical design of the Child and Adolescent Trial for Cardiovascular Health (CATCH). Controlled Clinical Trials 1995; 16: 96–118. 17. COMMIT Research Group, Community Intervention Trial for Smoking Cessation (COMMIT): I. Cohort results from a four-year community intervention. Amer. J. Public Health 1995; 85: 183–192. 18. M. H. Gail, D. P. Byar, T. F. Pechacek, and D. K. Corle, Aspects of the statistical design for the Community Health Trial for Smoking Cessation (COMMIT). Controlled Clinical Trials 1992; 13: 6–21.

7

19. D. M. Zucker, Cluster randomization. In: N. Geller (ed.), Contemporary Biostatistical Methods in Clinical Trials. New York: Marcel Dekker, 2003. 20. T. Braun and Z. Feng, Optimal permutation tests for the analysis of group randomized trials. J. Amer. Stat. Assoc. 2001; 96: 1424–1432. 21. M. H. Gail, S. D. Mark, R. J. Carroll, and S. B. Green, On design considerations and randomization-based inference for community intervention trials. Stat. Med. 1996; 15: 1069–1092. 22. J. P. Romano, On the behavior of randomization tests without a group invariance assumption. J. Amer. Stat. Assoc. 1990; 85: 686–692.

Pharmacoepidemiology, Overview Pharmacoepidemiology, the study of patterns of medication use in the population and their effects on disease, is a new field. The need for this area of research became evident in 1961, during the thalidomide catastrophe, when it was realized that drugs prescribed for therapeutic purposes could produce unexpected risks. The entry of thalidomide, a new hypnotic drug, on the market was accompanied by a sudden sharp increase in the frequency of rare birth defects, characterized by the partial or complete absence of limbs [29, 31]. Consequently, several countries either instituted agencies to regulate drugs or expanded the mandate of existing agencies [51]. These agencies were previously interested only in the demonstration of a drug’s efficacy, but now required proof of a drug’s safety before it was tested in humans let alone before it was marketed for use by the general population (see Drug Approval and Regulation). These proofs of safety, based on toxicologic and pharmacologic studies, were necessary before randomized controlled trials (RCTs) (see Clinical Trials, Overview) could be conducted on human subjects, primarily to demonstrate the efficacy of a drug. The use of the epidemiologic approach to characterize population patterns of medication use and to assess their effects developed as a complement to RCTs for several reasons. First, RCTs were designed to assess the efficacy and effectiveness of a drug, providing as well some data on its safety with respect to commonly arising side-effects. However, rare sideeffects typically cannot be identified in clinical trials because of their small size. For example, to detect a relative risk of 2, for a side-effect having an incidence of 1 per 100, we would require a two-arm trial with over 3000 subjects per arm (α = 0.05, β = 0.1). If the incidence of the side-effect is 1 per 10 000, the sample size per arm would need to be over 300 000. Clearly, these sample sizes are rarely if ever used in RCTs, yet the number of people who will be using these drugs will be in the millions. Secondly, RCTs usually restrict the study subjects to people without coexisting disease, who are therefore not taking other medications that could interact with the study medication. They are also

restricted with respect to age, rarely including children and elderly subjects. Yet, the elderly will be major consumers of most of these medications, along with other medications they are using for coexisting diseases (see Co-morbidity; Drug Interactions). Thirdly, RCTs will usually be based on a short follow-up that typically assesses medication use for a period of 3–12 months. Yet, again, subjects may be using these medications for years, so that the effect of the prolonged use of these medications remains clearly unknown from the RCT data. Finally, there are situations where the RCT is either unethical or inapplicable. For example, it would be ethically unacceptable today in North America or Europe to assess the long-term effects of a new anti-hypertensive agent against placebo in an RCT, although this has been done in China [12] (see Ethics of Randomized Trials). Yet, a large number of hypertensive patients from the general population are either untreated or do not comply with their treatment (see Compliance Assessment in Clinical Trials). They could be used as the reference group for a nonrandomized study based on a cohort study design. Although pharmacoepidemiology can be simply regarded as an application of epidemiologic principles and methods to the field of medications, it is now developing as a discipline of its own because of the special nature of drugs. Indeed, the ways by which drugs are prescribed, employed, marketed and regulated impose certain constraints on epidemiologic research into their use and effects. This field poses challenges that often require special solutions not found in other domains of application, such as cancer, cardiovascular, occupational or infectious disease epidemiology; medications are marketed rapidly, practice patterns of prescribing by physicians are variable and profiles of drug use by patients are complicated by varying compliance patterns. This complex and dynamic context in which pharmacoepidemiology is situated, as well as the available sources of data, have given rise to unique statistical challenges. The fact that the lifetime of a drug on the market is relatively short and can suddenly be shortened still further by a regulatory or corporate withdrawal, often imposes major constraints on studies of its effects. These studies must be conducted rapidly and use existing data in an efficient way without compromising validity (see Validity and Generalizability in Epidemiologic Studies).

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

2

Pharmacoepidemiology, Overview

In this article we describe several areas where biostatistical input has served to advance pharmacoepidemiology. Although the last two decades have witnessed an explosion of methodologic advances put forward by biostatistics in the design and analysis of epidemiologic studies, most of these have been fundamental to the field of epidemiology in general. We do not discuss these areas here since they are dealt with extensively elsewhere (see Analytic Epidemiology; Descriptive Epidemiology). Instead, we focus on the biostatistical aspects that produced unique methodologic advances, specific to problems posed by pharmacoepidemiologic research.

The Case–Crossover Design When conducting a case–control study, the selection of controls is usually the most challenging task. The fundamental principle is that selected controls should be representative of the source population which gave rise to the cases [33] a principle often difficult to implement in practice, especially when dealing with acute adverse events and transient exposures. For example, we may wish to study the risk of ventricular tachycardia in association with the use of inhaled beta agonists in asthma. This possible effect has been hypothesized on the basis of clinical study observations of hypokalemia and prolonged Q-T intervals as measured on the electrocardiogram in patients after beta agonist exposure [1]. These unusual cardiac deviations were observed only in the 4-hour period following drug absorption. Thus, a case–control study of this issue would first select cases with this adverse event and investigate whether the drug was taken during the 4-hour span preceding the event. For controls, on the other hand, the investigator must define a time point of reference for which to ask the question about use of this drug in the “past 4 hours”. However, if, for example, the drug is more likely to be required during the day, but controls can only be reached at home in the evening, the relative risk estimate will be biased by the differential timing of responses for cases and controls. Consequently, when dealing with the study of transient drug effects on the risk of acute adverse events, Maclure [30] proposed the case–crossover design, which uses the cases as their own controls. The case–crossover design is simply a crossover study in

the cases only. The subjects alternate at varying frequencies between exposure and nonexposure to the drug of interest until the adverse event occurs, which does for all subjects in the study since all are cases by definition. Each case is investigated to determine whether exposure occurred within the predetermined effect period, namely within the 4 hours previous to the adverse event in our example. This occurrence is then classified as having arisen either under drug exposure or nonexposure on the basis of the effect period. Thus, each case is either exposed or unexposed. For the reference information, data on the average drug use pattern are necessary to determine the probability of exposure in the time window of effect. This is done by obtaining data for a sufficiently long period of time to derive a stable estimate. In our example, we might determine the average number of times a day each case has been using beta agonists (two inhalations of 100 µg each) in the past year. This will allow us to estimate the proportion of time that each asthmatic is usually spending as “exposed” in the 4-hour effect period. This proportion is then used to obtain the number of cases expected on the basis of time spent in these “at-risk” periods, for comparison with the number of cases observed during such periods. This is done by forming a two-by-two table for each case, with the corresponding control data as defined above, and combining the tables using the Mantel–Haenszel technique as described in detail by Maclure [30]. odds ratio is

The resulting then given by OR = ai N0i / (1 − ai )N1i , where ai is 1 if case i is exposed, 0 if not, N0i is the expected number of unexposed periods and N1i the expected number of exposed periods during the reference time span. Table 1 displays hypothetical data from a case–crossover study of 10 asthmatics who experienced ventricular tachycardia. These were all queried regarding their use of two puffs of inhaled beta agonist in the last 4 hours and on average over the past year. The fact of drug use within the effect period is defined by ai with three cases having used beta agonists in the 4-hour period prior to the adverse event. The usual frequency of drug use per year is converted to a ratio of the number of exposed periods to the number of unexposed periods, the total number of 4-hour periods being 2190 in one year. Using the Mantel–Haenszel formula to combine the 10 two-bytwo tables, the estimate of the odds ratio is 3.0, and the 95% confidence interval (1.2, 7.6).

Pharmacoepidemiology, Overview Table 1 Hypothetical data for a case–crossover study of beta agonist exposure in last 4 hours and the risk of ventricular tachycardia in asthma Beta agonist Usual beta Periods usea in agonist Periods of of nonlast 4 hours use in exposure exposure Case no. (ai) last year (N1i ) (N0i ) 1 2 3 4 5 6 7 8 9 10 a

0 1 0 1 0 0 0 1 0 0

1/day 6/year 2/day 1/month 4/week 1/week 1/month 2/month 2/day 2/week

365 3 730 12 208 52 12 24 730 104

1825 2184 1460 2178 1982 2138 2178 2166 1460 2086

Inhalations of 200 µg: 1 = yes, 0 = no

The case–crossover design depends on several assumptions to produce unbiased estimates of the odds ratio. Greenland [14] presented examples where the odds ratio estimates from this approach can be biased. For example, the probability of exposure cannot vary over the time. Similarly, confounding factors must be constant over time. Finally, there cannot be interaction between unmeasured subject characteristics and the exposure. Nevertheless, this approach is being used successfully in several studies [38, 43]. It has also been adapted for application to the risk assessment of vaccines [11].

Confounding Bias Because of the lack of randomization, the most important limitation of observational studies in pharmacoepidemiology is whether an important confounding factor is biasing the reported relative risk estimate. A factor is considered a confounder if it is associated, at each level of drug exposure, with the adverse event, and with exposure to the drug itself. Two approaches exist to address the problem of confounding variables, which we describe in the context of the case–control design, although they apply equally well to a cohort design. The first is to select controls that are matched to the cases with respect to all confounding factors and to use the appropriate corresponding techniques of analysis for matched data, usually conditional logistic regression (see Matched Analysis; Matching).

3

This approach, although often appropriate, has been shown to be susceptible to bias from residual confounding due to coarse matching [5]. The second solution is to select controls unmatched with respect to these confounding factors, but to measure these confounders for all study subjects and use statistical techniques based on either stratification or multiple regression, permitting removal of their effect on the risk from the effect of the drug under study. This approach can also lead to residual confounding if the confounder data are not analyzed properly. For example, the risk of venous thromboembolism has been found to be higher among users of newer oral contraceptive drugs than users of older formulations [22, 41, 52], after controlling for the effect of age. A recent study, however, showed that when confounding by the woman’s age is analyzed using finer age bands, the relative risk is substantially reduced [10]. Beyond such difficulties generic to epidemiology, the context of pharmacoepidemiology has produced several situations where confounding requires particular statistical treatment. Some are described in this section.

Missing Confounder Data It is at times impossible to obtain data on certain important confounding variables. A frequent situation encountered in pharmacoepidemiology is that of complete data for the cases and incomplete data for the controls of a case–control study. This is often encountered in “computerized database” studies based on administrative databases where cases have likely been hospitalized and thus have an extensive medical dossier. For these cases, the investigator will thus have access to ample information on potential confounding variables. However, if the controls are population-based (see Case–Control Study, Population-based), it is unlikely they were hospitalized and will not provide comparable data on confounders in the absence of medical charts. Consequently, confounder data will only be available in the cases, and not in the controls. We can assess whether a factor is a confounder on the basis of data available solely for the cases, so that if the factor is deemed not to be a confounder, then the final analysis of the risk of the drug under study will not need to be adjusted. The approach is described by Ray & Griffin [37] and was

4

Pharmacoepidemiology, Overview

used in the context of a study of nonsteroidal antiinflammatory drug (NSAID) use and the risk of fatal peptic ulcer disease [15]. The strategy is based on the definition [5] of a confounder C (C+ and C− denote presence and absence) in the assessment of the association between a drug exposure E (E+ and E− denote exposure or not to the drug) and an adverse condition D (D+ and D− denote cases and controls, respectively). Confounding is present if both following conditions are satisfied: 1. C and E are associated in the control group (in D−). 2. C and D are associated in E+ and in E−. Assuming, in the absence of effect modification, a common odds ratio between E and D (ORED ) in C+ and in C−, condition 1 becomes equivalent to: C and E are associated in the case group (D+). Thus, if in the cases we find no association between the potential confounder and drug exposure, confounding by this factor can be excluded outright, without having to verify condition 2. In this instance, the analysis involving drug exposure in cases and controls can be performed directly without any concern for the confounding variable. This strategy for assessing confounding is extremely valuable for several case–control studies in pharmacoepidemiology, since if confounding is excluded by this technique, crude methods of analysis can be used to obtain a valid estimate of the odds ratio. However, this is not often the case. As an example, we use data from a case–control study conducted using the Saskatchewan computerized databases to assess whether theophylline, a drug used to treat asthma, increases the risk of acute cardiac death [46]. In this study, the 30 cases provided data on theophylline use, as well as on smoking, possibly an important confounder. However, the 4080 controls only had data available on theophylline use and not on smoking. Table 2 displays the data from this study. The crude odds ratio between theophylline use and cardiac death is 4.3((17/13)/(956/3124)). Because of the missing data on smoking, it is only possible to partition the cases, but not the controls, according to smoking. The odds ratio between theophylline use and smoking among the cases is estimable and found to be 7.5((14/5)/(3/8)), thus indicating that smoking is indeed a strong confounder.

Table 2 Data from a case–control study of theophylline use and cardiac death in asthma, with the smoking confounder data missing for controls Cases

Notation: Combined Stratified by smoking: Smokers Nonsmokers Data: Combined Stratified by smoking: Smokers Nonsmokers a

Controls

E

E

E

E

a

c

b

d

a0 a1

c0 c1

a

a

a

a

17

13

956

3124

14 3

5 8

a

a

a

a

These frequencies are missing for controls.

An approach was recently developed to permit the estimation of the adjusted odds ratio of the theophylline by cardiac death association, in the absence of confounder data among the controls [45]. The adjusted odds ratio is given by ORadj =

P0 (w − y) , (1 − P0 )y

(1)

where y = {v − [v 2 − 4(r − 1)rwx]1/2 }/[2(r − 1)], v = 1 + (r − 1)(w + x) when r
= 1 (and y = wx when r = 1), r is the odds ratio between exposure and confounder among the cases, x is the probability of exposure among the controls, P0 is estimated by a0 (a0 + c0 ), and w is the prevalence of the confounder among the controls [45]; w is the only unknown and must be estimated from external sources. An estimate of the variance of ORadj in (1) exists in closed form. For the illustrative data, an external estimate of smoking prevalence among asthmatics, obtained from a Canadian general population health survey, is 24%. Using this estimate, the adjusted odds ratio is 2.4, much lower than the crude estimate of 4.3, with 95% confidence interval (1.0, 5.8). This statistical approach was developed specifically to address the frequent problem of missing confounder data in pharmacoepidemiology. When using computerized databases, these data are more often missing only in the control series of a case–control study. This technique, based on statistical reasoning, allows us to derive adjusted estimates of relative risk with few assumptions. Extensions of this type

Pharmacoepidemiology, Overview of approach to a regression context will expand its usefulness.

The Case–Time–Control Design Case–control studies in pharmacoepidemiology that assess the intended effects of drugs are often limited by their inability to obtain a precise measure of the indication of drug exposure. Adjustment for this crucial confounding factor becomes impossible and an unbiased estimate of the drug effect is unattainable [49]. This bias, arising from confounding by indication, is a major source of limitation in pharmacoepidemiology [32]. Here again, a within-subject approach, similar to the case–crossover design, has been developed. By using cases and controls of a conventional case–control study as their own referents, the case–time–control design eliminates the biasing effect of unmeasured confounding factors such as drug indication [44]. This approach is applicable only in situations where exposure varies over time, which is typically often the case for medications. The correct application of the case–time–control design is based on a specific model for the data, a model that entails inherent assumptions and imposes certain conditions for the approach to be valid. The model, based on a case–control sampling design, is presented for a dichotomous exposure that varies over time and that is measured only for two consecutive time periods, the current period and the reference period. The logit of exposure Lij kl = logit{Pr(Eij kl = 1)}, is given by Lij kl = µ + Sil + πj + Θk

(2)

where Eij kl represents the binary exposure for group i, period j , outcome k and subject l within group i, µ represents the overall exposure logit, Sil is the effect of study subject l in group i, πj is the effect of period j and Θk is the effect of event occurrence k. More specifically, i = 0, 1 denotes the case–control group (1 = case subjects, 0 = control subjects), j = 0, 1 denotes the period (1 = current period, 0 = reference period), k = 0, 1 denotes the event occurrence (1 = event, 0 = no event) and l = 1, . . . , ni designates the study subject within group i, with n1 case subjects and n0 control subjects. The confounding effect of unmeasured severity or indication is inherently accounted for by Sil .

5

The period effect, measured by the log of the odds ratio, is given by δπ = π1 − π0 and estimated from the control subjects. The net effect of exposure on event occurrence is given by δΘ = Θ1 − Θ0 . The case subjects permit one to estimate the sum δΘ + δπ so that the effect of exposure on event occurrence δΘ , is estimable by subtraction. The estimation of the odds ratio is based on any appropriate technique for matched data, such as conditional logistic regression. Three basic assumptions are inherently made by this logit model. The first is the absence of effect modification of the exposure–outcome association by the unmeasured confounder, i.e. the exclusion of the Sil Θk interaction term in model (2). The second is the absence of effect modification of the exposure–outcome association by period, i.e. a null value for the πΘk interaction term. The third is the lack of effect modification of the exposure–period association by the confounder, represented by the absence of an Sil πj interaction term in model (2). Greenland [14] presented examples of the bias that can occur with this approach when the model contains the latter interaction. The approach is illustrated with data from the Saskatchewan Asthma Epidemiologic Project [40], a study conducted to investigate the risks associated with the use of inhaled beta agonists in the treatment of asthma. Using databases from Saskatchewan, Canada, a cohort of 12 301 asthmatics was followed during 1980–87. All 129 cases of fatal or near-fatal asthma and 655 controls were selected. The amount of beta agonist used in the year prior to the index date was found to be associated with the adverse event. In comparing low (12 or less canisters per year) with high (more than 12) use of beta agonists. The crude odds ratio for high beta-agonist use is 4.4, with 95% confidence interval (2.9, 6.7). Adjustment for all available markers of severity, such as oral corticosteroids and prior asthma hospitalizations as confounding factors, lowers the odds ratio to 3.1, with 95% confidence interval (1.8, 5.4), the “best” estimate one can derive from these case–control data using conventional tools. The use of inhaled beta agonists, however, is known to increase with asthma severity which also increases the risk of fatal or near-fatal asthma. It is therefore not possible to separate the risk effect of the drug from that of disease severity. To apply

Risk Functions Over Time Most epidemiologic studies assessing a risk over time routinely assume that the hazards are constant or proportional. Rate ratios are then estimated by Poisson regression models or Cox’s proportional hazards model [6]. Often, deviations from these simplifying assumptions are addressed at the design stage, by restricting the study to a specific follow-up period where the assumptions are satisfied. For instance, to study the risk of cancer associated with an agent considered to be an initiator of the disease, the first few years of follow-up after the initiation of exposure will not be accounted for in the analysis, to allow for a reasonable latency period. On the other hand, if the agent is suspected to be a cancer promoter, these same first years will be used in the analysis. Since such considerations are mostly dealt with at the design stage, little attention has been paid to the analytic considerations of this issue. In pharmacoepidemiology, the risk of an adverse event often varies strongly with the duration of use

(a)

Duration of drug use

Hazard ratio

the case–time–control design, exposure to beta agonists was obtained for the 1-year current period and the 1-year reference period. Among the 129 cases, 29 were currently high users of beta agonists and were low users in the reference period, while 9 cases were currently low users of beta agonists and were high users previously. Among the 655 controls, 65 were currently high users of beta agonists and were low users in the reference period, while 25 were currently low users of beta agonists and were high users previously. The case–time–control odds ratio, using these discordant pairs frequencies for a matched pairs analysis, is given by (29/9)/(65/25) = 1.2, with 95% confidence interval (0.5, 3.0). This estimate, which excludes the effect of unmeasured confounding by disease severity, indicates a minimal risk for these drugs. The case–time–control approach provides an unbiased estimate of the odds ratio in the presence of confounding by indication, a common obstacle in pharmacoepidemiology. This is possible despite the fact that the indication for drug use (in our example, disease severity) is not measured, because of the within-subject analysis. Nevertheless, as mentioned above, its validity is subject to several strict assumptions. This approach must therefore be used with caution.

Hazard ratio

Pharmacoepidemiology, Overview

(b)

Duration of drug use

Hazard ratio

6

(c)

Duration of drug use

Figure 1 Different risk profiles by duration of drug use: (a) acute effect; (b) increasing risk; (c) constant risk

of a drug. Figure 1 shows three different risk profiles, typical of drug exposure. Figure 1(a) displays the usual profile of risk associated with an acute effect. The drug will affect susceptible subjects early, reflected by the early sharp rise in the curve, and once these subjects are eliminated from the cohort, the remaining subjects will return to some lower constant baseline risk. The peak can occur almost immediately, as with allergic reactions to antibiotics, or may take a certain time to affect the organ, as with gastrointestinal hemorrhage subsequent to NSAID use. This profile of risk was used to explain variations in the risk of agranulocytosis associated with the

Pharmacoepidemiology, Overview use of the analgesic dipyrone [16]. It has recently been used to assess the risk profile of oral contraceptives [47]. Figure 1(b) shows a gradually increasing model of risk, associated with diseases of longer latency such as cancer. Figure 1(c) displays the constant hazard model, after a rapid rise in the risk level. Unfortunately, such graphs are not part of the analysis plan of most pharmacoepidemiologic studies at this point, despite the existence of appropriate techniques [9, 16]. The next few years should see an increasing use of spline functions and other similar tools to model the risk of drugs by their duration of use. The wider access to newer statistical software such as S-PLUS [42] among researchers in pharmacoepidemiology, as well as the publication of papers that simplify the understanding of these sophisticated approaches [13, 16], will encourage their wider use in a field where they are clearly pertinent.

Probabilistic Approach for Causality Assessment The traditional epidemiologic approach to assess whether a drug causes an adverse reaction is based on the Hill criteria [18], that require the association to be biologically plausible, strong, specific, consistent and temporally valid (see Hill’s Criteria for Causality). These criteria are applied to the results of pharmacoepidemiologic studies and, depending on the number of criteria satisfied, provide a level of confidence regarding causality of the drug (see Causation). The result of this exercise will be, if a drug is judged to cause an adverse reaction, that all exposed cases, or at least some etiologic proportion of these cases [39], are due to the drug. This approach is valuable for inferences to the population, but does not allow cases to be assessed individually, does not incorporate the specifics of the case, and does not entirely address the unique features of drugs as an exposure entity in epidemiology. The study of individual cases of adverse reactions has been the mainstay of national pharmacovigilance centers throughout the world for several decades [3, 51] (see Postmarketing Surveillance of New Drugs and Assessment of Risk). When a case report of a suspected drug-associated adverse

7

event is received, the natural question is whether the drug actually caused the event. Several qualitative approaches have been proposed to answer this question [20, 25]. Recently, however, a formal quantitative approach using biostatistical foundations has been put forward [26–28]. It is based on Bayes’ theorem, which can be used in the following way: Pr(D → E|B, C) =

Pr(D → E|B) Pr(C|D → E, B) , Pr(C)

where D → E denotes that the drug D causes the adverse event E, B represents the background characteristics of the case that are known to affect the risk, while C represents the case information. This Bayes’ theorem approach allows us to estimate the posterior probability that an adverse event was caused by a drug by separating the problem into two components. The first component is the prior probability of the event given the baseline characteristics of the patients. This is estimated from existing data obtained from clinical trials or epidemiologic studies. The second component is the probability of case information given that the drug caused the event. The primary limitation of this approach is the scarcity of data available to estimate the two components. In many instances, it is difficult to find the clinical and epidemiologic data necessary to estimate the prior probability, especially for rare clinical conditions that have not been the object of extensive population-based research. The same limitation applies to the second probability component because of the problems of finding cases relevant to proven drug causation. Case series that apply directly to the case being assessed are often difficult to find. These limitations are real but not limited to the Bayesian approach – they are a general problem in the assessment of individual cases. The authors of these methods suggest that the primary purpose of the Bayesian approach is to provide a framework in which subjective judgments relevant to assessment of an individual case are coherently combined. To facilitate the use of the Bayesian method, the equation is usually expressed in terms of odds rather than probabilities. This formulation simplifies somewhat the need for data, since epidemiologic studies more frequently report odds ratios than absolute probabilities. The relative likelihood may also be easier

8

Pharmacoepidemiology, Overview

to estimate subjectively. This equation formulated in terms of odds is given as posterior odds = Pr(D → E|B, C)/ Pr(D
→ E|B, C) = [Pr(D → E|B)/ Pr(D
→ E|B)] prior odds ×[Pr(C|D → E, B)/ Pr(C|D
→ E, B)] likelihood ratio. This approach was applied on several occasions [19, 23, 24, 34] and was recently made user-friendly by computerizing [21]. The increasing amount of new epidemiologic data on disease distribution and risk factors combined with new clinical and pharmacologic insights on drug effects will make this probabilistic approach more effective in future uses.

Methods Based on Prescription Data One of the distinguishing features of pharmacoepidemiology is the use of computerized administrative health databases to answer research questions reliably and with sufficient rapidity. The usual urgency of concerns related to drug safety makes these databases essential to perform such risk assessment studies. In particular, databases containing only information on prescriptions dispensed to patients, and no outcome information on disease diagnoses, hospitalizations or vital status, have been the object of interesting statistical developments. These standalone prescription drug databases, that do not require to be linked to outcomes databases, are more numerous and usually more easily accessible than the fully linked databases. They provide a source of data that allows the investigation of patterns of drug use that can yield some insight into the validity of risk assessment studies as well as generate and test hypotheses about these risks. In this section we briefly review some of the resourceful uses of these drug prescription databases in pharmacoepidemiology. A technique that was developed specifically for the context of drug databases is prescription sequence analysis [36]. Prescription sequence analysis is based on the situation when a certain drug A is suspected of causing an adverse event that itself is treated by a drug B. To apply this technique, the computerized drug database is searched for all patients using drug A. For these subjects, all patients prescribed drug B in the course of using drug A are

identified and counted. Under the null hypothesis that drug A does not cause the adverse event treated by drug B, this number of subjects should be proportional to the duration of use of drug A relative to the total period of observation. This extremely rapid method of assessing the association between drug A and drug B is assessed for its random error with a Monte Carlo simulation analysis. This technique was applied to assess whether patients using the antivertigo or antimigraine drug flunarizine (drug A) causes mental depression, as measured by the use of antidepressant drugs (drug B). The authors found that the number of patients starting on antidepressant drugs during flunarizine use was in fact lower than expected [36]. They thus concluded, using this rapid approach based solely on drug prescription data, that this drug probably does not cause mental depression. An extension of prescription sequence analysis, called prescription sequence symmetry analysis, was recently proposed [17]. Using a population of new users of either drug A or B, this approach compares the number of subjects who used drug A before drug B to that using B before A. Under the null hypothesis, this distribution should be symmetrical and the numbers should be equal. Another function of these databases is to use the prescriptions as covariate information to explain possible confounding patterns. The concept of channeling of drugs was put forward as an explanation of unusual risk findings [48]. For example, a case–control study conducted in New Zealand found that fenoterol, a beta agonist bronchodilator used to treat asthma attacks, was associated with an increased risk of death from asthma [8]. Using a prescription drugs database, it was found that severe asthmatics, as deemed from their use of other asthma medications prescribed for severe forms of the disease, were in fact channeled to fenoterol, probably because fenoterol was felt by prescribers to be a more potent bronchodilator than other beta agonists [35]. This phenomenon of channeling can be assessed rapidly in such databases, provided medications can be used as proxies for disease severity. This approach can be subject to bias, however, as it has been used with cross-sectional designs that cannot differentiate the directionality of the association. An application of channeling using a longitudinal design was recently presented [4]. It indicated that channeling can vary according to the timing of exposure, i.e. that disease severity was not associated with first-time use of a

Pharmacoepidemiology, Overview drug, but subsequently severe patients were more likely to be switched to that drug. This type of research into patterns of drug prescribing and drug use can be very useful in understanding the results of case–control studies with limited data on drug exposures and subject to confounding by indication (see Pharmacoepidemiology, Adverse and Beneficial Effects). These prescription drug databases have also been used to study patterns of interchange in the dispensing of NSAIDs. Such research is important because switching patterns permits the identification of brands that may not be well tolerated and result in the prescription of another agent. By using a stochastic approach, Walker et al. [50] estimated the transition probabilities from one NSAID to another. For a set of k different brands of NSAID, they derived the expected marginal distributions of the transition matrix that corresponds to a global equilibrium state by solving a system of k + 1 equations with k + 1 unknowns. By comparing these expected values with the observed marginals of the transition matrix, it was possible to assess whether this population had reached this stable state and for which drugs. Such models can be used rapidly to assess patterns of interchange and identify potentially harmful agents. Finally, these prescription drugs databases may in certain situations provide all the necessary data for a conventional cohort or case–control study. For instance, the use of beta blockers to treat hypertension and other cardiac diseases has been hypothesized to cause depression. A prescription for an antidepressant drug can be used as a proxy for the outcome of depression. In this way, a standalone prescription drug database can provide data on exposure to beta blockers, on the outcome of depression, as well as on covariate information from other medications [2, 7].

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

Acknowledgments Samy Suissa is a senior research scholar supported by the Fonds de la Recherche en Sant´e du Quebec (FRSQ). The McGill Pharmacoepidemiology Research Unit is funded by the FRSQ and by grants from the National Health Research and Development Programme (NHRDP) and the Medical Research Council (MRC) of Canada.

[13]

[14]

[15]

References [1]

Aelony, Y., Laks, M.M. & Beall, G. (1975). An electrocardiographic pattern of acute myocardial infarction

[16]

9

associated with excessive use of aerosolized isoproterenol, Chest 68, 107–110. Avorn, J., Everitt, D.E. & Weiss, S. (1986). Increased antidepressant use in patients prescribed beta-blockers. Journal of the American Medical Association 255, 357–360. Baum, C., Kweder, S.L. & Anello, C. (1994). The spontaneous reporting system in the United States, in Pharmacoepidemiology, 2nd Ed, B.L. Strom, ed. Wiley, New York, pp. 125–138. Blais, L., Ernst, P. & Suissa, S. (1996). Confounding by indication and channeling over time: the risks of beta-agonists, American Journal of Epidemiology 144, 1161–1169. Breslow, N. & Day, N.E. (1980). Statistical Methods in Cancer Research. Vol. I: The Analysis of Case-Control Studies. International Agency for Research on Cancer, Lyon. Breslow, N. & Day, N.E. (1987). Statistical Methods in Cancer Research, Vol. II: The Design and Analysis of Cohort Studies, 2nd Ed. International Agency for Research on Cancer, Lyon. Bright, R.A. & Everitt, D.E. (1992). Beta-blockers and depression: evidence against an association, Journal of the American Medical Association 267, 1783–1787. Crane, J., Pearce, N., Flatt, A., Burgess, C., Jackson, R., Kwong, T., Ball, M. & Beasley, R. (1989). Prescribed fenoterol and death from asthma in New Zealand 1981–1983: case-control study, Lancet 1, 917–922. Efron, B. (1988). Logistic regression, survival analysis, and the Kaplan-Meier curve, Journal of the American Statistical Association 83, 414–425. Farmer, R.D.T., Lawrenson, R.A., Thompson, C.R., Kennedy, J.G. & Hambleton, I.R. (1997). Populationbased study of risk of venous thromboembolism associated with various oral contraceptives, Lancet 349, 83–88. Farrington, C.P., Nash, J. & Miller, E. (1996). Case series analysis of adverse reactions to vaccines: a comparative evaluation, American Journal of Epidemiology 143, 1165–1173. Gong, L., Zhang, W., Zhu, Y., Zhu, J., Kong, D., Page, V., Ghadirian, P., Lehorier, J. & Hamet, P. (1996). Shanghai trial of Nifedipine in the elderly (STONE), Journal of Hypertension 19, 1–9. Greenland, S. (1995). Dose-response and trend analysis in epidemiology: alternatives to categorical analysis, Epidemiology 6, 356–365. Greenland, S. (1996). Confounding and exposure trends in case-crossover and case-time-control design, Epidemiology 7, 231–239. Griffin, M.R., Ray, W.A. & Schaffner, W. (1988). Nonsteroidal anti-inflammatory drug use and death from peptic ulcer in elderly persons, Annals of Internal Medicine 109, 359–363. Guess, H.A. (1989). Behavior of the exposure odds ratio in a case-control study when the hazard function is not

10

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26] [27] [28]

[29]

[30]

[31]

Pharmacoepidemiology, Overview constant over time, Journal of Clinical Epidemiology 42, 1179–1184. Hallas, J. (1996). Evidence of depression provoked by cardiovascular medication – a prescription sequence symmetry analysis, Epidemiology 7, 478–484. Hill, A.B. (1965). The environment and disease: association or causation?, Proceedings of the Royal Society of Medicine 58, 295–300. Hutchinson, T.A. (1986). A Bayesian approach to assessment of adverse drug reactions: evaluation of a case of acute renal failure, Drug Information Journal 20, 475–482. Hutchinson, T.A., Leventhal, J.M., Kramer, M.S., Karch, F.E., Lipman, A.G. & Feinstein, A.R. (1979). An algorithm for the operational assessment of adverse drug reactions. II. Demonstration of reproducibility and validity, Journal of the American Medical Association 242, 633–638. Hutchinson, T.A., Dawid, A.P., Spiegelhalter, D.J., Cowell, R.G. & Roden, S. (1991). Computer aids for probabilistic assessment of drug safety I: A spread-sheet program, Drug Information Journal 25, 29–39. Jick, H., Jick, S.S., Gurewich, V., Myers, M.W. & Vasilakis, C. (1995). Risk of idiopathic cardiovascular death and nonfatal venous thromboembolism in women using oral contraceptives with differing progestogen components, Lancet 346, 1589–1593. Jones, J.K. (1986). Evaluation of a case of StevensJohnson syndrome, Drug Information Journal 20, 487–502. Kramer, M.S. (1986). A Bayesian approach to assessment of adverse drug reactions: evaluation of a case of fatal anaphylaxis, Drug Information Journal 20, 505–518. Kramer, M.S. Leventhal, J.M., Hutchinson, T.A. & Feinstein, A.R. (1979). An algorithm for the operational assessment of adverse drug reactions. I. Background, description, and instructions for use, Journal of the American Medical Association 242, 623–632. Lane, D.A. (1984). A probabilist’s view of causality assessment, Drug Information Journal 18, 323–330. Lane, D.A. (1986). The Bayesian approach to causality assessment, Drug Information Journal 20, 455–461. Lane, D.A., Kramer, M.S., Hutchinson, T.A., Jones, J.K. & Naranjo, C.A. (1987). The causality assessment of adverse drug reactions using a Bayesian approach, Journal of Pharmaceutical Medicine 2, 265–283. Lenz, W. (1966). Malformations caused by drugs in pregnancy, American Journal of Diseases of Children 112, 99–106. Maclure, M. (1991). The case-crossover design: a method for studying transient effects on the risk of acute events, American Journal of Epidemiology 133, 144–153. McBride, W.G. (1961). Thalidomide and congenital abnormalities, Lancet ii, 1358.

[32]

[33]

[34]

[35]

[36]

[37]

[38]

[39] [40]

[41]

[42] [43]

[44] [45]

[46]

[47]

[48]

Miettinen, O.S. (1983). The need for randomization in the study of intended effects, Statistics in Medicine 2, 267–271. Miettinen, O.S. (1985). Theoretical Epidemiology: Principles of Occurrence Research in Medicine. Wiley, New York. Naranjo, C.A., Lanctot, K.L. & Lane, D.A. (1990). The Bayesian differential diagnosis of neutropenia associated with antiarrhythmic agents, Journal of Clinical Pharmacology 30, 1120–1127. Petri, H. & Urquhart, J. (1991). Channeling bias in the interpretation of drug effects, Statistics in Medicine 10, 577–581. Petri, H., De Vet, H.C.W., Naus, J. & Urquhart, J. (1988). Prescription sequence analysis: a new and fast method for assessing certain adverse reactions of prescription drugs in large populations, Statistics in Medicine 7, 1171–1175. Ray, W.A. & Griffin, M.R. (1989). Use of Medicaid data for pharmacoepidemiology, American Journal of Epidemiology 129, 837–849. Ray, W.A., Fought, R.L. & Decker, M.D. (1992). Psychoactive drugs and the risk of injurious motor vehicle crashes in elderly drivers, American Journal of Epidemiology 136, 873–883. Rothman, K.J. (1986). Modern Epidemiology. Little, Brown, & Company, Boston. Spitzer, W.O., Suissa, S., Ernst, P., Horwitz, R.I., Habbick, B., Cockcroft, D., Boivin, J.F., McNutt, M., Buist, A.S. & Rebuck, A.S. The use of beta-agonists and the risk of death and near death from asthma, New England Journal of Medicine 326, 501–506. Spitzer, W.O., Lewis, M.A., Heinemann, L.A., Thorogood, M. & MacRae, K.D. (1996). Third generation oral contraceptives and risk of venous thromboembolic disorders: an international case-control study. Transactional Research Group on Oral Contraceptives and the Health of Young Women, British Medical Journal 312, 83–88. StatSci (1995). S-PLUS Version 3.3. StatSci, a division of MathSoft, Inc., Seattle. Sturkenboom, M.C., Middelbeek, A., de Jong, L.T., van den Berg, P.B., Stricker, B.H. & Wesseling, H. (1995). Vulvo-vaginal candidiasis associated with acitretin, Journal of Clinical Epidemiology 48, 991–997. Suissa, S. (1995). The case-time-control design, Epidemiology 6, 248–253. Suissa, S. & Edwardes, M. (1997). Adjusted odds ratios for case-control studies with missing confounder data in controls. Epidemiology 8, 275–280. Suissa, S., Hemmelgarn, B., Blais, L. & Ernst, P. (1996). Bronchodilators and acute cardiac death, Journal of Respiratory and Critical Care Medicine 154, 1598–1602. Suissa, S., Blais, L., Spitzer, W.O., Cusson, J., Lewis, M. & Heinemann, L. (1997). First-time use of newer oral contraceptives and the risk of venous thromboembolism, Contraception 56, 141–146. Urquhart, J. (1989). ADR crisis management, Scrip 1388, 19–21.

Pharmacoepidemiology, Overview [49]

Walker, A.M. (1996). Confounding by indication, Epidemiology 7, 335–336. [50] Walker, A.M., Chan, K.W.A. & Yood, R.A. (1992). Patterns of interchange in the dispensing of non-steroidal anti-inflammatory drugs, American Journal of Epidemiology 45, 187–195. [51] Wiholm, B.E., Olsson, S., Moore, N. & Wood, S. (1994). Spontaneous reporting systems outside the United States, in Pharmacoepidemiology, 2nd Ed., B.L. Strom, ed. Wiley, New York, pp. 139–156. [52] World Health Organization Collaborative Study of Cardiovascular Disease and Steroid Hormone Contraception,

11

(1995). Venous thromboembolic disease and combined oral contraceptives: results of international multicentre case-control study. World Health Organization Collaborative Study of Cardiovascular Disease and Steroid Hormone Contraception, Lancet 346, 1575–1582.

(See also Drug Approval and Regulation; Pharmacoepidemiology, Study Designs) S. SUISSA

PHARMACOVIGILANCE

2

The ultimate purpose of pharmacovigilance is to minimize, in practice, the potential for harm that is associated with all active medicines. Although data about all types of adverse drug reactions (ADRs) are collected, the main focus is on preventing those that are defined to be serious, i.e., fatal or life-threatening or causing hospitalization or long-term disability. Pharmacovigilance may be seen as a public health function in which reductions in harm are achieved by measures that promote the safest possible use of medications and/or provide specific safeguards against known hazards (e.g., monitoring white blood cell counts to detect agranulocytosis in users of clozapine). To minimize harm, there is first a need to identify and assess the impact of unexpected potential hazards. For most medications, serious ADRs are rare; otherwise their detection could result in the drug not reaching or being withdrawn from the market. For products that do reach the market, almost by definition, serious hazards are seldom identified during clinical trials because sample sizes are almost invariably too small to detect hazards occurring at a frequency of 1 in 1000 patients or less. In addition, the prevailing conditions of clinical trials— selected patients, short durations of treatment, close monitoring, and specialist supervision— almost invariably mean that they will underestimate the frequency of ADRs relative to that occurring in ordinary practice. Thus, the need for post-marketing safety monitoring is well recognized even though the observational methods used provide a lower standard of evidence for causality than that derived from clinical trials. During pre-marketing clinical development, the aims of pharmacovigilance are different than the broad public health function described above. In clinical trials, there is a need to protect patients being exposed, often in a context of initial uncertainty, as to whether an individual has been randomized to the treatment in question. There is also a need to gather information on harmsthat

PATRICK WALLER Southampton United Kingdom

STEPHEN EVANS London School of Hygiene and Tropical Medicine United Kingdom

1

PURPOSES OF PHARMACOVIGILANCE

DEFINITION AND SCOPE

The term ‘‘pharmacovigilance’’ originated in France and became more widely used during the 1990s. Various definitions have been proposed (1, 2), but there is no generally accepted version, which probably reflects differing perspectives on the scope of this activity. The common ground is that pharmacovigilance has a safety purpose in relation to use of medications and involves patients, health professionals, pharmaceutical companies, and regulatory authorities. At its most narrow, pharmacovigilance may be equated with spontaneous reporting of suspected adverse reactions for marketed medicines. However, broader approaches are gaining acceptance in which pharmacovigilance is viewed as being the whole process of clinical risk management in relation to medications. The point at which medications are authorized for use in the general population may be considered as the starting point. However, this boundary is increasingly becoming blurred by recognition that the clinical development process for safety needs to be seamless, and that there is a need to plan post-authorization pharmacovigilance activities based on a clear delineation of what is already known, and what is not known, about safety. For the purpose of this article, we will consider pharmacovigilance to embrace all safety-related activity beyond the point at which humans are first exposed to a medicinal drug. For many trialists, pharmacovigilance is seen as the process of reporting SUSARs (serious, unexpected, suspected adverse reactions), although this is neither the only nor the most useful aspect of pharmacovigilance.

1

2

PHARMACOVIGILANCE

occur in order to make a provisional assessment of safety and to plan for post-marketing safety development. 3

HISTORICAL BACKGROUND

The seminal event that led to the development of modern pharmacovigilance was the marketing of thalidomide in the late 1950s. A teratogen that affects limb development and classically produces phocomelia (3), thalidomide was used widely by pregnant women for the treatment of morning sickness in many countries (but not in the United States) without having first been subjected to reproductive toxicologic screening, without regulatory review of its safety, and without any mechanism for identifying unexpected hazards being in place. About 10,000 fetuses were harmed, and thalidomide remained available for several years before the association was recognized. Aside from the development of statutory drug regulation across the developed world, the thalidomide disaster led to development of spontaneous ADR reporting schemes (4) (see below), based on the notion that collating the suspicions of clinicians about ADRs would lead to ‘‘early warnings’’ of unexpected toxicity. By 1968, several countries had established such schemes and their value was becoming apparent, leading the World Health Organization to establish an international drug monitoring program (5). In the 1970s, another drug safety disaster occurred, the multisystem disorder known as the oculomucocutaneous syndrome caused by practolol (a beta-blocker used to treat angina and hypertension) (6). This disaster led to the recognition that spontaneous ADR reporting alone is insufficient as a means of studying post-marketing safety. In the late 1970s, various schemes designed to closely monitor the introduction of new drugs were suggested but not widely implemented (7). The 1980s saw the development of prescription-event monitoring (8), an observational cohort method in which patients are identified from prescriptions of a particular drug, with data on subsequent adverse events being obtained from prescribing doctors. Also, pharmaceutical companies started to conduct postmarketing surveillance studies, but the value

of such studies was often limited (9). The scientific discipline of pharmacoepidemiology developed strongly during the 1990s with the increasing use of computerized databases containing records of prescriptions and clinical outcomes for rapid and efficient study of potential hazards, which have often been identified through spontaneous ADR reporting. In some instances, prescriptions are in a separate database to clinical events and linkage between the two databases needs to be achieved through some common identifier in the two sets of data in order to study adverse events at an individual level. Confidentiality legislation has made this type of record linkage more difficult (10). Since the late 1980s, collaboration between the pharmaceutical industry and regulators under the auspices of the Council for the International Organizations of Medical Sciences (CIOMS) has, to date, resulted in six published reports (11–16) proposing international standards for, inter alia, spontaneous ADR reporting, periodic safety update reports (PSURs), core clinical safety information, risk–benefit evaluation, and the management of safety information from clinical trials. A seventh CIOMS working group is in progress that will make proposals for developmental safety update reports. Recent years have also seen an increased regulatory focus on pharmacovigilance within many countries. Relevant standards for electronic ADR reporting and PSURs have been accepted through the International Conference on Harmonisation (ICH) (which involves the United States, the European Union, and Japan) (17), and further ICH guidelines are in preparation. During the 1990s, fuelled by electronic communication, international collaboration in pharmacovigilance increased markedly. However, for reasons that are not fully understood, divergent decisions around the world on the safety of particular medications have continued to be taken (18). In part this lack of consensus is probably because subjective judgments have to be made in the presence of uncertainty about the balance of benefits and risks.

PHARMACOVIGILANCE

4

METHODOLOGIES

4.1 Spontaneous ADR Reporting Spontaneous ADR reporting is a means of centrally collating clinical suspicions that, in individual cases, a drug may have been responsible for an adverse event. Its primary purpose is to detect ‘‘signals’’ of unrecognized drug toxicity. In general, such signals then require confirmation or refutation using other sources of data. Spontaneous reporting systems are in operation in many countries around the world with some variations. Until recently, reporting has, in most countries, been restricted to health professionals, but reporting by patients is increasingly being accepted although its value remains uncertain. To be considered valid, a report must originate from an identifiable reporter, relate to an individual patient identifiable to the reporter, and contain by way of minimum information a drug and a suspected reaction. Information about temporal relationships between drug and reaction, concomitant medication, other diseases, and outcome is usually also requested. Patient-specific information on age, weight, and gender is also useful for interpretation, but concerns over privacy can lead to this information being omitted. Important or incomplete reports may need to be pursued with the reporter to obtain missing or more detailed clinical and follow-up information. Using standard dictionaries for coding, in particular, MedDRA, which is now the international standard medical terminology (19), information from spontaneous ADR reports is entered onto a database that is then subjected to regular screening for new signals. Recent years have seen major advances in the use of statistical approaches to provide indications as to whether the number of reports of a particular drug/reaction combination is in excess of expectations (20). These approaches do not require drug exposure data (e.g., numbers of prescriptions) and are analogous to using a proportional mortality ratio in population epidemiology when accurate denominators are unavailable. The basic idea is to compare the proportion of all reports that relate to a specific medical

3

term between a particular drug and all other drugs in a database. There are several such methods of testing for ‘‘disproportionality,’’ and they yield broadly similar results (21). The database used may be one held by a national regulatory authority, the database of a pharmaceutical company, or the worldwide database held by the World Health Organization (5). There are extensive regulatory requirements around the world relating to spontaneous ADR reporting (22). Reporting is mostly voluntary (and unpaid) for health professionals (mandatory reporting being unenforceable and therefore unlikely to lead to a substantial increase in reporting rates) but a statutory requirement for pharmaceutical companies. A general rule is that serious (in the regulatory definition) suspected ADRs must be reported by companies to regulatory authorities within 15 calendar days of receipt. It is generally accepted that spontaneous ADR reporting is an effective means of detecting signals and the cornerstone of pharmacovigilance. Nevertheless, both false positives and false negatives occur fairly frequently and there are many well-recognized limitations. In particular, substantial underreporting is the norm; causality cannot usually be inferred, and there is great potential for inappropriate interpretation of the data. 4.2 Randomized Controlled Trials Generally, the reporting of adverse effects in clinical trials is deficient in quality, and to improve this, various suggestions have been made (23). Prior to submission of a formal report, there is a need to be vigilant about harms during the conduct of a trial. If it is a new product, with little known of its safety, and if other trials are ongoing, it can be very important to communicate information about any new risks. In extreme cases, there may be a need to stop all trials if a new and serious hazard is found. Within the context of randomized, blinded clinical trials, there are particular difficulties in conducting pharmacovigilance relating to uncertainty about drug exposure. There may be a need to unblind individual cases with serious adverse events in order to manage patients,

4

PHARMACOVIGILANCE

make an evaluation of the safety of the drug, and meet regulatory requirements. However, tension exists with the need to ensure that the integrity of the trial is not compromised by unblinding. Most major trials use a data monitoring committee to oversee all aspects of safety, and when this so, it is best placed to examine unblinded data. The general principle must be that the safety of patients in the trial is paramount. Unblinding should be kept to a minimum, and when it is necessary, there should be clear separation between personnel with knowledge of exposure and those involved with the ongoing conduct of the trial. An EU guidance issued in April 2004 relating to the European Union Directive on Clinical Trials (2001/20/EC) indicates that sponsors should break treatment codes when an unexpected serious suspected adverse reaction occurs. This action seems unfortunate, and it would have been better to recommend that the sponsor delegates this activity to an independent committee. Current legislation seems to ignore two further aspects. First, that a randomized controlled trial (RCT) is the best source of deciding on causality and that individual cases are not usually clearly attributable to a drug under trial. There are exceptions to this, and the TGN1412 tragedy (24) underlines the possibility that individual cases can be unequivocally regarded as caused by a drug. Generally, it is the final summary of data from the RCT, with appropriate statistical analysis that allows for stronger inference on causality for adverse effects that can occur in the absence of a drug. The second aspect is a neglect of the proportionality of the reporting requirements to the types of risk encountered in the trial. Again, the TGN1412 trial illustrates this point. Here was a new, innovative biotech product about which nothing was known in terms of its effect in humans; risks were high. To take a contrast, the megatrials of aspirin in prevention of further heart attacks were studying a product for which there were years of knowledge and experience about its safety and hazards; risks were low. The legislation does not acknowledge such differences, and processes are the same in both types of trial. Although the term ‘‘unexpected’’ may be used to minimize the reporting of individual events in such trials,

there is a danger that the regulatory processes do not take into account the different proportionality of risk to alter the amount of reporting. The interpretation of ‘‘unexpected’’ is not always clear. The CIOMS VI report (16) attempts to reduce the requirements for expedited reporting to a wide range of people and suggests that DSMBs/DMCs are used in lieu of expedited reporting where appropriate. Unfortunately, regulatory authorities and companies are currently inconsistent in their interpretation of even the EU Directive (see the CIOMS VI report), and where other regulations are in force, it is even more difficult. 4.3 Epidemiological Studies Observational epidemiologic research methods may be used to detect drug toxicity, but increasingly they have become established as a means of testing hypotheses and providing evidence to support causation, measure incidence, and identify risk factors. Both cohort and case-control study designs may be used. The nested case-control study, in which cases and controls are drawn from a defined cohort, usually within a database, combines positive aspects of both approaches and is particularly valuable. The ‘‘self-controlled case series’’ method is increasing in importance in assessing the safety of medications, particularly vaccines (25). This method depends on obtaining all cases of a disease, independently of the putative exposure, and uses these cases as their own controls. Nowadays it is relatively unusual for pharmacoepidemiologic research to be conducted on datasets gathered specifically for this purpose. Many databases around the world contain records of prescriptions and medical outcomes or enable linkage of these parameters in individual patients (26). It is generally accepted that information from such a database alone is frequently insufficient and that access to detailed medical records is also needed to support the validity of such research, although this process may lead to bias (27). 4.4 Other Methods The model whereby signals are detected from spontaneous ADR reports and then investigated using epidemiologic methods is, in

PHARMACOVIGILANCE

practice, too simplistic, and in any phase of clinical development, other types of data may be needed to investigate and successfully manage a particular pharmacovigilance issue. These data include preclinical studies, mechanistic studies, clinical trials, and meta-analysis. The latter has tended to be underused in the context of safety and pharmacovigilance, and its use in the context of observational data is controversial (28). Good systematic reviews, leading to a metaanalysis, can be useful because of the rarity of adverse events, although the reporting of them needs to be improved (23). 5

SYSTEMS AND PROCESSES

As indicated, the overall process of pharmacovigilance is one of monitoring and managing risks. This cycle involves the identification and investigation of potential hazards, re-evaluation of the risk-benefit balance, tailored actions designed to minimise risk, and communication of the necessary safeguards to users (Fig. 1) (29). All aspects of this cycle are undertaken by both pharmaceutical companies and regulatory authorities within an increasingly complex framework of legislation and guidelines. Since the mid-1990s, periodic safety update reporting (12) by companies has become an accepted part of the process although its value is not yet clear. Evaluation of safety data in the postmarketing setting may also be hampered by uncertainty about drug exposure, but mostly the limitations relate to details such as dose, duration of treatment, and concomitant medication, which are often missing. Frequently there is also uncertainty about the validity of the clinical diagnosis of individual

adverse events. Observational data, whether derived from case reports or epidemiologic studies, require particularly careful interpretation with regard to key factors such as causality and frequency. Chance, bias, and confounding almost invariably need to be considered as alternative explanations to the causal hypothesis. There are many methods for assessing cause and effect from single cases (30), but in the context of multiple data sources, the criteria proposed by BradfordHill more than 35 years ago (31), in the context of smoking and lung cancer, are still generally accepted as valid. These criteria are strength, consistency, specificity, temporality, biological gradient, biological plausibility, coherence, experimental evidence, and analogy. With regard to biological plausibility, when present, this may be strongly supportive of a causal association, but the converse is not true of its absence because, almost inevitably, many newly identified ADRs cannot be anticipated from prior knowledge. Once safety has been evaluated, there is a need to reassess the balance of benefit and risk, a complex judgmental process for which there is an established framework (14). In terms of actions to prevent ADRs, there is increasingly a focus on developing specific programs designed to minimize risks (examples include hematological monitoring with clozapine and pregnancy prevention with isotretinoin) (32, 33). When new hazards are identified, regulators and manufacturers act to amend product information by adding warnings and appropriate safeguards. Dosage changes after marketing are often introduced. If the balance of risk and benefit for a product can no longer be considered positive, there is a need to remove it

Identify

Investigate

Inform Manage

Figure 1. Pharmacovigilance: the process of risk management

5

Act

Evaluate

6

PHARMACOVIGILANCE

from the market. Overall, about 3% of marketed drugs are prematurely withdrawn from the market for safety reasons (34). Any significant action relating to safety will require communication of the hazard and the proposed measures to users (prescribers and/or patients). It has been increasingly recognized that this is an important aspect of pharmacovigilance that could be improved (35). Standard vehicles for communication include letters to health professionals and drug safety bulletins, but recently use has been made of electronic communication and the Internet. The timing of such communications often represents a considerable challenge in terms of ensuring that health professionals are in a position to advise patients as soon as information becomes available in the lay media. We have no mechanism for communicating information to health professionals that does not, within a matter of hours if not minutes, also reach the lay media if the issue is of wide interest. The other major challenge is achieving a balance between promoting the necessary behavioral change to safeguard patients and not provoking unwarranted scare stories in the media that may result in some patients inappropriately stopping treatment. 6

LIMITATIONS OF PHARMACOVIGILANCE

Since the thalidomide disaster, more than 40 years ago, considerable strides have been

made in all aspects of the risk management process for medications. The drugs in use now are quite different, many new classes of medication having been developed and proven to be acceptably safe in practice. Nevertheless there is little doubt that adverse drug reactions remain an important cause of morbidity and mortality in the developed world (36, 37). Much of this harm remains potentially preventable, and our current inability to prevent it is likely to directly reflect the limitations of existing systems. However, currently, methods for assessing the effects of interventions and safeguards are primitive, and until rectified, this is likely to hamper process improvement. Most pharmacovigilance issues are based on data drawn from a low level in the evidence hierarchy and are therefore amenable to considerable debate and controversy. Although there will always be a need for observational data in pharmacovigilance, it should be possible to gain better evidence through greater use of large simple trials and meta-analysis. Once a decision has been made to take action to warn of a hazard and promote safe use, the tools presently available to minimize risk seem imperfect. Certainly they do not achieve the goal of maximal prevention of serious ADRs. Some of this inadequacy is likely to relate to the unavailability of the necessary information when it is needed and could be counteracted by development

Best evidence

Culture of scientific development

Robust scientific decision-making

Outcome measures and audit

Tools for protecting public health

Measurable performance in terms of public health benefit Figure 2. Future model for pharmacovigilance

PHARMACOVIGILANCE

7

Table 1. Key Points Underpinning Future Model for Pharmacovigilance • Pharmacovigilance should be less focussed on finding harm and more on extending knowledge of safety. • There should be a clear starting point or ‘‘specification’’ of what is already known at the time of licensing a medicine and what is required to extend safety knowledge post-authorization. • Complex risk–benefit decisions are amenable to, and likely to be improved by, the use of formal decision analysis. • A new approach to provision of safety information that allows greater flexibility in presenting key messages based on multiple levels of information with access determined by user requirements. • Flexible decision support is the most likely means of changing the behavior of health professionals in order to promote safer use of medications. • There is a need to put in place outcome measures that indicate the success or failure of the process. These should include hard end-points indicating the impact on mortality and morbidity. Surrogates, such as the impact on prescribing of medications, are more readily available and are potentially valuable. • Systematic audit of pharmacovigilance processes and outcomes should be developed and implemented based on agreed standards (‘‘good pharmacovigilance practice’’). • Pharmacovigilance should operate in a culture of scientific development. This requires the right balance of inputs from various disciplines, a stronger academic base, greater availability of basic training and resource that is dedicated to scientific strategy.

of intelligent decision support systems for prescribing which focus on safety. 7

FUTURE DEVELOPMENTS

Developments in molecular biology and genetics are expected to have a considerable impact on pharmacovigilance within this decade. The nature and usage of new active substances is changing substantially (with mostly niche biological products becoming available), providing new challenges for those involved in monitoring safety. Genetic markers may predict the safety of many drugs (38), with potential implications for their practical use. Adverse drug reactions could be preventable through recording of personal pharmacogenetic profiles and development of guidance recommending use or avoidance of drugs, or tailored dosage regimens, for patients with specific genotypes. To address the limitations described above, a model for the future conduct of pharmacovigilance was proposed in 2003 (39) (Fig. 2). The key points underpinning this model are summarized in Table 1. A proposal for a document summarizing safety at the time of authorization allied to a

plan for extending safety knowledge postauthorization has already been developed as ICH guideline E2E and implemented into European legislation (40). It is important, however, that regulatory requirements are not simply increased but rather that they are refocused toward activities that have measurable public health gain. In the future, further harmonization of regulatory activities, including shared international databases and processes for common decisions, may be possible. To know that such developments represent true progress toward safer medications, it is vital that outcome measures are developed that measure the success or failure of these processes.

REFERENCES 1. R. D. Mann and E. B. Andrews, (eds.), Pharmacovigilance— Introduction. Chichester: Wiley, 2002, pp. 3–9. 2. B. B´egaud, Dictionary of Pharmacoepidemiology. Chichester: Wiley, 2000. 3. T. Stephens and R. Brunner, Dark Remedy. The Impact of Thalidomide ond its Revival as a Vital Medicine. New York: Perseus Books, 2002.

8

PHARMACOVIGILANCE 4. M. D. Rawlins, Spontaneous reporting of adverse drug reactions. Br. J. Clin. Pharmacol. 1988; 26: 1–11.

20.

5. S. Olsson, The role of the WHO programme on international drug monitoring in coordinating worldwide drug safety efforts. Drug Safety 1998; 19: 1–10. 6. P. Wright, Untoward effects associated with practolol administration: oculomucocutaneous syndrome. BMJ 1975; i: 595–598.

21.

7. A. B. Wilson, New surveillance schemes in the United Kingdom. In: W. H. W. Inman (ed.), Monitoring for Drug Safety. Lancaster, United Kingdom: MTP Press, 1986. 8. S. A. W. Shakir, PEM in the UK. In: R. D. Mann and E. B. Andrews (eds.), Pharmacovigilance. Chichester: Wiley, 2002, pp. 333–344.

22.

9. P. C. Waller, S. M. Wood, M. J. S. Langman, A. M. Breckenridge, and M. D. Rawlins, Review of company postmarketing surveillance studies. BMJ 1992; 304: 1470–1472.

23.

10. Walley T. (2006). Using personal health information in medical research BMJ, 332: 130–131. 11. CIOMS Working Group I, International Reporting of Adverse Drug Reactions. Geneva: CIOMS, 1990. 12. CIOMS Working Group II, International Reporting of Periodic Drug-Safety Update Summaries. Geneva: CIOMS, 1992.

24. 25.

26.

13. CIOMS Working Group III, Guidelines for Preparing Core Clinical Safety Information on Drugs. Geneva: CIOMS, 1995. 14. CIOMS Working Group IV, Benefit-Risk Balance for Marketed Drugs: Evaluating Safety Signals. Geneva: CIOMS, 1998.

27.

15. CIOMS Working Group V, Current Challenges in Pharmacovigilance: Pragmatic Approaches. Geneva: CIOMS, 2001.

28.

16. CIOMS Working Group VI, Management of Safety Information from Clinical Trials. Geneva: CIOMS, 2005.

29.

17. P. F. D’Arcy and D. W. G. Harron (eds.), Proceedings of the Third International Conference on Harmonisation Yokohama 1995, Queen’s University, Belfast, Ireland, 1996. 18. M. Fung, A. Thornton, K. Mybeck, J. H. Wu, K. Hornbuckle, and E. Muniz, Evaluation of the characteristics of safety withdrawal of prescription drugs from worldwide pharmaceutical markets—1960 to 1999. Drug Inform. J. 2001; 35: 293–317. 19. E. G. Brown, Dictionaires and coding in pharmacovigilance. In: J. C. C. Talbot and P. C. Waller (eds.), Stephens’ Detection of New

30.

31.

32.

Adverse Drug Reactions, 5th ed. Chichester: Wiley, 2003. S. J. W. Evans, Statistics: analysis and presentation of safety data. In: J. C. C. Talbot and P. C. Waller (eds.), Stephens’ Detection of New Adverse Drug Reactions, 5th ed. Chichester: Wiley, 2003. E. P. van Puijenbroek, A. Bate, H. G. M. Leufkens, M. Lindquist, R. Orre, and A. G. Egberts, A comparison of measures of disproportionality for signal detection in spontaneous reporting systems for adverse drug reactions. Pharmacoepidemiol. Drug Safety 2002; 11: 3–10. B. Arnold, Global ADR Reporting Requirements. Richmond, VA: Scrip Reports, PJB Publications Ltd., 1997. J. P. Ioannidis, S. J. Evans, P. C. Gotzsche, R. T. O’Neill, D. G. Altman, K. Schulz, and D. Moher, with the CONSORT Group (2004). Better reporting of harms in randomized trials: An extension of the CONSORT statement. Ann Intern Med. 2004; 141: 781–788. M. Day, Agency criticises drug trial. BMJ. 2006; 32: 1290. H. J. Whitaker, C. P. Farrington, and P. Musonda, Tutorial in biostatistics: The selfcontrolled case series method. Stat. Med. 2006; 25: 1768–1797. B. L. Strom, How should one perform pharmacoepidemiology studies? Choosing among the available alternatives. In: B. L. Strom (ed.), Pharmacoepidemiology, 3rd ed. Chichester: Wiley, 2000, pp. 401–413. J. M. M. Evans and T. M. MacDonald, Misclassification and selection bias in case-control studies using an automated database. Pharmacoepidemiol. Drug Saf , 1997; 6: 313–318. R. Temple, Meta-analysis and epidemiologic studies in drug development and postmarketing surveillance. JAMA 1999; 281: 841–844. P. C. Waller and H. Tilson, Managing drug safety issues with marketed products. In: J. C. C. Talbot and P. C. Waller (eds.), Stephens’ Detection of New Adverse Drug Reactions, 5th ed. Chichester: Wiley, 2003. S. A. W. Shakir, Causality and correlation in pharmacovigilance. In: J. C. C. Talbot and P. C. Waller (eds.), Stephens’ Detection of New Adverse Drug Reactions, 5th ed. Chichester: Wiley, 2003. A. Bradford-Hill, The environment and disease: association or causation? Proc. Roy. Soc. Med. 1965; 58: 295–300. J. Munro, D. O’Sullivan, C. Andrews, A. Arana, A. Mortimer, and R. Kerwin, Active

PHARMACOVIGILANCE monitoring of 12,760 clozapine recipients in the UK and Ireland. Beyond pharmacovigilance. Br. J. Psychiatry 1999; 175: 576–580. 33. S. E. Perlman, E. E. Leach, L. Dominguez, A. M. Ruszkowski, and S. J. Rudy, Be smart, be safe, be sure. The revised pregnancy prevention program for women on isotretinoin. J. Reprod. Med. 2001; 46(2 Suppl): 179–185. 34. K. E. Lasser, P. D. Allen, S. J. Woolhandler, D. U. Himmelstein, S. M. Wolfe, and D. H. Bor, Timing of new black box warnings and withdrawals for prescription medications. JAMA 2002; 287: 2215–2220. 35. The Uppsala Monitoring Centre, Effective Communications in Pharmacovigilance: The Erice Report. Uppsala, Sweden: Uppsala Monitoring Centre, 1998. 36. J. Lazarou, B. H. Pomeranz, and P. N. Corey, Incidence of adverse drug reactions in hospitalized patients. JAMA 1998; 279: 1200–1205. 37. M. Pirmohamed, S. James, S. Meakin, C. Green, A. K. Scott, T. J. Walley, K. Farrar, B. K. Park, and A. M. Breckenridge, Adverse drug reactions as cause of admission to hospital: Prospective analysis of 18 820 patients BMJ. 2004; 329: 15–19. 38. V. Ozdemir, N. H. Shear, and W. Kalow, What will be the role of pharmacogenetics in evaluating drug safety and minimising adverse effects? Drug Safety 2001; 24: 75–85. 39. P. C. Waller and S. J. W. Evans, A model for the future conduct of pharmacovigilance. Pharmacoepidemiol. Drug Safety 2003; 12: 17–29. 40. ICH guideline E2E. (2004). Pharmacovigilance planning. Available at: http://www.ich. org/LOB/media/MEDIA1195.pdf (Accessed July 6, 2006).

9

PHASE 2/3 TRIALS

2 and phase 3 trials of a traditional clinical development program. The first stage data, which are available through a planned interim analysis, are used to stop a trial for lack of efficacy or unacceptable safety profile, or to redesign stage 2 of the trial. Potential redesign decisions include the selection of treatment arms to carry forward and adjustment of the sample size necessary to ensure adequate data for overall benefit–risk evaluations. At the end of the trial, data from both stages are combined for an integrated analysis of efficacy and safety. Operationally, patient enrollment continues seamlessly (i.e., without any delay) after patients in the first stage are enrolled, even without having to wait for the interim analysis. This process eliminates the gap in time between the phase 2 and phase 3 trials of the traditional clinical development paradigm. Thus, in comparison, clinical development programs that incorporate phase 2/3 designs may have the potential for bringing optimal treatments to patients earlier and more efficiently. When a lack of efficacy is observed or if unacceptable safety issues develop, the total number of patients enrolled with a phase 2/3 trial can be substantially smaller than the number of a phase 3 trial that is designed without knowledge of phase 2 results. However, many different methods can be used to set up a phase 2/3 trial, which largely depend on the underlying clinical applications. A particular phase 2/3 design, which is tailor-made for a limited or similar clinical application, is not likely to be applicable to other clinical situations. For this reason, it is not possible to provide general designs for a wide variety of clinical applications in a single article. Instead, we focus on the legal foundations and common issues relevant for all phase 2/3 designs. These issues are discussed in depth in different sections to provide the reader with a more clear idea of how to develop a phase 2/3 design for a particular clinical application.

QING LIU Johnson and Johnson Pharmaceutical Research and Development L.L.C Raritan, New Jersey

It is well known that in the past decade, the cost of conducting clinical trials has increased exponentially, whereas the success rate for phase 3 development has dropped to an all time low of about 50%. Part of the reason for such a low success rate is that drug companies are facing increasing time pressures to meet patient demands or to satisfy performance goals set by shareholders. Because phase 2 (dose-response) trials are often perceived to delay the timeline, many clinical development programs either omit or conduct phase 2 trials in parallel with phase 3 trials. Traditionally, phase 2 trials have served the purpose of informing the early halt of a clinical development program or providing critical information for moving the clinical development forward to large-scale phase 3 trials. Without data from phase 2 trials, larger phase 3 clinical programs are at an increased risk for attrition related to problems of lack of efficacy or unacceptable safety concerns; in addition, phase 3 designs and statistical analyses (e.g., choice of dose or hypothesis testing strategy for handling multiplicity) are often suboptimal. A potential solution to these problems, while still limiting the overall development time and cost, is to incorporate the phase 2 component directly into a phase 3 trial, so that needed learning and redesigning take place while all relevant data contribute to integrated efficacy and safety analyses. Designs that formalize the approach to maintain scientific and statistical rigor are known as phase 2/3 combination designs or phase 2/3 designs herein. Typically, the adjective ‘‘seamless’’ is added to the description to emphasize the operational feature that transitioning from phase 2 to phase 3 is seamless with respect to patient enrollment; this aspect is described in more detail below. Typically, a phase 2/3 trial is divided into two stages, which correspond to the phase

1

DESCRIPTION AND LEGAL BASIS

To avoid causing more confusion about phase 2/3 designs, it is useful to clarify the

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

PHASE 2/3 TRIALS

notion of the traditional clinical development paradigm, and in particular, the concepts of phase 2 and phase 3 trials. This description is given in the legal framework of Code of Federal Regulations (1) of the U.S. Federal Government. 1.1 Descriptions of the Traditional Paradigm According to Temple (2), a traditional clinical development program typically consists of phase 1, phase 2, and phase 3 trials. Reference 1, part 312.21 provides general descriptions of the three phases. Phase 2 includes the controlled clinical studies conducted to evaluate the effectiveness of the drug for a particular indication or indications in patients with the disease or condition under study and to determine the common short-term side effects and risks associated with the drug. Phase 2 studies are typically well controlled, closely monitored, and conducted in a relatively small number of patients, usually involving no more than several hundred subjects. Phase 3 studies are expanded controlled and uncontrolled trials. They are performed after preliminary evidence suggesting effectiveness of the drug has been obtained, and are intended to gather the additional information about effectiveness and safety that is needed to evaluate the overall benefit-risk relationship of the drug and to provide an adequate basis for physician labeling. Phase 3 studies usually include from several hundred to several thousands subjects.

Temple (2) made even more clarifications. Phase 2 trials ‘‘often include dose response studies,’’ and ‘‘are well controlled studies and can be part of substantial evidence.’’ ‘‘People have redefined ‘phase 2’ into ‘2A and 2B’.’’ Phase 3 trials are ‘‘longer-term studies’’ with ‘‘more and long-term safety,’’ ‘‘more D/R (dose response),’’ and ‘‘broader (real life) population.’’ The definition of the term ‘‘substantial evidence’’ is given by the Federal Food, Drug, and Cosmetic Act (3), and it will be discussed at greater length at the end of this section. 1.2 Inadequacy of the Traditional Paradigm The high phase 3 attrition rate reflects two fundamental issues with the traditional paradigm: 1) the designs of phase 3 trials are either deficient to meet the stated objectives

or unnecessarily expose patients to ineffective or potentially unsafe drug or treatments and 2) clinical holds under 21CFR 312.42 (1) may not be enforced often enough when, ‘‘The plan or protocol for the investigation is clearly deficient in design to meet its stated objectives.’’ Typically, the sample size for a phase 3 design is based solely on power and type I error rate considerations related to primary endpoints. These considerations alone can be deficient for evaluating the overall benefit–risk relationship. Such deficiencies cannot be addressed unless the sample size is reliably calculated on the basis of the totality of adequate efficacy (primary and secondary) and tolerability/safety data, which include laboratory values. Because phase 3 trials are expanded clinical investigations beyond the scope of short-term phase 2 trials, unexpected tolerability/safety issues or lack of efficacy may occur later in the treatment or follow-up periods. Because most phase 3 trials have no provisions for prospective interim analyses, it is difficult to design phase 3 trials adequately a priori to ensure that: a) patients do not unnecessarily receive ineffective, intolerable, or unsafe treatments, and b) the designs meet the ultimate objective of obtaining adequate data for benefit–risk evaluations. Another deficiency is that many phase 3 trials do not study enough doses. There is no regulatory mandate for phase 3 doseresponse information, although the ICH E8 (4) advocates that, ‘‘Studies in Phase III may also further explore the dose-response relationship.’’ Because of cost and time concerns, one or two doses are often selected for phase 3 as the best-predicted doses from limited and short-term phase 2 trials. This practice has led to numerous failures in later safety studies or high-profile withdrawals of drugs from the market because of emerging safety risks to patients. Not studying enough doses also limits the populations that might experience a drug’s therapeutic benefits. In clinical practice, a patient’s treatment usually starts and continues with a low dose. Use of lower doses will generally tend to reduce the frequency

PHASE 2/3 TRIALS

of adverse effects. However, a higher dose, with greater potential for side effects, may be justified when the intended efficacy could not be reached with the low dose. Deciding on the correct dose depends on having adequate information about both benefits and side effects across a range of potential doses. 1.3 Descriptions of Phase 2/3 Designs Clinical development programs are usually divided into early and late stages. Under the traditional clinical development paradigm, an early clinical development program typically starts with phase 1 safety and tolerability trials, for which healthy volunteers or patients with terminal disease are enrolled. Then a proof-of-concept, or phase 2a, trial is conducted, either in patients with the target disease or in patients in a different disease area. For many clinical applications, the latestage clinical development program consists of a phase 2b dose-response trial, one or two phase 3 trials, and open label extension trials. Phase 3b or phase 4 trials are usually conducted after a drug is approved for marketing. A phase 2b trial often has more treatment arms than a phase 3 trial and uses short-term (intermediate) endpoints as compared with phase 3 clinical endpoints that are defined over a longer treatment and evaluation period. If a phase 2b trial is elevated to employ the same entry criteria for patient enrollment as in a phase 3 trial, then it is possible to combine both phases of the late stage development into a single confirmatory trial. Other conditions are also necessary. To integrate statistical inference, patients in phase 2b must receive the same treatment, follow-up, and clinical and laboratory evaluations as patients in phase 3, even though short term efficacy and safety evaluations of phase 2b patients are used to redesign the phase 3 part of the trial. The second aspect of phase 2/3 designs regards statistical inference for integrating phase 2 and phase 3 data for evidentiary evaluation of data, which is discussed later. The crucial and perhaps feasibility-limiting aspect is logistical planning to ensure seamless patient enrollment, timely data collection, prompt interim analysis and decision making, and

3

proper implementation of interim decisions. A design that is concerned with clinical, statistical, and logistical integration of phase 2b and phase 3 trials is called a phase 2/3 design. A phase 2/3 design provides more longterm benefit–risk data than a clinical development program under a traditional paradigm, because patients in the phase 2 stage are treated and followed for their longterm benefit–risk data, while patients in a traditional phase 2b trial are typically not followed for their long-term benefit–risk data. Thus, a clinical development program that employs a phase 2/3 design can provide more benefit–risk data with fewer patients than a clinical development program under a traditional paradigm (5). To apply a phase 2/3 design effectively, that is, to limit the number of treatment arms and to reduce unnecessary enrollment of patients in case the trial is stopped for lack of efficacy or safety concerns, it is important that the early clinical development program provides enough safety and efficacy, and preferably at least limited dose-response data, before engaging in late stage clinical development. Operationally, the second phase 3 trial, when required, can start after the interim analysis results of the phase 2/3 trial are known. Phase 2/3 designs are fundamentally different from those adaptive designs that employ only immediate efficacy and safety endpoints. In a phase 2/3 trial, long-term efficacy and safety data from both phases are used for inference. Short-term efficacy and safety from phase 2b are used for adaptation. Following the descriptions of phase 2 and phase 3 trials in 21CFR 312.21 (1), the ‘‘seamless phase 2/3 designs’’ that employ immediate endpoints should properly be renamed as ‘‘adaptive phase 2b designs.’’ Clarifying the distinctions and providing proper naming conventions for the two different types of designs are necessary to avoid additional and unnecessary confusion as the underlying design issues and statistical methodologies are very different. Naturally, clarification of these design distinctions also leads to a focus on most phase 3 trials that require long-term treatment and follow-up, for which substantial efforts are needed to develop applicable methodologies.

4

PHASE 2/3 TRIALS

1.4 Legal Basis for Phase 2/3 Designs The current laws provide the legal basis for phase 2/3 combination designs. The regulations in 12CFR 312.23 state (1): In phases 2 and 3, detailed protocols describing all aspects of the study should be submitted. A protocol for a phase 2 or 3 investigation should be designed in such a way that, if the sponsor anticipates that some deviation from the study design may become necessary as the investigation progresses, alternatives or contingencies to provide such deviation are built into the protocols at the outset. For example, a protocol for a controlled short-term study might include a plan for an early crossover of nonresponders to an alternative therapy.

The unpredictable nature of phase 3 trials often leads to deviations. To ensure, as noted above, that patients do not unnecessarily receive ineffective, intolerable, or unsafe treatments, and that the designs meet the ultimate objective of obtaining adequate data for benefit–risk evaluations, 12CRF 312.23 (1) permits contingencies for deviations as long as they are built into the protocols at the outset. With current statistical theory for adaptive designs, the contingencies are known as adaptations for an ongoing trial to deal with unexpected outcomes (or deviations) from prospectively planned interim analyses. A key design aspect is the sample size, which is often determined on the basis of primary endpoints, whereas other efficacy or safety endpoints are totally ignored. To ensure the sample size is adequate, datadependent evaluation and modification are necessary according to Section 355 (b)(5)(C) of the Federal Food, Drug and Cosmetic Act (3): Any agreement regarding the parameters of the design and size of clinical trials of a new drug under this paragraph that is reached between the Secretary and a sponsor or applicant shall be reduced to writing and made part of the administrative record by the Secretary. Such agreement shall not be changed after the testing begins, except --(i) with the written agreement of the sponsor or applicant; or

(i) pursuant to a decision, made in accordance with subparagraph (D) by the director of the reviewing division, that a substantial scientific issue essential to determining the safety or effectiveness of the drug has been identified after the testing has begun.

Per Section 355 (b)(5)(C) (see Reference 3), trial sample size can be changed for either safety or efficacy reasons alone, or for evaluation of benefit and risk relationships. Of particular note is that, according to Section 355 (b)(5)(B) (Reference 3), the clinical trials in consideration are those ‘‘intended to form the primary basis of an effectiveness claim.’’ An essential aspect of any phase 2/3 design is that the phase 2 stage is, in every way except the sample size, adequate and well controlled according to the rigor of the phase 3 stage. Thus, the results of phase 2 stage are part of substantial evidence and are combinable with phase 3 results. In fact, a major legal motivation for phase 2/3 combination designs is Section 355 (d) of the Federal Food, Drug, and Cosmetic Act (3), which states that the term ‘substantial evidence’ means evidence consisting of adequate and well-controlled investigations, including clinical investigations, by experts qualified by scientific training and experience to evaluate the effectiveness of the drug involved, on the basis of which it could fairly and responsibly be concluded by such experts that the drug will have the effect it purports or is represented to have under the conditions of use prescribed, recommended, or suggested in the labeling or proposed labeling thereof.

The definition of ‘‘substantial evidence’’ is consistent with a general scientific principle that all relevant data should be included to evaluate evidence in support of a hypothesis. 2 BETTER DOSE-RESPONSE STUDIES WITH PHASE 2/3 DESIGNS Hung et al. (6) raise the fundamental question: ‘‘Why can the accumulating information at an interim time of a trial suddenly gain sufficient insights for predicting benefit/risk ratio?’’ This question underlies a fundamental difficulty of the traditional phase 3 trials, in which interim monitoring of safety

PHASE 2/3 TRIALS

and efficacy is not employed. From experience with group sequential trials (see Reference 7, pp. 29–39), conducting interim analyses allows assessing safety and efficacy of data, and therefore, when new safety signals develop, a Data Safety and Monitoring Committee (DSMB) can make appropriate decisions, which include extending the trial for better evaluation of benefit–risk ratio. However, this experience does not guarantee that a phase 2/3 design is superior to a traditional well-planned multiple-dose phase 3 design. Rather, most proposed phase 2/3 designs, which are listed in the Further Reading section, are in fact inferior to a typical multiarm phase 3 trial because their practical use would lead to a higher probability of selecting a single dose that has unacceptable safety risks. A common feature of these phase 2/3 designs is that only one treatment arm, which is most often selected on the basis of the maximum test statistic for efficacy, is carried forward so as to produce the illusion of saving cost and time. Clinical issues and regulatory ramifications of selecting one dose for phase 3 trials under the traditional paradigm are described clearly and eloquently by Hemmings (8, pp. 30, 46, 47). From a regulatory point of view, selecting only one dose in phase 2/3 trials would certainly decrease long-term dose-response information, and therefore, reduce regulatory agencies’ ability to make better benefit–risk evaluations. These single-dose designs exacerbate the regulatory difficulties because the statistical procedure to select a dose that exhibits the maximum test statistic for efficacy tends also to select the dose with the poorest tolerability and/or the highest risk of adverse events, in general. Hence, practical applications of the single-dose designs can paradoxically lead to longer and more costly overall development, a higher latestage development attrition rate, or a greater probability of harming patients once the drug is on the market. This outcome is a realistic scenario that is even supported by simulation studies of some proposed designs. For example, for designs with three treatment arms and a single control arm, it has been demonstrated that the probability of selecting a particular dose is only one third of the

5

overall power when in fact all three treatments are equally efficacious. If the overall power for demonstrating statistical significance for any selected treatment is 90% and if only one of the three treatments has an acceptable long-term benefit–risk profile, then the probability of that treatment being selected is only 30%. This level of probability of success is worse than the unacceptable lower probability bound of 1/3 = 33.33% of the subset selection criterion of Bechhofer et al. (9, p. 9). In contrast, a phase 3 trial with a subset of three properly selected doses can increase the probability substantially that one of the three doses is close to being optimal. From a broader clinical development perspective, it is not difficult to perform simulation studies to show that with these singledose phase 2/3 designs the late stage attrition rate would be far greater than the current attrition rate of 50%. With some proposed designs, it is possible to select more than one dose to carry forward, without inflation of type I error rates caused by multiplicity. In such cases, the conclusion of time and cost savings is misleading. To fulfill the promise to bring safe and effective treatments to patients, phase 2/3 trials should achieve two basic objectives: 1) that patients do not receive ineffective, intolerable, or unsafe treatments unnecessarily; and 2) that adequate data for benefit–risk evaluations can be obtained without compromising existing regulatory and scientific standards. These objectives cannot be achieved with traditional phase 3 trials by design (see the section entitled ‘‘Inadequacy of the traditional paradigm’’). It is only in pursuing these objectives that the additional benefits of time and cost savings for the overall development program can be meaningfully assessed. For placebo controlled parallel group dose-response trials, Liu and Pledger (5) proposed a ‘‘drop-the-loser’’ approach in which low doses that meet a specified futility guideline for short-term efficacy endpoints and high doses with safety and tolerability concerns are dropped. The remaining dose range, which has a high probability (e.g., 90% as compared with 30% of single-dose designs) of covering the optimal treatment, is then carried forward for long-term evaluations. In addition, patients already randomized in the

6

PHASE 2/3 TRIALS

phase 2 stage continue to receive treatment and follow-up. The design produces maximum long-term dose-response information that is not available from typical phase 3 trials and saves overall development cost and time. It is important to note that this approach already implements the idea recommended by Hemmings (8). 3

PRINCIPLES OF PHASE 2/3 DESIGNS

Prospective planning is necessary to ensure that statistical inference from the trial results is valid and interpretable. Details are discussed below. 3.1 Endpoints and Feasibility The first step is to assess the operational feasibility of a phase 2/3 trial. Two major factors that determine the feasibility is how quickly patients can be enrolled to meet the sample size requirement and how long the treatment and follow-up period are required to evaluate the clinical endpoints. For most phase 3 trials, the time required to enroll patients can be short relative to the time required for treatment and followup. If the interim analysis had to be based on the clinical endpoint, then a maximum sample size could be reached long before the interim analysis is even performed. Any decisions made at the interim analysis would not lead to any savings of time and resources. To reflect the traditional approach for late stage development, a phase 2/3 design can use short-term endpoints in the phase 2b portion for decisions regarding treatment arm selection and sample size adjustment. To ensure reliable decision making, it is important to select the type and timing of the short-term endpoints carefully so that the trial is not prematurely terminated and the likelihood is maximized to carry the optimal treatment arm forward to the second stage. An exception is when the clinical efficacy is measured by a time-to-event endpoint, and it is acceptable to assume a proportional hazards model for statistical inference. In this case, the traditional group sequential designs with type I error rate spending and nonbinding futility boundaries can be modified to incorporate other adaptive features such

as dropping treatment arms and sample-size adjustment. However, for many such applications, the required number of patients can be enrolled relatively fast. It usually takes a longer time to accumulate enough events for interim analyses. Thus, a group-sequential type of phase 2/3 designs can only reduce the time to complete the trial. 3.2 Hypotheses and Interpretability To qualify a phase 2/3 trial as a pivotal trial, it is critical that the trial results are interpretable with respect to the hypotheses specified in advance in the development program. By the general theory of adaptation developed by Liu et al. (10), it is possible to adapt a trial ‘‘on-the-fly’’ to an unspecified hypothesis without inflating the type I error rate. However, a significant P-value that combines data from both stages in this case only provides a qualitative evaluation that a new treatment is somehow different from the control. More specific evaluations are necessary to describe more precisely how patients will benefit from the new treatment, so the new treatment can be labeled properly in the package insert. 21CFR 312.6 (1) states that The label or labeling of an investigational new drug shall not bear any statement that is false or misleading in any particular and shall not represent that the investigational new drug is safe or effective for the purposes for which it is being investigated.

Apparently, more specific evaluations can solely be based on data from the second stage, which reflects exactly the traditional clinical development paradigm when hypotheses of relevance for phase 3 trials are formulated only after the results of phase 2 trials are known. The relevance of hypotheses for a treatment indication depends on consensus of the medical community and has to be agreed to by regulatory agencies. Under the traditional clinical development paradigm, agreements on designs of phase 3 trials are typically reached through the end-of-phase-2 meetings. To ensure relevance of hypotheses and seamless execution of a phase 2/3 trial, it is essential to specify the hypotheses up front in the study protocol.

PHASE 2/3 TRIALS

3.3 Types of Adaptations The set of prespecified hypotheses limits the types of adaptations admissible under a phase 2/3 design. After the interim analysis, the trial can focus on testing more specific hypotheses when other hypotheses become unlikely to be true. If a type of adaptation can potentially lead to a hypothesis outside what is agreed on, then the trial results may be difficult to interpret. In addition, the multiple type I error rates may not be properly controlled because of enlarging the set of hypotheses. Depending on the clinical applications, relevant hypotheses may be different, and therefore, admissible types of adaptations may also be different. However, it is useful to classify the types of adaptations according to their specificities to the underlying clinical applications: type A reflects goals of primary adaptation, which often leads to change in the hypotheses; type B pertains to sample size adjustment, which does not change the hypotheses or their interpretation; type C covers adaptive hypothesis testing methods for handling global null hypotheses. For a dose-response trial, Liu and Pledger (5) proposed a phase 2/3 design that covers all three types of adaptations. Specifically, type A adaptations cover dropping low doses for lacking efficacy and high doses that are potentially not safe, which is a form of primary adaptation; type B adaptations provide methods for sample size adjustment according to dose selections made; and type C adaptations provide an adaptive trend testing procedure derived by modeling the dose-response curve using data from the first stage. Type A adaptations are restricted to endpoint selection or subpatient population selection for trials with multiple endpoints or subpatient populations, respectively, for which the goals of primary adaptation are endpoint selection and patient population selection, respectively. 3.4 Adaptation Rules versus Guidelines Although it is critical to specify the types of adaptations to ensure interpretability and validity of results, it is also important (see

7

21CFR 312.23, Reference 1) to provide guidelines for adaptations in the study plan. Generally, adaptation guidelines are suggested plans for possible adaptations that are based on observations of short-term efficacy and perhaps safety data. The adaptation guidelines, which may not be as explicit as algorithms that lead to specific adaptations, embody basic principles for adaptations, which facilitate intelligent interim decision making by a DSMB. In contrast, adaptation rules, which are algorithms specified a priori that apply to only a few endpoints and do not cover every aspect of the ongoing trial, are meant to be followed only to maintain certain statistical properties of inadequate adaptive designs. By 21CFR 312.21 (1), phase 3 trials are expanded studies for collecting additional information on effectiveness and safety for evaluating overall benefit–risk, which are not already available from phase 2 trials. Therefore, additional uncertainty and risk to patients occurs in phase 3 trials that are not predictable from phase 2 trials. In wellplanned phase 2/3 trials, adaptation guidelines facilitate intelligent and responsible interim decision making by a DSMB, whose goals are to protect patients and to ensure that the ultimate trial objective of collecting adequate data to evaluate benefits and risks is achieved. Canner (11) eloquently stated that: Decision-making in clinical trials is complicated and often protracted . . . no single statistical decision rule or procedure can take the place of well-reasoned consideration of all aspects of the data by a group of concerned, competent and experienced persons with a wide range of scientific backgrounds and points of view.

DeMets (12), in discussing group sequential trials, pointed out that for practical applications the stopping boundaries are guidelines, whereas other relevant secondary efficacy and safety data as well as external information must also be taken into account in decisions to stop the trial early. In fact, it is now standard practice to provide to a DSMB a comprehensive list of trial related summaries (see Reference 7, Tables 5.1 and 5.2 within). The totality of accumulating data may lead to

8

PHASE 2/3 TRIALS

unplanned but necessary changes. An excellent example, which is described in Ellenberg et al. (7, pp. 36, 37), is the fluconazole trial for preventing serious fungal infection in patients with AIDS. Even though the primary endpoint of serious fungal infections clearly crossed the O’Brien-Fleming boundary in favor of fluconazole, the DSMB recommended continuation of the trial after observing a large excess of deaths on the fluconazole regimen. At the completion of the trial, patients on fluconazole continued to show lower rate of serious fungal infections and greater death rates. More fluconazole patients died without experiencing a serious fungal infection. This outcome suggested the need to explore a possible unintended mechanism of fluconazole that could adversely affect survival. It must be pointed out that under the traditional development paradigm, designs of phase 3 trials occur after phase 2 results are available. The underlying principles for designing phase 3 trials after examining phase 2 results are essentially adaptation guidelines as described. Thus, employing adaptation guidelines is an inherent requirement for phase 2/3 designs. 3.5 Ethical Considerations For life-threatening diseases, it is imperative to make the best treatment available as early as possible. This goal is often achieved in a context of a single pivotal trial in which effects of new treatments are tested at multiple interim analysis times. Once a new treatment is shown to be superior to a control, it is no longer ethically acceptable to subject patients to inferior treatments. In a trial with multiple treatment regimens, patients who are randomized to treatment arms that are eliminated either because of lack of efficacy or safety concerns are typically treated with the best treatment available, which can be either a new treatment regimen in the trial or a standard treatment outside the trial. When treatment eliminations are based on short-term data, long-term clinical efficacy and safety data of the underlying treatment do not even exist for patients enrolled in the first stage of a phase 2/3 trial. This issue causes bias for estimating treatment effects

of remaining treatment arms. Because both efficacy and safety data are used for dose selection, it is difficult to qualify the direction of the bias. Gallo and Maurer (13) point out that there is no means to correct the bias, even for clinical development under the traditional paradigm. For non-life-threatening diseases, multiple placebo-controlled phase 3 trials and large, simple open-label extension trials are often necessary to provide substantial evidence for efficacy and safety. With a phase 2/3 design, there are fewer ethical concerns to continuing treating and following patients already on treatment arms that are dropped for lack of efficacy as long as the placebo control is still present (and no harm results). It is often appropriate to continue treating and following patients on their randomized treatment arms that are dropped for tolerability reasons, while permitting decisions of early withdrawal on an individual caseby-case basis. For this setting, statistical procedures for testing, estimation, and confidence intervals already exist (5,14). 4 INFERENTIAL DIFFICULTIES 21CFR 312.6 (1) states that The label or labeling of an investigational new drug shall not bear any statement that is false or misleading.

And Section 355 (d)(5) of the Federal Food, Drug, and Cosmetic Act (3) provides a clause for the Secretary of DHHS (the Department of Health and Human Services) to issue an order refusing to approve an application if evaluated on the basis of the information submitted to him as part of the application and any other information before him with respect to such drug, there is a lack of substantial evidence that the drug will have the effect it purports or is represented to have under the conditions of use prescribed, recommended, or suggested in the proposed labeling thereof

Thus, in a general sense, a key inferential objective for conducting clinical trials and performing statistical analysis is to provide evidence in support of labeling statements

PHASE 2/3 TRIALS

of a drug’s effectiveness and safety. It is up to the Secretary of DHHS to decide whether the evidence presented is substantial according to established regulatory guidance [see Section 355 (b)(5)(A), Reference 3]. Under the traditional clinical development paradigm in which phase 3 trials of fixed sample size are used, statements of the drug’s effectiveness are evaluated mainly through significance tests where P-values are used as measures of strength of evidence. The P-values are then used to determine whether the evidence presented in support of a statement is substantial according to the conventional choice of type I error rates. For settings in which parametric models can be correctly specified, significance tests are typically derived according to the sufficiency principle such that no evidence of relevance for the parameter in question is left unused. Among an infinite number of ways of combing sufficient statistics, minimum sufficient statistics are used in significance tests, mainly according to the NeymanPearson theory, under which the power of significance tests at any significance level is maximized. Refinements to statements of the drug’s effectiveness based on effect sizes are often provided through point estimates and confidence intervals in journal publications. However, estimates and confidence intervals are rarely presented in package inserts of labeling; instead, summary tabulations are used. For adaptive designs, several testing procedures allow various adaptations and multiple comparisons. These adaptive testing methods are based on traditional ad-hoc tests proposed for combining results of different experiments (15–17). The combination tests belong to a class of more general conditional tests proposed by Proschan and Hunsberger (18), which are rigorously justified by Liu et al. (10) on probability measure-theoretic grounds. Although it seems that these combination tests are adequate for controlling the type I error rates at a specified level, raging and confusing debates argue whether they can lead to scientifically sound evidentiary evaluation of a drug’s effectiveness. Central to these debates is the claim that most adaptive designs do not follow the sufficiency principle (19–21), by which inference

9

regarding an unknown parameter should be based on the ‘‘minimal sufficient statistic.’’ However, most literature retains the original definition of the sufficiency principle by Fisher (22,23) that any inference should only be based on sufficient statistics. The essence of Fisher’s definition is that no data of relevance should be left out in the inference, which is exactly the emphasis of adaptive designs. It is true that adaptive designs that maintain flexibility (i.e., relying on adaptation guidelines as described earlier) do not use minimum sufficient statistics to combine data. This finding occurs simply because an α-level adaptive test that uses minimum sufficient statistics while maintaining flexibility does not exist (24). It must be pointed out that using minimum sufficient statistics is the consequence of applying the NeymanPearson theory in the setting of fixed sample size designs. It is not at all clear at this time whether, or in what sense, minimum sufficient statistics are optimal for adaptive designs. Despite this uncertainty, apparent inferential difficulties are noted with common combination tests when derived P-values or other aspects of the tests are used for evidentiary evaluations of the data. Liu and Pledger (25, p. 1981) give an example to illustrate a nonstochastic curtailing problem with the Pvalues of Brannath et al. (26) for Fisher’s combination tests. They also point out that the P-values from their recursive combination tests inherit a problem of stage-wise ordering of the sample space for group sequential designs. After a boundary is crossed, additional data that are obtained either because of natural overrunning or intentional trial extension to obtain more data for other endpoints are not incorporated in the final analysis. Therefore, the intent-to-treat principle for data analysis is violated. A recent paper by Burman and Sonesson (19) highlights logical difficulties with weighted test statistics, as well as their resulting point estimates and confidence intervals. With no explicit reference to minimum sufficiency, Chen et al. (27) suggest that an adaptive test statistic should give an equal weight to all patients according to the idea of ‘‘one-patient-one-vote.’’ It must be emphasized that these difficulties with adaptive tests neither nullify

10

PHASE 2/3 TRIALS

the legal basis for phase 2/3 combination designs nor negate their promise to bring optimal treatment to patients. Rather, these difficulties reinvigorate classic criticisms of significance tests seeking to maintain the size of type I error rates regardless of the sample size while ignoring the fundamental issue of evidentiary evaluation of data (28, pp. 61–82). According to Proschan (24), an α-level adaptive test that uses minimum sufficient statistics while maintaining flexibility does not exist. Thus, any effort to contrive adaptive tests of fixed significance level is bound to cause additional inferential or logical difficulties in evidentiary evaluation of data. Frisen (29) has concluded that To gain control over the significance level is not good enough. There are very misleading procedures that have a controlled significance level. The concentration on a solution to get a fixed significance level takes the focus away from the important issue of evaluating information.

Not surprisingly, this conclusion coincides with Section 355 (d)(5) of the Federal Food, Drug, and Cosmetic Act (3), by which the FDA’s approval process consists of application review, including assessing the strength of evidence, and then approval decisions, for which a lack of substantial evidence can be the basis for the Secretary of DHHS to issue an order refusing to approve an application. Hopefully, with this enlightenment, statistical research can focus on evidentiary evaluation of data.

It is argued that phase 2/3 designs should be used to improve on what the current phase 3 designs are lacking. The difficulties with existing adaptive methods are highlighted. It is concluded that to resolve these difficulties, statistical research should focus on evidentiary evaluation of data, not fixed-size significance tests. Of particular note is that standard criteria for properly controlling type I error rates need still be followed. Extended logistical and implementation issues are not discussed at all. Nevertheless, the readers should find that various discussions and recommendations are available in the reading list provided.

6 ACKNOWLEDGMENTS The author would like to thank two referees and an associate for their comments.

REFERENCES 1. 21CFR312. Available at: http://www. accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/ CFRSearch.cfm?CFRPart=312. 2. R. Temple, Phases of drug development and reviewer role in CDER/FDA, New Reviewers’ Workshop. 1997. 3. TITLE 21–Food and Drugs. Federal Food, Drug, and Cosmetic Act. Available at: http:// www.access.gpo.gov/uscode/title21/chapter9− subchapterv− .html. 4. ICH E8. Available at: http://www.ich.org/ cache/compo/475-272-1.html.

5

SUMMARY

The focus of the article is on descriptions and U.S. legal foundations for phase 2/3 designs. By the U.S. constitution, the FDA interprets and executes laws but has no authority to make laws. Thus, this article serves the purpose of informing regulators from the FDA of their responsibilities to implement the laws despite their personal preference, bias, or opinion. The author is unfamiliar with the legal bases for regulatory agencies of other countries, and therefore, would like to encourage readers to research and report the findings.

5. Q. Liu and G. W. Pledger, Phase 2 and 3 combination designs to accerlerate drug development. J. Am. Stat. Assoc. 2005; 100: 493–502. 6. H. M. J. Hung, R. T. O’Neill, S. J. Wang, and J. Lawrence, A regulatory view on adaptive/flexible clinical trial design. Biometrical J. 2006; 48: 565–573. 7. S. S. Ellenberg, T. R. Fleming, and D. L. DeMets, Data Monitoring Committees in Clinical Trials, A Practical Perspective. West Sussex, UK: Wiley, 2002. 8. R. Hemmings, Philosophy and methodology of dose-finding – a regulatory perspective. In: S. Chevret (ed.), Statistical Methods for DoseFinding Experiments. New York: John Wiley & Sons, 2006.

PHASE 2/3 TRIALS 9. R. E. Bechhofer, T. J. Santner, and D. M. Goldsman, Design and Analysis of Experiments for Statistical Selection, Screening, and Multiple Comparisons. New York: John Wiley & Sons, 1995. 10. Q. Liu, M. A. Proschan, and G. W. Pledger, A unified theory of two-stage adaptive designs. J. Am. Stat. Assoc. 2002; 97: 1034–1041. 11. P. L. Canner, Monitoring of the data for evidence of adverse or beneficial treatment effects. Control. Clin. Trials 1983; 4: 467–483. 12. D. L. DeMets, Stopping guidelines vs stopping rules: a practitioner’s point of view. Commun. Statist. 1984; 13: 2395–2418. 13. P. Gallo and W. Maurer, Challenges in implementing adaptive designs: comments on the viewpoints expressed by regulatory statisticians. Biometrical J. 2006; 48: 591–597. 14. M. A. Proschan, Q. Liu, and S. A. Hunsberger, Practical midcourse sample size modification in clinical trials. Control. Clin. Trials 2003; 24: 4–15. 15. P. Bauer and K. Khone, Evaluations of experiments with adaptive interim analyses. Biometrics 1994; 50: 1029–1041. 16. L. Cui, H. M. J. Hung, and S. J. Wang, Modification of sample size in group sequential clinical trials. Biometrics 1999; 55: 853–857. 17. W. Lehmacher and G. Wassmer, Adaptive sample size calculations in group sequential trials. Biometrics 1999; 55: 1286–1290. 18. M. A. Proschan and S. A. Hunsberger, Designed extension of studies based on conditional power. Biometrics 1995; 51:1315–1324. 19. C. F. Burman and C. Sonesson, Are flexible designs sound? Biometrics 2006; 62: 664–669. 20. D. R. Cox and D. V. Hinkley, Theoretical Statistics. London: Chapman & Hall, 1974. 21. O. Barndorff-Nielsen, Information and Exponential Families in Statistical Theory. New York: Wiley, 1978. 22. R. A. Fisher, On the probable error of a coefficient of correlation deduced from a small sample. Metron 1921; 1: 3–32. 23. R. A. Fisher, Theory of statistical estimation. Proc. Cambridge Philos. Soc. 1925; 22: 700–725. 24. M. A. Proschan, Discussions on ‘‘Are flexible designs sound?’’ Biometrics 2006; 62: 674–676. 25. Q. Liu and G. W. Pledger, On design and inference for two-stage adaptive clinical trials with dependent data. J. Statist. Plan. Inference 2006; 136: 1962–1984.

11

26. W. Brannath, M. Posch, and P. Bauer, Recursive combination tests. J. Amer. Statist. Assoc. 2002; 97: 236–244. 27. Y. H. J. Chen, D. L. DeMets, and K. K. G. Lan, Increasing the sample size when the unblended interim result is promising. Stats. Med. 2004; 23: 1023–1038. 28. R. Royall, Statistical Evidence: A Likelihood Paradigm. London: Chapman & Hall, 1997. 29. M. Frisen, Discussions on ‘‘Are flexible designs sound?’’ Biometrics 2006; 62: 678–680.

FURTHER READING P. Bauer and M. Kieser, Combining different phases in the development of medical treatments within a single trial. Stats. Med. 1999; 18: 1833–1848. W. Bischoff and F. Miller, Adaptive two-stage test procedures to find the best treatment in clinical trials. Biometrika 2005; 92: 197–212. F. Bretz, H. Schmidli, F. Konig, A. Racine, and W. Maurer, Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: general concepts. Biometrical J. 2006; 48: 623–634. D. A. Follman, M. A. Proschan, and N. L. Geller, Monitoring pairwise comparisons in multi-armed clinical trials. Biometrics 1994; 50: 325–336. L. Y. T. Inoue, P. F. Thall, and D. A. Berry, Seamlessly expanding a randomized phase II trial to phase III. Biometrics 2002; 58: 823–831. P. J. Kelly, N. Stallard, and S. Todd, An adaptive group sequential design for phase II/III clinical trials that select a single treatment from several. J. Biopharm. Statist. 2005; 15: 641–658. F. Konig, P. Bauer, and W. Brannath, An adaptive hierarchical test procedure for selecting safe and efficient treatments. Biometrical J. 2006; 48: 663–678. J. Maca, S. Bhattacharya, V. Dragalin, P. Gallo, and M. Krams, Adaptive seamless phase II/III designs – background, operational aspects, and examples. Drug Inf. J. 2006; 40: 463–473. M. Posch, F. Koenig, W. Brannath, C. DungerBaldauf, and P. Bauer, Testing and estimation in flexibile group sequential deisgns with adaptive treatment selection. Stats. Med. 2005; 24: 3697–3714. M. A. Proschan and S. A. Hunsberger, Combining treatment selection and definitive testing. Biometrical J. 2006; 48: 690–692.

12

PHASE 2/3 TRIALS

A. R. Sampson and M. W. Sill, Drop-the-losers design: normal case. Biometrical J. 2005; 3: 257–268. D. J. Schaid, S. Wieand, and T. M. Therneau, Optimal two stage screening designs for survival comparisons. Biometrika 1990; 77: 659–663. H. Schmidli, F. Bretz, A. Racine, and W. Maurer, Confirmatory seamless phase II/III clinical trials with hypotheses selection at interim: applications and practical considerations. Biometrical J. 2006; 48: 635–643. N. Stallard and S. Tood, Point estimates and confidence regions for sequential trials involving selection. J. Statist. Plan. Inference 2005; 135: 402–419.

P. F. Thall, R. Simon, and S. S. Ellenberg, A two-stage design for choosing among several experimental treatments and a control in clinical trials. Biometrics 1989; 45: 537–547. S. Todd and N. Stallard, A new clinical trial design combining phase II and III: sequential designs with treatment selection and a change of endpoint. Drug Inf. J. 2005; 39: 109–118. J. Wang, An adaptive two-stage design with treatment selection using the conditional error function approach. Biometrical J. 2006; 48: 679–689.

PHASE I/II CLINICAL TRIALS

with stable disease. However, Horstmann et al. (5) also pointed out the considerable heterogeneity among phase I trials and the fact that the great majority of patients entering into a phase I trial hold out hope for some benefit, despite its a priori small probability. Based on several phase I studies, Kurzrock and Benjamin (6), in line with other investigators engaged in phase I trials, concluded that reporting of the observed responses was needed. Muggia (7) concluded that ‘‘prior estimates of the risks and benefits of phase I oncology trials need updating and insistence on not conveying therapeutic intent in the informed consent process in all instances is misplaced.’’ To get some insights in the practice of phase I/II trials, a total of 96 English language papers were selected from Medline using ‘‘phase I/II’’ in their title, published from January 1, 2005, to July 1, 2006. They included (1) 32 phase I dose-finding clinical trials using the toxicity as primary endpoint and the efficacy as secondary endpoint; (2) 28 studies first conducted according to a phase I dose-finding, then followed by a phase IIA clinical trial in which an additional subset of patients were treated at the selected MTD or the next lower dose level to get an estimation of the efficacy at that dose level; (3) 27 phase IIA clinical trials using the efficacy as primary endpoint and the toxicity as secondary endpoint; and (4) 9 phase IIB randomized clinical trials using the efficacy as primary endpoint and the toxicity as secondary endpoint. In all cases, toxicity and efficacy were modeled separately (8). Despite discussion about the need to evaluate both toxicity and efficacy in the same trial, in practice, phase I clinical trials are conducted as dose-finding, and phase II trials are designed to examine potential efficacy or any responsive activity based on one dose level recommended from the phase I (9). Recently, there has been increasing research into the development of dose-finding methodologies based on joint modeling of toxicity and efficacy outcomes. Gooley et al. (10) seem to be the first to consider two dose-outcome curves. Over the last 10 years, there has been

SARAH ZOHAR Inserm U717, Biostatistics Department Inserm CIC9504, Clinical Research Center Saint-Louis Hospital Paris, France

Traditional phase I clinical trials aim to identify the maximum tolerated dose (MTD), and phase II clinical trials aim to evaluate the potential clinical activity of a new drug based on the MTD obtained from phase I. Due to the limited sample size in phase I, the MTD might not be obtained in a reliable way, thus effecting the subsequent phases II and III. The final objective is to select an optimal dose, a dose associated with the highest efficacy, given a tolerable toxicity (1). In addition, in early phase dose-finding clinical trials, investigators are increasingly interested in selecting not only a dose level but also a treatment schedule. If two different treatment schedules show the same efficacy, the schedule with less toxicity is preferred (2). Historically, for cancer chemotherapy or cytotoxic agents, the recommendation of the dose level for further studies is based on toxicity considerations. However, this practice is only valid under the assumption that the dose-toxicity and the dose-efficacy relationships are monotonous and increasing; that is, the treatment activity increases with the dose as does potential toxicity (3). Therefore, the MTD estimated in phase I dosefinding trials is assumed to be the dose level associated with the most promising rate of efficacy. Phase I trials are not limited to toxicity observation alone, but frequently the therapeutic response is also recorded even if it is not considered as the primary outcome (4). In a recent review phase I clinical trials, Horstmann et al. (5) concluded that response rates in these trials were almost always recorded and were higher than previously believed. Although reported response rates in the past have been around 4% to 6% with toxicity-related death rates of 0.5% or lower, recently published phase I trials have shown response rates exceeding 10%

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

PHASE I/II CLINICAL TRIALS

a growing body of literature proposing new methodologies using either binary outcomes for toxicity and efficacy (1, 11–17), among others, or a continuous variable for treatment activity while toxicity is assumed to be binary (16, 18, 19). In this article, sequential dose-finding desi gns for phase I/II clinical trials are presented, and prospective or retrospective examples are given as illustration for these methods. 1

TRADITIONAL APPROACH

Traditional phase I/II clinical trials combine a phase I and a phase II trial of the same treatment into a single protocol. They are conducted in two stages, in which the first stage consists in a phase I dose-finding design and the second stage a single-arm phase II approach. The first stage aims at determining the MTD and recommends a dose level for the second stage. In the first stage, a ‘‘standard’’ or 3 + 3 dose-finding algorithm is usually used; the most common and widespread algorithm is as follows. Patients are treated in cohorts of three patients, with a maximum of six patients treated at any dose level and starting at the lowest dose level. For each cohort at a given dose level, (1) if no toxicity is observed over three patients then the next cohort is treated at the next higher dose level; (2) if one toxicity is observed over three patients then the next cohort of three patient is treated at the same dose level; (3) if two toxicities over six are observed then the current dose level is selected as the MTD; or (4) if two or more toxicities are observed over three patients, then the MTD is exceeded and the recommended dose level is the next lowest dose. At the end of the first stage, the dose-finding algorithm is stopped and the recommended dose level from first stage is then used in the second stage. The second stage is a phase II single-arm design in which a predetermined sample size is included. The aim of the second stage is to measure the treatment effect at the recommended dose level from the first stage (20).

2 RECENT DEVELOPMENTS 2.1 Continual Reassessment Method Approaches The continual reassessment method (CRM) was proposed for phase I dose-finding clinical trials by O’Quigley et al. (21). It provides an estimation of the MTD from a fixed set of dose levels, based on sequentially administering the dose level closest to the currently estimated MTD to each newly included patient up to a predetermined number of patients. This method has been shown to be efficient and unbiased (22). Recently, based on this method, a phase I/II design was proposed by O’Quigely et al. in the human immunodeficiency virus (HIV) setting (13). The aim of this method was to (1) model jointly the toxicity and the efficacy of a new treatment; (2) minimize the number of patients treated at unacceptably high toxic dose levels; (3) minimize the number of patients treated at inefficient dose levels; (4) rapidly either escalate dose levels in the absence of indication of drug activity or de-escalate dose levels in the presence of unacceptably high levels toxicity; (5) stop the trial early for efficiency; and (6) minimize the number of patients needed to complete the study. Further, it is assumed that there exists a dose level at which the treatment would be considered as a success in terms of toxicity and efficacy; this dose level is named the most successful dose (MSD). The patient outcome is modeled with two binary random variables: the first corresponds to a toxicity variable, whatever the treatment response is, and the second corresponds to the treatment response conditionally on the absence of toxicity. Before the trial onset, the investigators have to specify an efficacy target (i.e., a minimum response probability) and a toxicity target (i.e., a maximal toxicity probability). Two mathematical modeling procedures have been proposed. In the first, the dose-toxicity relationship and the dose-efficacy relationship are separately modeled using two oneparameter–dependent working models. In the second procedure, a CRM-derived joint

PHASE I/II CLINICAL TRIALS

model is introduced using only one parameter. In both procedures, a sequential ratio test is used as a stopping rule. For the sake of simplicity, only the algorithm of the latter compromise model is exposed: 1. The first cohort is administered at the dose level selected by the investigator. An initial toxicity target is selected. 2. A traditional 3 + 3 escalating scheme is initially selected until heterogeneity in terms of toxicity is reached; that is, at least both one toxicity and one absence of toxicity have been observed. 3. Once heterogeneity has been observed, the subsequent cohorts receive CRMdriven selected doses. 4. In absence of observed efficacy, the current dose and all the doses lower than the current dose are dropped out of the dose range, and the toxicity target is increased within the limits of a prespecified maximum toxicity target level. 5. The recommended dose for subsequent trials corresponds to the dose with both the highest efficacy probability and the toxicity below the accepted maximum threshold. This method was extended by Zohar and O’Quigley (17) in the cancer setting by adding stopping criteria for toxicity. In the Bayesian approach proposed by Ivanova (12), a dose de-escalation is proposed when a toxicity has been observed, whereas a dose-escalation is proposed in absence of both toxicity and efficacy. Finally, Fan and Wang (23) proposed a general theoretic-decision framework that generalizes the CRM and provides two computationally compromised strategies for practical implementation, the One-Step Look Ahead (OSLA) and the Two-Step Look Ahead (TSLA). 2.2 Thall and Russell (TR) and Efficacy-Toxicity Trade-Off Approaches The Thall and Russell (TR) (15) and the tradeoff (14) approaches are based on Bayesian criteria that aim at finding a dose level that satisfies both toxicity and efficacy criteria. In the TR method, the patient’s outcomes are

3

jointly modeled by means of a trinary ordinal variable, corresponding respectively to ‘‘toxicity’’ (whatever the treatment response is) and ‘‘treatment positive response’’ in the absence of toxicity and absence of both treatment response and toxicity. Before the trial onset, the investigators have to specify a minimum efficacy probability target and a maximum toxicity probability target. An underlying mathematical model expresses the probabilities of treatment response and toxicity as interdependent functions of doses (15, 20). The trial consists of sequentially treating cohorts of patients up to a maximum sample size or earlier if stopping criteria are fulfilled. Formally, the trial algorithm is as follows: 1. The first cohort is administered the first dose level, and subsequent cohorts receive dose levels according to Bayesian posterior estimates of efficacy and toxicity outcome probabilities. 2. No dose level skipping occurs unless previous cohorts have been treated at all intermediate dose levels. 3. If the current dose is too toxic and (1) is not the lowest dose level, then the next cohort receives the next lower dose level or (2) is the lowest dose level, then the trial is stopped for unacceptable toxicity for all of the dose range. 4. If the current dose has acceptable toxicity but appears inefficient and (1) the next higher dose level has acceptable toxicity, then the next cohort receives the next higher dose level; or (2) the next higher dose level has unacceptable toxicity, then the trial is stopped for inefficiency under toxicity restrictions; or (3) the current dose is the highest dose level, then the trial is stopped for inefficiency for all of the dose range. 5. If the current dose is associated with acceptable toxicity and efficacy probabilities, the next cohort receives the next higher dose level associated with acceptable toxicity, but with a higher efficacy probability than the current dose. This method allows ending the trial before the inclusion of all of the predetermined number of patients. When either (for all dose

4

PHASE I/II CLINICAL TRIALS

levels) the efficacy remains lower than the desirable target or (for all dose levels) the toxicity remains higher than the maximum acceptable toxicity, or an optimal dose is found. This method was next extended and modified by Zhang et al. (24) to estimate the optimal dose in term of toxicity and efficacy in the setting of specific biologic targets. Because the TR method was limited to trinary outcome, Thall and Cook proposed a Bayesian method in which the outcome can be either bivariate binary (where patients may experience both events—toxicity and efficacy) or trinary (14). In this method, the efficacy probabilities are not necessary monotone or increasing with the dose. Doses are selected for each cohort of patients based on a set of efficacy-toxicity trade-off contours that partition the two-dimensional outcome probability domain (14). As with the TR approach, this method is based on three basic components: (1) a Bayesian model for the joint probability of toxicity and efficacy as a function of dose; (2) decision criteria based on acceptable/unacceptable efficacy or toxicity; and (3) several elicited (toxicity, efficacy) probability pairs of targets. In this case, the targets are used to construct a family of toxicity–efficacy trade-offs contours that provide a basis for quantifying the desirability of each dose; as a result, each newly included cohort receives the most desirable dose computed for posterior estimations (25, 26). Formally, the trial algorithm is as follows: 1. The first cohort is administered the dose level chosen by the physician. The subsequent cohorts receive a dose level according to Bayesian posterior estimates of a trade-off between efficacy and toxicity, given all the observed previous outcomes. 2. No untried intermediate dose may be skipped when escalating. 3. In the absence of an acceptable dose level, the trial is stopped for an unacceptable either low efficacy or high toxicity. 4. In presence of at least one acceptable dose level, the next cohort receives the dose level associated with the maximum desirability among the acceptable doses.

5. If the trial was not stopped earlier, the dose level recommended at the end of the trial is the dose level associated with the maximum desirability among the acceptable doses. The implementation of the method can be easily achieved using dedicated freely available software developed by the authors (27) at the MD Anderson Cancer Center, Houston, Texas. 2.3 Methods Using Continuous Response Outcome In many situations, the treatment activity corresponds to a continuous variable rather than a binary variable, whereas the toxicity is assumed to be binary. A Bayesian dosefinding procedure has been proposed (19, 28) in which the latter binary variable is modeled using a logistic regression function, and the former continuous variable is modeled using a linear regression model. Formally, the trial algorithm is as follows: 1. The total number of subjects is divided into cohorts of equal size c. The first cohort receives a dose level derived from the maximization of a Bayesian prior estimate of a gain function whereas the subsequent cohorts receive dose levels based on successive posterior estimates of the same gain function, given the past outcomes. 2. When the current prior or posterior probability of a dose-dependent event exceeds some predefined value, the dose is not considered as acceptable and is dropped out. 3. The trial is stopped either when the maximum allowed number of subjects is reached, or when none of the selected doses are deemed safe to administer the agent to, or when the procedure is able to select an optimal acceptable dose with sufficient accuracy, whichever event occurs first. At the same time, Bekele and Chen (18) have proposed a dose-finding design for biomarkers, in which both the biomarker

PHASE I/II CLINICAL TRIALS

expression and the toxicity are jointly modeled to determine the optimal dose level. The toxicity is modeled as a binary variable, whereas the treatment effect is modeled as a latent continuous normally distributed variable. 2.4 Other New Approaches Several other approaches were recently published addressing the issue of optimal dosefinding in phase I/II clinical trials. Loke et al. (29) proposed a weighted dose-finding design, based on a parametric utility function associating a real value to each dose level. After the inclusion of each cohort, given the observed outcomes, the expected utility under the posterior distribution is maximized to find the optimal dose for the next cohort. By contrast, Yin et al. (1) have chosen nonparametric functions to model both the dose-toxicity and dose-efficacy relationships. They proposed an adaptive decision procedure based on odds ratio equivalence contours between toxicity and efficacy. 3

ILLUSTRATIONS

3.1 The Cisplatin Combined with S-1 Trial—Illustration of the Traditional Approach A phase l/II clinical trial in advanced gastric cancer was conducted to assess the MTD, the recommended dose, and the objective response rate at the recommended dose of cisplatin combined with S-1, an oral dihydropyrimidine dehydrogenase inhibitory fluoropyrimidine (30). This trial used the standard method as the dose-allocation procedure. Three dose levels of cisplatin were planned: 60, 70, or 80 mg/m2 , all associated with S-1 given orally at 40 mg/m2 . The MTD was defined as the dose at which 33% or more patients experienced dose-limiting toxicities (DLT) during the first course. The first 12 patients were entered into the phase I stage, and the next 13 patients were entered into the phase II stage to confirm the toxicities and efficacy at the recommended dose. All patients were eligible for toxicity evaluation in any course and objective response evaluations. In the phase I stage at level 1, one patient had a DLT, but the other two

5

patients in the same cohort showed no DLT. An additional three patients were enrolled for the safety evaluation, but overall only one of the total of six patients developed a DLT at 60 mg/m2 of cisplatin. At dose level 2, two of six patients exhibited DLTs. Based on these results, dose level 2 was declared as the MTD, and level 1 was declared as the recommended dose in the phase II stage. An additional 13 patients were treated with the same cisplatin dose in the phase II stage. The response rate at the recommended dose at the end of the trial was 73.7% (14 out of 19; 95% confidence interval [CI], 48.8%–90.9%). 3.2 The Interferon Trial—Illustration of the TR Approach A phase l/II clinical trial in metastatic melanoma was conducted to assess the optimal dose in terms of toxicity and efficacy of interferon alpha 2a. This trial used a modified TR method as dose-allocation procedure (15). Four daily dose levels of interferon alpha 2a were planned: 3, 6, 9, and 12 mIU/day. A maximum number of 21 patients was initially allowed. The patient’s tumor response and DLT were jointly modeled using a trinary ordinal variable: DLT, tumor response without toxicity, and absence of both toxicity and tumor response. The target for the minimal efficacy probability was set to 0.20, and the maximal DLT probability was fixed at 0.30. Five cohorts of patients were enrolled in the trial for a total of 16 patients. Three patients were included in each cohort except for the 9 mIU/day cohort which enrolled four patients. Table 1 reports the dose-allocation procedure along the trial. Given the outcomes of these 16 included patients, no acceptable dose level could be allocated, as they were either too toxic or insufficiently efficient without toxicity (Figure 1). Therefore, the trial could be stopped before the inclusion of the maximum predetermined number of patients. At the end of the trial, the estimated posterior limiting toxicity probabilities were 16%, 36.4%, 62.3%, and 81.1%, for the dose levels 3, 6, 9, and 12 mIU/day, respectively; those with efficacy probabilities without limiting toxicities were at 8.2%, 12.3%, 10.7%, and 6.5%, respectively. The dose level of

6

PHASE I/II CLINICAL TRIALS

Doss-efficacy relationship Doss-toxicity relationship Efficacy target Toxicity target

Probability

0.8

0.4

0.0 6

3

9

12

Dose (a)

Probability

0.8

0.4

Probability of too low efficacy Probability of unacceptable toxicity Stopping threshold

0.0 6

3

9

12

Dose (b) Figure 1. Illustration of the TR approach in interferon trial. Dose-efficacy and dose-toxicity relationships were computed at the end of the trial as well as the computation of the stopping criteria. (A) The dose-efficacy and dose-toxicity relationships. (B) The stopping criteria in relation to the dose levels. Table 1. Illustration of the TR Approach: Dose-Allocation Procedure of the Interferon Alpha 2a Trial Cohort 1 2 3 4 5

Number of patients

Dose level (mIU/day)

Outcome by patient


3 3 3 3 4

3 6 6 6 9

0/1/0 2/0/2 0/2/0 0/0/0 2/2/0/0


0 = No dose-limiting toxicity and no tumor response; 1 dose-limiting toxicity; 2 = dose-limiting toxicity.

6 mIU/day was well tolerated but not sufficiently efficient whereas the dose level of 9 mIU/day was too toxic. REFERENCES 1. G. Yin, Y. Li, and Y. Ji, Bayesian dose-finding in phase I/II clinical trials using toxicity

=

Tumor response and no

and efficacy odds ratios. Biometrics. 2006; 62: 777–787. 2. Y. Li, B. Bekele, Y. Ji, and J. D. Cook, DoseSchedule Finding in Phase I/II Clinical Trials Using Bayesian Isotonic Transformation. UT MD Anderson Cancer Center Department of Biostatistics Working Paper Series, Working Paper 26. Berkeley, CA: Berkeley Electronic Press, May 2006. Available at:

PHASE I/II CLINICAL TRIALS http://www.bepress.com/mdandersonbiostat/ paper26 3. W. M. Hryniuk, More is better. J Clin Oncol. 1988; 6: 1365–1367. 4. J. O’Quigley and S. Zohar, Experimental design for phase i and phase i/ii dose-finding studies. Br J Cancer. 2006; 94: 609–613. 5. E. Horstmann, M. S. McCabe, L. Grochow, S. Yamamoto, L. Rubinstein, et al., Risks and benefits of phase 1 oncology trials, 1991 through 2002. N Engl J Med. 2005; 352: 895–904. 6. R. Kurzroc and R. S. Benjamin, Risks and benefits of phase 1 oncology trials, revisited. N Engl J Med. 2005; 352: 930–932. 7. F. M. Muggia, Phase 1 clinical trials in oncology. N Engl J Med. 2005; 352: 2451–2453. 8. S. Zohar and S. Chevret, Recent developments in adaptive designs for phase I/II dosefinding studies. J Biopharm Stat. 2007; 17(6): 1071–83. 9. B. E. Storer, Phase I. In: P. Armitage and T. Colton, (eds.), Encyclopedia of Biostatistics. Chichester, UK: Wiley, 1999, pp. 3365–3375. 10. T. A. Gooley, P. J. Martin, L. D. Fisher, and M. Pettinger, Simulation as a design tool for phase I/II clinical trials: an example from bone marrow transplantation. Control Clin Trials. 1994; 15: 450–462. 11. T. M. Braun, The bivariate continual reassessment method extending the CRM to phase I trials of two competing outcomes. Control Clin Trials. 2002; 23: 240–256. 12. A. Ivanova, A new dose-finding design for bivariate outcomes. Biometrics. 2003; 59: 1001–1007. 13. J. O’Quigley, M. D. Hughes, and T. Fenton, Dose-finding designs for HIV studies. Biometrics. 2001; 57: 1018–1029. 14. P. F. Thall and J. D. Cook, Dose-finding based on efficacy-toxicity trade-offs. Biometrics. 2004; 60: 684–693. 15. P. F. Thall and K. E. Russell, A strategy for dose-finding and safety monitoring based on efficacy and adverse outcomes in phase I/II clinical trials. Biometrics. 1998; 54: 251–264. 16. J. Whitehead, Y. Zhou, J. Steven, and G. Blakey, An evaluation of a bayesian method of dose escalation based on bivariate binary responses. J Biopharm Stat. 2004; 14: 969–983. 17. S. Zoha and J. O’Quigley, Identifying the most successful dose (MSD) in dose-finding studies in cancer. Pharm Stat. 2006; 5: 187–199.

7

18. B. N. Bekele and Y. Shen, A Bayesian approach to jointly modeling toxicity and biomarker expression in a phase I/II dosefinding trial. Biometrics. 2005; 61: 343–354. 19. J. Whitehead, Y. Zhou, J. Stevens, G. Blakey, J. Price, and J. Leadbetter, Bayesian decision procedures for dose-escalation based on evidence of undesirable events and therapeutic benefit. Stat Med. 2006; 25: 37–53. 20. P. F. Thall, E. H. Estey, and H. G. Sung, A new statistical method for dose-finding based on efficacy and toxicity in early phase clinical trials. Invest New Drugs. 1999; 17: 155–167. 21. J. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: a practical design for phase 1 clinical trials in cancer. Biometrics. 1990; 46: 33–48. 22. L. Z. Shen and J. O’Quigley, Consistency of continual reassessment method under model misspecification. Biometrika. 1996; 83: 395–405. 23. S. K. Fan and Y. G. Wang, Decision-theoretic designs for dose-finding clinical trials with multiple outcomes. Stat Med. 2006; 25: 1699–1714. 24. W. Zhang, D. J. Sargent, and S. Mandrekar, An adaptive dose-finding design incorporating both toxicity and efficacy. Stat Med. 2006; 25: 2365–2383. 25. P. F. Thall, J. D. Cook, and E. H. Estey, Adaptive dose selection using efficacy-toxicity trade-offs: illustrations and practical considerations. J Biopharm Stat. 2006; 16: 623–638. 26. P. F. Thall and J. D. Cook, Using both efficacy and toxicity for dose-finding. In: S. Chevret (ed.), Statistical Methods for Dose-Finding Experiments. Chichester, UK: Wiley, 2006, pp. 275–285. 27. Department of Biostatistics and Applied Mathematics, MD Anderson Cancer Center, University of Texas. Efftox [software]. Updated August 15, 2006. Available at: http://biostatistics.mdanderson.org/SoftwareDownload/ 28. Y. Zhou, J. Whitehead, E. Bonvin, and J. W. Stevens, Bayesian decision procedures for binary and continuous bivariate doseescalation studies. Pharm Stat. 2006; 5: 125–133. 29. Y. C. Loke, S. B. Tan, Y. Cai, and D. Machin, A Bayesian dose finding design for dual endpoint phase I trials. Stat Med. 2006; 25: 3–22. 30. W. Koizumi, S. Tanabe, K. Saigenji, A. Ohtsu, N. Boku, et al., Phase I/II study of S-1 combined with cisplatin in patients with

8

PHASE I/II CLINICAL TRIALS advanced gastric cancer. Br J Cancer. 2003; 89: 2207–2212.

CROSS-REFERENCES Phase I trials Phase II trials Maximum tolerable dose (MTD) Continual reassessment method (CRM)

PHASE III TRIALS

Regulatory agencies are charged with evaluating the sponsor’s evidence, and ultimately have the authority to approve a drug for marketing. Because all compounds carry some risk when administered, evaluations must weigh the risk versus the benefit of administering the drug to patients in the indicated population. We will describe considerations involved in optimizing studies to facilitate these evaluations. The design of phase III clinical trials requires careful consideration of multiple factors, which include basic scientific principles as well as the specific medical context within which a drug is developed, including disease characteristics and outcomes as well as other available treatments. We will review some of these factors in our discussion of the appropriate experimental control.

CRAIG MALLINCKRODT VIRGIL WHITMYER Eli Lilly and Company Indianapolis, Indiana

The key to rational drug development is to ask important questions and answer them with appropriate studies in a sequence such that the results of early studies can influence decisions regarding later studies. The sequence of studies in drug development is typically described as having four temporal phases (phases I to IV). The objective of phase III clinical trials is to confirm the preliminary but inconclusive evidence obtained from phase II studies that a drug is safe and effective for the intended indication in the relevant patient population. (The terms ‘‘study’’ and ‘‘trial’’ can be used interchangeably, and we will use them both throughout this article.) Phase III studies provide the primary basis upon which drugs are approved for use by the various regulatory agencies. This article focuses on the key aspects of the design, conduct, and analysis of phase III trials necessary to yield such confirmatory evidence. Many of the concepts discussed in this article are also covered in guideline documents produced by the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) (1). The ICH guidelines describe general considerations used by the various regulatory agencies around the world when evaluating potential new drugs. These guidance documents therefore also provide the primary criteria used by drug developers when designing their research programs. Drugs may be indicated for use in the treatment of a disease (e.g., antibiotics or statins for hypercholesterolemia), the management of symptoms of a disease (e.g., many pain medications that do not treat the underlying cause of pain), or even the management of side effects of another drug (e.g., prevention of nausea caused by chemotherapy).

1

CHOICE OF CONTROL GROUP

Studies are designed to show that a drug is beneficial, where benefit is defined relative to some alternative. If the outcome and time course of a disease is very predictable, the comparison may be to what has happened in the past, in either treated or untreated patients. Such a study is said to have historical control. Itraconazole (Sporanox) was approved for the treatment of blastomycosis and histoplasmosis based on a comparison with the natural history of untreated cases of these conditions (2, 3). Historical control may also be used if no other treatment for the disease in question is available and the consequences of a disease are severe (4). In another example, weekly risedronate was compared with matched historical controls for the prevention of fractures in patients with osteoporosis (5). Matching is a variation on the historical control in which each patient is compared with a patient in a database who had similar relevant illness, prognostic, and/or demographic characteristics. For a study with historical control to provide good evidence of efficacy, the disease outcome should not be affected by the expectations of the patients who receive the drug,

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

PHASE III TRIALS

and the historical outcomes must be good predictors of current outcomes. If evidence existed that the spores causing blastomycosis had developed into a tougher strain than that inhaled by the historical control group, then an historical control group would not have provided valid comparisons in the itraconazole trials. Because the course of many diseases is not uniform or predictable, the benefits of a drug usually cannot be evaluated fairly using an historical control. Therefore, phase III studies typically include a concurrent control. The most common type is placebo control, which has been called the gold standard of experimental design (6, 7). In a placebo-controlled trial, subjects are assigned either to the test treatment or to an indiscernible treatment that does not contain the test drug. The drug and placebo groups are designed to be as similar to one another as possible in every respect, with the exception that the control group recieves an inactive agent in place of the drug received by the test group. Use of a placebo control group allows researchers to differentiate the causal drug effect from those effects resulting from the expectation of improvement associated with taking a drug (placebo effect) and from other supportive care provided in the study. Placebo-controlled phase III studies are quite common across nearly every indication, and examples are found in any recent medical journal. The use of placebo control does not imply that the control group is untreated. The test drug and placebo can each be added to a common standard therapy (add-on therapy), which might be pharmacologic or nonpharmacologic (e.g., counseling). In many disease states, placebo-treated patients show appreciable improvement even when placebo is used alone (not as an add-on therapy). An interesting variation on the use of placebo is the use of ‘‘active placebo.’’ An active placebo is a drug that has some important side-effect usually associated with the test drug, but is known to have no effect on the disease for which the test drug is being tested. Though not a phase III study, low-dose lorazepam was used as an active placebo in a study examining the potential synergy between morphine and gabapentin

for the treatment of pain (8). The investigators argued that the sedating effect of a low dose of lorazepam made it a better choice than inert placebo because sedation was so predictable in the test drugs that its absence would allow investigators and study participants to identify those patients assigned to placebo. An alternative that falls somewhere between historical control and placebo control is to use no-treatment concurrent control, in which subjects are assigned either to test treatment or to no treatment at all. Such a design leaves open the possibility that improvement in the test group is due to features of treatment other than the drug itself, such as nondrug supportive care a patient receives in the study. One reason to accept this reduction in inferential certainty is difficulty in creating a placebo version of the test drug. Dini et al (9) describe a phase III trial using no-treatment concurrent control for the evaluation of a type of physical therapy called pneumatic compression for patients affected by postmastectomy lymph edema. It was not possible to construct an otherwise identical procedure that did not include the intervention under investigation. An important part of the sponsor’s responsibility is to ensure the ethical conduct of trials. For some disease states, exposing patients to placebo in the absence of other treatment or supportive care is clearly not ethical. There are multiple reasons why this may be the case. Studies in oncology, for example, often do not use placebo-only control because it is unethical to withhold treatment from a patient with cancer due to the poor expected outcome without treatment. Similarly, when an adequate standard of care is available, it may become less acceptable to use a placebo control. In the case of the risedronate trial with historical control described previously (5), the investigators argued that placebo control was unethical because adequate treatment alternatives existed. A similar, but more controversial, case has been made in the treatment of major depressive disorder (MDD). Because patients with MDD are at risk for suicide, the cost of leaving MDD untreated can be high. Because the standard of care for MDD includes multiple agents that are effective in the treatment

PHASE III TRIALS

of MDD, some argue that placebo-controlled trials are no longer justified, and that active controlled trials ought to be used in the future for phase III studies (10, 11). Others have concluded that placebo-treated patients are not at increased risk for suicide in MDD clinical trials (12) and that alternatives to placebo control do not, for reasons we will explain, provide adequate evidence of efficacy in MDD. Although the use of placebo or alternatives is controversial in MDD, situations exist where clearly alternatives to placebo are useful. In such cases, a sponsor might choose to conduct a study using an active or positive control, in which a test drug is compared with a known active drug (13, 14). However, difficulties in interpreting results can arise in active comparator studies. For example, assume a test drug was compared with a known effective drug in a two-arm study with the intent of showing the test drug to be superior to the control. Assume, however, that the results suggested that the drugs were not different. The results tell us very little about the efficacy of the test drug. The equivocal result could arise either because the drugs truly had a similar effect or because the drugs were in fact different but the study was, for some reason, unable to detect that difference. A study may lack assay sensitivity—the ability to detect a true difference—for a variety of reasons. Imprecise estimates of treatment effects (due primarily to an unduly small sample size), flawed methods, or high placebo response are common factors that limit assay sensitivity. Lack of assay sensitivity is common, especially for some psychiatric indications, no matter how carefully planned. For example, in phase III antidepressant clinical trials submitted to regulatory agencies to support drug approvals (and therefore designed in accordance with the ICH guidelines) and for compounds that were ultimately found to be efficacious, only about half the comparisons showed a difference between the known effective antidepressants and placebo (15). Note that it is possible, as in Goldstein et al. (13), to include both a placebo control and an active control within the same study. The inclusion of a placebo arm in a study with an active control allows investigators to

3

assess whether a failure to distinguish test drug from the active comparator implies ineffectiveness of the test drug or lack of assay sensitivity. If the two active drugs appear to have similar efficacy but at least one of them is significantly different from placebo, this increases confidence that the two drugs are not substantially different because we have evidence that the study was capable of finding differences. On the other hand, if none of the treatments is significantly different from any of the others, this constitutes evidence that the study lacked assay sensitivity because we know that in fact the active comparator is significantly different from placebo. A different type of active control is doseresponse concurrent control, in which subjects are assigned to one of several dosages of the same drug. A demonstration of a direct correspondence between higher dose and greater effect is a strong argument for a drug’s efficacy. Dose-response studies can simultaneously demonstrate that a treatment effect exists and establish the relationship between dose and effect. Such a study would be appropriate to demonstrate efficacy in a situation in which placebo control is deemed unethical and no other active comparator is available. A variation of this design involves comparing dosing regimens rather than dosages, such as once versus twice daily dosing. As with other active control studies, this type of control can be combined with others, such as a placebo control. 2

CONTROLLING BIAS

The discussion so far has centered on the types of comparison that can show a drug provides some benefit relative to other options. An adequate control group is necessary but not sufficient to ensure an adequate trial. One must also ensure that the test group and the control group are similar at the start of the trial so that differences observed during the trial are due to the effect of the test drug, not differences in the populations studied. If patients receiving test drug have a milder form of the disease than patients in the control group, then the test group may look better at endpoint due to the preexisting difference rather than a difference in

4

PHASE III TRIALS

the test drug’s efficacy. To avoid such selection bias, patients are typically assigned to groups using randomized allocation. In fact, because many factors, known and unknown, might influence treatment outcome, assigning patients randomly helps ensure that the groups are evenly balanced with respect to the factors that influence outcome. Blinding is another important tool used to prevent differences between test and control groups that are not due to the efficacy of the test drug. To blind a subject is to prevent the subject from knowing which treatment group she is in. The idea behind blinding is to reduce the effect of expectation bias—the phenomenon whereby a subject’s expectations influence her outcome. If a patient knows that she is being treated with placebo, for example, she may be less likely to report that she is feeling better. Blinding is particularly important when the outcome of a study is based on subjective interpretations, such as rating pain on a scale of 0 to 10. In a single-blind study, the treatment assignment is not known by the study participant but is known to those conducting the study. In a double-blind study, the treatment assignment is not known by the study participant, the investigator, or sponsor’s staff involved in the study. Double blinding can help avoid tendencies to provide preferential treatment to one group. For example, if investigators knew which patients were on the test drug, a tendency might exist to do things to help the test drug look better or to provide more supportive care for patients on placebo because it was known these patients were not receiving an active medication. Either behavior would skew the results of a study, though in different directions. 3

EVALUATION OF TREATMENT EFFECT

When a sponsor decides that drug Y is likely to be safe and effective for the treatment of

Fact

disease Z, the burden of proof lies with the sponsor. A regulatory agency assumes that drug Y is not effective until it is proven to be effective beyond a reasonable doubt. The fundamental premise behind this system is that it is better to miss out on a safe and effective drug than it is to approve a drug that is not safe and effective. We have described a number of features of adequate study design. Recall that the objective of a phase III study is to show that a test drug provides some benefit relative to some alternative. Let us begin with the gold standard case, the double-blind, randomized, placebo-controlled study. The primary objective of such a study is to confirm the efficacy of drug Y by showing patients treated with drug Y have superior outcomes compared to patients treated with placebo. The null hypothesis, or the starting assumption that the sponsor must disprove, is that patients treated with drug Y do not have superior outcomes to patients treated with placebo. Of course, the sponsor’s actual belief, based on the tentative results of smaller phase II clinical trials, is in the alternative hypothesis that drug Y is effective. The sponsor’s burden is to demonstrate beyond a reasonable doubt that the null hypothesis is false and therefore that the alternative hypothesis is true. To clarify what constitutes reasonable doubt, consider the possible correct or incorrect decisions that can be made regarding the null hypothesis. Based on the possible outcomes we can construct a grid of decisions versus facts, with two possible decisions and two possible facts, as illustrated below: Obviously we want to make the correct decision. There are costs for both erroneous decisions either because patients are not given the option to be treated with an efficacious drug (upper right) or because they take on the risk and cost of treatment with a drug that is not efficacious (lower left). However, the need for certainty must be balanced against the

Decision made Drug effective

Drug effective Drug not effective

True positive (no error) False positive (type I error)

Drug not effective False negative (type II error) True negative (no error)

PHASE III TRIALS

time, expense, and ethics associated with the clinical trial. Requiring a very high degree of certainty, such that the probabilities of making a false-positive or false-negative decision are very low, means we will need to do an extremely large study that will take a long time, be very expensive, and require more patients to be exposed to unproven therapies. Over time, some generally accepted norms have emerged in regards to how certain we need to be in our decisions regarding the outcomes of phase III clinical trials. Most individual phase III clinical trials are set up so that the maximum risk of a falsepositive result favoring drug over control that we are willing to tolerate is 2.5% (1 in 40). This is the requirement of statistical significance, meaning that the P-value obtained from the appropriate statistical test must be 0.05 or less in order to conclude drug is different from control. A P-value of 0.05 equates to a false-positive rate of 5%, but half the false-positive results would favor the control and half would favor the test drug (0.05/2.00 = 0.025, which = 1/40). Given the public health concern associated with falsepositive results, regulatory agencies typically require that a drug be proven effective in two independent studies to be approved for marketing. This independent replication lowers the overall risk of a false-positive result to 1/40 × 1/40 = 1/1600. The science of statistics provides the means for estimating the magnitude of the treatment effect, for assessing the uncertainty in the estimate, and for determining the probability of a false-positive result. If the statistical analyses indicate that the probability of a false-positive result is 5% or less (2.5% favoring drug over control), then that phase III clinical trial is declared to be positive and counts as one of the two trials needed for approval. If the risk of a falsepositive result is greater than 5%, the study is declared to be not positive. If the sponsor believes this result is a false-negative result, they may pursue additional studies to prove the drug is effective. On the other hand, the risk of a falsenegative result in phase III trials is typically capped at 10% to 20%. Statistics also provides the methods for determining how many subjects need to be studied to yield results

5

with sufficient certainty to provide the proper control of the rate of false-positive and falsenegative results. This is referred to as ensuring that the study has adequate power to detect a difference between test drug and placebo. The power of a study is increased by increasing the number of patients or by decreasing the variability in patient outcome. Because the former is more easily controlled, more power is typically achieved by adding more patients. Powering of studies is often dictated by practical and ethical considerations. Increasing the size of the study by a given number of patients has an ever decreasing effect on the power as power increases. Thus, at some point, appreciably increasing power requires large increases in the size of the study, thereby markedly increasing the cost and duration of the study, and requiring many more patients to be exposed to an unproven treatment (and perhaps placebo). 4

CHOICE OF COMPARISON

Thus far, we have considered proving a drug to be effective beyond a reasonable doubt based on demonstrating its superiority to placebo or an active drug, or by demonstrating a dose-response relationship. However, a new drug may be an important advance if it has efficacy similar to a current drug but has improved safety, tolerability, or dosing. A new drug might also be an important advance if it is, on average, similar in effect to the current drug but is more beneficial in certain patient groups. Such a case is especially possible if the new drug works by a different mechanism of action than the current drug. Accordingly, an alternative to establishing the test drug as superior to placebo or an active treatment is to establish the test drug as not inferior to an active treatment. Examples where establishing noninferiority might constitute an important advance include a new treatment for cancer that targets a specific action in cancer cells and has a response rate similar to but with much milder side-effects than a cytotoxic chemotherapy that attacks all fast growing cells. Another example might be treatments for type II diabetes that provide glucose control similar to standard treatments but do not increase

6

PHASE III TRIALS

weight, a side effect that can have multiple consequences, including making it harder in the long run to control the diabetes. Noninferiority trials seek to establish that the test drug is not worse than the active control. Because two treatments are not likely to have exactly the same effects, this is considered to be the case if the outcomes are within some clinically relevant margin of one another (16). An important difference exists between superiority and noninferiority trials with respect to the issue of proof beyond a reasonable doubt. In general, lack of assay sensitivity tends to mute differences between treatments. In superiority trials, lack of assay sensitivity increases the rate of false-negative results. In contrast, lack of assay sensitivity in noninferiority trials tends to bias results toward a conclusion that the test drug is not different from the active control, which in this case would inflate the rate of false-positive results. Recall that in the decision-making framework it is better to have a false-negative result than a false-positive result. Hence, to guard against the possibility of falsely concluding a test drug to be noninferior to an active comparator because the trial lacked assay sensitivity, it is often necessary for a noninferiority trial to explicitly demonstrate assay sensitivity. A common way to do this is to include a placebo control in addition to the active control. If the active control is found to be different from placebo, this proves that the study had enough sensitivity to find a difference as great as that between active control and placebo. This supports the further conclusion that if the difference between test drug and active control were equally great, it too would have been detected. The use of both active and placebo controls also allows pursuit of multiple goals in one trial. For example, a three-arm study with test drug, active, and placebo control could establish superiority of the test drug to placebo and simultaneously evaluate the degree of similarity of efficacy and safety to the active comparator. 5

TYPE OF DESIGN

In addition to controlling bias and determining the type of control and comparison to use,

another important aspect of phase III trials is the type of design. ‘‘Design’’ refers to the pattern of treatments and the order in which subjects receive them over time. 5.1 Parallel Group Design The most common clinical trial design for phase III is the parallel group design, in which subjects are randomized to one of two or more arms and remain on that treatment throughout the trial. Such a design simply assumes that the characteristics of the populations in the various arms are such that, were the treatments equivalent, patient outcomes would be equivalent. Accordingly, any difference at endpoint must be due to some difference between test drug and control. 5.2 Crossover Design In the most simple crossover design, which compares two treatments, each subject receives both treatments in random order, and hence acts as his own control for treatment comparisons. Intuitively, having each patient take both drug A and B and comparing the effects (crossover design) is more reliable than comparing drug A in one group and drug B in a completely separate group (parallel design), because it is more plausible that for the same patient, her outcome would be the same if the two test drugs were equivalent. Indeed, the crossover designs can reduce the number of subjects required to achieve the same level of precision compared with a parallel group design. The crossover design can be used to compare three or more treatments, in which case each patient gets multiple test treatments, though not necessarily all of them. Treatment periods in a crossover design are usually separated by a washout period where patients take no test medications so that they can return to the baseline condition. However, in many disease states and for many medications, carryover effects exist that make it difficult for subjects to return to their baseline condition, which can invalidate results from the later phases of the study. Therefore, in order for a cross-over design to be valid the disease under study must be chronic and stable, the effects of

PHASE III TRIALS

the drug must fully develop within the treatment period, and the washout periods must be sufficiently long for complete reversal of the drug effect. Asthma clinical trials can satisfy these assumptions, and crossover designs have been used in this domain (17). 5.3 Factorial Arrangement of Treatments In a factorial design, two or more treatments are evaluated within a single study by varying combinations of the treatments. The simplest example is the 2 × 2 factorial design in which subjects are randomly allocated to one of the four possible combinations of two treatments, A and B. These combinations are A alone, B alone, both A and B, and neither A nor B. In many cases, this design is used for the specific purpose of examining the way medications A and B interact with each other (synergistic or antagonistic effects). A factorial arrangement of treatments can be used with either a parallel design or crossover design. The Bypass Angioplasty Revascularization Investigation 2 Diabetes study (18) is an example of a parallel group 2 × 2 factorial design. This study compared revascularization combined with aggressive medical treatment versus aggressive medical treatment alone (factor A), while simultaneously comparing two glycemic control strategies, insulin sensitization versus insulin provision (factor B) in patients with type 2 diabetes mellitus and angiographically documented stable coronary artery disease. 6

EVALUATING RISK

The features of study design that have been discussed thus far have generally been related to the demonstration of benefit, by way of demonstrating efficacy for some indication. Although demonstrating efficacy is the primary objective of many phase III studies, a drug’s benefit must be weighed against the risk associated with the compound as well. Many drugs are associated with common adverse events (as or more frequently than 1/10 patients taking the drug), which might include nausea, somnolence, headache, excessive sweating, or any number of other conditions. When the consequences of these events

7

are not severe, the adverse events are noted but not regarded by regulatory agencies as grounds for nonapproval because the benefits of the drug outweigh the risks. Other adverse events may be more severe, and their consequences can outweigh the benefit from treating the disease. In these cases, regulators are likely not to approve the drug for marketing. In such cases, the frequency of an adverse event might be quite low (on the order of 1/10,000), but the cost when it does occur may be so great that the agency nevertheless will not approve the drug. For example, a 10% risk of transient pruritus might be acceptable, but a much smaller risk of seizure or fatal cardiac arrhythmia may be deemed unacceptable. A 1% risk of seizures might be acceptable in a treatment for a highly fatal cancer, but it would likely be unacceptable for less dangerous indications (hair loss, erectile dysfunction) or in diseases for which approved treatments without such risks are available. Although a detailed discussion of assessing the safety of drugs is beyond the scope of this article, it is important to bear in mind a few important facts. Regulatory agencies make decisions about the approval or nonapproval of drugs from a public health standpoint. Is the overall health of the public best served by having this drug available as a treatment option, or is it best served by not having the drug as an option? It would certainly be comforting if regulatory agencies could guarantee the safety of medicines. But quantifying safety risks is extremely difficult, especially for rare events. The ‘‘rule of threes’’ is a useful guide for determining the likelihood of detecting rare events. For example, based on statistical considerations, if the true rate of some serious adverse event is 1 in 1000, you would need to observe 3000 patients to be fairly certain to have observed that one event. Alternatively stated, you need to observe 3000 patients without seeing a single case of a particular adverse event in order to reasonably conclude the incidence of that adverse event is less than 1/1000. This means that to quantify all the risks, especially for rare events, within the context of controlled clinical trials, requires more patient exposure than is practically possible

8

PHASE III TRIALS

before approving a drug. There are always risks associated with medicines, some known and others not. We can reduce the unknown risks by withholding access to drugs until they have been used in very large numbers of patients. For example, we could wait to approve a drug until after it had been approved and used in another country or region for several years. But in the mean time, patients with the disease would not have access to the benefits of the drug. The choice of being an early adopter or a late adopter of drugs is a policy debate, not a scientific debate. To have useful discussions on this topic, it is important to realize that phase III clinical trials simply cannot ‘‘guarantee’’ that a medicine is safe. 7

DISCUSSION

Our discussion of research methods in phase III clinical trials has raised a number of considerations. We’d prefer to not expose patients to placebo, but placebo control enhances our ability to interpret treatment effects whereas active control may lead to more difficulty in interpretation. A crossover design can yield reliable information using fewer patients than a parallel design, but carryover effects limit the applicability of crossover designs. We’d like to be sure of the safety of new drugs, but gathering the enormous volume of data to achieve reliable estimates of risk for rare events could mean withholding a useful treatment from patients who would benefit from it. Clearly, no universally best approach to phase III research exists. Optimizing phase III research relies on management of the inevitable trade-offs in study design, to the extent this is possible. This can involve choice of design features as well as careful consideration of features such as the study population. For example, the risk of suicide is a concern in MDD clinical trials, so we don’t want to expose patients to ineffective therapies. However, placebo enhances our ability to understand treatment effects. This tradeoff can be managed by including placebo but rigorously screening patients at study entry and admitting only those deemed to not be at risk of suicide. Such a study has a rigorous

control group, but may not be representative of a portion of the population with MDD. For reasons such as this, even the best designed trials have limitations. Any one trial may address its primary hypotheses with little or no ambiguity while other important questions remain unresolved. Although we may wish that a single trial could answer all the important questions, this is simply not possible. Rather, we use information from the current trial coupled with knowledge from earlier trials to continually reevaluate what is known. With this knowledge, we then reconsider what important questions remain unresolved, and continue to plan and conduct appropriate studies to address the unresolved questions.

REFERENCES 1. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. Efficacy Guidelines. Updated March 9, 2007. Available at: http://www.ich.org/cache/compo/475272-1.html 2. Sporanox (itraconazole) capsules. Prescribing information. Titusville, NJ: Janssen Pharmaceutica Products, June 2006. 3. A. Glasmacher, C. Hahn, E. Molitor, T. Sauerbruch, I. G. Schmidt-Wolf, and G. Marklein. Fungal surveillance cultures during antifungal prophylaxis with itraconazole in neutropenic patients with acute leukaemia. Mycoses. 1999; 42: 395–402. 4. E. A. Gehan. Design of controlled clinical trials: use of historical controls. Cancer Treat Rep. 1982; 66: 1089–1093. 5. N. B. Watts, R. Lindsay, Z. Li, C. Kasibhatla, and J. Brown, Use of matched historical controls to evaluate the anti-fracture efficacy of once -a-week risedronate. Osteoporos Int. 2003; 14: 437–441. 6. D. Hauschke and I. Pigeot, Establishing efficacy of a new experimental treatment in the ‘‘gold standard’’ design. Biom. J. 2005; 47: 782–786; discussion 787–798. 7. J. C. Rains and D. B. Penzien,Behavioral research and the double-blind placebocontrolled methodology: challenges in applying the biomedical standard to behavioral headache research. Headache. 2005; 45: 479–486.

PHASE III TRIALS 8. I. Gilron, J. Bailey, T. Dongsheng, R. Holden, D. Weaver, and R. Houlden, Morphine, gabapentin, or their combination for neuropathic pain. N. Engl. J. Med. 2005; 352: 1324–1334. 9. D. Dini, L. Del Mastro, A. Gozza, R. Lionetto, O. Garrone, et al., The role of pneumatic compression in the treatment of postmastectomy lymphedema. A randomized phase III study. Ann. Oncol. 1998; 9: 187–190. 10. M. Enserink, Psychiatry: are placebocontrolled drug trials ethical? Science. 2000; 288: 416. 11. KJ. Rothman and KB. Michels, The continuing unethical use of placebo controls. N. Engl. J. Med. 1994; 331: 394–398. 12. A. Khan, H. Warner, and W. Brown, Symptom reduction and suicide risk in patients treated with placebo in antidepressant clinical trials. Arch. Gen. Psychiatry. 2000; 57: 311–317. 13. D. J. Goldstein, C. H. Mallinckrodt, Y. Lu, and M. A. Demitrack, Duloxetine in the treatment of major depression: a double-blind clinical trial. J. Clin. Psychiatry. 2002; 63: 225–231. 14. R. Kane, A. Farrell, R. Sridhara, and R. Pazdur, United States food and drug administration approval summary: bortezomib for the treatment of progressive multiple myeloma after one prior therapy. Clin. Cancer Res. 2006; 12: 2955–2960.

9

15. A. Khan, M. Detke, S. Khan, and C. Mallinckrodt, Placebo response and antidepressant clinical trial outcome J. Nerv. Ment. Dis. 191: 211–218. 16. L. L. Laster and M. F. Johnson, Noninferiority trials: the ‘‘at least as good as’’ criterion. Stat. Med. 2003; 22: 187–200. 17. J. D. Brannan, S. D. Anderson, C. P. Perry, R. Freed-Martens, A. R. Lassig, B. Charlton, and the Aridol Study Group. The safety and efficacy of inhaled dry powder mannitol as a bronchial provocation test for airway hyperresponsiveness: a phase 3 comparison study with hypertonic (4.5%) saline. .Respir. Res. 2005; 6: 144. 18. B. M. Mori, R. L. Frye, S. Genuth, K. M. Detre, R. Nesto, et al., for the BARI 2D Trial. Hypotheses, design, and methods for the Bypass Angioplasty Revascularization Investigation 2 Diabetes (BARI 2D) trial. Am. J. Cardiol. 2006; 97(Suppl): 9G–19G.

CROSS-REFERENCES Phase I Trials Phase II Trials Phase IV Trials Non-inferiority Trials

PHASE II TRIALS

1

OLIVER N. KEENE

PROOF-OF-CONCEPT (PHASE IIA) TRIALS

The initial proof that a new chemical entity (NCE) provides benefit to patients with the disease of interest is often referred to as ‘‘proof of concept.’’ This is an important step for pharmaceutical companies as the level of investment to take an NCE to the next stages (phase IIb dose-ranging trials, phase III trials) is considerable. If development of a drug is halted at an early stage, then considerable savings are made compared with ceasing development later in the process (1, 2). An important issue in proof-of-concept studies is regression to the mean. This occurs because companies will tend to progress the drugs with larger observed effect sizes compared with those that have smaller ones; for example, in a particular indication if a drug with an effect size of x needs an effect size of y to be progressed, then the average effect size of drugs that are progressed will be > x in their proof-of-concept studies. This effect is hard to quantify as the extent of the issue will vary; nevertheless, the tendency will be for proof-of-concept studies to overstate the subsequent effect size observed later.

GlaxoSmithKline Research and Development Greenford, Middlesex, United Kingdom

Design of phase II studies represents a challenge. For proof of concept, sufficient data are needed to allow further progression without imposing impractical targets at this stage of development. For dose-ranging studies, a broad exploration of the dose-response curve is required. Opportunities exist within phase II trials to make use of sequential and adaptive designs where important changes to the trial such as dropping treatment arms or reevaluating sample size are made as a result of interim analyses. Traditional approaches to the analysis of phase II trials that rely on significance testing to compare doses and/or establish efficacy are increasingly being replaced by methods that model the dose-response curve and by analysis based on Bayesian methods. Phase II trials typically represent the first time a new molecule or intervention is investigated in patients with the disease of interest. Before this, the compound will usually have been studied in healthy volunteers (phase I trials), and there may be some proof of pharmacological effect, but it is not until phase II that efficacy in the disease can be explored. The role of phase II trials is to provide initial proof of efficacy, reassurance on safety, and guidance on appropriate doses and dosing regimens before more comprehensive testing of the intervention in phase III trials. Trials in this phase are considered exploratory or for learning, in contrast to phase III trials which are confirmatory. Studies that specifically address the issue of initial evidence of efficacy and safety in the disease in question are often referred to as proof of concept or phase IIa trials; those that investigate different doses or dosing regimens are correspondingly called dose-ranging or phase IIb trials (although these definitions are not universal). Both aims can sometimes be addressed within a single study. These issues are, however, separated for simplicity in this discussion.

1.1 Design Phase IIa trials are typically smaller than phase III trials (20 to 200 patients). They attempt to give the NCE its best chance of showing efficacy compared with a control or placebo treatment. The duration of a phase IIa trial is limited even for a chronic disease and would typically be 1 month or less. Such studies may be completed in a single center or a small number of centers. The use of a single dose reduces study size and can be appropriate where the objective is solely to assess whether the drug has potential and where a later dose-ranging study is planned. This needs to be assessed against the extra information provided from including more than one dose. Doses for phase II studies are chosen based on pharmacokinetic (PK) exposure, results of animal models, and effects on biomarkers or surrogate endpoints in healthy volunteers.

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

PHASE II TRIALS

1.2 Sample Size

1.4 Interpretation of Efficacy Results

Determining the sample size for a proofof-concept study is not straightforward (3). The size of difference to be detected may be unclear, or it may be that power calculations imply a recruitment target that is impractical at this stage of the drug’s development. Increasing the sample size in principle increases the amount of information obtained from the trial, but ‘‘it would not make sense to increase the sample size to the point comparable to that for a confirmatory trial’’ (3). Proof-of-concept studies need to be viewed in the wider context of the clinical development plan for the drug (4). It is helpful to predefine the criteria in terms of efficacy and safety outcomes that would support further progression of the compound (‘‘go/no-go’’ criteria). Such criteria can also be informative in terms of sample size, in that the probability of a ‘‘go’’ decision in a particular trial can be calculated based on predictions of the true efficacy of the drug. Before a phase IIa study, safety data are only available from volunteers. Therefore, the populations initially studied will typically exclude patients who are more at risk of adverse events, such as the elderly or those with other serious diseases. Assessment of safety in these more selected phase II populations can enable the inclusion of more vulnerable patients in phase III.

Interpretation of the outcome of an individual proof-of-concept study involves considerably more than a simple significance test on the primary endpoint. Proof-of-concept studies should emphasize both type I and type II errors. An appropriate balance should be struck between the risk of progressing a nonefficacious drug and the risk of discarding a drug of potential benefit. P-value comparison with significance levels greater than the conventional two-sided 5% level (one-sided 2.5%) may be required. For example, use of a onesided 10% level may be justified (7). Increasing the type I error rate can also allow trials to progress with the reduced sample size required in early phase studies. Estimates of effect size and confidence intervals are critical. For positive studies, these describe the range of treatment effects consistent with the observed data and can guide the team toward a true understanding of the value of the opportunity. Similarly for less positive or negative studies, confidence intervals play a critical role in assessing the risk of stopping development by providing an estimate of the drug’s potential. They are important when design assumptions were overly optimistic, and they can expose the limitations of small studies. If Bayesian rather than frequentist inference is employed, then posterior probabilities play a similar role. A full exploration of the data from an individual phase II study is generally required in addition to analysis of the primary endpoint. Interpretation of significance on secondary endpoints or on subgroups needs to be guided by the extent to which the analyses are preplanned. Where a rigorous analysis plan has been put in place, which appropriately accounts for multiplicity, then such analyses are reliable. Where extensive post hoc analysis has been performed or where there is no explicit plan to account for multiplicity, it is important for all findings from such analyses to be placed in the context of the number of investigations. Because an appropriate assessment of risk is difficult in these situations, careful consideration should always be given to running a second proof-of-concept study to confirm these exploratory analyses.

1.3 Safety and Pharmacokinetic Data Collection of safety data at this stage is key. Often these trials will include intensive safety monitoring, particularly if there are any unresolved safety issues from the preclinical or phase I data. It is also advantageous to include collection of pharmacokinetic samples to enable development of a model for the relationship between the pharmacokinetics (PK) of the drug and the pharmacodynamic (PD) outcome. Trials will often include interim analyses. These analyses allow examination of accumulating safety data to ensure that the safety of the participants is protected. In addition, there may be preplanned futility analyses so that a trial can be stopped if the efficacy is particularly unpromising (5, 6).

PHASE II TRIALS

3

1.5 Role of Bayesian Methods

2.1 Design

Bayesian analysis has potential to aid interpretation of phase II trials (8). Explicit quantification of probabilities that the drug can achieve given effect sizes are especially informative. The credibility of these analyses is enhanced if the methodology is applied with the same rigor as is seen for frequentist analyses (i.e., prespecification of priors and models). Choice of prior distribution might be seen as problematic when little is known on the effect of the drug. An appropriate strategy is to use so-called archetypal priors (9), one representing skepticism and one representing enthusiasm. The extent to which the results of analyses using the two priors converge allows confidence regarding the outcome of the trial.

Dose-response information has been typically obtained from parallel group trials where each patient is given a different dose or placebo. Other designs used include crossover studies where each patient receives more than one dose in a random order (11) or titration designs where patients receive doses of increasing magnitude (12). A key decision in dose-response studies is the number of doses to include. To fully investigate the top and bottom of the doseresponse curve, it is necessary to study a range of doses. A key shortcoming of clinical drug development is often insufficient exploration of the dose response (13). A study with two doses can provide some information, but in practice a minimum of three doses is needed. Preferably however, a dose-ranging study should study at least four doses as well as placebo to fully explore the dose-response curve and to better assess the PK/PD relationship. To justify a particular dose, it is generally necessary to show that a lower dose would not provide similar efficacy, which means in general the chosen dose cannot be the lowest dose used in the study. One common problem is that doses are chosen to be too close together to distinguish responses (10). As well as the treatment arms of an experimental drug, a placebo or comparator is usually included to measure the absolute size of the drug effect (10).

1.6 Interpretation of Safety Results Before further progression, a comprehensive review of safety data is essential. Safety signals can appear from adverse event data, laboratory data, vital signs, and other specific measurements. Where major problems are identified, this can lead to termination of the drug. When definitive evidence of causation of the signal by the intervention is not available, expert evaluation of this emergent safety data is required to ensure that appropriate measures are taken in future studies to minimise the risk to patients and to further evaluate the signal.

2.2 Interpretation of Efficacy Results 2

DOSE-RANGING (PHASE IIB) TRIALS

The purpose of dose-ranging trials is to investigate how the dose of the drug affects both efficacy and safety outcomes. The outcome of a successful dose-ranging study is to determine the dose or doses to be taken forward to phase III trials. A particular difficulty, however, is that for an efficacious drug ‘‘any given dose provides a mixture of desirable and undesirable effects, with no single dose necessarily optimal for all patients’’ (10). As well as providing information on dose, these trials can also address the appropriate dosing regimen, such as whether a drug is better given once a day or twice a day.

Analysis of dose-response trials is controversial. Simple analysis compares doses on a pairwise basis. This approach does not make full use of the information in the trial as it takes no account of the ordering of the doses. However, regulatory requirements can be interpreted as requiring this; the relevant guidance states, ‘‘It should be demonstrated . . . that the lowest dose(s) tested, if it is to be recommended, has a statistically significant . . . effect’’ (10). In principle, if the study includes four or more doses, more information can come from applying a model to the dose-response relationship (14). If such a model establishes that increasing dose leads to increasing efficacy,

4

PHASE II TRIALS

then this should be sufficient (provided that the differences in efficacy observed are of clinical interest). Use of a modeling approach allows use of a greater number of doses as opposed to a pairwise comparison approach which needs sufficient sample size at each individual dose to have adequate power. One strategy is to employ both analyses, with one designated as the primary analysis and the other as a supporting analysis. 2.3 Dropping/Adding Treatment Arms A limitation of a traditional dose-ranging trial is the need to fix a specific limited number of doses in advance. A more flexible approach to dose ranging can be achieved by using what has become known as an ‘‘adaptive design.’’ In this approach, a trial will have one or more interim analyses before the final analysis. At the interim stage, doses can be added or dropped (13, 15). Most examples published to date have been of trials that started with a range of doses and then chose one or two more to progress to the final stage (16). In this situation, as Wang et al. (17) note, ‘‘distributing the remaining sample size of the discontinued arm to other treatment arms can substantially improve the statistical power of identifying a superior treatment arm.’’ The most innovative and well-quoted example of using an adaptive design for dose ranging is the Acute Stroke Therapy by Inhibition of Neutrophils (ASTIN) trial (18, 19) in stroke, which not only adaptively selected doses at multiple interim analyses but also employed Bayesian rather than traditional (frequentist) statistical analysis. The trial terminated early for futility, saving considerable development costs. 2.4 Interpretation of Safety Results As for proof-of-concept trials, a detailed assessment is required, particularly of any safety signals identified. Where possible, the relationship of these signals to the dose administered needs to be reviewed in conjunction with the associated efficacy relationship to determine the optimum doses to take forward into phase III.

3 EFFICACY ENDPOINTS Efficacy endpoints in phase II are often different from standard phase III outcomes. To be useful, they need to be capable of predicting the response in phase III; such endpoints are typically described as surrogate endpoints (20–22). The value of a surrogate is to be less variable and/or to provide a more speedy outcome than a corresponding longer term clinical endpoint. One particular issue with a surrogate endpoint is defining the size of a clinically relevant effect. Another common approach is to use the same endpoints as phase III but measured at an earlier time point after a shorter treatment duration. In some disease areas, where surrogate endpoints are not available, phase II studies can be large (n > 100) and rely on designs that are similar to confirmatory phase III studies—that is, same primary endpoint, similar inclusion criteria, and similar length of treatment and length of assessment. In these circumstances, where clear evidence of useful effect is shown, such a study has the potential to become pivotal in a regulatory submission. Such an interpretation is aided when the potential for the study to become pivotal is identified in advance to regulatory agencies. A trial that simultaneously combines dose selection (possibly at an interim analysis) and provides evidence of efficacy in a clinically important outcome is often described as a phase II/III trial (23). 4 ONCOLOGY PHASE II TRIALS Phase II trials in oncology have a different background from trials in other areas. Because of the special issues involved, they have historically developed in different ways (24, 25). In particular, oncology trials make much greater use of single–arm, open-labeled studies to determine the initial proof of efficacy and the appropriate doses for phase III. Bayesian methods have proved particularly valuable in this area (26). REFERENCES 1. S. Senn, Some statistical issues in project prioritization in the pharmaceutical industry. Stat Med. 1996; 15: 2689–2702.

PHASE II TRIALS 2. S. Senn, Further statistical issues in project prioritization in the pharmaceutical industry. Drug Inf J. 1998; 32: 253–259. 3. C. Chuang-Stein, Sample size and the probability of a successful trial. Pharm Stat. 2006; 5: 305–309. 4. S. A. Julious and D. Swank, Moving statistics beyond the individual clinical trial: applying decision science to optimise a clinical development plan. Pharm Stat. 2005; 4: 37–46. 5. J. M. Lachin, A review of methods for futility stopping based on conditional power. Stat Med. 2005; 24: 2747–2764. 6. S. Snappin, M. G. Chen, Q. Jiang, and T. Koutsoukos, Assessment of futility in clinical trials. Pharm Stat. 2006; 5: 273–281. 7. R. M. Simon, S. M. Steinberg, M. Hamilton, A. Hildesheim, S. Khleif, et al., Clinical trial designs for the early clinical development of therapeutic cancer vaccines. J Clin Oncol. 2001; 19: 1848–1854. 8. N. Stallard, J. Whitehead, and S. Cleall, Decision-making in a phase II clinical trial: a new approach combining Bayesian and frequentist concepts. Pharm Stat. 2005; 4: 119–128. 9. D. Spiegelhalter, L. S. Freedman, and M. K. B. Parmar, Bayesian approaches to randomised controlled trials. J R Stat Soc Ser A Stat Soc. 1994; 157: 357–416. 10. International Conference on Harmonization (ICH). E4: Dose response information to support drug registration. Fed Regist. 1994; 59: 55972–55976. 11. S. J. Ruberg, Dose response studies. I. Some design considerations. J Biopharm Stat. 1995; 5: 1–14. 12. L. Z. Shen, M. S. Fineman, and A. Baron, Design and analysis of dose escalation studies to mitigate dose-limiting adverse effects. Drug Inf J. 2006; 40: 69–78. 13. B. Gaydos, K. Krams, I. T. Perevozskaya, F. Bretz, Q. Liu, et al., Adaptive dose-response studies. Drug Inf J. 2006; 40: 451–461. 14. S. J. Ruberg, Dose response studies. II. Analysis and interpretation. J Biopharm Stat. 1995; 5: 15–42. 15. A. J. Phillips and O. N. Keene, Adaptive designs for pivotal trials: discussion points from the PSI adaptive design expert group. Pharm Stat. 2006; 5: 61–66. 16. N. Stallard and S. Todd, Sequential designs for phase III clinical trials incorporating treatment selection. Stat Med. 2003; 22: 689–703.

5

17. S. J. Wang, H. M. J. Hung, and R. T. O’Neill, Adapting the sample size planning of a phase III trial based on phase II data. Pharm Stat. 2006; 5: 85–97. 18. M. Krams, K. Lees, W. Hacke, A. Grieve, J. M. Orgogozo, and G. Ford, Acute Stroke Therapy by Inhibition of Neutrophils (ASTIN): an adaptive dose-response study of UK-279,276 in acute ischemic stroke. Stroke. 2003; 34: 2543–2548. 19. D. A. Berry, P. Mueller, A. P. Grieve, M. K. Smith, T. Parke, and M. Krams, Bayesian designs for dose-ranging drug trials. In: C. Gatsonis, B. Carlin, and A. Carriquiry (eds.), Case Studies in Bayesian Statistics V. New York: Springer-Verlag, 2001, pp. 99–181. 20. T. R. Fleming, Surrogate endpoints in clinical trials. Drug Inf J. 1996; 30: 545–551. 21. L. J. Lesko and A. J. Atkinson, Jr., Use of biomarkers and surrogate endpoints in drug development and regulatory decision making: criteria, validation, strategies. Annu Rev Pharmacol Toxicol. 2001; 41: 347–66. 22. V. G. De Gruttola, P. Clax, D. L. DeMets, G. J. Downing, S. S. Ellenberg, et al., Considerations in the evaluation of surrogate endpoints in clinical trials: summary of a National Institutes of Health workshop. Control Clin Trials. 2001; 22: 485–502. 23. B. W. Sandage, Balancing phase II and III efficacy trials. Drug Inf J. 1998; 32: 977–908. 24. T. R. Fleming, One sample multiple testing procedures for phase II clinical trials. Biometrics. 1982; 43: 143–151. 25. R. Simon, Optimal two-stage designs for Phase II clinical trials. Control Clin Trials. 1989; 10: 1–10. 26. P. F. Thall and R. Simon, Practical Bayesian guidelines for phase IIB clinical trials. Biometrics. 1994; 50: 337–349.

FURTHER READING S. Senn, Statistical Issues in Drug Development. Chichester, UK: Wiley, 1997, Chapter 20. S. J. Pocock, Clinical Trials: A Practical Approach. Chichester, UK: Wiley, 1983.

CROSS-REFERENCES Dose-response trial Phase II/III trials Adaptive designs End of phase II meeting (FDA)

PHASE I TRIALS

small group of people, a slightly larger dose to the next group, and so on, while closely monitoring the patients for side effects. Escalation continues until a predefined percentage of patients experience unacceptable toxic effects at a given dose level. Nevertheless, such rule-guided designs have been criticized because of potential excessive toxicity. By contrast, model-based designs have been proposed to estimate the MTD directly as the highest dose at which a specified probability of toxic response is not exceeded, which is some prespecified percentile of the monotonic dose-response curve, to be estimated from data. The most popular model-based design is the Continual Reassessment Method (CRM) (4), with a wide literature devoted to extensions and improvements of its original design. Finally, whatever their design, cancer Phase I trials are mainly intended to determine the MTD and the toxicity, not to show whether the treatment is effective. Nevertheless, because patients are used, the most recent dose-finding trials in cancer have attempted to use combined Phase I/II designs (5). We present the Phase I clinical trials conducted in healthy volunteers and oncology patients, respectively, highlighting their differences. Some of the most often used designs are described.

SYLVIE CHEVRET U717 Inserm, France

Phase I includes the initial introduction of an investigational new drug into humans, aiming at developing a safe and efficient drug administration. These first-in-human studies are closely monitored and are usually conducted in an inpatient clinic, where a small number of healthy volunteer subjects (usually, 20–80) can be observed by full-time medical staff. The subject is usually observed until several half-lives of the drug have passed. These studies are designed to determine the metabolic and pharmacologic actions of the drug in humans, the side effects associated with increasing doses, and, if possible, to gain early evidence on effectiveness. Thus, sufficient information about the drug’s pharmacokinetics and pharmacological effects should be obtained during these trials to permit the design of well-controlled, scientifically valid, Phase II studies. However, some circumstances can occur when toxicity is anticipated and patients are used, such as with oncology (cancer) and HIV drug trials (1,2). In these settings, risk is regarded as a necessary price to pay for the chance of benefit (3). Thus, drug therapy is considered tolerable if the toxicity is acceptable. Therefore, the specific goal of such Phase I clinical trials is to find the maximum tolerated dose (MTD) of the new drug for a prespecified mode of administration. Because cancer Phase I trials are considered among the most risky in all of medicine, the design of these trials is surrounded by ethics. Actually, cancer Phase I trials are usually offered to patients who have had other types of therapy and who have few, if any, other treatment choices. They normally include dose-finding studies so that doses for clinical use can be refined, although some other Phase I trials test new combinations or new dose schedules of drugs that are already shown to be effective. The tested range of doses is usually a small fraction of the dose that causes harm in animal testing. Scientists start by giving a low dose to a

1

PHASE I IN HEALTHY VOLUNTEERS

For non-life-threatening diseases, Phase I trials are usually conducted on healthy volunteers. For many compounds, it is anticipated that adverse events will be few, and they will be noted if they occur, but the principal observations of interest will be pharmacokinetic (PK) and pharmacodynamic (PD) measures. They will allow identifying evidence of the intended physiological effect. Little consensus is observed in the design of these first-in-human studies. Buoen et al. (6) reviewed a total of 3323 healthy volunteers trials published in the five major clinical pharmacology journals since 1995. The average trial was placebo controlled, double blind, including 32 subjects at five dose levels, with large heterogeneity in sample size and

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

PHASE I TRIALS

design. Recently, Ting (7) published a book devoted on dose finding in drug development, with large emphasis on Phase I designs in healthy volunteers. 1.1 Single-Dose Studies The parallel single-dose design is still the most common design in healthy volunteer studies, although the use of crossover designs has been reported to increase over time (6). In the latter case, before proceeding to the next treatment period, a washout period is mandatory to avoid carryover effects from one period to another, which requires drugs with long half-lives. Both designs are usually placebo controlled and blinded, with evaluation of data on safety, tolerability, pharmacokinetics, and pharmacodynamics.

the multiple responses of each subject are correlated, and mixed-effects models with random subject effects are used to fit an overall dose-response relationship. By expressing prior information as pseudo-data, the same methodology can be used to perform a Bayesian analysis and to determine posterior modal estimates for the model parameters (8). Such Bayesian decision procedures have been previously proposed for dose-escalation studies in healthy volunteers based on a fixed number of patients, to be treated one at a time (9). The careful selection of these PD along with PK data obtained in Phase I studies allows understanding the dose response data to increase the likelihood of a successful Phase II trial.

1.2 Multiple Ascending Dose Studies

2 PHASE I IN CANCER PATIENTS

The healthy volunteers may be given more than one dose in separate periods with washouts. Designs for dose allocation are either fixed, in which doses are randomly allocated to volunteers, or adaptive, through sequential allocation. Phase I studies that require multiple dose escalation steps have also led to the development of pharmacokinetically guided dose escalation (PGDE) strategies for conducting early clinical trials. They use PK parameters such as concentration-time curve (AUC) or the maximum concentration (Cmax) from the preceding dose group to rationalize the dose increments for escalation. Such PGDE may provide a more rapid and safe completion of the trial. In these studies, the trial starts with a cohort of size 4, three patients treated at the lowest dose level and one on placebo (8). The same cohort participates in three more treatment periods, each individual receives placebo once and otherwise receives the first three dose levels in ascending order. The dose is subsequently escalated up to a predetermined level of plasma concentration. Samples (of blood and other fluids) are collected at various time points and analyzed to understand how the drug is processed within the body. Usually, the PK responses, such as the AUC or Cmax, are, after logarithmic transformation, assumed normally distributed. As each subject is observed more than once,

When dealing with cytotoxic drugs in cancer patients, toxicity is usually thought of as a preliminary desire for increasing efficacy, and it defines the main endpoint of Phase I trials. Patient outcome is usually characterized by a single binary response that indicates whether toxicity occurs within a relatively short time period from the start of therapy. It is assumed that the probability of toxicity increases monotonically with dose of the drug. Several toxicity criteria have been published such as the National Cancer Institute Common Toxicity Criteria Grades or similar scales such as the United States cooperative group or World Health Organization (WHO) scales that can be used in assessing toxicity. Most dose-allocation procedures dichotomize toxicity grades based on being dose limiting. According to symptom/organ categories, investigators might choose any toxicity of WHO grade 3 or above, for instance, for the purpose of dose escalation. This factor is usually referred in the protocol as dose-limiting toxicity (DLT). However, wide variations in the methods used to identify dose-limiting toxicities have been reported (10,11). Conventional Phase I designs are most naturally sequential or adaptive, based on dose allocation rules. Actually, in this oncology setting, the safe dose for additional studies is often interpreted as being the highest

PHASE I TRIALS

dose associated with no adverse outcome, or with ‘‘acceptable’’ toxicity, and it is referred as the MTD (12). Conventionally, the MTD is the highest dose with a sufficiently small risk of DLT. The value of such a dose depends on the design and the sample size used in the trial, so that the MTD is a sample statistic identified from the data. However, from a statistical point of view, the MTD should be thought as a population characteristic rather than calculated from the sample (13). Thus, it is commonly defined as a percentile of the dose-toxicity relationship, for instance the ‘‘toxic dose 20,’’ which is the dose for which the probability of toxicity is 0.20. The choice of the target mostly depends on the drug and pathology under study. See the MTD section for more details. All oncology Phase I designs tend to have formal rules concerning dose escalation following adverse events. Many dose allocation rules have been proposed, and extensive reviews have been already published (14–16). Briefly, two main approaches are commonly reported and opposed, so-called ‘‘algorithm-based,’’ ‘‘rule-guided,’’ or ‘‘datadriven’’ methods on one hand and ‘‘modelbased’’ or ‘‘model-guided’’ methods on the other hand. Although they are obviously different in terms of the underlying philosophy with regard to dose finding and interpretation of the MTD, they share certain designs characteristics. 2.1 Common Characteristics of Phase I Designs in Cancer The ethical basis of Phase I cancer trials has been widely questioned, in part because these trials involve potentially vulnerable patients (17–19). However, most included patients had a good baseline performance status (20). Nevertheless, sample heterogeneity in Phase I sample is commonly observed. A recent review of Phase I oncology trials of cytotoxic agents published from 2002 through 2004 (21) reported that patients included in such trials predominantly had solid tumors (90%), with 9% of studies including only hematologic or lymphatic malignancies, and 1% of studies enrolling both. Among the cancers, five tumor types are mostly encountered, namely cancers of colon or rectum, lung, kidney, breast, and prostate (20).

3

Whatever the rule for dose escalation, a series of ascending dose levels are prespecified. The starting dose, which is the lowest, is usually determined from preclinical animal toxicology data (usually mouse or dog) or on previous clinical experience when available, as the safest. Subsequent dose levels can be either equally spaced or not, increasing the first dose by 100%, then 66%, 50%, 40%, and 33%, according to a ‘‘modified Fibonacci’’ sequence that is used in one quarter of reported Phase I oncology trials (21). In cancer, single ascending dose studies are usually conducted. The objective is to escalate doses until a dose level is identified as suitable for use in later Phase II trials, based on ‘‘acceptable’’ safety. Such single ascending dose studies are those in which cohorts of three or six patients are given a small dose of the drug concurrently, and are observed for a specific period of time. Data from each cohort are collected and analyzed prior to choosing the next dose for administration. If they do not exhibit any adverse side effects, then a new group of patients is given a higher dose. This method continues until intolerable side effects start to develop, at which point the drug is said to have reached the MTD. The mean number of patients included in such trials usually is about 30, with most trials escalating drug doses in four to eight cohorts prior to establishing the MTD or stopping the trial (20). 2.2 Standard ‘‘3 + 3’’ Design The most popular design for dose escalation in cancer Phase I is known as the ‘‘3 + 3’’ design, in which the MTD is identified from the data (22). It is based on the inclusion of three patient cohorts, that is, treating three patients at the same time. Escalation proceeds starting at the lowest available dose until DLT is observed within the cohort. If DLT is observed in two or three patients in the cohort, then the trial is terminated. If only one patient experiences a DLT, then another cohort of three patients is included at the same dose level; if more than two DLTs are observed among the six patients, then the trial is stopped. Otherwise, the escalation continues in the same way. The final

4

PHASE I TRIALS

recommended MTD is the dose one level lower than the final dose used or the stop dose if no more than two DLT were observed on that dose. Of note, at the end of the dose escalation, investigators occasionally enroll additional patients at the MTD to explore the tolerability of the regimen with no explicit statistical justification for doing so. Although the ‘‘3 + 3’’ design has been used for many years and is still widely used, it is only recently that its statistical properties have been explored using exact computation (23). Surprisingly, the identified MTD has unpredictable statistical properties, which tend to give between 10% and 29% of the population toxicity. Indeed, the MTD is not a model-based estimate of the true dose that would yield the targeted dose-limiting toxicity rate. Because the MTD is a sample statistic, it depends on the design and sample size used, with poor properties. In the early 1990s, it seemed that some quantitative assessment design with model-guided determination of the MTD will identify the MTD with fewer patients, more precision, and fewer patients exposed to suboptimal doses.

2.3 Up and Down Designs Otherwise, up-and-down designs assign patient to the next lower dose if toxicities are observed among the group. Such ‘‘A + B’’ designs (24) could be considered in the wider family of random walk rules for the sequential allocation of dose levels to patients in a Phase I clinical trial. Patients are sequentially assigned the next higher, same, or next lower dose level according to some probability distribution, which may be determined by ethical considerations as well as the patient’s response. These designs are simple and do not require any parametric assumptions on the dose-toxicity relationship other than that the probability of toxicity is nondecreasing with dose. Maximum likelihood estimators are derived under a two-parameter logistic distribution (25) so that these designs could appear as intermediate between fully ruleand model-based approaches.

2.4 Continual Reassessment Method Among the model-based designs, the Continual Reassessment Method (CRM) was developed originally to estimate the MTD of a new drug, defined as any targeted quartile of the dose-toxicity relationship (4). Before the trial begins, the investigators express their opinions of the probability of toxicity at each dose level, and these anticipated values (pi ) refer at the ‘‘working model.’’ Although these initial guesses may not be accurate, they provide guidance about dose escalation. The CRM dose allocation rule is as follows: The first cohort of patients, originally one patient at a time, is administered either the initial MTD guess (4) or the first dose level (26–29). Then, the dose level assigned to the next patient cohort is the dose level associated with the actual toxicity probability closest to the target. These probabilities are estimated iteratively after each inclusion from a one-parameter model for the relationship between dose di and probability of toxicity, such as the power model: P(toxicity)= (pi )a , or the logistic model, logit(P(toxicity))= a0 + a di , where the scale parameter a0 is fixed. This model is rerun until the prespecified fixed sample size (usually around 20–30) is reached. Although it has been recommended to be used routinely for cancer Phase I trials, it has not been as widely used as it could be, possibly because of its statistical surrounds (30). The initial CRM actualized toxicity probabilities through the Bayes theorem, which modifies investigator’s prior knowledge with the observed data to derive so-called posterior probabilities (4,26–29). In this Bayesian framework, a prior distribution is required for model parameter, usually noninformative priors from the Gamma family, with the special case exponential for simplicity. Usage of animal toxicity data in determining the prior distribution for the CRM has been proposed (31). Thereafter, O’Quigley and Shen (32) also derived a likelihood-based estimator of the model parameter for the so-called ‘‘likelihoodCRM,’’ and the CRM is no longer a Bayesian method. Of note, maximum likelihood estimation requires a set of heterogeneous responses, so that the first stage is a rulebased design that uses three patient cohorts

PHASE I TRIALS

5

that ends during observation of the first toxicity. To handle ethical concerns better about the nature of potential toxic reactions at nonpreviously tested doses, it has been modified to move more cautiously upward toward the MTD such as never skipping a dose during the escalation (26–28). See O’Quigley et al. (33) and Zhou (15) for a review. Stopping rules have been proposed for the CRM, for instance that allow the trial to stop when the width of the posterior 95% probability interval for the MTD becomes sufficiently narrow, or when no change in selected MTD is expected (14,34–36). The CRM focuses on a single dose very quickly, and it is actually the dose level closest to the MTD with high probability. By contrast to the ‘‘3 + 3’’ design, it has been shown to allocate fewer than 10% patients a dose equal to or exceeding the MTD (4). Of note, because most patients were administered doses around the MTD, the predicted toxicity probability given by the CRM for doses that are not close to the MTD are generally inaccurate.

accruing data during the trial, choosing doses for successive patient cohorts, and selecting a maximum tolerable dose, they seem to be closely related.

2.5 Bayesian Decision Procedures

3 PERSPECTIVES IN THE FUTURE OF CANCER PHASE I TRIALS

Actually, although the CRM gives good estimates of the MTD, it does not capture the whole dose-toxicity curve. Thus, Bayesian decision-theoretic approaches have been proposed to estimate the whole dose-response curve while focusing attention on the target dose (9,37). Therefore, a two-parameter model is assumed to describe how the probability of toxicity increases with dose, usually the logistic model in log-dose (9,38). Prior information with regard to the model parameters is often elicited from investigators, expressed as if ni ‘‘pseudo-observations’’ at dose di , which results in ti toxicities, i = −1, 0, had already been observed. The dose allocation rule is based on the computation of gain functions (38). Among those, note that the ‘‘patient gain,’’ in which each patient within the cohort receives the dose at which the posterior probability of toxicity is as close to the target as possible, is similar to the dose selection criterion used in the CRM. Because both methods use Bayesian probability models as a basis for learning from

2.6 Other Extensions Many proposals have been published that allow the CRM to take into account additional information, accounting for about 100 references in a PubMed search. They incorporate PK data (39), multiple-dose administrations (40) that include that of three different percentiles (denoted low, medium, high) ‘‘LMH-CRM’’ (41), how long the drug has been administered [‘‘TITE-CRM’’ (42)], grade information about the severity of toxicity [‘‘Quasi-CRM’’ (43)], delayed patient outcome (44), as well as two groups of patients (45). At last, but not at least, much research has been devoted to the joint modeling of toxicity and efficacy. As model-based approaches, they could be related to either the CRM [‘‘bivariate CRM’’ (46)] or to Bayesian decision procedures (47). Such combined Phase I/II trials are described elsewhere (5).

Some researchers considered the percentile definition of MTD as inadequate in addressing the ethical question of protecting the patients from severe toxicity. This finding leads to the suggestion by Babb et al. (48) that the MTD should be chosen with toxicity not exceeding the tolerable toxicity with a high probability, which imposes a safety constraint on the overdose control. Over the last decade, cancer drugs under development have become more targeted, and the clinical research environment has become more scrutinized. A few authors have suggested intrasubject dose escalation until the desired biological effect is obtained, even for cancer Phase I trials. Accelerated titration, that is, rapid intra-patient drug dose escalation designs, seems to be of interest in this context (49). This method will allow subjects in the trial to benefit more and providing information on inter-subject variability. For targeted agents with well-defined pharmacodynamic markers that have little or no

6

PHASE I TRIALS

toxicity in the therapeutic dose range, the definition of a maximum tolerated dose can be avoided. This determination has raised questions over the best study designs for Phase I studies, which concerns the design of early phase dose-finding clinical trials with monotone biologic endpoints, such as biological measurements, laboratory values of serum level, and gene expression. A specific objective of this type of trial could be to identify the minimum dose that exhibits adequate drug activity and shifts the mean of the endpoint from a zero dose to the so-called minimum effective dose. Such designs would benefit from research on Phase II designs. This is a promising research area for the next future Phase I trials. 4

DISCUSSION

Phase I oncology trials have been the subject of considerable ethical (18,50) and statistical debate (19,51) for the past 20 years. Whatever the design used, ethical concerns seem fulfilled, and the overall toxic death rate for cancer Phase I studies published in peer-reviewed journals has been reported as low as 0.49% (52) or 0.54% (20), which has decreased over the study period, from 1.1% over 1991–1994 to 0.06% over 1999– 2002 (20). Surprisingly, only a few trials are based on novel designs and such formal methods have received little implementation (11,13). Based on a recent review, only 1.6% of the reported Phase I cancer trials (20 of 1235 trials) followed a model-based design proposed in one of the statistical studies (53). Although simulation studies have shown that up-anddown designs treated only 35% of patients at optimal dose levels versus 55% for Bayesian adaptive designs, the former are still widely used in practice (10). Actually, the practical simplicity of the ‘‘3 + 3’’ approach is likely to be its best pro. By contrast, modelguided escalation appears difficult and practical advice for choosing design parameters and implementing the CRM have been long mandatory. Recently, tutorials that should be helpful have been published (13,30,54). Moreover, the availability of specific software for dose finding should allow developing their use in practice [see the review of

Websites and software to increment Phase I approaches by Zohar (55)]. Finally, most limitations will be overcome best by a close collaboration between clinical pharmacologists, clinicians, and statisticians. REFERENCES 1. B. Storer, Phase I. In: P. Armitage and T. Colton (eds.), Encyclopedia of Biostatistics. Chichester, UK: John Wiley & Sons, 1999, pp. 3365–3375. 2. B. Storer, Phase I trials. In: T. Redmond (ed.), Biostatistics in Clinical Trials. Chichester, UK: John Wiley & Sons, 2001, pp. 337–342. 3. W. M. Hryniuk, More is better. J. Clin. Oncol. 1988;6:1365–1367. 4. J. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: a practical design for phase 1 clinical trials in cancer. Biometrics 1990;46:33–48. 5. S. Zohar and S. Chevret, Recent developments in adaptive designs for phase I/II dose-finding studies. J. Biopharm. Stat. 2007;17:1071–1083. 6. C. Buoen, O. Bjerrum, and M. Thomsen, How first-time-in-human studies are being performed: a survey of phase I doseescalation trials in healthy volunteers published between 1995 and 2004. J. Clin. Pharmacol. 2005;45:1123–1136. 7. N. Ting, Dose finding in drug development. In: M. Gail, et al. (ed.), Statistics for Biology and Health. New York: Springer. 2006. 8. S. Patterson, et al., A novel Bayesian decision procedure for early-phase dose-finding studies. J. Biopharm. Stat. 1999;9:583–597. 9. J. Whitehead and H. Brunier, Bayesian decision procedures for dose determining experiments. Stat. Med. 1995;14:885–893. 10. K. Margolin, et al., Methodologic guidelines for the design of high-dose chemotherapy regimens. Biol. Blood Marrow Transpl. 2001;7:414–432. 11. S. Dent and E. Eisenhauer, Phase I trial design: are new methodologies being put into practice? Ann. Oncol. 1996;7:561–566. 12. B. Storer, Design and analysis of phase I clinical trials. Biometrics 1989;45:925–937. 13. J. Whitehead, et al., Learning from previous responses in phase I dose-escalation studies. Br. J. Clin. Pharmacol. 2001;52:1–7. 14. D. Potter, Adaptive dose finding for phase I clinical trials of drugs used for chemotherapy of cancer. Stat. Med. 2002;21:1805–1823.

PHASE I TRIALS

7

15. Y. Zhou, Choice of designs and doses for early phase trials. Fundam. Clin. Pharmacol. 2004;18:373–378.

30. E. Garrett-Mayer, The continual reassessment method for dose-finding studies: a tutorial. Clin. Trials 2006;3:57–71.

16. S. Chevret, Statistical methods for dosefinding experiments. In: S. Senn (ed.), Statistics in Practice. Chichester, UK: John Wiley & Sons, XXXX.

31. N. Ishizuka and Y. Ohashi, The continual reassessment method and its applications: a Bayesian methodology for phase I cancer clinical trials. Stat. Med. 2001;20:2661– 2681.

17. C. Daugherty, Ethical issues in the development of new agents. Invest. New Drugs 1999;17:145–153. 18. E. Emanuel, A phase I trial on the ethics of phase I trials. J. Clin. Oncol. 1995;13:1049–1051. 19. M. Ratain, et al., Statistical and ethical issues in the design and conduct of phase I and II clinical trials of new anticancer agents. J. Natl. Cancer Inst. 1993;85:1637–1643.

32. J. O’Quigley and L. Shen, Continual reassessment method: a likelihood approach. Biometrics 1996;52:673–684. 33. J. O’Quigley, M. Hughes, and T. Fenton, Dosefinding designs for HIV studies. Biometrics 2001;57:1018–1029. 34. S. Zohar and S. Chevret, The continual reassessment method: comparison of Bayesian stopping rules for dose-ranging studies. Stat. Med. 2001;20:2827–2843.

20. T. J. Roberts, et al., Trends in the risks and benefits to patients with cancer participating in phase 1 clinical trials. JAMA 2004;292:2130–2140.

35. J. O’Quigley, Continual reassessment designs with early termination. Biostatistics 2002;3:87–99.

21. S. Koyfman, et al., Risks and benefits associated with novel phase 1 oncology trial designs. Cancer 2007;110:1115–1124.

36. J. Heyd and B. Carlin, Adaptive design improvements in the continual reassessment method for phase I studies. Stat. Med. 1999;18:1307–1321.

22. N. Geller, Design of phase I and II clinical trials in cancer: a statistician’s view. Cancer Invest. 1984;2:483–491. 23. Y. Lin and W. Shih, Statistical properties of the traditional algorithm-based designs for phase I cancer clinical trials. Biostatistics 2001;2:203–215. 24. A. Ivanova, Escalation, group and A + B designs for dose-finding trials. Stat. Med. 2006;25:3668–3678. 25. S. Durham, N. Flournoy, and W. Rosenberger, A random walk rule for phase I clinical trials. Biometrics 1997;53:745–760. 26. D. Faries, Practical modifications of the continual reassessment method for phase I cancer clinical trials. J. Biopharm. Stat. 1994;4:147–164. 27. S. Goodman, M. Zahurak, and S. Piantadosi, Some practical improvements in the continual reassessment method for phase I studies. Stat. Med. 1995;1:1149–1161. 28. E. Korn, et al., A comparison of two phase I trial designs. Stat. Med. 1994;13:1799–1806. 29. Møller, S., An extension of the continual reassessment methods using a preliminary up-and-down design in a dose finding study in cancer patients, in order to investigate a greater range of doses. Stat. Med. 1995;14:911–922.

37. C. Gatsonis and J. Greenhouse, Bayesian methods for phase I clinical trials. Stat. Med. 1992;11:1377–1389. 38. J. Whitehead and D. Williamson, Bayesian decision procedures based on logistic regression models for dose-finding studies. J. Biopharm. Stat. 1998;8:445–467. 39. S. Piantadosi and G. Liu, Improved designs for dose escalation studies using pharmacokinetic measurements. Stat. Med. 1996;15:1605–1618. 40. A. Legedza and J. Ibrahim, Longitudinal design for phase I clinical trials using the continual reassessment method. Control. Clin. Trials 2000;21:574–588. 41. B. Huang and R. Chappell, Three-dose-cohort designs in cancer phase I trials. Stat. Med. 2007. 42. Y. K. Cheung and R. Chappell, Sequential designs for phase I clinical trials with late-onset toxicities. Biometrics 2000;56:1177–1182. 43. Z. Yuan, R. Chappell, and H. Bailey, The continual reassessment method for multiple toxicity grades: a Bayesian quasi-likelihood approach. Biometrics 2007;63:173–179. 44. P. Thall, et al., Accrual strategies for phase I trials with delayed patient outcome. Stat. Med. 1999;18:1155–1169.

8

PHASE I TRIALS

45. J. O’Quigley and X. Paoletti, Continual reassessment method for ordered groups. Biometrics 2003;59:430–440. 46. T. Braun, The bivariate continual reassessment method. extending the CRM to phase I trials of two competing outcomes. Control. Clin. Trials 2002;23:240–256. 47. P. F. Thall, E. H. Estey, and H.-G. Sung, A new statistical method for dose-finding based on efficacy and toxicity in early phase clinical trials. Investigat. New Drugs 1999;17:155–167. 48. Babb, J., A. Rogatko, and S. Zacks, Cancer phase I clinical trials: efficient dose escalation with overdose control. Stat. Med. 1998;17:1103–1120.

52. E. Horstmann, et al., Risks and benefits of phase 1 oncology trials, 1991 through 2002. N. Engl. J. Med. 2005;352:895–904. 53. A. Rogatko, et al., Translation of innovative designs into phase I trials. J. Clin. Oncol. 2007;25:4982–4986. 54. X. Paoletti, et al., Using the continual reassessment method: lessons learned from an EORTC phase I dose finding study. Eur. J. Cancer 2006;42:1362–1368.

49. R. Simon, et al., Accelerated titration designs for phase I clinical trials in oncology. J. Natl. Cancer Inst. 1997;89:1138–1147. 50. M. Agrawal and E. Emanuel, Ethics of phase 1 oncology studies: reexamining the arguments and data. JAMA 2003;290:1075–1082. 51. E. Korn, et al., Clinical trial designs for cytostatic agents: are new approaches needed? J. Clin. Oncol. 2001;19:265–272.

CROSS-REFERENCES

55. S. Zohar, Websites and software. In: S. Chevret (ed.), Statistical Methods for DoseFinding Experiments. Chichester, UK: John Wiley & Sons, 2006, pp. 289–306.

Continual Reassessment Method (CRM) Maximal tolerable dose Dose escalation design

PHASE I TRIALS IN ONCOLOGY

is given in cycles with multiple administrations of the investigational drug per cycle, sometimes alone, other times in combination with other drugs that are already part of a standard regimen. Patients typically receive multiple cycles, especially if some benefit of the therapy is noted and the patient is tolerating the regimen. From a safety standpoint, toxicity is regularly assessed throughout the course of the patient’s participation in the trial—while the patient is exposed to the study drug as well as typically for at least 30 days after the last day of study drug exposure. However, to shorten the trial duration, toxicities in the first cycle or two are used to determine if the patient or patients in a dose cohort are tolerating the study drug at a particular dose level to allow administration of the next higher dose of study drug to the next cohort of patients. Toxicity in oncology trials is typically graded using the National Cancer Institute Common Terminology Criteria for Adverse Events version 3.0 (available online from the Cancer Therapy Evaluation Program website at http://ctep.cancer.gov). Toxicity is measured on a scale from 0 to 5. The dose-limiting toxicity (DLT) is usually defined as treatmentrelated nonhematologic toxicity of grade 3 or higher, or treatment-related hematologic toxicity of grade 4 or higher. The toxicity outcome is typically binary (DLT/no DLT), for the purpose of dose-escalation considerations. We will use the terms toxicity and DLT interchangeably in this article. The underlying assumption is that the probability of toxicity is a nondecreasing function of dose. From a statistical viewpoint, the maximum tolerable dose (MTD) is defined as the dose at which the probability of DLT is equal to the maximum tolerable level,
. The target level
is usually set to 0.2, though values from 0.1 to 0.35 are sometimes used. Often, no statistical definition of the MTD is given (18), and the MTD can be defined, for example, as the dose level just below the lowest dose where one or two or more patients out of a predetermined number of patients (such as six), experienced DLT.

ANASTASIA IVANOVA University of North Carolina at Chapel Hill Chapel Hill, North Carolina

According to ClinicalTrials.gov, an official clinical trial resource of the National Institutes of Health, phase I clinical trials are initial studies with volunteers or patients to determine the metabolism and pharmacologic actions of drugs in humans, to assess the side effects associated with increasing doses, and to gain early evidence of effectiveness or efficacy. Conducting phase I oncology trials with healthy volunteers would potentially expose them to severe toxicity events. Therefore, phase I trials in oncology are usually conducted with patients, often patients for whom other options do not exist. The goals of phase I trials in oncology are [1] to estimate the maximum tolerable dose of the investigational agent, [2] to determine dose-limiting toxicities, and [3] in many cases, to also investigate the pharmacokinetics and pharmacodynamics of the investigational drug. Most of the existing oncology therapies are cytotoxic. The word cytotoxic means toxic to cells, or cell-killing. Chemotherapy and radiotherapy are the examples of cytotoxic therapies widely used in oncology. A new class of cancer drugs acting as cytostatic agents is being developed. These agents usually target one specific process involved in malignant transformation of cells, and they result in growth inhibition rather than tumor regression. Based on their specific mechanism of action, these target-specific agents are expected to have a more favorable toxicity profile (8). The methods presented in the remainder of the article address mostly the development of cytotoxic agents. 1

DOSE-LIMITING TOXICITY

Most often new oncologic agents are administered intravenously; however, some agents are administered orally. Frequently, therapy

Wiley Encyclopedia of Clinical Trials, Copyright © 2007, John Wiley & Sons, Inc.

1

2

2

PHASE I TRIALS IN ONCOLOGY

STARTING DOSE

The murine LD10 is the dose with approximately 10% mortality, as established in preclinical studies in animals. One-tenth or twotenths of the equivalent of LD10, expressed in milligrams per meter squared, is often used as a starting dose in a phase I trials with agents that have not been tested in humans before. Other choices for the starting dose are possible; for example, a fraction of the toxic dose low—the dose that if doubled causes no lethality in preclinical studies in animals—can be used as a starting dose. Many phase I trials are conducted with agents that have been already studied in humans. In such trials, the starting dose is usually based on experience with the agent in previously conducted human studies.

3

DOSE LEVEL SELECTION

The dose levels are selected by the investigational team, which is typically composed of physicians, pharmacologists, and other clinical development scientists. They use the preclinical and, when available, the clinical data of the investigational product (or data from drugs of the same class), taking into account the particular characteristics of the population of patients who will receive the study agent. The dose levels are chosen before the onset of the trial. The set of doses can be chosen according to the modified Fibonacci sequence, in which higher escalation steps have decreasing relative increments (100%, 67%, 50%, 40%, and 30% thereafter). Other incremental sequences are used as well. Statistical methods have been proposed that will choose dose levels for assignment as long as the first dose level is specified. For example, in the continual reassessment method (16), subsequent dose levels can be computed (at least in theory) from the working dose-toxicity model and priors on model parameters based on the dose assignments and outcomes observed in the trial. Dose levels can also be selected using pharmacology, for example, to achieve a certain area under the curve (5).

4 STUDY DESIGN AND GENERAL CONSIDERATIONS Because most cancer drugs are highly toxic and phase I trials in oncology enroll patients rather than healthy volunteers, the study design in phase I oncology trials differs from phase I dose-finding trials for other therapeutic areas. Usually, the therapeutic effect of a drug is believed to increase with dose; however, the dose of a study drug that a patient can receive is often limited by toxicity. It is important to study as many patients as possible at or near the MTD while significantly minimizing the number of patients who are administered investigational drug at toxic doses. Therefore, in oncology phase I trials, the patients are assigned sequentially, starting with the lowest dose. Usually, in the initial stage, the patients are assigned in relatively small cohorts at increasing dose levels until the first toxicity is seen. Designs for dose finding are often classified as parametric and nonparametric designs. Designs such as the continual reassessment method (16) and the escalation with overdose control (1) are often referred to as parametric designs. Biased coin designs (6, 7), group up-and-down designs (21), and the traditional or 3 + 3 design (14, 19) and its extension A + B designs (15) are frequently referred to as nonparametric designs. Nonparametric designs are attractive to physicians because they are easy to understand and implement because the decision rule is intuitive and does not involve complicated calculations. This might explain the wide use of the traditional or 3 + 3 design. 5 TRADITIONAL, STANDARD, OR 3 + 3 DESIGN The most frequently used design in phase I trials in oncology is the traditional or standard or 3 + 3 design (14, 19). This design is often associated with the set of dose levels obtained using the modified Fibonacci sequence (19). Therefore, this design is sometimes also referred to as the Fibonacci escalation scheme, but this is a misnomer because the Fibonacci sequence defines spacing between doses and not the rule for when to

PHASE I TRIALS IN ONCOLOGY

repeat, increase, or decrease the dose level for the next cohort of patients. The 3 + 3 design can be used with any set of dose levels, regardless of the spacing. In the 3 + 3 design, subjects are assigned in groups of three starting with the lowest dose with the following provisions. If only three subjects have been assigned to the current dose so far, then: 1. If no toxicities are observed in a cohort of three, the next three subjects are assigned to the next higher dose level. 2. If one toxicity is observed in a cohort of three, the next three subjects are assigned to the same dose level. 3. If two or more toxicities are observed at a dose, the MTD is considered to have been exceeded. If six subjects have been assigned to the current dose, then: 1. If at most one toxicity is observed in six subjects at the dose, the next three subjects are assigned to the next higher dose level. 2. If two or more toxicities are observed in six subjects at the dose, the MTD is considered to have been exceeded. The estimated MTD is the highest dose level with observed toxicity rate less than 0.33. The 3 + 3 design does not require sample size specification: the escalation is continued until a dose with unacceptable number of toxicities is observed. The frequency of stopping escalation at a certain dose level in the 3 + 3 design depends on the toxicity rate at this dose as well as the rates at all lower dose levels. The operating characteristics of the 3 + 3 design have been extensively studied for a number of dose-toxicity scenarios by exact computations and simulations (12, 13, 15, 17). On average, the dose chosen by the 3 + 3 design as the MTD has the probability of toxicity of 0.2. Because three to six patients are treated at each dose in the 3 + 3 design, the estimate of the MTD is not precise. Also, the 3 + 3 design does not provide rules regarding the next assignment or how

3

to estimate the MTD if there are dropouts or additional patients are assigned. Designs similar to the 3 + 3 can be constructed for other quantiles
(see the article on escalation and up-and-down designs). Alternatively, designs similar to the 3 + 3 but with different cohort sizes can be constructed to target
= 0.2. 6 CONTINUAL REASSESSMENT METHOD AND OTHER DESIGNS THAT TARGET THE MTD The continual reassessment method (CRM) was developed with the goal of bringing experimentation to the MTD as soon as possible and assigning as many patients as possible to the MTD. The CRM is a Bayesian design proposed in 1990 by O’Quigley et al. (16). (See the article on the continual reassessment method for more details.) Another example of a model-based design for dose-finding studies is the escalation with overdose control (1). This design is from a class of Bayesian feasible designs. It uses a loss function to minimize the predicted amount by which any given patient is overdosed. Cumulative cohort design (9) is a nonparametric design that uses only one assumption that toxicity is nondecreasing with dose. This design can be viewed as a generalization of group designs (21). The decision to decrease, increase, or repeat a dose for the next patient depends on how far an estimated toxicity rate at the current dose is from the target. Stopping rules are not generally used with any of these designs, and hence the specification of the total sample size is required. 7

START-UP RULE

At the beginning of the trial, the 3 + 3 design assigns patients in groups of three at escalating doses until the first toxicity is observed. This is an example of a start-up rule. The start-up rule brings the trial near doses with a toxicity rate close to the target level. It can be advantageous to use a start-up rule before reverting to the CRM or other design selected for the study. One frequently used start-up rule assigns patients in groups of

4

PHASE I TRIALS IN ONCOLOGY

size s at escalating doses until the first toxicity is seen. It is important to avoid escalation that is too rapid or too slow (3). With that goal in mind, the group size s in the start-up should be chosen according to the target toxicity level
(11). Ivanova et al. (11) suggested choosing group size according to the following formula: s=
log(0.5) / log(1−
). For example, if
= 0.5, the start-up with s = 1 is used; if
= 0.3, the start-up with s = 2 is used; and if
= 0.2, the start-up with s = 3 is used. 8

PHASE I TRIALS WITH LONG FOLLOW-UP

Because many of oncology trials are multicycle, there is often a need to follow the patients for DLT for a long period of time. Another example of trials where long-term toxicities are likely are radiation therapy trials. For trials with long follow-up periods, Cheung and Chappell (4) proposed a time-to-event modification of the CRM that they called TITE-CRM. The TITE-CRM allows using the data from all patients in the clinical trial who have received exposure to the investigational agent, not simply the data from patients who completed the safety follow-up phase of the trial. 9

toxicity variable, and the MTD is defined as the dose with a certain weighted sum of probabilities of outcome categories. For example, in the trial described by Ivanova (10), the MTD was defined as the dose where the double rate of DLTs plus the rate of lower grade toxicities that cause dose reductions is equal to 0.5. Bekele and Thall (2) used more sophisticated weighting system and ordinal toxicity variables with up to four categories. They proposed a Bayesian design for such trials.

PHASE I TRIALS WITH MULTIPLE AGENTS

It is common in oncology to treat patients with drug combinations. Often, the dose of one agent is fixed, and the goal is to find the MTD of the other agent administered in combination. Designs for set-ups where doses of both agents vary can be found in the work of Thall et al. (20) and Wang and Ivanova (22). 10 PHASE I TRIALS WITH THE MTD DEFINED USING TOXICITY GRADES Toxicity in oncology is measured on a scale from 0 to 5. Usually the goal is to find a dose at which occurs a certain proportion of DLTs, where DLT is typically defined as study drug–related nonhematologic toxicity of grade 3 or higher or study drug–related hematologic toxicity of grade 4 or higher. In some cases, investigators consider ordinal

REFERENCES 1. J. Babb, A. Rogatko, and S. Zacks, Cancer phase I clinical trials: efficient dose escalation with overdose control. Stat Med. 1998; 17: 1103–1120. 2. B. N. Bekele and F. T. Thall, Dose-finding based on multiple toxicities in a soft tissue sarcoma trial. J Am Stat Assoc. 2004; 99: 26–35. 3. K. Cheung, Coherence principles in dose finding studies. Biometrika. 2005; 92: 863–873. 4. Y. K. Cheung and R. Chappell, Sequential designs for phase I clinical trials with late-onset toxicities. Biometrics. 2000; 56: 1177–1182. 5. J. M. Collins, C. K. Grieshaber, and B. A. Chabner, Pharmacologically-guided phase I trials based upon preclinical development. J Natl Cancer Inst. 1990; 82: 1321–1326. 6. S. D. Durham and N. Flournoy, Random walks for quantile estimation. In: S. S. Gupta and J. O. Berger (eds.), Statistical Decision Theory and Related Topics V. New York: SpringerVerlag, 1994, pp. 467–476. 7. S. D. Durham and N. Flournoy, Up-and-down designs I. Stationary treatment distributions. In: N. Flournoy and W. F. Rosenberger (eds.), Adaptive Designs. Hayward, CA: Institute of Mathematical Statistics, 1995, pp 139–157. 8. R. Hoekstra, J. Verweij, and F. A. Eskens, Clinical trial design for target specific anticancer agents. Invest New Drugs. 2003; 21: 243–250. 9. A. Ivanova, N. Flournoy, and Y. Chung, Cumulative cohort design for dose-finding. J Stat Plan Inference. 2007; 137: 2316–2327. 10. A. Ivanova, Escalation, group and A + B designs for dose-finding trials. Stat Med. 2006; 25: 3668–3678.

PHASE I TRIALS IN ONCOLOGY 11. A. Ivanova, A. Montazer-Haghighi, S. G. Mohanty, and S. D. Durham, Improved upand-down designs for phase I trials. Stat Med. 2003; 22: 69–82. 12. S. H. Kang and C. Ahn, The expected toxicity rate at the maximum tolerated dose in the standard phase I cancer clinical trial design. Drug Inf J. 2001; 35: 1189–1200. 13. S. H. Kang and C. Ahn, An investigation of the traditional algorithm-based designs for phase I cancer clinical trials. Drug Inf J. 2002; 36: 865–873. 14. E. L. Korn, D. Midthune, T. T. Chen, L. V. Rubinstein, M. C. Christian, and R. M. Simon, A comparison of two phase I trial designs. Stat Med. 1994; 13: 1799–1806. 15. Y. Lin and W. J. Shih, Statistical properties of the traditional algorithm-based designs for phase I cancer clinical trials. Biostatistics. 2001; 2: 203–215. 16. J. O’Quigley, M. Pepe, and L. Fisher, Continual reassessment method: a practical design for phase I clinical trials in cancer. Biometrics. 1990; 46: 33–48. 17. E. Reiner, X. Paoletti, and J. O’Quigley, Operating characteristics of the standard phase I clinical trial design. Comput Stat Data Anal. 1999; 30: 303–315. 18. W. F. Rosenberger and L. M. Haines, Competing designs for phase I clinical trials: a review. Stat Med. 2002; 21: 2757–2770.

5

19. B. E. Storer, Design and analysis of phase I clinical trials. Biometrics. 1989; 45: 925–937. 20. P. Thall, R. Millikan, P. Mueller, and S. Lee, Dose finding with two agents in phase I oncology trials. Biometrics. 2003; 59: 487–496. 21. G. B. Wetherill, Sequential estimation of quantal response curves. J R Stat Soc Ser B Methodol. 1963; 25: 1–48. 22. K. Wang and A. Ivanova, Two-dimensional dose finding in discrete dose space. Biometrics. 2005; 61: 217–222.

FURTHER READING S. Chevret, ed., Statistical Methods for Dose Finding. New York: Wiley, 2006. J. Crowley, ed., Handbook of Statistics in Clinical Oncology. New York/Basel: Marcel Dekker, 2006. J. F. Holland and E. Frei, eds., Cancer Medicine, 7th ed. Hamilton, Ontario/London: BC Decker, 2006. N. Ting, ed., Dose Finding in Drug Development. New York: Springer-Verlag, 2006.

CROSS-REFERENCES Escalation and up-and-down designs Continual reassessment method

PHASE IV TRIALS

of their greater variability in purpose, tend to exhibit greater variability in methodology. Defining phase IV only makes sense if we think about the regulatory, licensing, and marketing processes for new medicines. Strictly, phase IV trials are those carried out to investigate some aspect of a medicine that is already licensed. For example, the International Conference on Harmonisation (ICH) E8 guideline (1) describes phase IV studies as being all studies performed after drug approval but ‘‘related to the approved indication.’’ In Pocock’s seminal book on clinical trials (2), however, he tends to consider phase IV trials almost synonymously with postmarketing surveillance, and he implies that they are typically uncontrolled studies and, therefore, not particularly scientifically useful. In another text on the subject, Piantadosi (3) sees such trials as possibly uncontrolled (e.g., to estimate [in absolute terms] the rate of adverse events) or as trials initiated after marketing of a new drug. In contrast to Pocock’s rather negative view, Wang and Bakhai (4) acknowledge the crucial need for phase IV studies to gain additional safety data by examining long-term administration and possible drug interactions. The truly defining feature of a phase IV trial is that it occurs after a medicine has been licensed—Wang and Bakhai, for example, tend to take this point of view, as does Grieve (5). Everitt (6) similarly takes a very broad (and perhaps rather nonspecific) view of phase IV studies, defining them simply as ‘‘Studies conducted after a drug is marketed to provide additional details about its safety, efficacy and usage profile.’’ Perhaps the only point of debate in this definition is whether these studies are carried out after marketing or after initial licensing. However, even setting aside this refinement, this definition itself causes difficulties (as also noted by Grieve, though missed by ICH E8) because it does not differentiate whether a medicine is licensed or marketed in a particular country or region, or whether it is licensed for a particular indication or via a particular route of administration, with certain dosages and dose frequencies. Many clinical trials carried out with marketed products are in view of

SIMON DAY Roche Products Ltd. Welwyn Garden City, United Kingdom

To understand the place of phase IV trials, it is necessary to contrast them with phase III trials and the purpose of phase III. Strictly speaking, phase IV is the phase coming after phase III, but many phase IV studies have all the same look and feel (and in many cases, very similar purposes) as phase III studies. The definitions of different phases make most sense in the context of development of new medicines (typically, although not exclusively, by the pharmaceutical industry), where the end of phase III is generally the point at which an application to health authorities is made for a marketing authorization. However, many clinical trials (typically those using licensed drugs) are sponsored by governments, academia, or charities. Although these inevitably come after the point of initial authorization/ marketing (i.e., postmarketing), they often have objectives that are similar to a phase III study, and their design and execution are very similar to classic phase III studies. 1

DEFINITIONS AND CONTEXT

Before defining what a phase IV trial is, we have to be clear about what a trial is and what it is not. Clinical trials are typically thought of as randomized interventional experiments, where very specific types of patients are enrolled, treatments (or controls or placebos) are blinded, follow-up measures are carefully standardized, a primary endpoint is clearly defined, and so on. However, many of these features are not absolute requirements of a trial, even though they may be desirable attributes. Clinical trials may not be blinded, may not necessarily have a control arm (or arms), and may not have a clearly defined primary endpoint. The better trials typically do have these features, but they are not necessary requirements. Phase IV trials, because

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

PHASE IV TRIALS

expanding their indications or using them in different dosages or in combination with other products. All of these activities necessarily come after initial licensing or in the postmarketing period, but many such trials would look like mainstream phase III trials and might even be used for licensing new indications or dosages. 2 DIFFERENT PURPOSES FOR PHASE IV TRIALS A wide variety of studies might be carried out under the phase IV umbrella term, and they may have very different purposes. This section describes some of the most common scenarios where phase IV studies have been carried out. With the exception of the first example (seeding studies), all may address perfectly legitimate scientific questions for which a good evidence base may be highly desirable. This is not to say that all such questions are always addressed in a high quality scientific way, and we shall see an example of this later. 2.1 Seeding (and Some Other Marketing) Studies Seeding studies, which are rarely seen today, were run by the pharmaceutical industry under the guise of a scientific study as a means of getting physicians to use the product so that they would then later prescribe it to their patients. These studies were called ‘‘seeding’’ because they were akin to planting a seed and then watching it grow in the form of sales and profits. A small amount of data might have been collected to try to give the appearance that it was a useful scientific study, but often not even basic efficacy and safety data were collected. Such studies have been criticized as being marketing activities masquerading as science. Although they are much less common today, it would be na¨ıve to think that those old-fashioned types of marketing study carried out by commercial companies have completely disappeared; there are still examples to be found, which typically are not published because of their lack of any scientific merit. How much value they add varies and is hotly debated. A survey with a clearly defined

sampling frame to precisely (and in an unbiased way) estimate the incidence of particular events could be very valuable, but such studies are not common. Less formal survey-type studies in less clearly defined populations are more common, and their value is less clear. 2.2 Big Numbers, Rare Events A problem with any development plan and the licensing any new medicine is the real possibility that rare—but typically serious—adverse reactions may be experienced by only a small fraction of those taking the medication. Of the many examples, one well-known one was the association found between pediatric aspirin and Reye’s syndrome (7). As noted by Armitage et al. (8), the statistical ‘‘rule of 3’’ stemming from the Poisson distribution states that 300x cases must be observed to be 95% sure of seeing at least one event if the true incidence is x%. So in a typical drug development plan including 2000 to 5000 patients, it is quite plausible that reactions occurring in fewer than 1 in 1000 patients may not be observed. If they are observed, there may only be one case; it would be impossible to exclude, or even imply, causality of the drug under investigation. The rule of 3 makes it obvious that the chances of observing very rare events (e.g., those occurring in fewer than 1 in 10,000 patients) are extremely small. It is not until a new medicine is widely available on the market that such exposure figures are possible. A postmarketing surveillance phase IV study plays an important role in looking for such rare but medically very serious events. 2.3 Chronic Usage As with rare events, there is a limit to what can reasonably be investigated before licensing. Although treatments for long-term use need to be tested for much longer than those for short-term use (e.g., see ICH E1 [9]), it will never be possible to study novel treatments over decades, despite the fact that many patients may use those treatments for such long periods. Furthermore, prophylactic medicines may produce side effects many years after they have been taken. The only way that such possible long-term side effects

PHASE IV TRIALS

can reasonably be detected is after licensing; again, postmarketing surveillance has a useful place. 2.4 Dosage Regimes Although dose finding should be part of the early phase development (typically in phase II), there is often scope for further investigation of uses of different doses of a medicine once it is on the market. Sometimes this may be to investigate different patient-severity groups or for prophylactic use. If the study is carried out by the manufacturer, it may be to get other doses approved; if it is carried out by independent researchers, then it may simply be to better inform, guide, and influence prescribing practice. Both aims are perfectly legitimate and might result in very similar studies being designed and run. 2.5 New Indications As with dose finding, further indications for existing marketed products are a legitimate goal of investigation, either by commercial organizations with a view to being able to make further marketing claims or by noncommercial researchers. Perhaps one of the most widely investigated medicines in terms of the number of patients entering goodquality clinical trials has been aspirin, particularly for its cardioprotective effects (10–12). Because the patent for aspirin has long since expired, there is little commercial interest in further expanding the range of indications for which it has demonstrated efficacy and safety. However, from a public health perspective, there has been a great deal of interest and much resultant benefit. These studies have all been held in the postmarketing era of aspirin, and so they are considered phase IV studies. 2.6 Co-administration A common feature of studies carried out in phase IV is to investigate benefits of drugs used in combination with others. Interaction studies look at possible adverse reactions between drugs, such as interactions with other likely or possible long-term medications, and they are typically required by licensing authorities/health authorities be-

3

fore marketing. However, possible positive synergistic effects between existing medications are often not investigated before first marketing. Often, the possibility of a synergistic effect may only be noticed by serendipity and a few case studies or anecdotal observations may, quite appropriately, provide the impetus to carry out a new study. Examples of such interactions would include aspirin and streptokinase (11) and aspirin and heparin (12). Much of the development in oncology has come about through co-administration of licensed products. Similarly, much of the development of treatment in human immunodeficiency virus/acquired immunodeficiency syndrome (HIV/AIDS) has been through polypharmacy (13). 2.7 Comparisons with Other Medications As has been noted, in many ways the requirements of phase III trials may not answer all of the questions of every consumer of the research— including patients, prescribers, and purchasers. How a new treatment compares with a variety of existing treatments is an important question, but its relevance in drug licensing is not so clear cut. New treatments that are less efficacious than existing treatments may still have a place in the pharmacy; they may cause fewer side effects, or be more convenient to use or less expensive. Usually, each of these criteria can only be considered on an on-average basis; that is, a new treatment, perhaps for a mild but chronic condition, may be less efficacious than an existing one on average, but for some patients the new treatment may actually be better. Hence, absolute measures of efficacy and safety (and so risk–benefit) can be very helpful. Such absolute measures are best evaluated by comparison with placebo. Placebo-controlled trials are necessary for many licensing decisions, but they may not always be sufficient. A new treatment in a critical-care setting that is worse than an existing treatment (even though it may be better than placebo) may represent a health hazard because withholding the better treatment may lead to a fatal or irreversible outcome for the patient. So the reliance on placebo and the use of active comparators in phase III is a complex balance.

4

PHASE IV TRIALS

Notwithstanding these arguments, it is unlikely that, at the time of initial licensing, a new treatment will have been compared with all existing possible treatments for a particular indication. The need for such comparisons—often in very specific patient groups—is real, and phase IV is an ideal time to make such comparisons. 2.8 Quality of Life and Health Economics Quality of life and health economics are very different things but are put together here because they share a common difficulty. Both need to be assessed in as near ‘‘real life’’ conditions as possible. It is well recognized that most clinical trials (certainly most traditional phase III trials) are not carried out in realistic, lifelike, settings but rather in quite artificial settings. Trials are usually carried out by experienced investigators at specialist centers, and the results are sometimes criticized as not being applicable to most physicians and centers. Trials are typically carried out in relatively ‘‘clean’’ patient populations (often excluding the elderly, those on multiple co-medications, etc.), and it is claimed they may not be applicable to a more general patient population. Trials typically monitor patients more closely than would be the case in routine practice, and this may mean that patients’ satisfaction is influenced (either positively or negatively) and the cost of treating patients, even excluding specific tests and investigations required for the trial, may not be representative of routine clinical practice. All of these concerns fall under the heading of generalizability (14). The extent to which results from trials can be generalized is a point of debate, and for many purposes the lack of a formally defined sampling frame that would enable formal inference to an identifiable population may not be too important (15). Although clinical trials may use random allocation, that is not the same as and serves different purposes to random sampling. Phase IV studies aimed at investigating patient acceptability or health economics often have much simpler designs than those used to investigate safety and efficacy, and this simplicity may offer greater generalizability.

3 ESSENTIAL AND DESIRABLE FEATURES OF PHASE IV TRIALS Any piece of research should be fit for its purpose. Moreover, one might argue that particularly ethically sensitive experimental research should be at least adequate to reliably meet its objectives. Without belittling excellence in research, minimizing the unnecessary exposure of human or animal subjects as well as costs must also be important considerations. Hence, what is adequate for one trial may be inadequate or excessive for another. This applies to phase IV trials as well as trials in other phases, and other forms of research. Any study should be of an appropriate size and design to be able to reliably answer its intended question, or at least to contribute usefully to the current state of knowledge. In this respect, phase IV studies should not be seen as of second class importance when compared with phase III studies. Phase III trials often have regulatory hurdles to meet that may force them to be of a certain minimum standard; phase IV studies will not always have such hurdles, but they should still meet the scrutiny of ethics committees and of the general scientific community, including (although not restricted to) journal editors and referees. Hence, the features that we generally associate with a good study should be important for a phase IV study as well. However, that is not to say that randomized controlled trials must necessarily be considered the gold standard for phase IV studies, although they often are. 4 EXAMPLES OF PHASE IV STUDIES This section gives three published examples of studies that have been carried out after licensing of a drug, and so by implication they fall within the scope of phase IV. The first is an example of a ‘‘poor’’ study. Many such poor studies are published, and even more are conducted but not published. This example has not been picked out for any reason other than as representative of the type of study that is not very useful. The second and third examples are much better

PHASE IV TRIALS

although, as with any study at any stage of a drug’s development life-cycle, neither is perfect. 4.1 Trimipramine and Cardiac Safety Cohn et al. (16) studied the antidepressant trimipramine. These investigators were interested in its potential cardiotoxic effects, which seems a perfectly reasonable type of question to ask of a marketed drug and thus the type of question to be considered in phase IV. Sadly, their study has little value in answering such a question. They studied only 22 patients, had no control group, and seemingly had no specific prior hypothesis, only a very broad general objective. Their conclusions about the general cardiac safety of trimaprimine were based on 60 of 61 paired t-tests yielding ‘‘not statistically significant’’ results (P > 0.05). Trials such as this do nothing to advance the credibility, potential value, or importance of phase IV studies, in contrast to the next two studies. 4.2 Cimetidine for Reducing Gastric Acid Cimetidine was the first in its class as a treatment for reducing gastric acid, which meant it received a license for treating symptoms of gastric and duodenal ulcers, reflux, and other related indications. It was first licensed in the United States in August 1977. Because it was of a new class of treatment, there was an expectation that sales would take off quickly. Undoubtedly, the marketing group at the manufacturer, Smith Kline and French, were hopeful for a fast take up. However, very quick uptake of new medicines is always a cause for concern because it means there is not much time to identify possible safety signals before a large number of patients has been exposed. The study aimed to recruit 10,000 patients so that it had a 90% chance of observing at least one event if the true incidence was 1 in 4348; and an 80% chance of observing at least one event if the true incidence was 1 in 6250. (The rule of 3 tells us that the study had a 95% chance of observing at least one event if the true incidence was 1 in 3333.) The patients were recruited by physicians who were encouraged to take part in the study by sales representatives. The

5

physicians recorded all adverse events, hospitalizations, consultations, changes in concomitant diagnoses and therapies, and any new conditions being diagnosed after cimetidine had been prescribed. Data from 7248 patients who took part in the study were available for all of the analyses. A further 663 patients’ data were available for safety analyses. A full review of the results from the study is beyond the scope of this article—it is merely given for illustrative purposes. The results and more details of the design are given in Humphries et al. (17) 4.3 Salmeterol and Salbutamol in Asthma To illustrate that phase IV studies are not restricted to surveillance (or ‘‘survey’’) designs, a randomized, double-blind, parallelgroup trial (very much like one would expect to see as part of a phase III development) examined two asthma treatments developed by Glaxo. The study was not initiated because of any specific concern about these particular treatments but rather because of a general concern about the use of bronchodilators as maintenance treatment. The study was highly pragmatic; after participant randomization, the physicians could coprescribe whatever treatments they considered appropriate for their patients. The primary endpoint measures were any serious adverse events and any reasons (positive or negative) for withdrawal from the study. Over 25,000 patients throughout the United Kingdom were recruited and randomized in a 2:1 fashion to either Salmeterol (50 µg) twice daily or Salbutamol (200 µg) four times a day. Again, we will not consider the study’s results or their interpretation, but it is worth noting that, in addition to comparative statements about the two treatments, the investigators also commented that ‘‘Because of the large numbers we could compare our results with events related to asthma throughout the United Kingdom’’ (18). This, however, was not strictly correct. The representative nature of a sample cannot be ensured simply by collecting a large sample. As previously noted, random allocation is not the same as—and serves a different purpose from—random sampling.

6

5

PHASE IV TRIALS

CONCLUSION

A variety of studies fall under the umbrella term of phase IV. That breadth is partly due to the lack of a satisfactory, unique definition for phase IV studies. Given the scope of phase IV as suggested here—that is, studies that take place after initial regulatory approval has been given—it should be clear that the breadth of studies is so wide because many questions remain unanswered at the time of first licensing. Some of these questions could be critically important but impossible to realistically assess before licensing. Other questions may be of potential importance, such as those examined in trials concerning potential new or extended indications. Still other phase IV studies have no value, and they fail to adequately address questions that themselves may be of dubious merit. Thus, phase IV studies cover a very wide spectrum. REFERENCES 1. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E8 General Considerations for Clinical Trials. Current Step 4 version, July 17, 1997. Available at: http://www.ich.org/LOB/media/MEDIA484 .pdf 2. S. J. Pocock, Clinical Trials: A Practical Approach. Chichester, UK: Wiley, 1983. 3. S. Piantadosi, Clinical Trials: A Methodologic Perspective, 2nd ed. New York: Wiley, 2005. 4. D. Wang, A. Bakhai, Clinical Trials. A Practical Guide to Design, Analysis and Reporting. London: Remedica, 2006. 5. A. Grieve, Phase IV trials. In: B. S. Everitt and C. R. Palmer (eds.), Encyclopaedic Companion to Medical Statistics. London: Hodder Arnold, 2005, pp. 259–260. 6. B. S. Everitt, The Cambridge Dictionary of Statistics. 2nd ed. Cambridge: Cambridge University Press, 2002. 7. A. S. Monto, The disappearance of Reye’s syndrome—a public health triumph [editorial]. New Engl J Med. 1999; 340: 1423– 1424. 8. P. Armitage, G. Berry, and J. N. Matthews, Statistical Methods in Medical Research. 4th ed. Oxford: Blackwell Science, 2002.

9. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonised Tripartite Guideline: E1 The Extent of Population Exposure to Assess Clinical Safety for Drugs Intended for Long-Term Treatment of Non-Life-Threatening Conditions. Current Step 4 version, October 27, 1994. Available at: http://www.ich.org/LOB/ media/MEDIA435.pdf 10. Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto Miocardico (GISSI). Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Lancet. 1986; 1: 397–402. 11. ISIS-2 Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction. Lancet. 1988; 2: 349–350. 12. ISIS-3 Collaborative Group. ISIS-3: a randomised comparison of streptokinase vs tissue plasminogen activator vs anistreplase and of aspirin plus heparin vs aspirin alone among 41,299 cases of suspected acute myocardial infarction. Lancet. 1992; 339: 753–770. 13. E. Vittinghoff, S. Scheer, P. O’Malley, G. Colfax, S. D. Holmberg, and S. P. Buchbinder, Combination antiretroviral therapy and recent declines in AIDS incidence and mortality. J Infect Dis. 1999; 179: 717–720. 14. A. L. Dans, L. F. Dans, G. H. Guyatt, and S. Richardson, Users’ guides to the medical literature. XIV. How to decide on the applicability of clinical trial results to your patient. JAMA. 1998; 279: 545–549. 15. M. Bland, An Introduction to Medical Statistics. Oxford: Oxford University Press, 1987. 16. J. B. Cohn, C. S. Wilcox, and L. I. Goodman, Antidepressant efficacy and cardiac safety of trimaprimine in patients with mild heart disease. Clin Ther. 1993; 15: 114–126. 17. T. J. Humphries, R. M. Myerson, L. M. Giford, M. E. Acugle, M. E. Josie, et al., A unique postmarket outpatient surveillance program of cimetidine: report on phase II and final summary. Am J Gastroenterol. 1984; 79: 593–596. 18. W. Castle, R. Fuller, J. Hall, and J. Palmer, Serevent nationwide surveillance study: comparison of salmeterol with salbutamol in asthmatic patients who require regular bronchodilator treatment. BMJ. 1993; 306: 1034–1037.

PHASE IV TRIALS

FURTHER READING

CROSS-REFERENCES

S. C. Chow and J. P. Liu, Design and Analysis of Clinical Trials. Concepts and Methodologies. New York: Wiley, 1998. P. M. Fayers and D. Machin, Quality of Life. Assessment, Analysis and Interpretation. Chichester, UK: Wiley, 2007. S. Senn, Statistical Issues in Drug Development. Chichester, UK: Wiley, 2007. S. Senn, Pharmaceutical industry statistics. In: C. Redmond and T. Colton (eds.), Biostatistics in Clinical Trials. Chichester, UK: Wiley, 2001, pp. 332–337. B. L. Strom, ed. Pharmacoepidemiology, 4th ed. Chichester, UK: Wiley, 2006.

Active-controlled trial Clinical development plan Combination trials Noninferiority trial Phase III trials Postmarketing surveillance Preference trials Quality of life

7

PHYSICIANS’ HEALTH STUDY (PHS)

1.1 Aspirin Component During the 1970s, Nobel Prize-winning basic research demonstrated that, in platelets, small amounts of aspirin (50–80 mg/day) irreversibly acetylate the active site of the isoenzyme cyclooxygenase-1 (COX-1) (2), which is required for the production of thromboxane A2 (3), a powerful promoter of aggregation (4). This effect persists for the entire life of the platelet, approximately 10–12 days. The evidence of benefit from antiplatelet therapy in secondary prevention was mounting as randomized trials were completed (5–7). In addition, results from a casecontrol study (8) and cohort studies (9,10) were compatible with the possibility that aspirin might reduce risk among individuals without cardiovascular disease. However, all observational studies are subject to uncontrolled and uncontrollable confounding that has a magnitude potentially as large as the effects being sought. PHS I was the first randomized trial to test the postulated benefit of aspirin reliably on many asymptomatic, apparently healthy individuals who could maintain compliance with a regimen for at least 5 years.

J. MICHAEL GAZIANO Harvard Medical School Brigham and Women’s Hospital Massachusetts Veterans Epidemiology and Research Information Center (MAVERIC) Boston VA Healthcare System Boston, Massachusetts

CHARLES H. HENNEKENS Florida Atlantic University University of Miami NOVA Southeastern University Boca Raton, Florida

Launched in 1982, the Physicians’ Health Study (PHS I) was one of the first largescale randomized trials conducted entirely by mail. This primary prevention trial evaluated two hypotheses: whether low-dose aspirin could reduce the risk of cardiovascular disease events and whether beta carotene could lower cancer risk. It incorporated several design strategies to increase efficiency and decrease cost (1). These strategies included the choice of physicians as the study population, the use of a factorial design, the implementation of a prerandomization run-in phase, and the collection of prerandomization blood specimens. The design of PHS I enabled the maintenance of high compliance and long-term follow-up at a fraction of the usual cost of large-scale trials of primary prevention.

1

1.2 Beta Carotene Component By the 1970s, basic research had indicated that the antioxidant properties of beta carotene provided a plausible mechanism for prohibiting carcinogenesis, and observational studies suggested that people who ate more beta carotene-rich fruits and vegetables had a somewhat lower risk of cancer and cardiovascular disease. However, separating the effects of beta carotene from the potential effects of other nutrients, including other dietary antioxidants, is difficult or even impossible in such observational studies. Furthermore, other health factors associated with fruit and vegetable intake might lower risk. A large randomized trial of significant duration was necessary to test this hypothesis directly, and PHS I was the first to do so.

OBJECTIVES

This randomized double-blind, placebo-controlled trial was designed to test the benefits and risks of taking aspirin prophylactically (325 mg on alternate days) for the primary prevention of cardiovascular disease events and the prophylactic use of beta carotene (50 mg on alternate days) for the primary prevention of cancer. By using a 2 × 2 factorial design, it was possible to test two hypotheses simultaneously.

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

2

PHYSICIANS’ HEALTH STUDY (PHS)

STUDY DESIGN

The choice of physicians was integral to both the low cost and the high compliance and follow-up rates associated with this trial. PHS I investigators had conducted several pilot studies between 1979 and 1981 on random samples of U.S. physicians, which showed that doctors would exhibit excellent compliance with their assigned regimen and that they would complete the required questionnaires. The cost per randomized participant for previous primary prevention trials generally ranged from $3,000 to more than $15,000 (11). This cost compares with a cost of approximately $40 per participant per year for PHS I. 2.1 Physicians as Subjects Physicians make excellent study participants for several reasons. They have a clear understanding of informed consent and quickly can recognize any potential side effects from a study regimen. Also, because of their ability to collect and report medical information accurately, it is not only possible but also feasible to conduct a trial by mail. Lastly, physicians are less mobile and therefore easier to follow than the general population, which thereby increases the likelihood that complete morbidity and mortality data could be collected for all participants for the entire duration of a trial. The trial was limited to male physicians because at that

time, a limited number of female physicians were middle aged and older. In addition, the rates of cardiovascular disease events are higher among healthy middle-aged males than among females of the same age. 2.2 Use of a Factorial Design Because there was no reason to believe that aspirin and beta carotene would interfere or interact with one another, the researchers concluded that by incorporating a 2 × 2 factorial design, they could study both agents easily and inexpensively (12). A factorial trial enabled them to evaluate these two interventions, separately and in combination, against a control. PHS participants were assigned to one of four possible combinations: active aspirin and active beta carotene, active aspirin and beta carotene placebo, aspirin placebo and active beta carotene, or aspirin placebo and beta carotene placebo. 2.3 Run-in and Randomization In 1982, 261,248 male physicians between the ages of 40 and 84 years who were identified from an American Medical Association database were sent letters inviting them to participate in the trial along with a baseline questionnaire (Fig 1). A total of 112,528 responded, with 59,285 indicating they were willing to participate. Respondents were deemed ineligible if they had a history of cancer, myocardial infarction, transient ischemic attack, or stroke; had current

261,248 U.S. male MD’s, aged 40-84

112,528 respondents to questionnaires 59,285 willing to participate 33,223 willing and eligible MDs enrolled in run-in (18 weeks on active aspirin and beta carotene placebo) 22,071 randomized

Figure 1. More than 112,000 of the 261,248 male physicians who received a letter inviting them to participate in the trial responded. Of these physicians, 59,285 indicated they were willing to participate, and this group was culled down to 33,223 who were both willing and eligible. After an 18-week run-in phase designed to test compliance, a total of 22,071 physicians were randomized.

PHYSICIANS’ HEALTH STUDY (PHS)

3

Randomization Scheme 22,071 Randomized

11,037 Aspirin

5,517 Beta carotene

11,034 Aspirin placebo

5,520 5,519 Beta carotene Beta carotene placebo

5,515 Beta carotene placebo

Figure 2. Because the study incorporated a 2 × 2 factorial design, the 22,071 participants were assigned to one of four groups: active aspirin plus beta carotene, active aspirin plus beta carotene placebo, aspirin placebo plus beta carotene, or both placebos.

MI in the Physicians's Health Study 250 239 number of MIs

200

44% in first MI due to aspirin

150 139 100 50 0

Placebo (n = 11,034)

Aspirin (n = 11,037)

liver or renal disease; had peptic ulcer or gout; had contraindications to aspirin consumption; were currently using aspirin or other platelet-active drugs or nonsteroidal anti-inflammatory agents; or were currently taking a vitamin A or a beta carotene supplement. A total of 33,223 eligible physicians were enrolled and entered an 18-week ‘‘run-in’’ phase, a pretrial period intended to test compliance with the study protocol to cull the individuals unlikely to take the required pills regularly for a long period of time. Pilot studies in U.S. physicians conducted by the PHS researchers and the initial follow-up from the British Doctors Study (13) both showed that the largest drop in compliance took place during the first several months. PHS investigators felt that the run-in phase would be very beneficial in yielding a group of committed high compliers for the long duration necessary to accrue sufficient cardiovascular

Figure 3. A statistically extreme 44% reduction in the risk of a first myocardial infarction led the PHS I Data and Safety Monitoring Board to stop the aspirin arm of the trial early.

disease endpoints. During this phase, all of the physicians enrolled in the run-in received active aspirin and placebo beta carotene. On completion of the run-in, 11,152 men were excluded, and 22,071 were randomized into the study. The remaining 22,071 physicians were then randomly assigned to one of four groups: active aspirin and active beta carotene (n = 5,517), active aspirin and beta carotene placebo (n = 5,520), aspirin placebo and active beta carotene (n = 5,519), or aspirin placebo and beta carotene placebo (n = 5,515) (Fig. 2). Overall, men in the group that was randomized into the trial were older, leaner, less likely to smoke (11% were current smokers, and 39% were former smokers), more likely to exercise, and less likely to use aspirin. Few other differences existed between the physicians who successfully competed the run-in phase and those who didn’t. Despite that, surprisingly, an analysis showed that, after

4

PHYSICIANS’ HEALTH STUDY (PHS)

adjusting for age, men who were randomized were significantly less likely to die from any cause and significantly less likely to die from either a cardiovascular event or cancer than their peers who also participated in the runin phase but who were not randomized (14).

few events. A significant 18% reduction (P = 0.01) was found in the risk of important vascular events [combined endpoint consisted of nonfatal myocardial infarction (MI), nonfatal stroke, and cardiovascular disease (CVD) death].

2.3.1 Prerandomization Blood Sample Collection. During the 4-month run-in period, participants were sent kits that contained prepared Vacutainer tubes, cold packs for mailing, and prepaid shipping packs. They were asked to have their blood drawn into these tubes, to fractionate the blood by centrifugation, and to return the samples by overnight courier. On receipt in the PHS blood laboratory, samples were immediately frozen and stored in liquid nitrogen freezers. Almost 15,000 of the participants (68%) provided blood samples. After PHS I ended, participants were asked to provide a second blood sample. More than 11,000 participants did. These samples were frozen (after being processed and aliquoted) and used to explore several biochemical and genetic hypotheses.

4.2 Beta Carotene

3

FOLLOW-UP AND COMPLIANCE

The first follow-up questionnaires were sent at 6 and 12 months of treatment, then yearly. They included questions about compliance with the study treatments, use of nonstudy medications, occurrence of major illnesses or adverse effects, and other risk factor information. By December 31, 1995, the scheduled end of the beta carotene arm of the study, fewer than 1% had been lost to follow-up, and compliance was 78% in the group that received beta carotene. 4 RESULTS 4.1 Aspirin The Data and Safety Monitoring Board stopped the aspirin component of PHS I after 5 years primarily because aspirin had a significant effect on the risk of a first myocardial infarction— reducing it by 44% (P < 0.00001) (Fig. 3). Results for stroke and total mortality were uninformative because there were too

Results from the beta carotene arm were equally important and demonstrated that 12 years of supplementation with beta carotene produced neither benefit nor harm. No differences were found in cardiovascular mortality (RR, 1.09; 95% CI, 0.93–1.27), MI (RR, 0.96; 95% CI, 0.84–1.09), stroke (RR, 0.96; 95% CI, 0.83–1.11), or a composite of the three endpoints (RR, 1.00; 95% CI, 0.91–1.09) associated with beta carotene assignment. In addition, no relationship was found between beta carotene and cancer mortality, malignant neoplasms, or lung cancer. In analyses limited to current or former smokers, no significant early or late effects of beta carotene on any endpoint of interest were detected. 4.3 Other Findings from the PHS More than 250 papers based on data from the Physicians’ Health Study have been published. Other results that have emerged from PHS I include: • Regular use (more than 60 days/year) of

•

• •

•

•

ibuprofen and other nonsteroidal antiinflammatory drugs may interfere with the cardioprotective effects of aspirin. Aspirin does not cause kidney damage, even among individuals who, over time, consume large quantities of aspirin. Aspirin may reduce the risk of asthma. Exercise helps prevent type 2 diabetes, especially in men who work out at least five times a week. Individuals who smoke one or more packs of cigarettes a day have twice the risk of macular degeneration, an eye disease that can cause loss of vision. Men with high blood levels of homocysteine have an increased risk of cardiovascular disease.

PHYSICIANS’ HEALTH STUDY (PHS) • Participants who eat whole grain break-

fast cereals have reduced total and cardiovascular mortality. • Elevated levels of insulin-like growth factor (IGF-1) are a powerful risk factor for prostate cancer; IGF-1 is also strongly linked to colon cancer risk.

5

CONCLUSIONS

PHS I was the first randomized trial to demonstrate that daily aspirin produces a statistically significant and clinically important reduction in risk of a first myocardial infarction. The trial was also the first to show that beta carotene has no significant benefit or harm on cancer or cardiovascular disease. This finding is particularly informative for cancer as the 12-year duration of the trial is far longer than any other randomized trial of beta carotene. At the time that the results from the aspirin component of the trial were published in 1989, only one other primary prevention trial of aspirin had been completed. In contrast to the Physicians’ Health Study I, the British Doctors’ Trial (13) showed no significant differences for fatal or nonfatal myocardial infarction, but the 95% confidence intervals were very wide. In the intervening years, four more large-scale trials have assessed the benefits of low-dose aspirin in the prevention of CVD. Taken together, these six randomized trials indicate a benefit of prophylactic aspirin in primary prevention of myocardial infarction and possibly ischemic stroke (15). The 10-year risk of a first coronary event in the subjects in these trials is less than 10% in five of the six. Because the benefits of aspirin are likely to outweigh the risks in individuals with a 10-year risk of a first coronary event of 10% or greater, several trials are now beginning to address the balance of benefits and risks of aspirin in populations with a 10-year risk of a first coronary event between 10% and 19%. The results from PHS I demonstrated that beta carotene alone was not responsible for the health benefits seen among people who ate plenty of fruits and vegetables. It was only one of several large randomized,

5

placebo-controlled trials that have been conducted since the 1980s to assess the role of beta carotene in cancer prevention. PHS I also found that supplementation with beta carotene had no effect on cardiovascular disease. Taken as a whole, results from largescale randomized trials of beta carotene in the primary prevention of CVD and of cancer have been disappointing. However, it is interesting to note that a recent study based on long-term consumption of beta carotene—12 or more years—among PHS participants has found that individuals with high intakes of beta carotene had improved cognitive skills (16). Over time, it has become clear that the physicians who have participated in the PHS I are healthier than the general U.S. population and also healthier than their physician peers who were invited to participate in the study but, for a variety of reasons, either didn’t or couldn’t. Certainly, most physicians have the advantage of excellent access to health care, which gives them a significant leg up over nonphysicians, so it is not surprising that doctors would be healthier than other Americans. What was surprising, however, was the difference between the 22,071 willing and eligible physicians who successfully completed the pretrial runin phase and their willing and eligible compatriots—11,152 men—who did not and were excluded, therefore, from the trial. PHS I ended on December 31, 1995, 13 years after it began in earnest when a computer assigned a Florida physician to a combination of beta carotene and placebo. However, 7,641 of the original participants continued on as part of the ongoing Physicians’ Health Study II (PHS II), which began in 1997 with nearly 15,000 participants. As the Physicians’ Health Study marks its 25th anniversary in 2008, most of the PHS participants continue to provide information on their health status. Initially, PHS II was three pronged, evaluating vitamin E, vitamin C, and a multivitamin for the primary prevention of cardiovascular disease, cancer (in general and prostate cancer, in particular), and cataracts as well as the age-related eye disease macular degeneration. The vitamin E and vitamin

6

PHYSICIANS’ HEALTH STUDY (PHS)

C arms ended in 2007, but the multivitamin arm has been funded to continue until 2011. Results from the vitamin E and vitamin C arms are expected to be submitted for publication in 2008. REFERENCES 1. C. H. Hennekens and J. E. Buring, Methodologic considerations in the design and conduct of randomized trials: the U.S. Physicians’ Health Study. Control. Clin. Trials. 1989; 10: 142S–150S. 2. C. D. Funk, L. B. Funk, M. E. Kennedy, A. S. Pong, and G. A. Fitzgerald, Human platelet/erythroleukemia cell prostaglandin G/H synthase: cDNA cloning, expression, and gene chromosomal assignment. Faseb. J. 1991; 5: 2304–2312. 3. G. A. FitzGerald, Mechanisms of platelet activation: thromboxane A2 as an amplifying signal for other agonists. Am. J. Cardiol. 1991; 68: 11B–15B. 4. J. R. Vane, Inhibition of prostaglandin synthesis as a mechanism of action for aspirin-like drugs. Nat. New. Biol. 1971; 231: 232–235. 5. The Persantine-Aspirin Reinfarction Study Research Group, Persantine and aspirin in coronary heart disease. Circulation. 1980; 62: 449–461. 6. The Persantine-Aspirin Reinfarction Study Research Group, Persantine-aspirin reinfarction study. Design, methods and baseline results. Circulation. 1980; 62: II1–42. 7. Aspirin Myocardial Infarction Study Research Group, A randomized, controlled trial of aspirin in persons recovered from myocardial infarction. JAMA. 1980; 243: 661–669. 8. H. Jick and O. S. Miettinen, Regular aspirin use and myocardial infarction. Br. Med. J. 1976; 1: 1057. 9. C. H. Hennekens, L. K. Karlson, B. Rosner, A case-control study of regular aspirin use and coronary deaths. Circulation. 1978; 58: 35–38. 10. E. C. Hammond and L. Garfinkel, Aspirin and coronary heart disease: findings of a prospective study. Br. Med. J. 1975; 2: 269–271. 11. J. E. Buring and C. H. Hennekens, Cost and efficiency in clinical trials: the U.S. Physicians’ Health Study. Stat. Med. 1990; 9: 29–33.

12. M. J. Stampfer, J. E. Buring, W. Willett, B. Rosner, K. Eberlein, and C. H. Hennekens, The 2 × 2 factorial design: its application to a randomized trial of aspirin and carotene in U.S. physicians. Stat. Med. 1985; 4: 111–116. 13. R. Peto, R. Gray, R. Collins, et al., Randomised trial of prophylactic daily aspirin in British male doctors. Br. Med. J. (Clin. Res. Ed.) 1988; 296: 313–316. 14. H. D. Sesso, J. M. Gaziano, M. Van Denburgh, C. H. Hennekens, R. J. Glynn, and J. E. Buring, Comparison of baseline characteristics and mortality experience of participants and nonparticipants in a randomized clinical trial: the Physicians’ Health Study. Control. Clin. Trials. 2002; 23: 686–702. 15. J. S. Berger, M. C. Roncaglioni, F. Avanzini, I. Pangrazzi, G. Tognoni, and D. L. Brown, Aspirin for the primary prevention of cardiovascular events in women and men: a sex-specific meta-analysis of randomized controlled trials. JAMA. 2006; 295: 306–313. 16. F. Grodstein, J. H. Kang, R. J. Glynn, N. R. Cook, and J. M. Gaziano, A randomized trial of beta carotene supplementation and cognitive function in men: the Physicians’ Health Study II. Arch. Intern. Med. 2007; 167: 2184–2190.

FURTHER READING L. Hansson, A. Zanchetti, S. G. Carruthers, et al., HOT Study Group, Effects of intensive blood-pressure lowering and low-dose aspirin in patients with hypertension: principal results of the Hypertension Optimal Treatment (HOT) randomised trial. Lancet. 1998; 351: 1755–1762. Medical Research Council’s General Practice Research Framework, Thrombosis prevention trial: randomised trial of low-intensity oral anticoagulation with warfarin and low-dose aspirin in the primary prevention of ischaemic heart disease in men at increased risk. Lancet. 1998; 351: 233–241. G. de Gaetano, Collaborative Group of the Primary Prevention Project, Low-dose aspirin and vitamin E in people at cardiovascular risk: a randomised trial in general practice. Lancet. 2001; 357: 89–95. P. M. Ridker, N. R. Cook, I. M. Lee, et al., A randomized trial of low-dose aspirin in the primary prevention of cardiovascular disease in women. N. Engl. J. Med. 2005; 352: 1293–1304.

PHYSICIANS’ HEALTH STUDY (PHS)

CROSS-REFERENCES Cardiovascular (CV) Factorial Designs Over the Counter (OTC) Drugs Primary Prevention Trials Run-in Period

7

Placebos History of Placebo The placebo, which originates from Latin meaning “I shall please”, was used for centuries as therapy for patients whom physicians were unsure of how to treat or for whom no useful treatment was available [26]. Attitudes towards therapies in Western cultures have shifted dramatically over the last four decades. Principles of safety, efficacy, and informed consent of participants are now well entrenched (see Ethics of Randomized Trials); the randomized controlled trial (see Clinical Trials, Overview) dominates as the method for evaluating therapy and patients are increasingly involved in decisions about their care [38]. The role of the placebo has evolved in response to all of these factors. In research, the notion of placebo control arose to account for those beneficial or harmful effects not directly attributable to the therapy of interest. Despite some dissent, placebos and placebocontrolled trials remain the benchmark by which all new drugs are evaluated and regulated [50]. In clinical care, any part of an encounter can have some therapeutic value and influence the patient’s response, including taking a history, stating a diagnosis, or making repeated measurements (e.g. blood pressure) or assurances about prognosis. Knowledgeable clinicians are interested in determining which parts of the placebo effect they should implement to optimize their patients’ health.

Definitions Many, similar definitions for “placebo”, “placebo effect”, and “placebo response” exist. Here, we use the word placebo to mean an inert substance or sham procedure designed to appear identical to the active substance or procedure but without known therapeutic effect. Placebo effect or placebo response is the psychophysiologic effect associated with placebos, thought to be primarily operative through the expectations or symbolic meaning of the administered therapy [5, 36, 39]. This implies both positive and negative effects, although the latter are often referred to as the nocebo effect [19]. More recently, the placebo effect has been further characterized as a true placebo effect versus a perceived placebo effect.

The perceived placebo effect, which is the effect commonly quoted and discussed in the literature, includes the true placebo effect plus other nonspecific effects, including the natural course of disease, regression to the mean, unidentified co-intervention effects, and other time-dependent effects [15, 39]. The natural course of any disease or symptom will change in severity over time and may resolve on its own (see Postmarketing Surveillance of New Drugs and Assessment of Risk). Regression to the mean, where a follow-up measurement of a patient’s disease or symptoms gives a more normal reading, is a wellrecognized phenomenon. Patients enrolled in a trial may influence clinical outcomes by implementing unidentified parallel interventions (co-intervention), for example, by starting a weight-loss program while in a trial of a diabetes drug. Time-dependent effects might include the increasing skill of the investigator in measuring study endpoints, or the decreasing “white coat hypertension” effect as patients become used to having their blood pressure measured. Since these nonspecific effects are inherent in any therapy (i.e. active intervention, placebo, or no-treatment groups), it has been argued that we should be focusing on the true placebo effect as a measurement target [15, 23].

Magnitude of the Placebo Effect For decades, the placebo response has been assumed to be similar across disease categories at approximately 35% improvement from baseline [3]. This figure has been applied equally to the proportion of patients improving and to the degree of improvement per patient per outcome. More recently, doubt has been cast on the constant placebo response as evidence accumulates that both the true and perceived placebo effects are variable [15, 36, 55]. A review of 75 trials of antidepressant medications revealed that the placebo response rate varied from 12.5 to 51.8% and had increased over time [55]. A recent systematic review of randomized trials with placebo and no-treatment arms (see Meta-analysis of Clinical Trials), concluded that there was evidence of a mild true placebo effect for some subjective outcomes (e.g. pain and anxiety) but not for more objective outcomes (blood pressure, weight loss, asthma outcomes) [23]. For example, placebo was associated with a reduction in pain by a mean of 0.65 cm

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

2

Placebos

on a 10 cm visual analog scale, as compared with no treatment. The authors measured the difference in outcomes between the two arms at the end of the treatment period rather than the change in outcomes from baseline, thus attempting to control for the nonspecific effects and determine the true placebo effect [22]. Their concluding caution against the use of the placebo and its effects for therapeutic purposes outside of a controlled clinical trial raised a storm of criticism. Although the validity of the findings were questioned, in terms of widely varying populations and diseases, heterogeneity and low statistical power [1, 46], wrong choice of placebo [27], and contamination of the “no-treatment” arms [14], in our opinion, it is really the generalizability of the findings that is in question (see Validity and Generalizability in Epidemiologic Studies). Within randomized controlled trials, where patients acknowledge by their consent that they know they may not receive an active treatment, the perceived (total) placebo effect may be more muted than in clinical practice. Furthermore, even within randomized trials, unblinding can occur and bias clinician and patient – a situation bound to happen with a no-treatment arm. Finally, and most important, a finding that placebo therapies and notreatment arms do not differ does not negate the possibility that a true placebo effect occurred (and occurred equally) in both groups. It is important to keep in mind that placebos are not risk free. Placebos may be harmful if they delay access to effective therapy for the disease under investigation, if their nonspecific symptomatic effects mask a condition that has effective treatment or via direct nocebo effect [1]. Where placebos are used without patient consent, any revelation of the deception may seriously undermine patient–physician relationship, which is itself a powerful source of “placebo effect”.

Influences on the Placebo Effect The attitude and behavior of the clinician toward the treatment and the patient, the attitude of the patient toward her own health and the treatment, as well as external, cultural, family, and media influences, all influence the placebo effect [5, 13, 22, 28, 35, 36]. Treatment variables including appearance, invasiveness, impressiveness, perceived plausibility, past experience, and cost all appear to play a role.

Provider factors can produce major placebo effects and, although not well studied, are considered integral to the “art of medicine”. The white coat of the clinician, the stethoscope, the interest, empathy, authority, and compassion displayed in the interview with the patient and the motivation and skillfulness with which a diagnosis or therapeutic path is pursued, each may influence the patient [5, 36, 39]. In a study of 200 British patients presenting to a physician with abnormal symptoms for which no firm diagnosis could be made, the patients were randomly assigned to a negative or positive consultation, and placebo treatment or no treatment [7]. The positive consultation consisted of a firm diagnosis and reassurance that the patient would be better in a few days while in the negative consultation, the physician confessed uncertainly. Two weeks later, 64% of those who received a positive consultation reported improvement compared with 39% of those who received a negative consultation. The percentage of the placebo treated group who improved was not significantly different than the percentage in the untreated group. Patient expectations, prior experiences with similar therapies, severity of current complaints, and suggestibility likely play a role in the placebo effect but have not been rigorously studied [2, 17, 29, 34, 36, 41, 42]. In a 10-week study of exercise, 48 healthy young adults were randomized to a control aerobics training program or an exercise program with constant reminders of the aim to improve both aerobic capacity and psychological well-being [13]. After 10 weeks, significant increases in fitness levels were seen in both groups, however, self-esteem and psychological well-being were only significantly improved in the experimental group, not the control group. Cultural differences in placebo responsiveness have also been explored. A review of 117 studies of ulcer treatment, and 37 studies of treatment for anxiety, showed Germany had the highest placebo healing rates for ulcers, but experienced only moderate placebo rates for treatments of anxiety. In comparison, Italy had the lowest placebo effects for anxiety, while Brazil had the lowest rates for ulcers [35]. Similarly, a retrospective analysis of deaths attributed to lymphatic cancer found that Chinese-Americans who were born in “Earth years”, and therefore deemed by Chinese medical history to be especially susceptible to diseases involving lumps, nodules, or tumors, had a lower mean age of death than those born in other years. No

Placebos such relationship could be seen in Caucasian controls who died of similar causes [35, 36]. Patients who enter into trials compared with those who do not and patients who adhere to their treatment, whether placebo or not, compared with those who do not (see Compliance Assessment in Clinical Trials), may have different outcomes. In a large study of beta-blockers to prevent a recurrence of a myocardial infarction, it was found that those who took more than 80% of their medications had a lower mortality rate compared to poor adherers whether they received beta-blockers or placebo [35, 39]. A Cochrane methods review pilot looked at patients enrolled in phase III randomized controlled trials (see Clinical Trials, Overview) versus patients who were similar but not enrolled. In 23 out of 25 reports, better outcomes were documented for patients within the trials compared to those who were not. Overall, there were lower mortality and lower rates for complications of therapy [21]. An analysis of the CAMIAT (Canadian Amiodarone Myocardial Infarction Arrhythmia Trial) study [24] showed that adherence to placebo or amiodarone, both predicted mortality. Patients who received placebo and were considered noncompliant had over a two-fold increased risk of sudden cardiac death, cardiac mortality, and all-cause mortality compared to those placebo patients who were compliant. Drug and treatment characteristics, apart from the pharmacology of the active ingredient(s), can have powerful placebo effects. Studies suggest that subjects react to the “meanings” or suggestion of the color and quantity of drugs. Red suggests “hot” or “danger”, blue suggests “down” or “quiet”, and a quantity of two means “more than one” [6, 12, 35]. A group of medical students were told that they were participating in a single-blinded study (see Blinding or Masking) on the psychological and physiologic effects of two drugs, a sedative and a stimulant, and received one or two placebo capsules of either blue or pink without attribution of any effect [6]. The consent form they were asked to sign gave a brief description of the stimulant or sedative side effects they may expect to experience. Study results showed that the students tended to experience stimulant reactions to the pink capsules, while the blue capsules produced depressant effects. Two capsules tended to produce a greater effect than one for both psychological changes (e.g. drowsiness) and physiologic changes (e.g. pulse) [37]. In other words, analgesia associated with an injection of saline solution was reversed with

3

an opiate antagonist such as naloxone and enhanced with an opiate agonist such as proglumide, which suggests that these patients were experiencing a physiologic response such as the release of endogenous opioids [4, 35]. Furthermore, the placebo response can be shown to produce a typical pharmacokinetic profile of activity [56]. Surgery and other mechanical or invasive procedures have been thought to produce exaggerated placebo responses, ever since the original studies of sham internal mammary artery ligation for the treatment of angina, where 80% of patients in the sham treatment arm reported substantial improvement [9]. More recently, the value of an extremely common procedure, arthroscopic lavage and debridement in patients with osteoarthritis of the knee, was questioned when it was found no better than sham surgery for outcomes of pain and physical function [37]. Reviews of trials examining placebo or comparing different placebo routes indicate that injections are more powerful than pills, and devices or procedures, such as sham ultrasound or sham acupuncture seem to be associated with stronger placebo effects than oral placebos [15, 28]. However, the extent to which the response is influenced by the procedure administrator versus the procedure itself remains unclear.

Ethics of Employing Placebo in Research Many questions have been raised about the ethics of using placebos in human research. Strong critics of the placebo argue that it is not ethical to assign subjects to any intervention that has even the potential of being less efficacious than current therapy (see Ethics of Randomized Trials). The opposing view asserts that very few “standard” treatments have been proven effective by today’s research standards, that placebo-controlled trials are a necessary first step with the smallest sample size to establish whether further research with a drug or treatment is warranted and that patients tend to improve within these trials regardless of allocation because of the close attention and follow-up. Certain situations, such as life-threatening conditions where a proven effective treatment exists, are agreed to be inappropriate for placebo-only allocation [50]. One of the areas of medicine where the use of placebo has been debated widely is in depression. Critics worried that randomization to placebo might lead

4

Placebos

to serious harm including suicide. When this was reviewed, not only were completed and attempted suicide rates similar for placebo, standard antidepressants (imipramine, amitriptyline, trazodone), and the investigational antidepressants (fluoxetine, sertraline, paroxetine), but also the clinical responses were not very different among the three groups either [30]. Another large review noted that the response to placebo in depressed patients was highly variable (10–50%) between trials, and has increased over the years with increasing trial length [55]. All of these factors suggest that the use of active treatment controls instead of placebo would make accurate evaluation of interventions difficult. Although it has been well argued that research subjects, by way of informed consent, indicate their willingness to freely participate in a placebocontrolled trial, there is some evidence that patients may not be fully informed about risks and benefits of the treatment options [25, 43]. Studies of consent documents have shown that they sometimes overstate the benefits and understate the risks of research protocols and are frequently written beyond the reading level of patients [11, 33, 49]. Even when consent forms are accurate and appropriate, patients may confuse treatment in a clinical trial with that of individualized medical care, overestimate the benefits of participating in a trial, and underestimate the risks they may be involved in [20]. A recent survey of patients participating in cancer trials showed that while they were satisfied with the consent form and considered themselves well informed, a large percentage of these patients did not recognize that the treatment being described was not a standard therapy, that there was potential risk to themselves, and that the study drug was unproven [25]. This research suggests that some of the placebo effect is already evident at this early stage before any treatment is actually given – the trust in trial clinicians, the hope for therapeutic benefit, and so on may color the patient’s memory of consent.

Guidelines for the Use of Placebos in Research Several prominent guidelines provide some direction for the use of placebos in research. All agree on basic principles such as noncoercion and fully informed consent for participants. In other matters, the guidelines differ somewhat. The Declaration of Helsinki

has the strictest guidelines and is often cited by critics of the placebo. First ratified by the World Medical Association (WMA) in 1964, it cites “The benefits, risks, burdens and effectiveness of a new method should be tested against those of the best current prophylactic, diagnostic and therapeutic methods. This does not exclude the use of placebo, or no treatment, in studies where no proven prophylactic, diagnostic, or therapeutic methods exists” [57]. However, its stand has been criticized as too strict as “best . . . methods” may not be the same as “best available” or “most cost-effective” comparators [40] and many standard therapies have not been rigorously compared to determine which is the best. The International Conference on Harmonization (ICH), a committee with representatives from the drug industry and regulatory authorities internationally, has published ICH E10 – Guideline for Choice of Control Group and Related Issues in Clinical Trials [53]. These guidelines, citing the unique scientific usefulness and general safety of placebos, allow the use of placebo controls even when effective treatment exists unless subjects would be exposed to an unacceptable risk of death or permanent injury or if the toxicity of standard therapy is so severe that “many patients have refused to receive it”. ICH E10 advocates the use of modified study designs whenever possible, such as add-on studies, factorial designs, or “early escape” from ineffective therapy [53]. Various national regulatory and research bodies have formulated their own guidelines. The TriCouncil Policy Statement (TCPS) from Canada’s three main research granting agencies, requires that grant recipients abide by the Declaration of Helsinki, prohibiting the use of placebo-controlled trials if there is an existing effective therapy available. However, it allows for broad exemptions, for example, for exceptional circumstances where effective treatment is not available to patients due to cost constraints or short supply, use in refractory patients, for testing add-on treatment to standard therapy, where patients have provided an informed refusal of standard therapy for a minor condition for which patients commonly refuse treatment, or when withholding such therapy will not lead to undue suffering or the possibility of irreversible harm [51]. The US Food and Drug Administration’s Code of Federal Regulations (CFR), revised in April 2002, accepts placebocontrolled trials and does not stipulate any restrictions but strongly advocates the need for informed

Placebos consent [10, 18]. The primacy of informed patient consent in placebo-controlled trials has spurred a number of critiques [25, 32, 48], challenges [52], and recommendations to improve processes of obtaining consent [11, 31, 44, 47, 54]. Patients themselves have become involved. The National Depressive and Manic-depressive Association, a large patient-directed illness-specific organization in the United States, recently developed a consensus development panel to discuss the controversy regarding the use of placebo [8]. In their guidelines, they acknowledge that mood disorders are episodic, chronic conditions that are associated with considerable morbidity and have no curative or fully preventative treatments. Despite the Declaration of Helsinki, it was agreed that mood disorder research was not at the point where noninferiority trials (trials designed only to rule out that the new therapy is worse than the control) (see Equivalence Trials) involving active controls could be considered scientifically valid designs. Therefore, the guidelines cite “placebo is justified when testing a new antidepressant with a novel mechanism of action that has a substantial probability of efficacy with an acceptable adverse effect risk. However, placebo is also ethical in studies of new drugs in a class because the newer members may offer important advantages over the original drug.” Despite the guidelines that currently aid researchers regarding the use of placebos in research, there are ongoing issues that need to be addressed. One could debate the “proof” of efficacy compared to placebo for many drugs in current use, such that the counterargument has been made that it might be unethical to insist on the use of these drugs in control groups. The Declaration of Helsinki’s view on placebo was established in an attempt to deter pharmaceutical companies and research organizations from exploiting people in poorer populations, who may not have access to proven treatments. This concern is not relevant in many countries. Advocating the use of placebos only when no standard treatment exists leaves the efficacy of drugs in certain patient groups largely unknown. Many clinicians need to know whether a therapy is “better than nothing” and could therefore be an alternative for patients who do not respond to the conventional treatment or cannot tolerate it [45]. In chronic diseases such as hypertension, diabetes, and vascular disease, combinations of therapies are increasingly prevalent. It is likely a more

5

efficient assessment of a new drug to test it against placebo initially before embarking on multiple, more complicated, larger sample size, drug add-on trials.

Innovations to Improve Research Involving Placebo Apart from improving the process of informed consent as described above, several themes of innovative trial design are developing. All attempt to minimize patient exposure to placebo only or increase the efficiency of the design to lower sample size. The first design, now commonly used, is the add-on trial where one group gets both the standard therapy and the new therapy while the other group gets the standard therapy only. The difference between the mean response of the combination standard/new therapy group and the mean of the standard therapy group alone is a reasonable estimate of effect of the new therapy provided the two therapies do not interact with each other. The second design was developed for antidepressant drug trials to address the high placebo response rates [55]. This two-phase randomized crossover method initially randomizes more patients to placebo than active therapy and, in the second phase, crosses over only placebo nonresponders [16]. It is intended to minimize the overall placebo response rate and thus the sample size required to show a clinically important difference (see Sample Size Determination for Clinical Trials). The third design avoids the bias of patient selection according to placebo response. A 22 factorial design involving standard and new therapies has four groups: double placebo, new therapy, standard therapy, and combination standard therapy/new therapy. With a 1 : 1 : 1 : 1 allocation ratio, there are two estimates of the efficacy of the new therapy–the difference between the new therapy group mean and the double placebo group mean, and the difference between the combination standard therapy/new therapy group mean and the standard therapy group mean. These two estimates are pooled together to measure the overall efficacy of the new therapy by computing the mean of the two estimates. These two estimates should be similar to each other provided the new therapy does not interact with the standard therapy. However, one half of the patients are given either the double placebo or the new therapy of yet unproven efficacy. This may present an ethical problem if the

6

Placebos

standard therapy has excellent efficacy and clinicians are reluctant to deny patients access to it. If the investigator were to drop these two groups from the third design, the result is the first, add-on design, and the measure of interaction between the two therapies has to rely on theory, not data. A fourth design is possible where there are two known-to-be effective therapies available and where some clinicians might use the first standard, some might use the second standard, and some might use both standards together. Here, a 23 factorial design should be considered: triple placebo, new therapy, first standard therapy, second standard therapy, double combination of new therapy and the first standard therapy, double combination of the new therapy with the second standard therapy, double combination of the first standard and second standard therapies, triple combination of the new therapy, the first standard therapy and the second standard therapy with 8 groups and an equal allocation ratio. The design would provide four estimates of the efficacy of new therapy, namely: (1) the difference in the mean of new therapy group and the mean of the triple placebo group, (2) the difference in the mean of the double combination of new therapy and first standard group and the mean of the first standard group, (3) the difference in the mean of the double combination of the new therapy and second standard therapy and the mean of the second standard group and (4) the difference between the mean of the triple combination of new therapy, first standard therapy and second standard therapy and the mean of the double combination of the first and second standard therapies. These four estimates should be similar to each other provided the new therapy does not interact with either of the first or second standard therapies, an assumption that can be checked. Finally, if the two ethically challenged groups, placebo and new therapy, are dropped from the fourth design, then the remaining fractional factorial design has six groups that permit three estimates of efficacy of the new therapy, namely, (2), (3), and (4). Again these three estimates should be similar to each other provided the two standard therapies do not interact with new therapy, and these three estimates should be similar to the one estimate of the efficacy of the new therapy that it is not possible to obtain from the six groups, namely (1). This six-group fractional factorial design still permits the efficacy of the new therapy to be estimated by pooling together the three

estimates with the mean of (2), (3), and (4). Provided the interaction assumption is reasonable, this threeterm mean should provide adequate evidence of the efficacy of the new therapy. This fifth design should be ethically acceptable since no patient is being denied the benefit of a known therapy.

Summary The placebo’s role in medicine has been and continues to be in transition. For decades, the dispute regarding the placebo revolved around its ability (or not) to induce psychological or physiological effects. More recently, the debate has focused on the utilization of the placebo-controlled trial, its usefulness, and whether, despite informed consent, it impinges on patient autonomy and the practice of beneficence. There are currently several international guidelines that direct the researchers on the use of placebos; however, there are prominent differences between them. The future is likely to bring developments in trial designs where a new therapy’s effect can be compared to placebo yet no individual subject is exposed only to placebo, designs that lower the placebo response rate so that treatment effect may be ascertained more efficiently, and designs that allow further exploration and exploitation of factors influencing the placebo effect.

Acknowledgments Dr. Holbrook is the recipient of a Canadian Institutes for Health Research Career Investigator award.

References [1]

[2]

[3] [4]

[5]

Bailar, J.C. III. (2001). The powerful placebo and the Wizard of Oz, The New England Journal of Medicine 344, 1630–1632. Barsky, A.J., Saintfort, R., Rogers, M.P. & Borus, J.F. (2002). Nonspecific medication side effects and the nocebo phenomenon, JAMA 287, 622–627. Beecher, H.K. (1955). The powerful placebo, JAMA 159, 1602–1606. Benedetti, F. (1996). The opposite effects of the opiate antagonist naloxone and the cholecystokinin antagonist proglumide on placebo analgesia, Pain 64, 535–543. Benson, H. & Friedman, R. (1996). Harnessing the power of the placebo effect and renaming it “remembered wellness”, Annual Review of Medicine 47, 193–199.

Placebos [6]

[7] [8]

[9]

[10]

[11]

[12]

[13]

[14]

[15] [16]

[17]

[18]

Blackwell, B., Bloomfield, S.S. & Buncher, C.R. (1972). Demonstration to medical students of placebo responses and non-drug factors, Lancet 1, 1279–1282. Brody, H. & Waters, D.B. (1980). Diagnosis is treatment, The Journal of Family Practice 10, 445–449. Charney, D.S., Nemeroff, C.B., Lewis, L., Laden, S.K., Gorman, J.M., Laska, E.M., Borenstein, M., Bowden, C.L., Caplan, A., Emslie, G.J., Evans, D.L., Geller, B., Grabowski, L.E., Herson, J., Kalin, N.H., Keck, P.E., Jr., Kirsch, I., Krishnan, K.R., Kupfer, D.J., Makuch, R.W., Miller, F.G., Pardes, H., Post, R., Reynolds, M.M., Roberts, L., Rosenbaum, J.F., Rosenstein, D.L., Rubinow, D.R., Rush, A.J., Ryan, N.D., Sachs, G.S., Schatzberg, A.F., Solomon, S. (2002). National depressive and manic-depressive association consensus statement on the use of placebo in clinical trials of mood disorders, Archives of General Psychiatry 59, 262–270. Cobb, L., Thomas, G.I., Dillard, D.H., Merendino, K.A. & Bruce, R.A. (1959). An evaluation of internalmammary artery ligation by a double blind technic, The New England Journal of Medicine 260, 1118. Code of Federal Regulations. Title 21 Volume 5. Revised April 1, 2002 Accessed on November 18, 2002 at http://www.accessdata.fda.gov/scripts/ cdrh/cfdocs/cfcfr/CFRSearch.cfm? FR = 314.126. 2002. Coyne, C.A., Xu, R., Raich, P., Plomer, K., Dignan, M., Wenzel, L.B., et al. (2003). Randomized, controlled trial of an easy-to-read informed consent statement for clinical trial participation: a study of the eastern cooperative oncology group, Journal of Clinical Oncology 21, 836–842. de Craen, A.J., Roos, P.J., Leonard d, V. & Kleijnen, J. (1996). Effect of colour of drugs: systematic review of perceived effect of drugs and of their effectiveness, BMJ 313, 1624–1626. Desharnais, R., Jobin, J., Cote, C., Levesque, L. & Godin, G. (1993). Aerobic exercise and the placebo effect: a controlled study, Psychosomatic Medicine 55, 149–154. Einarson, T.E., Hemels, M. & Stolk, P. (2001). Is the placebo powerless? The New England Journal of Medicine 345, 1277–1279. Ernst, E. & Resch, K.L. (1995). Concept of true and perceived placebo effects, BMJ 311, 551–553. Fava, M., Evins, A.E., Dorer, D.J. & Schoenfeld, D.A. (2003). The problem of the placebo response in clinical trials for psychiatric disorders: culprits, possible remedies, and a novel study design approach, Psychotherapy and Psychosomatics 72, 115–127. Greene, P.J., Wayne, P.M., Kerr, C.E., Weiger, W.A., Jacobson, E., Goldman, P., Kaptchuk, T. (2001). The powerful placebo: doubting the doubters. Advance in Mind-Body Medicine 17, 298–307. Guidance for institutional review boards and clinical investigators. 21 CFR Part 50. 1998 Update. Accessed

[19]

[20] [21]

[22]

[23]

[24]

[25]

[26]

[27] [28]

[29]

[30]

[31]

[32]

[33]

7

on November 18, 2002 at http://www.fda.gov/ oc/ohrt/irbs/appendixb.html. 1998. Hahn, R.A. (1997). The nocebo phenomenon: concept, evidence, and implications for public health, Preventive Medicine 26, 607–611. Horng, S. & Miller, F.G. Is placebo surgery unethical? The New England Journal of Medicine 347, 137–139. How do the outcomes of patients treated within randomized control trials compare with those of similar patients treated outside these trials? http://hiru. mcmaster.ca/ebm/trout/, accessed 2002-11-17. 2001. Hrobjartsson, A. (2002). What are the main methodological problems in the estimation of placebo effects? Journal of Clinical Epidemiology 55, 430–435. Hrobjartsson, A. & Gotzsche, P.C. (2001). Is the placebo powerless? An analysis of clinical trials comparing placebo with no treatment, The New England Journal of Medicine 344, 1594–1602. Irvine, J., Baker, B., Smith, J., Jandciu, S., Paquette, M., Cairns, J., et al. (1999). Poor adherence to placebo or amiodarone therapy predicts mortality: results from the CAMIAT study. Canadian Amiodarone Myocardial Infarction Arrhythmia Trial, Psychosomatic Medicine 61, 566–575. Joffe, S., Cook, E.F., Cleary, P.D., Clark, J.W. & Weeks, J.C. (2001). Quality of informed consent in cancer clinical trials: a cross-sectional survey, Lancet 358, 1772–1777. Kaptchuk, T.J. (1998). Powerful placebo: the dark side of the randomised controlled trial, Lancet 351, 1722–1725. Kaptchuk, T.J. (2001). Is the placebo powerless? The New England Journal of Medicine 345, 1277–1279. Kaptchuk, T.J., Goldman, P., Stone, D.A. & Stason, W.B. (2000). Do medical devices have enhanced placebo effects? Journal of Clinical Epidemiology 53, 786–792. Khan, A., Leventhal, R.M., Khan, S.R. & Brown, W.A. (2002). Severity of depression and response to antidepressants and placebo: an analysis of the food and drug administration database, Journal of Clinical Psychopharmacology 22, 40–45. Khan, A., Warner, H.A. & Brown, W.A. (2000). Symptom reduction and suicide risk in patients treated with placebo in antidepressant clinical trials: an analysis of the Food and Drug Administration database, Archives of General Psychiatry 57, 311–317. Lavori, P.W., Sugarman, J., Hays, M.T. & Feussner, J.R. (1999). Improving informed consent in clinical trials: a duty to experiment, Controlled Clinical Trials 20, 187–193. Lilford, R.J. (2003). Ethics of clinical trials from a Bayesian and decision analytic perspective: whose equipoise is it anyway? BMJ 326, 980–981. Macklin, R. (1999). The ethical problems with sham surgery in clinical research, The New England Journal of Medicine 341, 992–996.

8 [34]

[35]

[36]

[37]

[38]

[39]

[40] [41]

[42]

[43]

[44] [45] [46]

Placebos Mataix-Cols, D., Rauch, S.L., Manzo, P.A., Jenike, M.A. & Baer, L. (1999). Use of factor-analyzed symptom dimensions to predict outcome with serotonin reuptake inhibitors and placebo in the treatment of obsessivecompulsive disorder, American Journal of Psychiatry 156, 1409–1416. Moerman, D.E. (2000). Cultural variations in the placebo effect: ulcers, anxiety, and blood pressure, Medical Anthropology Quarterly 14, 51–72. Moerman, D.E. & Jonas, W.B. (2002). Deconstructing the placebo effect and finding the meaning response, Annals of Internal Medicine 136, 471–476. Moseley, J.B., O’Malley, K., Petersen, N.J., Menke T.J., Brody, B.A., Kuykendall, D.H., Hollingsworth, J.C., Ashton, C.M., Wray, N.P. (2002). A controlled trial of arthroscopic surgery for osteoarthritis of the knee. The New England Journal of Medicine 347, 81–88. O’Connor, A.M., Rostom, A., Fiset, V., Tetroe, J., Entwistle, V., Llewellyn-Thomas, H., et al. (1999). Decision aids for patients facing health treatment or screening decisions: systematic review, BMJ 319, 731–734. Papakostas, Y.G. & Daras, M.D. (2001). Placebos, placebo effect, and the response to the healing situation: the evolution of a concept, Epilepsia 42, 1614–1625. Riis, P. (2000). Perspectives on the fifth revision of the declaration of Helsinki, JAMA 284, 3045–3046. Rochon, P.A., Binns, M.A., Litner, J.A., Litner, G.M., Fischbach, M.S., Eisenberg, D., Kaptchuk, T.J., Stason, W.B., Chalmers, T.C. (1999). Are randomized control trial outcomes influenced by the inclusion of a placebo group?: a systematic review of nonsteroidal antiinflammatory drug trials for arthritis treatment, Journal of Clinical Epidemiology 52, 113–122. Rosenberg, N.K., Mellergard, M., Rosenberg, R., Beck, P. & Ottosson, J.O. (1991). Characteristics of panic disorder patients responding to placebo, Acta Psychiatrica Scandinavica, Supplementum 365, 33–38. Rothman, K.J. & Michels, K.B. (1994). The continuing unethical use of placebo controls, The New England Journal of Medicine 331, 394–398. Slater, E.E. (2002). IRB reform, The New England Journal of Medicine 346, 1402–1404. Solomon, D.A. (1995). The use of placebo controls, The New England Journal of Medicine 332, 62. Spiegel, D., Kraemer, H. & Carlson, R.W. (2001). Is the placebo powerless? The New England Journal of Medicine 345, 1276–1279.

[47]

[48]

[49] [50]

[51]

[52]

[53]

[54]

[55]

[56]

[57]

St. Joseph’s Health care/McMaster University/Hamilton Health Sciences. Informed consent checklist. http: //www.fhs.mcmaster.ca/csd/forms/ic-chk list.pdf. 7-30-2003. Steinbrook, R. (2002). Protecting research subjects–the crisis at Johns Hopkins, The New England Journal of Medicine 346, 716–720. Tattersall, M.H. (2001). Examining informed consent to cancer clinical trials, Lancet 358, 1742–1743. Temple, R. & Ellenberg, S.S. (2000). Placebo-controlled trials and active-control trials in the evaluation of new treatments. Part 1: ethical and scientific issues, Annals of Internal Medicine 133, 455–463. Tri-Council Policy statement. Ethical conduct for research involving humans. Updated November 21, 2000. Accessed on November 20, 2002 at. http: //www.nserc.ca/programs/ethics/english/ policy.htm. 2000. Truog, R.D., Robinson, W., Randolph, A. & Morris, A. (1999). Is informed consent always necessary for randomized, controlled trials? The New England Journal of Medicine 340, 804–807. U.S. Department of Health and Human Services Food and Drug Administration. Guidance for Industry: E 10 Choice of Control group and Related Issues in Clinical Trials. Available at: http://www.fda.gov/cder/ guidance/4155fnl.pdf., accessed July 24, 2003. 2001. Verheggen, F.W., Jonkers, R. & Kok, G. Patients’ perceptions on informed consent and the quality of information disclosure in clinical trials, Patient Education and Counseling 96 A.D.29, 137–153. Walsh, B.T., Seidman, S.N., Sysko, R. & Gould, M. (2002). Placebo response in studies of major depression: variable, substantial, and growing, JAMA 287, 1840–1847. Weiner, M. & Weiner, G.J. (1996). The kinetics and dynamics of responses to placebo, Clinical Pharmacology and Therapeutics 60, 247–254. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects, JAMA (2000). 284, 3043–3045.

ANNE HOLBROOK, CHARLES H. GOLDSMITH & MOVA LEUNG

PLANNING A GROUP-RANDOMIZED TRIAL

know as much as possible about the problem before they plan their trial. Having become experts in the field, the investigators should choose the single question that will drive their GRT. The primary criteria for choosing that question should be: (1) Is it important enough to do?, and (2) Is this the right time to do it? Reviewers will ask both questions, and the investigators must be able to provide well-documented answers. Most GRTs seek to prevent a health problem, so that the importance of the question is linked to the seriousness of that problem. The investigators should document the extent of the problem and the potential benefit from a reduction in that problem. The question of timing is also important. The investigators should document that the question has not been answered already and that the intervention has a good chance to improve the primary endpoint in the target population, which is most easily done when the investigators are thoroughly familiar with previous research in the area; when the etiology of the problem is well known; when a theoretical basis exists for the proposed intervention; when preliminary evidence exists on the feasibility and efficacy of the intervention; when the measures for the dependent and mediating variables are well-developed; when the sources of variation and correlation as well as the trends in the endpoints are well understood; and when the investigators have created the research team to carry out the study. If that is not the state of affairs, then the investigators must either invest the time and energy to reach that state or choose another question. Once the question is selected, it is very important to put it down on paper. The research question is easily lost in the dayto-day details of the planning and execution of the study, and because much time can be wasted in pursuit of issues that are not really central to the research question, the investigators should take care to keep that question in mind.

DAVID M. MURRAY Division of Epidemiology School of Public Health The Ohio State University, Columbus Ohio

1

INTRODUCTION

Planning a group-randomized trial (GRT) is a complex process. Readers interested in a more detailed discussion might wish to consult Murray’s text on the design and analysis of GRTs (1), from which much of this article was abstracted. Donner and Klar’s text is another good source of information (2). 2

THE RESEARCH QUESTION

The driving force behind any GRT must be the research question. The question will be based on the problem of interest and will identify the target population, the setting, the endpoints, and the intervention. In turn, those factors will shape the design and analytic plan. Given the importance of the research question, the investigators must take care to articulate it clearly. Unfortunately, that does not always happen. Investigators may have ideas about the theoretical or conceptual basis for the intervention, and often even clearer ideas about the conceptual basis for the endpoints. They may even have ideas about intermediate processes. However, without very clear thinking about each of these issues, the investigators may find themselves at the end of the trial unable to answer the question of interest. To put themselves in a position to articulate their research question clearly, the investigators should first document thoroughly the nature and extent of the underlying problem and the strategies and results of previous efforts to remedy that problem. A literature review and correspondence with others working in the field are ingredients essential to that process, as the investigators should

3

THE RESEARCH TEAM

Having defined the question, the investiga-

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

PLANNING A GROUP-RANDOMIZED TRIAL

tors should determine whether they have expertise sufficient to deal with all the challenges that are likely to occur as they plan and execute the trial. They should identify the skills that they do not have and expand the research team to ensure that those skills are available. All GRTs will need expertise in research design, data collection, data processing and analysis, intervention development, intervention implementation, and project administration. As the team usually will need to convince a funding agency that they are appropriate for the trial, it is important to include experienced and senior investigators in key roles. No substitute exists for experience with similar interventions, in similar populations and settings, using similar measures, and similar methods of data collection and analysis. As those skills are rarely found in a single investigator, most trials will require a team, with responsibilities shared among its members. Most teams will remember the familiar academic issues (e.g., statistics, data management, intervention theory), but some may forget the very important practical side of trials involving identifiable groups. However, to forget the practical side is a sure way to get into trouble. For example, a school-based trial that does not include on its team someone who is very familiar with school operations is almost certain to get into trouble with the schools. A hospital-based trial that does not include on its team someone who is very familiar with hospital operations is almost certain to get into trouble with the hospitals. The same can be said for every other type of identifiable group, population, or setting that might be used. 4

THE RESEARCH DESIGN

The fundamentals of research design apply to GRTs as well as to other comparative designs. As they are discussed in many familiar textbooks (3–7), they will be reviewed only briefly here. Additional information may be found in two recent textbooks on the design and analysis of GRTs (1, 2). The goal in the design of any comparative trial is to provide the basis for valid inference that the intervention as implemented caused

the result(s) as observed. To meet that goal, three elements are required: (1) there must be control observations, (2) there must be a minimum of bias in the estimate of the intervention effect, and (3) there must be sufficient precision for that estimate. The nature of the control observations and the way in which the groups are allocated to treatment conditions will determine, in large measure, the level of bias in the estimate of the intervention effect. Bias exists whenever the estimate of the intervention effect is different from its true value. If that bias is substantial, the investigators will be misled about the effect of their intervention, as will the other scientists and policy makers who use their work. Even if adequate control observations are available so that the estimate of the intervention effect is unbiased, the investigator should know whether the effect is greater than would be expected by chance, given the level of variation in the data. Statistical tests can provide such evidence, but their power to do so will depend heavily on the precision of the intervention effect estimate. As the precision improves, it will be easier to distinguish true effects from the underlying variation in the data.

5 POTENTIAL DESIGN PROBLEMS AND METHODS TO AVOID THEM For GRTs, the four sources of bias that are particularly problematic and should be considered during the planning phase are selection, differential history, differential maturation, and contamination. Selection bias refers to baseline differences among the study conditions that may explain the results of the trial. Bias caused by differential history refers to some external influence that operates differentially among the conditions. Bias caused by differential maturation reflects uneven secular trends among the groups in the trial favoring one condition or another. These first three sources of bias can mask or mimic an intervention effect, and all three are more likely given either non-random assignment of groups or random assignment of a limited number of groups to each condition.

PLANNING A GROUP-RANDOMIZED TRIAL

The first three sources of bias are best avoided by randomization of a sufficient number of groups to each study condition, which will increase the likelihood that potential sources of bias are distributed evenly among the conditions. Careful matching or stratification can increase the effectiveness of randomization, especially when the number of groups is small. As a result, all GRTs planned with fewer than 20 groups per condition would be well served to include careful matching or stratification before randomization. The fourth source of bias is caused by contamination, which occurs when interventionlike activities find their way into the comparison groups; it can bias the estimate of the intervention effect toward the null hypothesis. Randomization will not protect against contamination; although investigators can control access to their intervention materials, they can often do little to prevent the outside world from introducing similar activities into their control groups. As a result, monitoring exposure to activities that could affect the trial’s endpoints in both the intervention and comparison groups is especially important in GRTs, which will allow the investigators to detect and respond to contamination if it occurs. Objective measures and evaluation personnel who have no connection to the intervention are also important strategies to limit bias. Finally, analytic strategies, such as regression adjustment for confounders, can be very helpful in dealing with any observed bias. 6 POTENTIAL ANALYTIC PROBLEMS AND METHODS TO AVOID THEM The two major threats to the validity of the analysis of a GRT that should be considered during the planning phase are misspecification of the analytic model and low power. Misspecification of the analytic model will occur if the investigator ignores or misrepresents a measurable source of random variation, or misrepresents the pattern of any over-time correlation in the data. To avoid model misspecification, the investigator should plan the analysis concurrent with

3

the design, plan the analysis around the primary endpoints, anticipate all sources of random variation, anticipate the error distribution for the primary endpoint, anticipate patterns of over-time correlation, consider alternative structures for the covariance matrix, consider alternative models for time, and assess potential confounding and effect modification. Low power will occur if the investigator employs a weak intervention, has insufficient replication, has high variance or intraclass correlation in the endpoints, or has poor reliability of intervention implementation. To avoid low power, investigators should plan a large enough study to ensure sufficient replication, choose endpoints with low variance and intraclass correlation, employ matching or stratification before randomization, employ more and smaller groups instead of a few large groups, employ more and smaller surveys or continuous surveillance instead of a few large surveys, employ repeat observations on the same groups or on the same groups and members, employ strong interventions with good reach, and maintain the reliability of intervention implementation. In the analysis, investigators should employ regression adjustment for covariates, model time if possible, and consider post hoc stratification. 7 VARIABLES OF INTEREST AND THEIR MEASURES The research question will identify the primary and secondary endpoints of the trial. The question may also identify potential effect modifiers. It will then be up to the investigators to anticipate potential confounders and nuisance variables. All these variables must be measured if they are to be used in the analysis of the trial. In a clinical trial, the primary endpoint is a clinical event, chosen because it is easy to measure with limited error and is clinically relevant (5). In a GRT, the primary endpoint need not be a clinical event, but it should be easy to measure with limited error and be relevant to public health. In both clinical and GRTs, the primary endpoint, together with its method of measurement,

4

PLANNING A GROUP-RANDOMIZED TRIAL

must be defined in writing before the start of the trial. The endpoint and its method of measurement cannot be changed after the start of the trial without risking the validity of the trial and the credibility of the research team. Secondary endpoints should have similar characteristics and also should be identified before the start of the trial. In a GRT, an effect modifier is a variable whose level influences the effect of the intervention. For example, if the effect of a schoolbased drug-use prevention program depends on the baseline risk level of the student, then baseline risk is an effect modifier. Effect modification can be seen intuitively by looking at separate intervention effect estimates for the levels of the effect modifier. If they differ to a meaningful degree, then the investigator has evidence of possible effect modification. A more formal assessment is provided by a statistical test for effect modification, which is accomplished by including an interaction term between the effect modifier and condition in the analysis and testing the statistical significance of that term. If the interaction is significant, then the investigator should present the results separately for the levels of the effect modifier. If not, the interaction term is deleted and the investigator can continue with the analysis. Proper identification of potential effect modifiers comes through a careful review of the literature and from an examination of the theory of the intervention. Potential effect modifiers must be measured as part of the data-collection process so that their role can later be assessed. A confounder is related to the endpoint, not on the causal pathway, and unevenly distributed among the conditions; it serves to bias the estimate of the intervention effect. No statistical test exists for confounding; instead, it is assessed by comparing the unadjusted estimate of the intervention effect to the adjusted estimate of that effect. If, in the investigator’s opinion, a meaningful difference exists between the adjusted and unadjusted estimates, then the investigator has an obligation to report the adjusted value. It may also be appropriate to report the unadjusted value to allow the reader to assess the degree of confounding. The adjusted analysis will not be possible unless the potential confounders are measured. Proper identification

of potential confounders also comes through a careful review of the literature and from an understanding of the endpoints and the study population. The investigators must take care in the selection of potential confounders to select only confounders and not mediating variables. A confounder is related to the endpoint and unevenly distributed in the conditions, but is not on the causal pathway between the intervention and the outcome. A mediating variable has all the characteristics of a confounder but is on the causal pathway. Adjustment for a mediating variable, in the false belief that it is a confounder, will bias the estimate of the intervention effect toward the null hypothesis. Similarly, the investigator must take care to avoid selecting as potential confounders variables that may be affected by the intervention even if they are not on the causal pathway linking the intervention and the outcome. Such variables will be proxies for the intervention itself, and adjustment for them will also bias the estimate of the intervention effect toward the null hypothesis. An effective strategy to avoid these problems is to restrict confounders to variables measured at baseline. Such factors cannot be on the causal pathway, nor can their values be influenced by an intervention that has not been delivered. Investigators may also want to include variables measured after the intervention has begun, but will need to take care to avoid the problems described above. Nuisance variables are related to the endpoint, not on the causal pathway, but evenly distributed among the conditions. They cannot bias the estimate of the intervention effect, but they can be used to improve precision in the analysis. A common method is to make regression adjustment for these factors during the analysis so as to reduce the standard error of the estimate of the intervention effect, thereby improving the precision of the analysis. Such adjustment will not be possible unless the nuisance variables are measured. Proper identification of potential nuisance variables also comes from a careful review of the literature and from an understanding of the endpoint. The cautions described above for the selection of potential

PLANNING A GROUP-RANDOMIZED TRIAL

confounding variables apply equally well to the selection of potential nuisance variables. 8

THE INTERVENTION

No matter how well designed and evaluated a GRT may be, strengths in design and analysis cannot overcome a weak intervention. Although the designs and analyses employed in GRTs were fair targets for criticism during the 1970s and 1980s, the designs and analyses employed more recently have improved, with many examples of very well-designed and carefully analyzed trials. Where intervention effects are modest or short-lived, even in the presence of good design and analytic strategies, investigators must take a hard look at the intervention and question whether it was strong enough. One of the first suggestions for developing the research question was that the investigators become experts on the problem that they seek to remedy. If the primary endpoint is cigarette smoking among ninth-graders, then the team should seek to learn as much as possible about the etiology of smoking among young adolescents. If the primary endpoint is obesity among Native American children, then the team should seek to learn as much as possible about the etiology of obesity among those young children. If the primary endpoint is delay time in reporting heart attack symptoms, then the team should seek to learn as much as possible about the factors that influence delay time. And the same can be said for any other endpoint. One of the goals of developing expertise in the etiology of the problem is to identify points in that etiology that are amenable to intervention. Critical developmental stages, or critical events or influences that trigger the next step in the progression, may exist or it may be possible to identify critical players in the form of parents, friends, coworkers, or others who can influence the development of that problem. Without careful study of the etiology, the team will largely be guessing and hoping that their intervention is designed properly. Unfortunately, guessing and hoping rarely lead to effective interventions. Powerful interventions are guided by good theory on the process for change, combined

5

with a good understanding of the etiology of the problem of interest. Poor theory will produce poor interventions and poor results, which was one of the primary messages from the community-based heart diseaseprevention studies, where the intervention effects were modest, generally of limited duration, and often within chance levels. Fortmann et al. (8) noted that one of the major lessons learned was how much was not known about how to intervene in whole communities. The theory that describes the process of change in individuals may not apply to the process of change in identifiable groups. If it does, it may not apply in exactly the same way. Good intervention for a GRT will likely need to combine theory about individual change with theory about group processes and group change. A good theoretical exposition will also help identify channels for the intervention program. For example, strong evidence exists that recent immigrants often look to longterm immigrants of the same cultural group for information on health issues, which has led investigators to try to use those longterm immigrants as change agents for the more recent immigrants. A good theoretical exposition will often indicate that the phenomenon is the product of multiple influences and so suggest that the intervention operate at several different levels. For example, obesity among school children appears to be influenced most proximally by their physical activity levels and by their dietary intake. In turn, their dietary intake is influenced by what is served at home and at school and their physical activity is influenced by the nature of their physical activity and recess programs at school and at home. The models provided by teachers and parents are important both for diet and for physical activity. This multilevel etiology suggests that interventions be directed at the school food-service, physical education, and recess programs; at parents; and, possibly, at the larger community. GRTs would benefit by following the example of clinical trials, where some evidence of feasibility and efficacy of the intervention is usually required before launching the trial. When a study takes several years to

6

PLANNING A GROUP-RANDOMIZED TRIAL

complete and costs hundreds of thousands of dollars or more, that expectation seems very fair. Even shorter and less expensive GRTs would do well to follow that advice. What defines preliminary evidence of feasibility? It is not reasonable to ask that the investigators prove that all intervention and evaluation protocols can be implemented in the population and setting of interest in advance of their trial. However, it is reasonable to ask that they demonstrate that the major components of the proposed intervention can be implemented in the target population, which can be done in a pilot study. It is also reasonable to ask that the major components of the proposed evaluation are feasible and acceptable in the setting and population proposed for the trial, which also can be done in a pilot study. What defines preliminary evidence of efficacy? It is not fair to ask that the investigators prove that their intervention will work in the population and setting of interest in advance of their trial. However, it is fair to ask that they provide evidence that the theory supporting the intervention has been supported in other situations. It is also fair to ask that the investigators demonstrate that similar interventions applied to other problems have been effective. Finally, it is reasonable to ask that the investigators demonstrate that the proposed intervention generates short-term effects for intermediate outcomes related to the primary and secondary endpoints and postulated by the theoretical model guiding the intervention. Such evidence provides reassurance that the intervention will be effective if it is properly implemented. 9

POWER

A detailed exposition on power for GRTs is beyond the scope of this article. Excellent treatments exist, and the interested reader is referred to those sources for additional information. Chapter 9 in the Murray text (1) provides perhaps the most comprehensive treatment of detectable difference, sample size, and power for GRTs. Even so, a few points bear repeating here. First, the increase in between-group variance

caused by the ICC in the simplest analysis is calculated as 1 + (m − 1)ICC, where m is the number of members per group; as such, ignoring even a small ICC can underestimate standard errors if m is large. Second, although the magnitude of the ICC is inversely related to the level of aggregation, it is independent of the number of group members who provide data. For both of these reasons, more power is available given more groups per condition with fewer members measured per group than given just a few groups per condition with many members measured per group, no matter the size of the ICC. Third, the two factors that largely determine power in any GRT are the ICC and the number of groups per condition. For these reasons, no substitute exists for a good estimate of the ICC for the primary endpoint, the target population, and the primary analysis planned for the trial, and it is unusual for a GRT to have adequate power with fewer than 8–10 groups per condition. Finally, the formula for the standard error for the intervention effect depends on the primary analysis planned for the trial, and investigators should take care to calculate that standard error, and power, based on that analysis. 10 SUMMARY GRTs are often complex studies, with greater challenges in design, analysis, and intervention than what is seen in other studies. As a result, much care and effort is required for good planning. Future trials will be stronger and more likely to report satisfactory results if they (1) address an important research question, (2) employ an intervention that has a strong theoretical base and preliminary evidence of feasibility and efficacy, (3) randomize a sufficient number of assignment units to each study condition so as to have good power, (4) are designed in recognition of the major threats to the validity of the design and analysis of group-randomized trials, (5) employ good quality-control measures to monitor fidelity of implementation of intervention and measurement protocols, (6) are well executed, (7) employ good process-evaluation measures to assess effects on intermediate endpoints, (8)

PLANNING A GROUP-RANDOMIZED TRIAL

employ reliable and valid endpoint measures, (9) are analyzed using methods appropriate to the design of the study and the nature of the primary endpoints, and (10) are interpreted in light of the strengths and weaknesses of the study. REFERENCES 1. D. M. Murray, Design and Analysis of GroupRandomized Trials. New York: Oxford University Press, 1998. 2. A. Donner and N. Klar, Design and Analysis of Cluster Randomization Trials in Health Research. London: Arnold, 2000. 3. R. E. Kirk, Experimental Design: Procedures for the Behavioral Sciences, 2nd ed. Belmont, CA: Brooks/Cole Publishing Company, 1982. 4. L. Kish, Statistical Design for Research. New York: John Wiley & Sons, 1987. 5. C. L. Meinert, Clinical Trials. New York: Oxford University Press, 1986. 6. W. R. Shadish, T. D. Cook, and D. T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin Company, 2002. 7. B. J. Winer, D. R. Brown, and K. Michels, Statistical Principles in Experimental Design. New York: McGraw-Hill, 1991. 8. S. P. Fortmann, J. A. Flora, M. A. Winkleby, C. Schooler, C. B. Taylor, and J. W. Farquhar, Community intervention trials: reflections on the Stanford Five-City Project experience. Amer. J. Epidemiol. 1995; 142(6): 576–586.

7

POISSON REGRESSION

stochastic process with constant intensity rate λ. In the Poisson regression model, it is assumed that the mean (expected value) parameter λ is a function of the covariates
x = (x1 , . . . , xp ) . The connection between λ
and x = (x1 , . . . , xp ) is established as a linear predictor η via a link function g, where,

EKKEHARD GLIMM and MICHAEL BRANSON Novartis Pharma AG Basel, Switzerland

One of the major limitations of conventional Poisson regression is the fact that model assumptions force the expected value and the variance of the counts to be equal. In practice, this assumption is often violated, the empirical variance of the counts being larger than their mean. Several methods are available to relax this restrictive condition. A popular approach is to introduce an overdispersion parameter. Also, the full distributional assumptions can be abandoned in favor of a model for first- and second-order moments only. Generalized estimating equations are used to fit such overdispersed ‘‘Poisson’’ models. Other methods to deal with overdispersion are zero-inflated Poisson models, the negative binomial model and mixed effectsand repeated-measures Poisson regression models. The latter constitute a field of much recent research that aims at a unification of linear model and GLM theory. This unification is also increasingly reflected in statistical software. 1

η = β0 +

βi xi and η = g(λ)

i=1

For the Poisson regression model, the log-link function g(λ) = log(λ) is used. Since λ must be positive and the support of η is the entire real line, therefore,  λ = exp β0 +

p


 βi xi

i=1

As the Poisson distribution is a member of the exponential family of distributions, the Poisson regression model is a special case of a generalized linear model (GLM) (1). In fact, it can be viewed as the prototype of a generalized linear model. The interest is in inference
on the model parameters β = (β 0 , . . . , β p ) , that is, estimation of these parameters and their variances from observed data, confidence intervals, testing for statistical significance, and so on. In many clinical trial applications, we are primarily interested in the event rate υ = λ/t, where t is the time a patient is exposed or ‘‘at risk’’ of having an event, for example, an epileptic fit in an epilepsy trial. Because log(λ/t) = log(λ) − log(t), we can easily incorporate this situation into the GLM formulation by inclusion of log(t) into the set of predictors, forcing its parameter β (t) , say, to be 1. In the next section, we introduce more specific details of the Poisson regression model. This section is followed by the outline of a case study, and aspects of Poisson regression modeling are described via the analysis of these data. In the application of Poisson regression, it is probably the norm rather than the exception to observe that the variance of counts is much larger than its mean. This

INTRODUCTION

Poisson regression is a method to relate the number of event counts or event rates to a set of covariates, for example, age, treatments under investigation, and so on. The Poisson regression model assumes that a number (Y) of counts follows a Poisson distribution with rate λ, denoted Po(λ). The Poisson distribution, sometimes called the ‘‘distribution of rare events’’, has the probability density function f (Y = y) =

p


exp(−λ) · λy y!

where y is the discrete number of event counts, y = 0, 1, . . . . Consequently, E(Y) = λ, and Variance(Y) = λ. The Poisson distribution develops naturally from a discrete

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

POISSON REGRESSION

phenomenon is called overdispersion and is discussed after the introduction of the case study. Finally, the discussion comments on statistical software for fitting Poisson regression models, current research trends, and related methods.

The solution of these equations are the MLˆ Birch (3) showed that the soluestimates β. tions are unique and maximize the (log) likelihood. The estimated covariance matrix of βˆ is given by the inverse of the observed Fisher information, −1 x1
 .  ˆ  = (x1 · · · xn )W  ..  xn


2 PARAMETER ESTIMATION, CONFIDENCE INTERVALS AND GOODNESS-OF-FIT 2.1 Parameter Estimation Nelder and Wedderburn (1,2) have developed the theory of maximum likelihood (ML)estimation for the exponential family of distributions. The general parameterization of the exponential family density is  f (y|θ , φ) = exp

yθ − b(θ ) + c(y, φ) a(φ)



where θ is called the canonical or natural parameter, and φ is called the dispersion parameter. If in a GLM, the link function g(.) is such that the linear predictor and the canonical parameter are identical (i.e., η = θ ), then the link is called canonical. Canonical links simplify likelihood inference. For the Poisson distribution the parameters are: φ = a(φ) = 1, θ = log(λ), b(θ ) = exp(θ ) and c(y,θ ) = − log(y!). Obviously, g(λ) = log(λ) is the canonical link of the Poisson distribution. Standard GLM procedures, in particular Fisher scoring, also called iterative reweighted least-squares (IWLS), are used to calculate the ML-estimates βˆi from the given data (yj , xj )j=1,...,n where yj denotes the number of event counts observed on sample unit
j and xj = (x1j , . . . , xpj ) is the vector of this sample unit’s covariates. If, for example, we consider patients in a clinical trial, then the sample units would be patients and the covariates may include characteristics like age, sex, allocated treatment group (or dose of therapy), and possibly interactions thereof. For the Poisson model, the likelihood equations are n
 j=1

 yj − λj xij = 0

(1)

Inf −1



ˆ is the diagonal matrix with elewhere W ments given by  w ˆj =

∂λj ∂ηj

2  Var(Yj )

ˆ For the evaluated at the ML-estimates β. Poisson regression model, we have w ˆ j = λˆ j . Usually, the solution of the likelihood equations [Equation (1)] requires an iterative algorithm. The preferred method is Fisher scoring, which is the same as Newton–Raphson for GLMs with a canonical link (Reference 4, section 4.6). As a by-product, Fisher scoring yields the asymptotic covariance matrix Inf −1 of the ML-estimates. Fisher scoring is implemented in all major statistics software packages. The fitting method is very reliable and fast irrespective of starting values. If the estimation procedure fails to produce convergence to finite ML-estimates, then the usual reason is an over-parameterized model (i.e., too many covariates) with an infinite likelihood. This is, of course, likely to happen with either many (especially highly discrete) covariates and/or with sparse data, that is, very few observed events in general. 2.2 Confidence Intervals Univariate confidence intervals for the parameters of the model can be constructed  as Wald’s (1−α)-confidence intervals βˆi ±  u1−α/2 · sˆ βˆi , where u1−α/2 is the (1 − α/2)quantile  the standard normal distribution  of and sˆ βˆi is the estimated standard deviation from Fisher scoring. However, confidence intervals that are based on the profile likelihood (i.e., the likelihood viewed as a function of β i with the other parameters

POISSON REGRESSION

β j , j = i kept fixed at their ML-estimates βˆj ) have better small-sample properties (see Reference 4, section 1.3; Reference 5, section 1.4). Historically, Wald confidence intervals were preferred because they do not require iterative calculations, as profile likelihood ratio-based confidence intervals do. With the arrival of more powerful software packages, the computation convenience of the Wald’s confidence interval has lost importance, such that, the latter are preferred in general. All of these confidence intervals are based on largesample asymptotic theory for ML-estimates (see Reference 6). 2.3 Goodness-of-Fit Inference on the goodness-of-fit of the model can be assessed using overall goodness-of-fit tests and review of the residuals. The general ideas are the same for all GLMs. Hence, they are summarized only briefly in the following. Overall goodness-of-fit tests compare nested models, that is, a model with a set of
parameters β = (β 0 , . . . , β p ) to a sub-model with 0 < q < p of the p parameters in β. For GLMs, three asymptotically equivalent tests are generally available: score, Wald, and likelihood ratio (LR) tests (7). Under fairly general regularity conditions, the test statistics have an asymptotic χ 2 distribution with (p − q) degrees of freedom, if the submodel is true. Thus, these tests can be used to select and decide about the validity of the submodel. As a special case of the LR test, a postulated (or candidate) model can be compared with the saturated model that has as many parameters as observations. The saturated model forces a perfect fit. The resulting statistic is called the deviance (Reference 4, chapter 4.1) and is a general measure of goodnessof-fit. An alternative and frequently used goodness-of-fit measure is the Pearson χ 2 statistic, 2 n 
yj − yˆ j yˆ j j=1

where yˆ j are the predictions from the model. Asymptotically, the Pearson χ 2 statistic has the same distribution as the other three test statistics, that is, a χ 2 distribution with (n − q) degrees of freedom where q is the

3

number of parameters in the model under investigation, provided this model is true. However, care is sometimes required in interpreting the deviance and Pearson statistics as following the χ 2 distribution with low counts (see Reference 8, section 14.4). Non-nested models are usually compared in an informal way by information criteria. A popular choice is Akaike’s information criterion (AIC) given by − 2 × log likelihood + 2p, where p is the number of parameters in the model. The smaller the AIC, the better the model. Other information criteria also exist and each are based on the same principle of combining the log likelihood of a model with a penalty for the number of parameters. A more detailed investigation of the goodness-of-fit of a model requires examining residuals, that is, deviations of observations from the model predictions. For this purpose, different types of residuals have been suggested. Let yj be the observed Poisson count and yˆ j be its prediction from the model. Then  • deviance residuals are given as dj · sign(yj − yˆ j ), where dj is the contribution of the jth observation to the deviance. • Pearson residuals are given as ej =    yj − yˆ j / yˆ j . In analogy, these are the individual contributions to the Pearson test statistic. • standardized Pearson residuals are  ej / 1 − hˆ j where hˆ j is the jth diagonal element of the so-called ‘‘Hat’’-matrix, the asymptotic covariance matrix of the   model predictions yˆ j j=1, ..., p . Standardized Pearson residuals are usually preferred because they have an approximate variance of 1. Just like in ordinary regression, these residuals are checked for values that deviate considerably from 0 to detect outliers in the data and plotted versus various characteristics like covariates, predictions of expected events counts, time, and so on, to determine whether they display nonrandom patterns that would indicate a lack of fit (see Reference 9, section 6.2, for residual analysis in a Poisson regression context). Note, however, that in cases where the distribution of the counts yj is far away from approximately normal, such plots may be of limited use.

4

POISSON REGRESSION

3 CASE STUDY: STATISTICAL ANALYSIS FOR THE INCIDENCE RATES OF ADVERSE EVENTS (NASOPHARYNGITIS) To fix ideas, we introduce a case study in which the incidence rate of an adverse event (nasopharyngitis) was of interest. A total of 710 patients [473 patients given the experimental treatment (E) and 237 patients given the control treatment (C), a 2:1 randomization ratio] participated in the trial. The total exposure time for each treatment was 384 years and 149 years for E and C, respectively. In 533 patients (349 patients in E, and 184 patients in C) no nasopharyngitis event occurred during the study period; 177 patients (124 in E, 53 in C) had at least one event, and 89 patients (62 in E, 27 in C) had more than one event with up to 21 events for a single patient. More specific details of the clinical trial cannot be disclosed for confidentiality reasons. The counts of nasopharyngitis events were modeled using a Poisson regression model. Thus, for patient i in treatment group j = E, C: yij ∼ Po(λij ) with log(λij ) = log(tij ) + β 0 + β j , where tij is the duration of exposure of the ith patient in group j. Note that just like in ordinary regression, we need a restriction on the parameters to make them identifiable. Here, β 2 = 0 such that in the general formulation, log(λij ) = log(tij ) + β0 + β1 · xj with  1, if j = E xj = 0, if j = C Therefore, the event rate for treatment group C is exp(β 0 ) and is exp(β 0 + β 1 ) for treatment E. The treatment difference as measured by the ratio of event rates (E:C) is therefore given by exp(β 1 ). This model can be easily fitted with many general statistical software packages. We used SAS PROC GENMOD (10). The estimate βˆ1 = −0.124 of the treatment effect indicates that the (exposure-adjusted) event rate is lower in E relative to C. In conjunction with the baseline estimate of βˆ0 = −6.04, this finding implies that the expected number of nasopharyngitis events in a year is 365 × exp(−6.04) = 0.87 in C, but it is

only 365 × exp(−6.04−0.124) = 0.77 in E. The 95% profile LR-based confidence interval and the Wald confidence interval for βˆ1 are (−0.33;0.09) and (−0.33;0.08) respectively, and include the null value 0. The Pearson χ 2 test statistic is 2587 with 708 degrees of freedom. This value is much larger than would be expected if the model were correct. 4 OVERDISPERSION The simple Poisson regression model has only one parameter, which causes the expected value and the variance to be same. Hence, it is inflexible regarding event rate intensity. For example, the Poisson assumption forces the event rate of the underlying stochastic process to be constant in time. In practice, the observed variance of the counts per patient in a clinical trial is often much larger than the observed mean count. This phenomenon, which is called overdispersion, is very common. Like in the case study introduced in the previous section, its root cause is often patient heterogeneity that is not captured in the model covariates. For example, it can be shown that overdispersion arises if counts which are assumed to be independent and from a common Poisson distribution are actually from a mixture of Poisson distributions with different event intensity rates λi . Of particular practical relevance is the fact that overdispersion develops naturally if patients are followed for time periods of different lengths. Overdispersion can be addressed in several ways, which are discussed in the following sections. 4.1 Zero-Inflated Poisson Model Sometimes, overdispersion is encountered because there are too many zeros relative to the nonzero counts in the data. This phenomenon sometimes affects clinical trials because events might have a higher probability to reoccur once they have occurred for the first time. This leads to the zero-inflated Poisson (ZIP) model. Early references for the ZIP model are References 11 and 12. Here, it is assumed that the nonzero counts follow a Poisson model, but that the zeros are a mixture of zeros from this model and an

POISSON REGRESSION

additional stochastic process producing extra zeros. Hence we have the model:  exp(−λ) · λy   ;y = 0 p0 + (1 − p0 ) · y! f (y) = y exp(−λ) · λ   ;y > 0 (1 − p0 ) ·  y! where p0 is the probability of obtaining an extra zero. The parameters p0 and λ are estimated using ML-methods. In the presence of covariates, p0 (x) is often modeled as a logistic regression model. See for example Reference 13. To illustrate the approach, a ZIP model is assumed for the case study previously introduced. The zero-inflation parameter is modeled as logit(p0ij ) = γ 0 + γ j ; γ 2 = 0 for all patients i in treatment group j and the Poisson parameter log(λij ) = log(tij ) + β 0 + β j as before. It fits better than the ordinary Poisson model with an offset (AIC = 1440 vs AIC = 1826). Hence, there is some indication of zeroinflation. The ML-estimates of the logit part are γˆ0 = −0.75 with a Wald 95% confidence interval of (0.40;1.10) and γˆ1 = −0.02 with a 95% confidence interval of (−0.43;0.40). It seems that the zero-inflation is the same in the two treatment groups. The estimated treatment effect is βˆ1 = −0.129 with a 95% Wald confidence interval of (−0.37, 0.11). This finding is very similar to the estimate from the ordinary Poisson regression model. However, −2 log likelihood = 1432 for the zero-inflated Poisson model and 449 for the saturated Poisson model, which indicates that the zero-inflation does not remove all of the overdispersion. 4.2 Quasi-Likelihood Overdispersion can also be tackled by introducing an additional dispersion parameter into the model and estimating it by quasilikelihood methods. This method implies that the Poisson assumption and, in fact, any explicit distributional assumption about the counts is abandoned and only the expected value and variance of the counts are assumed to follow a parametric model. Quasi-likelihood (QL) and its generalization, frequently called generalized estimating equations (GEE), solve a set of equations in the model parameters that ‘‘look like’’ the likelihood

5

equations from a GLM. To be more specific, the quasi-likelihood assumption in the Poisson regression model is that still E(Y) = λ, but that var(Y) = φλ. The extra parameter φ accounts for the overdispersion. In Poisson regression, the estimating equations for the model parameters β in the QL approach and the likelihood equations for β under a full Poisson-distributional assumption (φ = 1) are the same, so that the QL point estimates and the full likelihood point estimates of β coincide, and the same methods can be used to fit them (Reference 5, chapter 3). However, a Poisson assumption underestimates the variance of the point estimates. If we assume that the same overdispersion affects all observations independent of their covariates, then φ can be estimated by φˆ =  χ 2 / (k − p), where k is the number of observations, p the number of parameters in the model, and χ 2 is the value of the deviance or the Pearson goodness-of-fit test statistic. The approximate covariance matrix of the QL estimates βˆ is given by −1    n 
∂λj
∂λj    φλj  V= ∂β ∂β 

j=1

where λj = g(xj ) is the jth patient’s Poisson parameter. We can estimate V by plugging βˆ and φˆ into this formula, but that approach is sensitive to misspecification of the variance function var(Y). Therefore, the so-called robust or sandwich estimator is sometimes preferred. The sandwich estimator is obtained by replacing β and φ with its  2 estimates βˆ and φˆ and var(Y j ) with yj − λˆ j in   n 
∂λj
 −1 var(Yj ) φλj V˜ = V  ∂β j=1



φλj

−1



∂λj ∂β

 V.

ˆ even V˜ is the approximate covariance of β, if var(Y) is misspecified. If overdispersion depends on covariates, then a parametric model may be used to model φ as a function of the covariates. In general, although this approach is very powerful, it has some theoretical intricacies

6

POISSON REGRESSION

associated with the fact that without additonal restrictions on φ, the overdispersed Poisson model will have a distribution outside the exponential family of distributions (see Reference 5, sections 3.4–3.6). Reanalyzing the Case Study Data. The QL approach results in the estimation of φ  by φˆ = 2587/708 = 1.91 based on the Pearson test statistic. All standard errors of the parameter estimates have to be multiplied by this value to account for the overdispersion. Consequently, confidence intervals are wider. For example, the 95% profile likelihood confidence interval for the difference βˆ1 between treatments becomes (−0.51;0.28). The Wald 95% confidence interval is (−0.52;0.27). Both are Poisson likelihood-based confidence intervals and are not computed using the sandwich estimator for the for covariance matrix. Although this approach is adequate for estimation of a parameter of special interest, like the treatment effect in this example, other model inadequacies like the excessive number of zero counts remain unresolved. For prediction of the number of counts in a newly recruited patient, say, a more careful modeling effort, possibly including more covariates should be considered. 4.3 Negative Binomial Model Another method is to assume that the distribution of counts is a gamma-mixture of different Poisson distributions. If it is assumed that patients follow Poisson distributions with different parameters λj and these are in turn sampled from a gamma distribution, then it follows that the number of events has a negative binomial distribution (e.g., Reference 14, p. 122ff). This distribution has a variance that is larger than its expected value. In contrast to the QL approach, this approach is still based on a full parametric model for the counts. The negative binomial function is in the exponential family only if its dispersion parameter ξ is assumed fixed. The usual method to fit this model consists of solving the likelihood equation by IWLS for a fixed ξ (q) providing the parameter estimate βˆ (q) in the qth step, then to use Newton–Raphson with fixed βˆ(q) to update ξ (q) and alternate between the two until both ξ (q) and βˆ(q) have converged (Reference 4,

section 13.4). The negative-binomial model with zero inflation (ZINB) as a generalization of the ZIP model has been discussed in Reference 15. Reanalyzing the case-study data using a negative binomial model gives a Pearson statistic of 725, which is much better than the simple Poisson model and, in fact, reveals no signs of an inadequate fit. The treatment effect is estimated as −0.122 with a likelihood based 95% confidence interval of (−0.53;0.28), which is almost identical to the result obtained from applying the QL method. The ML-estimate of the dispersion parameter ξ is ξˆ = 4.1 with 95% profile likelihood confidence interval (3.2;5.3), which again indicates substantial overdispersion. 5 DISCUSSION In the analysis of categorical data, the counts in the cells of a contingency table are often assumed to follow a Poisson distribution, giving rise to Poisson loglinear models. These models can be interpreted as Poisson regression models with categorical covariates only. In that case, we do not have repeated counts from a single individual but rather counts per constellation of covariates. Although this situation at least formally constitutes a ‘‘Poisson regression model,’’ it is not usually treated under this heading. The analysis of categorical data arranged in contingency tables has many aspects to it that are out of scope of Poisson regression and vice versa. To give just one example, in contingency table analysis, it is usually irrelevant whether Poisson, multinomial, or independent multinomial sampling is assumed (16). For categorical data analysis, see References 17 or 4. In recent years, considerable advances have been achieved in aligning GLM theory with linear mixed model theory, resulting in generalized linear mixed models (GLMM). This is reflected in the fact that many software packages now have very similar syntax for mixed linear models and GLMMs (e.g., SAS PROC MIXED and GLIMMIX). Basically, regarding the linear model component, the same ideas and principles apply for both linear models and GLMs, especially with

POISSON REGRESSION

respect to notions such as random effects, marginal versus conditional models, and correlation structures. However, fitting these models is more complicated in the GLMM context, and some assumptions, especially on the covariance structure in marginal models, are difficult to assess. Also, in contrast to the ordinary linear mixed model, certain important equivalences of the parameters in a conditional and the corresponding marginal model do no longer hold. The interested reader is referred to chapters 11 and 12 in Reference 4, and Reference 18. An introduction to the random effects Poisson model is given by Reference 4, section 13.5. More detailed investigations have been performed in References 18 and 19. Fitting marginal and random effects GLM is complex because the likelihoods induced by the model assumptions are generally not tractable analytically. Numerical algorithms to fit these models are an active area of current research and a wide variety of approaches have been suggested. Many of these approaches concentrate on maximizing the likelihood directly by using numerical integration methods, for example, Gauss-Hermite quadrature, the EM algorithm, Markov Chain Monte Carlo methods, or Laplace transformations. See, for example, References 18 and 20 for recent overviews. Other approaches approximate the likelihood by a simpler function. The motivation for such approximations may be numerical tractability; relaxation of model assumptions (e.g., the GEE approach), or certain deficiencies of ML-estimates (e.g., penalized likelihood methods). Additionally, likelihoodbased methods have several variants, such as profile likelihood, marginal and conditional likelihood, and h-likelihood. See Reference 5 for an in-depth discussion of these concepts. Bayesian methods can also be applied to Poisson regression modeling, which frequently leverage MCMC to sample for the joint posterior distributions of parameters. The interested reader is referred to References 21–24. Most theory for Poisson regression is likelihood-based and relies heavily on the large sample properties of ML-estimates. Sometimes, this may be questionable because of small sample sizes or few counts. There is an

7

extensive literature on small-sample methods in contingency table analysis (including the case of Poisson-distributed counts), see for example, Reference 4 or 25. However, few investigations regarding sparse data have been made on Poisson regression with continuous covariates or the more complex models for handling overdispersion investigated in the previous section. A general approach to this is the application of resampling methods like the Bootstrap (26). A discussion of nonparametric regression and generalized linear models is also given in Reference 27. Poisson regression models are also closely related to multiple time-to-event methods, see References 28 and 29. REFERENCES 1. P. McCullagh and J. A. Nelder, Generalized Linear Models. London: Chapman & Hall, 1989. 2. J. A. Nelder and R. W. M. Wedderburn, Generalized linear models. J. Royal Stat. Soc. 2004; A135: 370–384. 3. M. W. Birch, Maximum-likelihood in threeway contingency tables. J. Royal Stat. Soc. 1963; B25: 220–233. 4. A. Agresti, Categorical Data Analysis, 2nd ed. New York: Wiley, 2002. 5. Y. Lee, J. A. Nelder, and Y. Pawitan, Generalized Linear Models with Random Effects. London: Chapman & Hall /CRC, 2006. 6. R. J. Serfling, Approximation Theorems of Mathematical Statistics. New York: Wiley, 1980. 7. D. R. Cox and D. V. Hinkley, Theoretical Statistics. London: Chapman & Hall, 1974. 8. P. Armitage, G. Berry, and J. N. S. Matthews, Statistical Methods in Medical Research. Oxford, UK: Blackwell, 2002. 9. A. Gelman and J. Hill, Data Analysis using Regression and Multilevel/Hierarchical Models. Cambridge, UK: Cambridge University Press, 2007. 10. SAS Institute. SAS Online Doc 9.1.3. 2007. Cary, NC. Available: http://support.sas.com/ onlinedoc/913/docMainpage.jsp. 11. J. Mullahy, Specification and testing of some modified count data models. J. Economet. 1986; 33: 341–365. 12. D. Lambert, Zero-inflated Poisson regression, with an application to defects in manufacturing. Technometrics 1992; 34: 1–14.

8

POISSON REGRESSION

13. M. S. Ridout, J. P. Hinde, C. G. B. Demetrio, A score test for testing a zero-inflated Poisson regression model against zero-inflated negative binomial alternatives. Biometrics 2001; 57: 219–223. 14. A. M. Mood, F. A. Graybill, and D. C. Boes, Introduction to the Theory of Statistics, 3rd ed. Singapore: McGraw-Hill, 1974. 15. D. B. Hall and K. S. Berenhaut, Score Tests for heterogeneity and overdispersion in zeroinflated poisson and binomial regression models. Canad. J. Stat. 2002; 30: 415–430. 16. J. Palmgren, The Fisher information matrix for log-linear models arguing conditionally in the observed explanatory variables. Biometrika 1981; 68: 563–566. 17. Y. V. V. Bishop, S. E. Fienberg, and P. W. Holland, Discrete Multivariate Analysis. Cambridge, UK: MIT Press, 1975. 18. G. Molenberghs and G. Verbeke, Models for Discrete Longitudinal Data. New York: Springer, 2005. 19. P. J. Diggle, P. J. Heagerty, K-Y. Liang, and S. L. Zeger, Analysis of Longitudinal Data, 2nd ed. Oxford, UK: Oxford Science Publications, 2002. 20. J. C. Pinheiro, D. M. Bates, Mixed Effects Models in S and S-Plus. New York: Springer, 2000. 21. D. K. Dey, S. K. Ghosh, and B. K. Mallick, Generalized Linear Models: A Bayesian Perspective. New York: Marcel Dekker, 2000. 22. P. Congdon, Applied Bayesian Modelling. New York: Wiley, 2003.

23. D. J. Spiegelhalter, J. K. Abrams, and R. P. Myles, Bayesian Approaches to Clinical Trials and Health-Care Evaluation. New York: Wiley, 2004. 24. A. Gelman, J. B. Carlin, H. B. Stern, and D. B. Rubin, Bayesian Data Analysis. London: Chapman & Hall/CRC, 1995. 25. K. F. Hirji, Exact Analysis of Discrete Data. London: Chapman & Hall/CRC, 2006. 26. B. Efron and R. Tibshirani, An Introduction to the Bootstrap. London: Chapman & Hall /CRC, 1994. 27. P. J. Green and B. W. Silverman, Nonparametric Regression and Generalized Linear Models. London: Chapman & Hall, 1994. 28. T. M. Therneau and P. M. Grambsch, Modeling Survival Data: Extending the Cox Model. New York: Springer, 2000. 29. P. K. Andersen, Ø. Borgan, R. D. Gill, and N. Keiding, Statistical Models based on Counting Processes. New York: Springer, 1993.

CROSS-REFERENCES Generalized Linear Models Generalized Estimating Equations Mixed Effects Models Regression Adverse Event Outcome Incidence Rate Time to Event

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

of patients, then considering the average drug effect is not sufficient. Because BSV is estimated in a population PKPD analysis, population PKPD models can predict the range of drug concentrations, drug effects, and therapeutic responses for the whole patient population of interest. Consequently, population PKPD models allow one to predict the results of clinical trials and to optimize future clinical trial designs. This article presents the concept, methods, and some applications at an introductory level. More in depth reports on population PKPD can be found in several reviews (4–8).

¨ JuRGEN B BULITTA

Department of Pharmaceutical Sciences, School of Pharmacy and Pharmaceutical Sciences State University of New York at Buffalo Buffalo, New York

NICHOLAS H G HOLFORD Department of Pharmacology and Clinical Pharmacology University of Auckland Auckland, New Zealand

Population pharmacokinetics (PK) and pharmacodynamics (PD) originated in the 1970s and has been adopted extensively in drug development and clinical pharmacology (1). Population PKPD can incorporate all available and relevant data from a clinical study in a single data analytical step (2, 3). Therefore, population PKPD methods provide a sound basis for data analysis and for interpretation of results. Population PKPD models allow one to study the effect of multiple covariates like renal function, body size, body composition, and age on PK and PD simultaneously. Population PKPD can be distinguished from hypothesis testing procedures such as analysis of variance (ANOVA) and analysis of covariance (ANCOVA) in that population. PKPD can account for the full time course of PK, PD, and disease progression. This difference becomes especially clear when attempting to make predictions. ANOVA-based methods cannot be extended to predict unstudied circumstances. As the time course of concentrations and the time course of effects is usually nonlinear, almost all PKPD models are nonlinear. ANOVA and ANCOVA can only account for nonlinearity by transformation of the independent or dependent variable(s). Because a substantial amount of between subject variability (BSV) often occurs, it is very important to account for BSV when describing PK and PD parameters. If the goal of therapy is to cure more than 95%

1

TERMINOLOGY

Precise use of terminology is important in population PKPD modeling. Related terms such as variability, and uncertainty as well as covariates and covariance are frequently confused. 1.1 Variable A variable is a quantity that can change its value. Variables can be dependent or independent. Independent variables are used as input for a model. It is usually assumed that independent variables are known or measured with negligible error (e.g., sampling time, dose, body weight, age, and creatinine concentration). A dependent variable is a function of the independent variable(s) and the model parameter(s). The value of a dependent variable is predicted by the model. Usually, dependent variables are associated with measurement error (e.g., drug concentration in plasma, amounts in urine, or pain score). A variable can be several different types: (1) continuous (e.g., a concentration represented as a floating point number), (2) count data (e.g., an integer for the number of counts from a radioactive decay or the number of headaches experienced by a migraine sufferer), (3) ordered categorical (e.g., a pain score from zero to five), or (4) unordered categorical (e.g., Asian, Caucasian, Polynesian).

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

1.2 Parameter

1.5 Covariate (also called covariable)

Parameters are components of the model equation(s) that characterize the underlying system. They are constants in the model but may change from person to person and over time within a person. Usually, parameters are assumed not to change during one dosing interval (during one occasion). The observations are used to estimate the model parameters.

A covariate is an independent variable that is used to modify (predict) the value of a parameter in the model equation(s). Renal function is a covariate that is used to modify the predicted value for renal clearance. (Other examples include effect of body size, body composition, and disease state on clearance and volume of distribution). 1.6 Covariance

1.3 Variability Variability describes the (true) diversity or heterogeneity of subjects. For example, if 100 subjects are studied, then each subject will have his/her own value for clearance and volume of distribution. The BSV describes the variability of clearance and volume of distribution between those 100 subjects. The BSV can be described by a distribution (e.g., by a normal distribution with a mean and standard deviation or by a log-normal distribution with a geometric mean and coefficient of variation, or by a nonparametric distribution). In a parametric population PK model, the BSV of clearance is an estimated model parameter. 1.4 Uncertainty Measured quantities and model parameters are only known with some degree of certainty. Uncertainty is described by the standard error of the estimate. However, because uncertainty is typically not distributed normally in nonlinear estimation problems, it is better to describe uncertainty with some other metric such as a confidence interval. Uncertainty must be distinguished from variability. For example, in a hypothetical study with a million patients, the average clearance of all patients will be known with negligible uncertainty, and the confidence interval for average clearance will be extremely narrow. Note that the BSV (variability) may be large (e.g., 30% coefficient of variation), but with very large numbers of patients its uncertainty will be very small (e.g., 90% confidence interval of 29.9% to 30.2%). The estimate of variability will not change systematically with the size of the study, but its uncertainty will become smaller as study size gets larger.

Covariance describes the common (joint) variability of two random effects. Apparent clearance and apparent volume of distribution may vary predictably (fixed effect) because of variability in extent of absorption, but commonly the interest lies in the apparently random association of BSV between model parameters (random effects). The joint distribution of BSV is described by the variance of each parameter and the covariance between the parameters. The variance and covariance can be combined to describe the correlation between the parameters. 1.7 Mixed Effects Model The term mixed effects model is used to explain sources of variability. The word effect does not refer to a drug effect. It refers to a factor that has an effect on variability. These effects are divided into those that have the same effect every time (i.e., they are predictable) and those that cannot be predicted. The first kind of effect is called a fixed effect and the second kind is called a random effect. When variability is described by a combination of fixed effects and random effects, the resulting model is called a mixed effects model. 1.8 Subject Variability Often, a marked variability occurs in pharmacokinetic concentration time profiles between various subjects, and often even more variability occurs in drug effects. The BSV in structural model parameters is used to describe a major part of variability. Besides variability between subjects, within subject variability (WSV) of model parameters also occurs. The sum of BSV and WSV is the population parameter variability (PPV) (9).

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

The WSV can be split into between occasion variability (BOV) and within occasion variability (WOV). An occasion is defined as the shortest time interval that allows one to estimate all model parameters reliably. Commonly this means one dosing interval. Figure 1 illustrates the components of PPV. If estimates for renal function are available on different occasions, BOV can also be split into a predictable and a random component. 2

FIXED EFFECTS MODELS

A fixed effects model describes the system of interest in a predictable (deterministic) manner. It contains no random effects. Fixed effects models comprise ‘‘input-output models’’ (which describe the overall response) and ‘‘group models’’ (which describe predictable effects on BSV). Covariates can be used to split BSV into a predictable component (BSVpredictable ) and a random component (BSVrandom ) as described below. Covariates like weight or renal function are used to distinguish BSVpredictable from BSVrandom . 2.1 Input-Output Models Input-output models describe the relationship between independent and dependent variables. In a PK model, this would be the relationship between time and concentration for the chosen dosage regimen. For a PD model, this might be the relationship between drug concentration and drug effect. Input-output models are also called structural models as they define the structure of the system. Most commonly, compartmental models are used as structural models in population PKPD analyses. However, physiologically based population PK models may be used more frequently to gain better understanding of the biological factors that affect drug disposition such as blood flow and organ specific elimination. 2.2 Group Models The group model incorporates the effect of one (or more) covariates on the parameters of the structural model. All subjects from the population who share the same set of covariates used to predict differences in a

3

parameter belong to the same group and have the same group estimate for clearance and volume of distribution. An allometric size model is often used to describe the effect of body size on clearance and volume of distribution. Allometry provides a strong theoretical basis for understanding how body function and structure change with body size (10). Total body weight (WT) is used in our example to describe body size. FWTCL,i and FWTV,i represent the fractional changes for the ith subject (with weight WTi ) relative to a subject with a standard weight (WTSTD ) of 70 kg. CLPOP is the population estimate for clearance, and VPOP is the population volume of distribution. CLGRP,WTi is the clearance, and VGRP,WTi is the volume of distribution for subjects with WTi (these group estimates are sometimes called typical values):
FWTCL,i =

WTi WTSTD

0.75 ;

CLGRP,WTi = CLPOP · FWTCL,i FWTV,i =

WTi ; WTSTD

VGRP,WTi = VPOP · FWTV,i The allometric exponent of 0.75 for clearance is important, especially if PK data in pediatric patients are compared with PK data in adult patients (11). Some authors estimate or assume exponents for clearance different from 0.75 without any clear justification for departing from well-based theory. The creatinine clearance [e.g., as estimated by the Cockcroft & Gault formula (12)] can be used to predict renal function. Renal function is often a useful predictor of renal clearance. Equation (12) predicts the creatinine clearance of the ith subject (CLcri ) for a nominal individual with weight 70 kg: 
140 − agei mL CLcri = SCri min SCri (mg/dL) is the serum creatinine concentration of the ith subject, and agei is the age of this subject in years. The intact nephron hypothesis (13) assumes that renal function (including glomerular filtration, secretion, and reabsorption) is proportional to

4

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

Figure 1. Different components of population parameter variability.

the glomerular filtration rate. Glomerular filtration rate is related closely to the renal clearance of creatinine. Therefore, CLcri can be used to predict renal function for the ith subject. A standard glomerular filtration rate (CLcrSTD ) of 100 mL/min for a 70 kg subject is used to scale GFRi : RFi =

CLcri CLcrSTD

The group renal clearance (CLGRP,R ) in subjects with WTi and RFi then becomes: CLGRP,R (WTi , RFi ) = CLPOP,R · RFi · FWTCL,i CLPOP,R is the population estimate of renal clearance in a subject of standard weight with RF of 1. Importantly, CLcri is calculated for a nominal subject with weight 70 kg, as FWTCL,i includes the effect of body size on clearance and as the effect of weight on renal clearance should only be included once. 3

RANDOM EFFECTS MODELS

The random effects model describes the unpredictable variability of the structural model parameters. Two levels of random effects are usually described for PKPD models. The first level describes random effects associated with structural model parameters (i.e., BSVRandom and WSVRandom ). Although these kinds of variability are considered to be random, this random variability ‘‘simply’’ represents a lack of knowledge of the underlying sources of variability.

The second level describes random effects associated with each prediction of an observation. The difference between the observations and model predictions (residuals) are described by the residual unidentified variability (RUV) or residual error. The distribution of PK parameters is often skewed to the right. Studies with small sample size neither allow one to determine the shape of the distribution reliably nor to estimate its variance precisely. As distributions of PK parameters are usually skewed to the right, it is often assumed that PK parameters are log-normally distributed, which means that the logarithm of the individual PK parameters is normally distributed. This distribution has reasonable properties, because some subjects show very large values for clearance, and only positive values for clearance are physiologically possible. The individual predictions of clearance and volume of distribution can be obtained from the group clearance by introducing a random effect: The random effects for clearance (CL) (ηCL,i ) and volume of distribution (ηV,i ) describe the difference (on log-scale) of the individual clearance and volume of distribution for the ith subject from their group estimates CLGRP and VGRP . It is assumed that ηCL and ηV are normally distributed with mean zero and variance BSV2 and BSV2 . CL V   CLi = CLGRP · exp ηCL,i   Vi = VGRP · exp ηV,i The individual clearances (CLi ) and individual volumes of distribution (Vi ) in the

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

equations shown above follow a log-normal distribution. The BSVCL and BSVV are standard deviations of a normal distribution on log-scale and are approximations to the apparent coefficient of variation (CV) on linear scale (see the comment from Beal (14) for a more in depth discussion). For example, a BSV2 estimate of 0.04 yields an apparent CL CV of 20% for clearance. A useful covariate should result in smaller estimates for the BSV of the respective PK parameter. Figure 2 shows an example for the variability of clearance and volume of central and peripheral compartment with or without body weight as covariate. As shown in Fig. 2, inclusion of WT as a covariate reduced the random (unexplained) variability in clearance and volume of distribution. The reduction of unexplained variability is expressed as a fraction of total variance. The relative variance shown in Fig. 2 is the variance that includes WT as covariate divided by the variance without WT. Inclusion of WT as a covariate reduced the unexplained variability by 27.9% for CL, by 29.1% for V1, and by 38.5% for V2.

3.1 Residual Error Another level of variability cannot be described by variability in PK parameters. This variability is called residual error or residual unidentified variability (RUV). The RUV describes the random deviation of an observation from its predicted value in the respective subject. Bioanalytical error, process noise, model misspecification, and WOV of fixed effect parameters contribute to the estimated RUV. Process noise may be caused by the fact that the study is performed in a different way than documented (e.g., a sample was recorded to be taken at 1 hour post dose although it was actually taken at 1.25 hours post dose). Commonly, the RUV is described by a combined additive and proportional error model. C denotes the individual predicted concentration in the absence of residual error, Y is the individual predicted concentration including an additive (SDC ), and a proportional (CVC )

5

residual error component. The random deviates εSDC and εCVC are assumed to be normally distributed with mean zero and standard deviation SDC and CVC . Y = C · (1 + εCVC ) + εSDC The residual variability is a different level of random effects than the random BSV. If one subject has 10 observed concentrations, this subject will have 10 random deviates for εSDC and 10 random deviates for εCVC (one for each predicted observation), but it will only have one random deviate for ηCL and ηV . 4 MODEL BUILDING AND PARAMETER ESTIMATION Model building is a nontrivial task and depends very much on the planned application of the final model. It is traditional to invoke the principle of parsimony (i.e., to use of the simplest model that fulfills the requirements of the planned applications and that is physiologically reasonable as the final model). Mechanistic models (if available) are expected to be superior to empiric models when making predictions outside of the observed range. Model building typically proceeds in steps based on hypothesis tests that compare one model with another. This method may be used in conjunction with informal decision based on the mechanistic plausibility of the model. Parameter estimation is part of the model building process. Each model that is evaluated will involve parameter estimation. The plausibility of the parameter estimates and their identifiability (related to uncertainty of the value) are often used in an informal way to guide model building. 4.1 Hypothesis Testing Hypothesis testing can be used to test the null hypothesis that a more complex model is indistinguishable from a simpler model. Hypothesis testing can be done by the likelihood ratio test (see objective function below) if both models are nested. Nested means that the more complex model becomes equivalent to the simpler model if a parameter in the complex model is fixed to some

6

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

Figure 2. Box-whisker plot for the BSV of CL and the volume of distribution for the central (V1) and peripheral (V2) compartment without and with total body weight as covariate (indicated by the index wt). Boxes show median and interquartile range, whiskers are the 10th to 90th percentiles, and the markers represent 5th to 95th percentiles.

value—usually zero. Examples for nested models are: 1. One, two, or three compartment models with or without lag-time. Multicompartment models become indistinguishable from a one-compartment model if the inter-compartmental clearance is fixed to zero. As lag-time approaches zero, models with lag-time converge to models without lag-time. 2. Models with a) first-order, b) mixedorder (Michaelis-Menten), or c) parallel first-order and mixed-order elimination are nested. • Model c) converges to model a) if the

maximum rate of the mixed-order process (Vmax) is estimated to be zero. • Model c) converges to model b) if the

first-order clearance of model c) is estimated to be zero. • Model b) approaches model a) if the

Michaelis-Menten constant (Km) is much higher (more than 10 × ) than the peak concentration (the ratio of Vmax and Km then equals the firstorder clearance). In a statistically strict sense, a first-order elimination model is not nested within a mixedorder (Michaelis-Menten) elimination model. However, if Km approaches infinity, the objective function of the first-order and the mixed-order model converge to the same value.

3. A model with zero-order elimination is nested within a mixed-order elimination model if Km is zero. Examples of models that are not nested: 1. A model with first-order input (absorption) is not nested within a model with zero-order input. 2. Models that use different covariates are not nested (e.g., a model with total body weight as size descriptor is not nested within a model with lean body mass as size descriptor).

4.2 Objective Function Population software packages calculate (or approximate) the log-likelihood. This likelihood is the probability of observing the data given the proposed model and parameters. In maximum likelihood estimation, the set of parameters (fixed and random effects) that maximizes this probability is searched. Three general likelihood calculation procedures exist: (1) exact solutions for exact likelihoods (only for nonparametric methods), (2) exact solutions to approximate likelihoods (e.g., NONMEM; GloboMax LLC, Hanover, MD), and (3) approximate solutions to exact likelihoods (e.g., MC-PEM). The exact or approximated likelihood is used as the objective function during the estimation process that searches for the highest likelihood of the data for a specific model and set of model parameters. Usually, population programs

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

either report the log-likelihood or report −2 × the log-likelihood (-2LL). The difference in -2LL between two models can be used to test the null hypothesis that the more complex model is equivalent to the simpler model. If one parameter is added to a model, then a drop by 3.84 points of -2LL is associated with a P-value of 0.05, under the assumption that the difference in -2LL is approximately chisquare distributed. This procedure is called ¨ a likelihood ratio test. Wahlby et al. (15–17) studied the type I error rates of the likelihood ratio test in NONMEM in detail. Population models use the observations from all subjects simultaneously, which is a considerable advantage over the standard two-stage approach. For example, when some subjects do not have data during the absorption phase, it is possible to deduce their absorption parameters based on other subjects with absorption phase data. This ‘‘borrowing’’ of information is a key feature of population models. 4.3 Bootstrap Bootstrapping is a powerful concept to derive the uncertainty of all parameters of a population model. Importantly, confidence intervals can be constructed for fixed effects, random effects, and for derived statistics. Confidence intervals derived via bootstrapping are believed to represent the uncertainty of model parameters better than confidence intervals based on asymptotic standard error estimates and assuming the uncertainty is normally distributed. Standard error estimates are often difficult to compute for some models, and it may not be possible to determine standard errors. A variety of bootstrap techniques can be used. In a parametric bootstrap, new datasets are generated by simulating from the final population model. These simulated datasets are then estimated by the final population PK model. This method is called parametric bootstrapping, because the new datasets are generated from parametric distributions for the random effects in the population model. A parametric bootstrap provides robust measures for the precision and bias of the population parameter estimates under the assumption that the chosen population model is correct.

7

A nonparametric bootstrap involves generation of new datasets by random samples from the original dataset with replacement (‘‘resampling with replacement’’). The same number of subjects as in the original dataset is randomly drawn for each new dataset. Each subject might be drawn multiple times. The population model parameters are reestimated for each of the bootstrap datasets and the median, and nonparametric 95% confidence interval (2.5th to 97.5th percentile) are calculated from the model parameter estimates of all bootstrap replicates. An example of the results of a nonparametric bootstrap is shown in Table 1 for a two-compartment model with zero-order input. The last two columns of Table 1 indicate the slight asymmetry of the confidence intervals (e.g., for random effects parameters). The BSV terms in Table 1 are the square roots of the estimated variance on log-scale. These square roots are commonly interpreted as an approximation to the coefficient of variation (CV) of a log-normal distribution. As pointed out by Beal (14). these quantities should be called apparent CVs, that is the apparent CV of CL was 30.7% (20.1% to 41.4%) [median (95% confidence interval)] in Table 1. The number of bootstrap samples depends on which statistics are to be derived from the bootstrap replicates. About 50 replicates may be sufficient to get a reasonable estimate of the median and standard error of population model parameters. However, 1,000 or 2,000 replicates are probably required to obtain robust estimates of a 90% or 95% confidence interval. 4.4 Bayesian Estimation The word ‘‘Bayesian’’ is used in various circumstances in PKPD. A key feature of Bayesian techniques is that prior knowledge is incorporated into the estimation of model parameters. Maximum a posteriori (MAP) Bayesian estimation as implemented in ADAPT II (or in the NONMEM POSTHOC step) assumes that the parameters of the population model (=priors) are known exactly (without uncertainty). If the individual PK parameters for a new subject are to be estimated, MAP-Bayesian estimation provides

8

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

Table 1. Population means and between subject variability (apparent \% CV) of a two compartment PK model Parameter abbreviation

Explanation

Point estimate & non-parametric 95% confidence interval Median P2.5 P97.5

Confidence interval limit divided by median P2.5 P97.5

Fixed effects CL (L h−1 ) V1 (L)

Total clearance Volume of central compartment Volume of peripheral comp. Intercompartmental clearance

9.52 9.45

8.37 8.59

11.0 10.5

88% 91%

116% 111%

28.4

23.7

33.4

83%

118%

4.73

4.24

5.35

90%

113%

BSV(CL) BSV(V1) BSV(V2) BSV(CLic )

Between subject variability of the respective PK parameter

0.307 0.243 0.380 0.251

0.201 0.414 66%∗ 0.151 0.321 62%∗ 0.159 0.499 42%∗ 0.188 0.315 75%∗ Correlations between random effects

r(CL, V1) r(V1, V2) r(V2, CLic )

Coefficient of correlation between pairs of random effects

0.423 0.091 −0.002

CVc SDc (mg/L)

Proportional error Additive error

0.134 0.0932

V2 (L) CLic (Lh−1 )

Random effects

−0.062 −0.330 −0.648

135%∗ 132%∗ 131%∗ 125%∗

0.809 0.454 0.740 Error model

0.124 0.0570

0.145 0.135

93%∗ 61%∗

109%∗ 145%∗

Data are medians and 95\% confidence intervals from a non-parametric bootstrap with 1,000 replicates.  These numbers are the 2.5th percentile and the 97.5th percentile divided by the median (see columns 3 to 5). These confidence intervals were also asymmetric, if ratios were calculated on variance scale.

the most likely individual parameter estimates given the prior knowledge ( = population PK model and its parameters) and the observations of the new subject. Software packages like BUGS (which includes PKBUGS and WinBUGS; MRC Biostatics Unit, Cambridge, UK) offer the possibility to account for uncertainty in the priors (i.e., to specify the uncertainty of the parameters of the population PK model). Consider the following scenario: A population PK model was derived in healthy volunteers based on a dataset with 24 healthy volunteers with frequent sampling. New data in patients with sparse sampling (1–5 samples per patient) become available. It is unknown whether the patient population has similar average population PK parameters and a similar variability in these parameters. Bayesian analysis (e.g., in BUGS) allows one to use the population

PK model in healthy volunteers for analysis of the sparse dataset in patients. It is often observed that the patient population has a larger volume of distribution and a lower clearance relative to the healthy volunteers. If enough information is in the patient dataset to estimate the differences in population PK parameters, a full Bayesian approach as implemented in BUGS® allows one to estimate population PK model parameters specific for the patient population. The population mean parameters and BSV can be different in the patient population compared with the healthy volunteers. The population PK model in healthy volunteers is used to support this estimation, and the population PK model parameters for the patients will depend on the estimates of the population PK parameters for the healthy volunteers ( = prior information with uncertainty).

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

4.5 Mixture Models If one estimates the PK parameters in each subject individually, one may observe that one group of subjects has high clearances, whereas the other group of subjects has lower clearances. This kind of bimodal distribution for clearance might be predictable using a fixed effects model based on a covariate such as genotype. However, if an explanatory covariate is not available (e.g., if the genotype cannot be tested), then a random effects model can be used. Mixture models offer the possibility to use a probabilistic (random effect) model to assign each subject to a particular category. This category does not rely on the existence of an observed covariate but uses the best fit of a subject’s data to alternative models to predict the most likely category. The category can be used to assign different parameters or even different models to describe the data of a particular subject. Mixture models are a powerful feature of population software packages. They are often useful when BSV seems to be very large and cannot be reduced by available covariates. They can also be applied when a multimodel distribution is known (e.g., a genetic polymorphism for clearance) but no genotypic or phenotypic information (covariate) is available. 5

SOFTWARE

Various population software packages available. Perhaps the most important differences among them are the estimation method, flexibility, robustness, and estimation time. Important issues with regard to flexibility are: • Number of dependent and independent

variables a program can handle. • Ability to handle cumulative outputs

(like excretion into urine). • Ability to invoke a program using a scripting mechanism (important for advanced meta-analysis applications such as bootstrapping). • Possibility to parallelize the computations in order to reduce overall runtime.

9

Some programs (and algorithms) for population analyses (in alphabetical order) are as follows: • Adapt V (EM algorithm, will be avail-

able soon) • BUGS (WinBUGS and PKBUGS) • Kinetica (includes P-Pharm) • Monolix (SAEM algorithm) • MW\Pharm & MultiFit • NLME in S-Plus • NLMIX • NONMEM (FO, FOCE, and LAPLA-

CIAN method) • NPML • PDx-MC-PEM (MC-PEM algorithm) • PEM • S-ADAPT (MC-PEM algorithm) • SAS (proc nlmixed; SAS Institute Inc.,

Cary, NC) • USC*PACK (NPEM, NPAG, and NPOD

algorithm) • WinNonmix

These estimation algorithms use either a parametric or a nonparametric parameter variability model. The programs calculate either the exact likelihood, an approximation to the true likelihood, or an approximate likelihood. The differences between those programs were reviewed by Aarons (18) and Pillai et al. (1). Girard and Mentr´e (19) presented a blinded comparison of various population software packages. Bauer et al. (20) systematically compared various estimation algorithms for population PKPD models. An in-depth discussion of the appropriateness of one or the other algorithm is beyond the scope of this article. We would like to mention only the following two points from our point of view: 1. Use of the so-called FO method (firstorder method) for population analyses seems only warranted if extremely long run times prevent the use of the FOCE method or algorithms that calculate the exact likelihood. Both the FOCE method (with or without interaction option) and the FO method use an

10

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

approximate likelihood. However, the approximation of the FOCE method is much better than the approximation of the FO method, especially if the BSV is large. 2. The programs and algorithms shown above have different capabilities of being parallelized. Parallel computing can offer an increase in computation speed but the only program currently available that offers this feature is SADAPT.

6

NONMEM

NONMEM was the first population software package and is currently the most widely used population program. Most experience with any population program is currently available for NONMEM. Its current version (NONMEM VI) offers the FO, FOCE, and LAPLACIAN method for estimation. NONMEM uses parametric parameter variability models and offers the possibility of specifying mixture models. NONMEM VI also offers the possibility to include a so-called ‘‘frequentists’ prior’’ to incorporate knowledge from a previous population PK analysis (e.g., from literature reports). Archives from a very active NONMEM user (NMusers) e-mail discussion group can be found under the following link: http://www.cognigencorp.com/nonmem/nm/. 6.1 Other Programs A list of other software packages and algorithms is shown above. Two key features of other programs are as follows: 1. Ability to specify the parameter variability model by a nonparametric distribution as implemented in the USC*PACK that uses the NPEM, NPAG, or NPOD algorithm. This method makes no assumptions on the distribution of parameters. 2. Incorporation of prior information in a full Bayesian analysis as implemented in BUGS® (including WinBUGS and PKBUGS). This method is flexible, and it allows incorporation of prior information (with uncertainty) from another

population PK analysis or to perform a meta-analysis.

7 MODEL EVALUATION Model evaluation depends on the planned application of the model. Sometimes, the terms validation or qualification are used. It is arguable whether a population model can be fully validated. The authors believe that evaluation of a population model for an intended application is probably the best one can show. Various methods of model evaluation are used. Some more popular methods are listed below. 7.1 Residuals Residuals are the differences between the observations and model predictions. It is often helpful to use both the individual predictions and the population predictions in a residual analysis. The latter are the predictions for an individual using the group parameter values, which use the covariates of the current subject but with all random effects, η and ε, set to zero. A residual analysis usually involves the following plots: 1. Observations versus individual predictions plot on linear scale and on log-log scale. 2. Observations versus population predictions plot on linear scale and on log-log scale (use of a log 2 scale may be advantageous). 3. Residuals versus time plot. 4. Residuals versus observations. Visual inspection of plots 1 and 2 on loglog scale is important, as plots on linear scale do not reveal misfits for low concentrations. Ideally the data points in plots 1 and 2 should fall around the line of identity without bias throughout the whole concentration range. If the observation versus individual prediction plot does not show any systematic bias, it does not always mean that the model has good predictive performance [see Bulitta and Holford (21) for details]. If this plot shows systematic bias, it usually means that the

Figure 3. Visual predictive check using an empirical effect compartment model to describe the delay in effect of warfarin on PCA. The lines are the median and the 90% prediction interval obtained by simulation from the PKPD model. The symbols are the observed PCA values.∗

11

Prothrombin Complex Activity

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

structural model is not flexible enough to describe the observed profiles. A systematic bias in the observation versus population prediction plot points to potential problems with the predictive performance of the model. 7.2 Predictive Checks Visual predictive checks are a tool used to compare the predictive performance of competing models visually. A visual predictive check involves a stochastic simulation of the individual predicted concentrations for a large number of subjects (usually between 1,000 and 10,000). It is often helpful to calculate these predicted concentrations at nominal time points (theoretical times). The percentiles of those 1,000 to 10,000 profiles are calculated and plotted on top of the observations of all subjects. More sophisticated applications of the predictive checks were presented by Yano et al. (22). Figure 3 shows an example for a visual predictive check. A PKPD model was used to describe the time course of prothrombin ∗ The CL is the clearance of warfarin and k e0 is a first-order transfer rate constant between the central compartment and the hypothetical effect compartment. The word ‘‘hypothetical’’ indicates that the amount in this compartment is assumed to be negligible. The E0 is the baseline, Emax is the maximum effect, and EC50 the concentration associated with half-maximal effect.

complex activity (PCA) after a large oral dose of warfarin. An empirical effect compartment model (23) was used to predict the delay in relation to warfarin plasma concentration. The model captures the central tendency of the raw data adequately, but the BSV is too large. Ideally, 10% of the observations should fall outside the 90% prediction interval (595% percentile) at each time point. Figure 4 shows the same observations but with a physiologically based turnover model (24) to describe the changes in clotting factors that determine PCA. It is expected to be a superior model on mechanistic grounds. A comparison with Fig. 3 demonstrates that the model misspecification from the effect compartment model causes an overprediction of variability. 7.3 Prediction Discrepancy A method related to the visual predictive check is the prediction discrepancy (25). The prediction discrepancy compares the distribution of observed and simulated values in a more formal way that does not rely on subjective visual assessment. 7.4 Cross-Validation and External Validation Cross-validation is a form of internal validation of a population model. The observations are randomly split into two parts: an index dataset (for example, that includes 67% of subjects) and a validation dataset (that

12

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

includes the remainder of subjects). The population model is estimated from the index dataset without using the validation dataset. This estimated population model is then used to predict the concentrations of the validation dataset. If no systematic bias is in the model predictions for the validation dataset, the population PK model derived from the index dataset is concluded to be appropriate for prediction of profiles for subjects of the validation dataset. The random splitting between the index and validation dataset can be repeated multiple times. External validation is supposed to be the gold standard of model validation. An external dataset from a similar subject population as used in the clinical trial from which the population model was estimated must be available. The concentrations are predicted for the external dataset and then compared with the observations of the validation dataset as for an internal validation. Criteria for validation should be specified prior to the analysis, but we are not aware of examples where it has been done. Almost always, the ‘‘validation’’ is claimed to be successful based on the authors’ assessment. Unfortunately, comparable clinical trials are not available in most cases. Differences in bioanalytical assays and between clinical sites may bias the results of an external validation. 8

STOCHASTIC SIMULATION

Deterministic simulations predict the concentration time profile of one specific patient based on one set of PK parameters. Stochastic simulations account for the true between patient variability and therefore allow one to predict the range of plasma concentrations for a whole patient population. Instead of the phrase ‘‘stochastic simulations’’ the phrase ∗ The CL is the clearance of warfarin that is assumed to inhibit the input of the prothrombin complex activity (PCA). The Imax is the maximum inhibition and IC50 is the warfarin concentration in the central compartment (C1) that results in half-maximal inhibition. The kin describes the zeroorder input into the PCA compartment, and kout is the first-order rate constant of loss from the PCA compartment.

‘‘Monte Carlo simulation’’ is frequently used. In this context, the term Monte Carlo is used to describe a simulation technique and not an estimation technique like in the Monte Carlo Parametric Expectation Maximization algorithm. 8.1 Experimental Design One of the greatest merits of population PKPD modeling is its ability to evaluate and to optimize clinical trial designs. Software packages like WinPOPT (26), PFIM (27), and others can provide optimal sampling times, the optimal number of treatment groups, and the optimal number of subjects per group. The associated standard errors for the population model parameters can be predicted. Clinical trial simulation also offers the ability to predict the power of clinical trials, but it is computationally more difficult because of the time consuming nature of the repeated simulation and analysis steps and the lack of an automated procedure for converging on an optimal design. Clinical trial simulation can be performed in any of the abovementioned software packages (e.g., NONMEM, S-Adapt, SAS, etc.). The Pharsight® Trial Simulator™ is available as a specialized tool for comparison of userspecified clinical trial designs. Optimization of clinical trial design is very valuable, when large clinical trials are planned. A large amount of resources can be saved by optimizing the experimental design. 9 APPLICATIONS Population PKPD and disease progression modeling offers a variety of applications for clinical drug development, regulatory decision-making, and understanding characteristics of the disease in the patient population of interest. 9.1 Drug Development Clinical drug development takes about 5–10 years and total preapproval costs amount to about 802 million US dollars [adjusted to the year 2000, (28)] per approved drug. About 89% (29) of the drugs that enter the clinical phase do not reach the market. Population

13

Prothrombin Complex Activity

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

Figure 4. Visual predictive check using physiologically based turnover model to describe the delay in effect of warfarin on PCA.∗

PKPD can accelerate drug development, for example, by design and analysis of clinical trials (30). Population PKPD modeling can be powerful if a large number of potentially important independent variables are used (patient covariates, co-medications, etc.) as is usually the case during phase II and III of clinical drug development. More rational dosage regimens can be derived by population PKPD techniques that improve the ability to individualize drug dosing and to include such recommendations in the drug label. Population PKPD modeling can also provide valuable information for selecting dosage regimens for phase II or III clinical trials and can support ‘‘go / no-go decisions.’’ The meaning of potentially important biomarkers and their relationship to the disease progression and drug response can be evaluated. After a predictable and meaningful relationship between a biomarker and clinical outcome is established, this biomarker can be used as surrogate endpoint to make regulatory decisions. A biomarker is a directly observable quantity such as blood pressure or an EEG response. A surrogate marker is a biomarker used for regulatory decisions. The clinical outcome can be thought of as something that

the patient feels or cares about (e.g., the probability of stroke, frequency of admission to hospital, or relief from pain). 9.2 Regulatory Science Population PKPD models support a more rational decision making, especially for large clinical trials. Population models consider all observations in a single data analytical step and account for the time course of concentrations and drug effect(s). These models provide a sound basis for regulatory decision making. Techniques like nonparametric bootstrapping allow one to estimate the confidence intervals of model parameters that can be used for statistical comparisons. A drug can be statistically shown to be effective if the confidence interval for efficacy (Emax: maximal drug effect) does not include zero. Population PKPD models are also very helpful for analysis of studies in patient populations with considerably different sets of covariates. This feature is used in bridging studies in which results in Caucasians may be translated to Japanese patients. 9.3 Human and Disease Biology Population PKPD modeling offers a possibility to improve our understanding of the mechanism and time course of disease progression. Such analyses are performed more

14

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS

rarely than a population PK analysis. However, disease progression models that incorporate the system biology will probably be important in the future and are likely to support clinical science and academic research. A more in-depth understanding of the human and disease biology offers the potential to guide development of new drugs. 10

ACKNOWLEDGMENT

¨ Jurgen Bulitta was supported by a postdoctoral fellowship from Johnson & Johnson. REFERENCES 1. G. C. Pillai, F. Mentre, and J. L. Steimer, Nonlinear mixed effects modeling - from methodology and software development to driving implementation in drug development science. J. Pharmacokinet. Pharmacodyn. 2005; 32: 161–183. 2. L. Zhang, S. L. Beal, and L. B. Sheiner, Simultaneous vs. sequential analysis for population PK/PD data I: best-case performance. J. Pharmacokinet. Pharmacodyn. 2003; 30: 387–404. 3. L. Zhang, S. L. Beal, and L. B. Sheiner, Simultaneous vs. sequential analysis for population PK/PD data II: robustness of methods. J. Pharmacokinet. Pharmacodyn. 2003; 30: 405–416. 4. L. B. Sheiner and J. L. Steimer, Pharmacokinetic/pharmacodynamic modeling in drug development. Annu. Rev. Pharmacol. Toxicol. 2000; 40: 67–95. 5. L. B. Sheiner and T. M. Ludden, Population pharmacokinetics/dynamics. Annu. Rev. Pharmacol. Toxicol. 1992; 32: 185–209. 6. R. Jelliffe, Goal-oriented, model-based drug regimens: setting individualized goals for each patient. Ther. Drug Monit. 2000; 22: 325–329. 7. H. Sun, E. O. Fadiran, C. D. Jones, et al., Population pharmacokinetics. A regulatory perspective. Clin. Pharmacokinet. 1999; 37: 41–58. 8. L. Sheiner and J. Wakefield, Population modelling in drug development. Stat. Methods Med. Res. 1999; 8: 183–193. 9. N. H. G. Holford, Target concentration intervention: beyond Y2K. Br. J. Clin. Pharmacol. 1999; 48: 9–13. 10. G. B. West, J. H. Brown, and B. J. Enquist, The fourth dimension of life: fractal geometry and allometric scaling of organisms. Science 1999; 284: 1677–1679.

11. B. J. Anderson, K. Allegaert, J. N. Van den Anker, V. Cossey, and N. H. Holford, Vancomycin pharmacokinetics in preterm neonates and the prediction of adult clearance. Br. J. Clin. Pharmacol. 2007; 63: 75–84. 12. D. W. Cockcroft and M. H. Gault, Prediction of creatinine clearance from serum creatinine. Nephron 1976; 16: 31–41. 13. J. M. Hayman, N. P. Shumway, P. Dumke, and M Miller, Experimental Hyposthenuria. J. Clin. Invest. 1939; 18: 195–212. 14. S. Beal, Apparent coefficients of variation. Available: http://gaps.cpb.ouhsc.edu/nm/ 91sep2697.html, 1997. 15. U. Wahlby, M. R. Bouw, E. N. Jonsson, and M. O. Karlsson, Assessment of type I error rates for the statistical sub-model in NONMEM. J. Pharmacokinet. Pharmacodyn. 2002; 29: 251–269. 16. U. Wahlby, E. N. Jonsson, and M. O. Karlsson, Assessment of actual significance levels for covariate effects in NONMEM. J. Pharmacokinet. Pharmacodyn. 2001; 28: 231–252. 17. U. Wahlby, K. Matolcsi, M. O. Karlsson, and E. N. Jonsson, Evaluation of type I error rates when modeling ordered categorical data in NONMEM. J. Pharmacokinet. Pharmacodyn. 2004; 31: 61–74. 18. L. Aarons, Software for population pharmacokinetics and pharmacodynamics. Clin. Pharmacokinet. 1999; 36: 255–264. 19. P. Girard and F. Mentr´e, A comparison of estimation methods in nonlinear mixed effects models using a blind analysis. Abstract 834. Abstracts of the Annual Meeting of the Population Approach Group in Europe2005, p. 14. 20. R. J. Bauer, S. Guzy, and C. Ng, A survey of population analysis methods and software for complex pharmacokinetic and pharmacodynamic models with examples. AAPS J. 2007; 9: E60–83. 21. J. B. Bulitta and N. H. G. Holford, Assessment of predictive performance of pharmacokinetic models based on plasma and urine data. Brisbane, Australia: PAGANZ: Population Approach Group in Australia & New Zealand, 2005. 22. Y. Yano, S. L. Beal, and L. B. Sheiner, Evaluating pharmacokinetic/pharmacodynamic models using the posterior predictive check J. Pharmacokinet. Pharmacodyn. 2001; 28: 171–192. 23. L. B. Sheiner, D. R. Stanski, S. Vozeh, R. D. Miller, and J. Ham, Simultaneous modeling of pharmacokinetics and pharmacodynamics:

POPULATION PHARMACOKINETIC AND PHARMACODYNAMIC METHODS application to d-tubocurarine. Clin. Pharmacol. Ther. 1979; 25: 358–371. 24. N. L. Dayneka, V. Garg, and W. J. Jusko, Comparison of four basic models of indirect pharmacodynamic responses. J. Pharmacokinet. Biopharm. 1993; 21: 457–478. 25. F. Mentre, S. Escolano, Prediction discrepancies for the evaluation of nonlinear mixedeffects models. J. Pharmacokinet. Pharmacodyn. 2006; 33: 345–367. 26. S. B. Duffull, N. Denman, J. A. Eccleston, and H. C. Kimko, WinPOPT - Optimization for Population PKPD Study Design. Available: www.winpopt.com, 2006. 27. S. Retout, F. Mentr´e, PFIM - Population designs evaluation and optimisation. Available: http://www.bichat.inserm. fr/equipes/Emi0357/ download.html. 28. J. A. DiMasi, R. W. Hansen, and H. G. Grabowski, The price of innovation: new estimates of drug development costs. J. Health Econ. 2003; 22: 151–185. 29. I. Kola, J. Landis, Can the pharmaceutical industry reduce attrition rates? Nat. Rev. Drug Discov. 2004; 3: 711–715. 30. P. Lockwood, W. Ewy, D. Hermann, and N. Holford, Application of clinical trial simulation to compare proof-of-concept study designs for drugs with a slow onset of effect; an example in Alzheimer’s disease. Pharm. Res. 2006; 23: 2050–2059.

FURTHER READING An introduction to pharmacometrics with many links to other sites. Available: http://www. health.auckland.ac.nz/pharmacology/staff/nholford/pkpd/pkpd.htm. A. J. Atkinson, C. E. Daniels, R. Dedrick, and C. V. Grudzinskas, Principles of Clinical Pharmacology. San Diego, CA: Academic Press, 2001. P. L. Bonate. Pharmacokinetic-Pharmacodynamic Modeling and Simulation. New York: Springer, 2005. H. C. Kimko and S. B. Duffull, Simulation for Designing Clinical Trials: A PharmacokineticPharmacodynamic Modeling Perspective. New York: Marcel Dekker, Inc., 2002. Lecture (including video and PowerPoint) by Dr. Raymond Miller in the Principles of Clinical Pharmacology course by Dr. Arthur J. Atkinson at the NIH. Available: http://www.cc. nih.gov/researchers/training/principles.shtml.

15

P. Macheras and A. Iliadis, Modeling in Biopharmaceutics, Pharmacokinetics and Pharmacodynamics: Homogeneous and Heterogeneous Approaches. New York: Springer, 2005. I. Mahmood, Interspecies Pharmacokinetic Scaling: Principles And Application of Allometric Scaling. Rockville, MD: Pine House Publishers, 2005. Pharmacometrics homepage of the American College of Clinical Pharmacology (AACP). Available: http://www.accp1.org/cgi-bin/start.cgi/ pharmacometrics/disclaimer.htm. PharmPK online archives including many links to population software packages and learning material. Available: http://www.boomer. org/pkin/. J. C Pinheiro and D. M. Bates, Mixed Effects Models in S and S-Plus. New York: Springer, 2002. D. J. Spiegelhalter, K. R. Abrams, and J. P. Myles, Bayesian Approaches to Clinical Trials and Health-Care Evaluation (Statistics in Practice). Chichester, UK: John Wiley & Sons, 2004.

CROSS-REFERENCES Phase II/III Trials Mixed Effects Models Bayesian Approach Meta-analysis Clinical Trial Simulation

POSTMENOPAUSAL ESTROGEN/PROGESTIN INTERVENTIONS TRIAL (PEPI)

1.1 Sponsorship PEPI was sponsored by the National Heart, Lung, and Blood Institute. The initial design was conceived by institute staff. A request for proposals was prepared to solicit interest from qualified institutions. Potential investigators prepared applications that addressed the scientific issues and proposed methods for the study; these documents were peerreviewed and scored for scientific merit. PEPI was conducted as a collaborative research agreement through the Coordinating Center at Bowman-Gray School of Medicine at Wake Forest University.

ROBERT D. LANGER DRSciences, LLC, Jackson, Wyoming and Geisinger Health System, Danville, Pennsylvania

During the 1980s, several observational studies of varying designs conducted in different populations found an association between estrogen replacement in postmenopausal women and lower rates of coronary heart disease (CHD) and possibly stroke (1–4). Because CHD was and remains the leading cause of death in women, there was substantial interest in testing this potential benefit. Nevertheless, there was cause for skepticism because studies that compared the characteristics of HRT users to nonusers suggested a potential healthy user bias (5). By the mid1980s, some investigators and clinicians were calling for a clinical trial to test whether hormone therapy prevented CHD, and a conference that explored this possibility was held under the auspices of the U.S. National Heart, Lung, and Blood Institute (6). For a variety of reasons, including the recognition that, CHD rates are relatively low in women near the age of menopause, and that the estrogen effect on CHD was believed to be mediated principally by lipid benefits—a mechanism that takes years to become evident in generally healthy people—a trial with CHD as an endpoint was projected to require upward of 30,000 women followed for about 10 years. Given the magnitude of resources required, this idea was tabled pending more evidence. The concept of the PEPI study took root in this environment. 1

1.2 Clinical Centers, Participants, and Investigators PEPI was conducted at seven clinical centers across the United States, which involved a cross-section of U.S. women. Clinical centers were located at Stanford University, the University of California—Los Angeles, the University of California—San Diego, the University of Texas at San Antonio, the University of Iowa, Johns Hopkins University, and George Washington University. The study enrolled 875 women ages 45 to 64 years old and between 1 and 10 years postmenopausal at study baseline (8). Investigators were drawn from diverse disciplines, which included gynecology, general and internal medicine, cardiovascular epidemiology, laboratory medicine, biostatistics, and public health. 1.3 Considerations in the Selection of Treatments Kennedy DL, Baum C, Forbes MB. Noncontraceptive estrogens and progestins: use patterns over time. The PEPI trial was planned at a time when hormone therapy was in flux (9). A major goal was to provide clinically relevant information on treatments expected to be mainstream when the study was completed. Until approximately a decade before, nearly all postmenopausal hormone therapy was estrogen alone. Typical doses were 1.25 to 2.5 mg of conjugated equine estrogens. In the mid-1970s,

DESIGN AND OBJECTIVES

The primary objective of PEPI was to compare a variety of treatment regimens expected to be in common clinical use when the results were published for their effects on cardiovascular risk factors. Secondary stipulated outcomes included endometrial safety, changes in bone density, and effects on quality of life (7).

Wiley Encyclopedia of Clinical Trials, Copyright  2008 John Wiley & Sons, Inc.

1

2

POSTMENOPAUSAL ESTROGEN/PROGESTIN INTERVENTIONS TRIAL (PEPI)

several studies that used different methods and that were conducted in different populations linked estrogen to endometrialcarcinoma in women with a uterus (10–13). Because approximately 60% of postmenopausal women are in this category, the recognition of this association resulted in a steep decline in estrogen use for several years, and a parallel search for ways to protect against the endometrial hyperplasia associated with this risk. By the early 1980s, several clinical trials had demonstrated that the addition of a progestin (a synthetic progesterone) for one third to one half of a monthly cycle of estrogen protected against endometrial hyperplasia (14–16). A variety of dosing schedules and drug combinations were developed for women with a uterus; the typical dose of estrogen was reduced to about half the prior amount with 0.625 mg daily becoming standard, and progestogens such as medroxyprogesterone acetate, megestrol, levonorgestrel, and norethindrone acetate were added for 8 to 15 days per monthly cycle. Some clinicians favored 21 to 25 days of estrogen per month, whereas others favored continuing estrogen daily (17). As PEPI was conceived at the NHLBI, evidence was developing to suggest that adding progestogen changed the effects of estrogens on lipids, particularly the effect on highdensity lipoprotein (HDL). Questions were generated regarding potential differences between progestogens in the magnitude of this effect (17). Clinicians recognized that, at least for some women, cyclical progestogen seemed to promote fluid shifts, breast tenderness, bloating, and irritability. This finding generated interest in altered dosing patterns that could still prevent endometrial hyperplasia. The concept of using a lower dose of progestogen continuously along with continuous estrogen began to emerge on the theory that this could induce endometrial atrophy (18,19). From a lipid perspective, the relative effect of a lower cumulative progestogen dose with continuous exposure compared with a higher episodic dose was unknown but worthy of study. Other emerging treatments, with limited or no availability in the United States at the time, included nonoral estrogens and progesterone.

2 STUDY DESIGN 2.1 Duration PEPI was planned as study of intermediate markers that were likely to represent mechanisms that linked hormone therapy and cardiovascular disease. Important among these were lipids, and based on data from lipid studies conducted in the same era, a treatment interval of 3 years was felt to be appropriate (7). This interval would provide adequate time for effects to reach steady state, and it would also provide adequate time to evaluate adverse effects, tolerability, and quality of life, as well as effects in organ systems that react more slowly, which include bone. 2.2 Trial Design PEPI was a randomized, double-blind, placebo-controlled clinical trial. 2.3 Treatments Treatments were chosen primarily based on their contribution to the data on which the hypothesis of coronary benefit was based, and on the principle that PEPI should focus on treatments that were likely to represent clinical use in the United States when the study was completed (7). Conjugated equine estrogens (CEE) were selected for the estrogen component of all active treatments because most observational data reflected the use of this compound, and because it was the dominant estrogen in the United States at the time. The CEE dose was set at 0.625 mg, which was the more conservative dose that had become common in the years just prior to the study. Two progestogens were selected: medroxyprogesterone acetate (MPA), which was the dominant drug in clinical use at the time, and micronized progesterone, which was not then available in the United States. MPA was tested in two patterns, the then common sequential regimen of 10 mg for 12 days per cycle, and the continuous pattern of 2.5 mg daily that was developing at that time. Micronized progesterone was selected because of concern that available synthetic

POSTMENOPAUSAL ESTROGEN/PROGESTIN INTERVENTIONS TRIAL (PEPI)

3

Table 1. PEPI Treatment Groups Estrogen

Progestogen

CEE 0.625 mg daily CEE 0.625 mg daily CEE 0.625 mg daily CEE 0.625 mg daily Placebo

Placebo MPA 10 mg for 12 days per cycle MPA 2.5 mg daily Micronized progesterone for 12 days per cycle Placebo

progestins, which include MPA, had androgenic effects that could attenuate the effects of estrogen on cardiovascular risk factors. Micronized progesterone is a C-21 compound identical to endogenous progesterone but prepared in a manner that created small particles in an oil base to facilitate absorption through the gut; it was given as 200 mg for 12 days per cycle. To address the possibility that progestogens might add benefit for some outcomes in addition to providing endometrial protection (e.g., bone, quality of life), women with and without a uterus were randomized equally to all treatments. Five treatment conditions were defined as shown in Table 1. 2.4 Eligibility Primary eligibility criteria were age between 45 and 64 years, and at least 1, but not more than 10 years postmenopausal. Menopausal status was assessed by time since last menstrual period or time since oophorectomy, as well as hormone levels (follicle stimulating hormone and estradiol). Exclusions included endometrial hyperplasia on baseline study biopsy; hypertension that required more than two drugs; coronary heart disease that required antiarrhythmic medication; congestive heart failure; insulin-dependent diabetes; any major cancer within 5 years of baseline; malignant melanoma, endometrial, or breast cancer ever; nontraumatic hip fracture; or a thromboembolic event associated with estrogen use. Women with myocardial infarction more than 6 months prior to baseline were eligible so long as they did not require management as noted above. Exclusions for potential adherence issues included severe vasomotor symptoms or failure to maintain adequate compliance on a (single blind) 1-month cycle of placebo medications (7).

3

OUTCOMES

3.1 Cardiovascular The primary motivation for studying hormone therapy regimens was the potential benefit in preventing CHD. Accordingly, cardiovascular risk factors were to be primary. It was recognized that effects could occur on multiple mechanisms and that these effects could even oppose one another (7,20). The hypothesis that estrogen could reduce CHD through increased HDL was prominent, and HDL was recognized to be the strongest lipid predictor in women. Accordingly, it was set as primary but with other lipids, which included low-density lipoprotein (LDL) and triglycerides, also to be assessed. It was decided to draw fasting blood for lipids at baseline and every 6 months, because this measurement would provide an adequate interval to observe changes and would yield 6 data points for compliant participants to assess longitudinal trends. However, it was clearly understood that lipids were unlikely to be the only important pathway that moderated hormone effects on CHD risk. Researchers were concerned that hormones might impact insulin resistance, which is a particularly strong cardiovascular disease risk factor in women. The most practical assessment for this factor on an outpatient basis was a glucose tolerance test. Striking a balance between the science and participant comfort, fasting, and 2-hour postglucose load samples for insulin and glucose were obtained at baseline and yearly. Insulin levels were stipulated as the primary factor. Researchers were suspicious that hormones could elevate systolic blood pressure, which could potentially worsen that risk factor. Accordingly, systolic and diastolic blood pressures were measured following a 5-minute rest using random-zero sphygmomanometers at each

4

POSTMENOPAUSAL ESTROGEN/PROGESTIN INTERVENTIONS TRIAL (PEPI)

visit (every 6 months). Finally, hemostatic effects were developing as a new class of cardiovascular risk factors, and the effects of hormone regimens on these markers were unknown. Fibrinogen was the most widely available assay and was measured at baseline, 1 year, and 3 years on all participants. Specimens for other markers of hemostasis, and for renin and aldosterone, were obtained on a subsample of women that represented three of the seven participating centers. 3.2 Endometrial Safety The primary safety outcome in PEPI was endometrial disease. A normal baseline endometrial biopsy was required for eligibility. As noted earlier, hormone therapy regimens were in flux (17). A variety of dosing patterns, compounds, and cumulative doses was in use, and a fairly radical new pattern— continuous progestin—was in development (18,19). No comparative data existed on the endometrial safety of these alternatives. Clearly, these data were necessary to help determine the relative merits of these treatments in clinical practice. Because not all disease is symptomatic, it was decided to obtain endometrial biopsies on all women with a uterus annually (even those on placebo to maintain treatment masking). 3.3 Bone Mineral Density Evidence from other clinical trials and observational studies documented maintenance of bone density with hormone therapy, whereas most women lost bone density in the absence of treatment (21,22). It was controversial whether hormone therapy could increase bone density and whether differences existed between treatment regimens in this effect. Based on the expectation that clinically meaningful changes in bone density require 1 to 2 years of treatment, PEPI obtained bone density measurements at baseline, 12 months, and 3 years. 3.4 Quality of Life Clinical experience suggested that hormone regimens could have diverse effects on quality of life. Potential areas affected included mood, sexual function, sense of well-being,

sleep, breast and pelvic symptoms, vaginal bleeding patterns, and physical activity. A comprehensive quality-of-life assessment was assembled to address these areas. Where possible, scales were drawn from existing standardized instruments. Otherwise, new scales were constructed with the assistance of content experts (23). 3.5 Other Measurements of waist, hip, and thigh circumference were obtained annually, as was recent dietary history using a food frequency questionnaire. Electrocardiograms were performed at baseline and at the end of study. 3.6 Sample Size and Power Sample size was estimated based on the desire to detect differences in HDL-cholesterol between the 5 study arms. The null hypothesis was that there would be no detectable differences in HDL between any of the 5 treatment groups. All pair-wise comparisons were considered, which yielded 10 potential comparisons. The result of subtracting the average of two prerandomization measures from the average of four postrandomization measures was defined as the treatment-related difference. The required sample size was calculated using standard formulas for z-tests that stipulated 80% power to detect a minimum of 5 mg/dl difference in HDL for each pair-wise comparison after 3 years of follow-up. Type I error was set at α = 0.05, and the Bonferroni approach was used so that α was divided by 40 (10 pair-wise comparisons times 4 endpoints). Potential attrition was considered, which includes differential loss of women with or without a uterus, and the number of subjects needed was increased to account for this loss with dropouts assigned values consistent with the null hypothesis. The cross-sectional standard deviation used for HDL was 17.9 mg/dl with a serial correlation of 0.72. Based on these assumptions, 840 women were required (120 at each of the 7 participating sites). That sample was projected to yield 92% power to detect differences of 5 mg/dl or greater in any pair-wise comparison after 3 years. Following determination of the sample size required for the HDL outcome, the minimum

POSTMENOPAUSAL ESTROGEN/PROGESTIN INTERVENTIONS TRIAL (PEPI)

detectable treatment effects at 80% and 90% power were estimated for the other 3 primary study outcomes. Estimated detectable differences for these at 80% power were fibrinogen 20.1 mg/dl, insulin 26.4 µU/ml, and systolic blood pressure 5 mmHg. 3.7 Follow-up Visits and Safety Assessments Participants were contacted at 3 months to assess any potential safety issues and to reinforce compliance if appropriate. Participants were examined in the clinic every 6 months. At those visits, a clinician took an interval history focused on potential adverse events; blood pressure, height, and weight were measured; and blood was drawn to be stored for possible future assays. All participants had an annual physical exam including pelvic, Pap smear, clinical breast exam, and mammography. Women with a uterus had an annual endometrial biopsy (7).

5

could be consulted regarding vaginal bleeding and could request treatment assignment within 3 groups—placebo, estrogen alone, or combined hormone therapy. This information allowed the consultant to determine the need for clinical evaluation or action while retaining the blind for all other parties in most cases (7). 4

RESULTS

PEPI was planned to enroll 840 participants: 120 per center and 168 per treatment arm. That recruitment goal was exceeded with 875 women enrolled as some centers were permitted to continue recruitment beyond their goal to ensure that adequate numbers were obtained. The primary results were published in January 1995 (20). Overall participant retention was excellent, with 97% of women completing the 36-month assessment.

3.8 Blinding

4.1 Cardiovascular Risk Factors

All participants, laboratory personnel, and clinical site investigators were blind to study treatment. Placebo and active drugs were prepared by the manufacturers in visually identical form. These elements were sent to a central facility that repackaged them in packages organized by cycle, labeled them with trial randomization codes, and shipped them to the clinical sites. Because of concern about possible endometrial hyperplasia, a system was designed to permit partial unblinding of a consulting gynecologist. This physician

HDL cholesterol, which was the primary stipulated outcome, was improved in all active treatment groups compared with placebo (20). Differences were observed between treatments in this effect; CEE alone and CEE plus micronized progesterone increased HDL significantly more than either of the CEE plus MPA regimens (Fig. 1). The two MPAcontaining regimens were also associated with an increase in 2-hour post-load glucose compared with placebo, whereas CEE alone and CEE plus micronized progesterone

PEPI: Mean % Change in High Density Lipoprotein Cholesterol % Placebo 70 14 CEE CEE + MPA sequential CEE + MPA continuous CEE + MP sequential

12 68 10 66 64

8 6 4 2

62

0 −2

Figure 1. PEPI: Mean percent change in High Density Lipoprotein Cholesterol.

60 −4 mg/dL 0

6

12

18 Months

24

30

36

6

POSTMENOPAUSAL ESTROGEN/PROGESTIN INTERVENTIONS TRIAL (PEPI)

PEPI: Mean % Change in Two-Hour Glucose 124

% 8 6

120 116

4 2 0

112 110 108

Placebo CEE CEE + MPA sequential CEE + MPA continuous CEE + MP sequential

−2 −4 −6 −8 0

mg/dL

12

24

Figure 2. PEPI: Mean percent change in 2-hour glucose.

36

Months PEPI: Mean % Change in Systolic Blood Pressure 122

% 6

120

4

118

2

116

0

Placebo CEE CEE + MPA sequential CEE + MPA continuous CEE + MP sequential

114 −2 112 −4 110 mm/Hg

−6 0

6

12

18

24

30

36

Months

were not. Estrogen alone was associated with a reduction in fasting glucose (Fig. 2). All active treatments had equivalent effects in a direction associated with lower cardiovascular risk on LDL and fibrinogen. All active treatments increased triglycerides in a similar manner. There were no significant effects on blood pressure (Fig. 3) (20).

Figure 3. PEPI: Mean percent change in systolic blood pressure.

study period (20). No endometrial carcinomas occurred in the ‘‘estrogen alone’’ arm, but there was an excess rate of hysterectomy. The three active treatments with a progestogen were equally effective in preventing endometrial disease. In all, two cases of endometrial carcinoma were observed: one in a participant on CEE plus micronized progesterone and one in the placebo group.

4.2 Endometrial Safety The annual rate of endometrial hyperplasia among women with a uterus who were randomized to CEE alone was over 20% (20,24). Women with significant hyperplasia as assessed by the local gynecologic consultant or their personal physician were withdrawn from study treatment. More than half of women with a uterus in this arm were lost to treatment for this reason over the 3-year

4.3 Bone Mineral Density All active treatments were associated with a modest increase in bone mineral density, and no significant differences were observed between treatments (Fig. 4) (20,25). This finding was of major importance because other studies had demonstrated independent effects on bone preservation for both estrogens and progestogens. PEPI demonstrated

POSTMENOPAUSAL ESTROGEN/PROGESTIN INTERVENTIONS TRIAL (PEPI) PEPI: % Change in Bone Mineral Density at 36 Months 6 5

Lumbar spine

Hip

4 3 2 1 0 CEE

CEE + MPA seq

CEE + MPA cont

CEE + MP seq

Figure 4. PEPI: Mean percent change in Bone Mineral Density at 36 months.

that, in typical clinical regimens, there was no meaningful benefit in bone mineral density associated with adding a progestogen to estrogen. So, with regard to bone, there was no reason for women without a uterus to use a progestogen. 4.4 Quality of Life All active treatments reduced vasomotor symptoms compared with placebo. Treatments with progestogens increased breast tenderness. No differences were observed in cognitive or affective measures (23,26). 4.5 Monitored Safety Outcomes Other than endometrial hyperplasia, no significant differences were observed in the incidence of any monitored outcome, which includes coronary events, stroke, gall bladder disease, and breast cancer. However, no incidences were expected given the sample size and duration of the study.

5

CONCLUSIONS

PEPI was an important intermediate step in the evaluation of the hypothesis that hormone therapy, particularly the estrogen component, could reduce coronary disease in women after menopause. Because coronary disease is the most common cause of death in women, with rates accelerating with age after menopause, such an effect could have enormous public health impact.

7

5.1 PEPI and the WHI PEPI demonstrated biological plausibility for intermediate markers of cardiovascular disease that supported the hypothesis that hormone therapy, particularly estrogen, could reduce the risk of coronary disease. It contributed to the body of knowledge that resulted in a large-scale test of hormone therapy effects on coronary events in one of the clinical trials of the Women’s Health Initiative (WHI), which was planned by members of the same NIH Institute (NHLBI) as the PEPI trial progressed (27). However, important differences were observed between the PEPI and WHI trials in participant characteristics. Women in PEPI were between 1 and 10 years postmenopausal and between the ages of 45 and 64 years at baseline. In contrast, women in the WHI were between 50 and 79 years old, and there were no criteria for years since menopause (28). The average time since menopause in the WHI was more than 12 years (29). The failure of the WHI to demonstrate a reduction in coronary heart disease for one combination regimen tested in PEPI (CEE plus continuous MPA 2.5 mg) caused a major reconsideration of this hypothesis (29). However, the PEPI finding of significant differences in HDL and glucose measures between CEE alone (and CEE plus micronized progesterone) and the two MPA-containing regimens was also partially replicated in the WHI finding that CEE alone was not associated with an increase in coronary events overall, and it was associated with a significant reduction in coronary events in women aged 50 to 59 years (30). 5.2 Definitive Answers to Important Secondary Questions PEPI provided unequivocal answers to two important secondary questions. First, it showed that there was no rationale for the use of a progestogen in women who did not need endometrial protection. There was no advantage for bone mineral density, and there was a potential disadvantage in HDL cholesterol, particularly with MPA. Second, it demonstrated that the rate of endometrial hyperplasia associated with even a mid-level

8

POSTMENOPAUSAL ESTROGEN/PROGESTIN INTERVENTIONS TRIAL (PEPI)

dose of unopposed estrogen was unacceptably high for routine clinical use. Yet it also found that hyperplasia rarely progressed to endometrial carcinoma with appropriate evaluation for unexpected bleeding and annual monitoring. 5.3 Contributions to Subsequent Women’s Health Research Because it was the first major NIH trial focused on older women, PEPI pioneered some history and exposure forms, as well as some quality of life scales, that have been adopted by a variety of studies since. Prior to PEPI, nearly all major long-term clinical trials had been conducted in men, and some investigators were concerned that women might not volunteer and participate in trials of this type. PEPI’s success in both recruitment and adherence laid these concerns to rest. REFERENCES 1. M. J. Stampfer, W. C. Willett, G. A. Colditz, et al., A prospective study of postmenopausal estrogen therapy and coronary heart disease. N. Engl. J. Med. 1985; 313: 1044–1049. 2. D. B. Petitti, J. A. Perlman, and S. Sidney, Noncontraceptive estrogens and mortality: long-term follow-up of women in the Walnut Creek Study. Obstet Gynecol. 1987; 70: 289–293. 3. T. L. Bush, E. Barrett-Connor, L. D. Cowan, et al., Cardiovascular mortality and noncontraceptive use of estrogen in women: results from the Lipid Research Clinics Program Follow-up Study. Circulation 1987; 75: 1102–1109. 4. A. Paganini-Hill, R. K. Ross, B. E. Henderson, Postmenopausal oestrogen treatment and stroke: a prospective study. BMJ 1988; 297: 519–522. 5. M. H. Criqui, L. Suarez, E. Barrett-Connor, J. McPhillips, D. L. Wingard, and C. Garland, Postmenopausal estrogen use and mortality. Results from a prospective study in a defined, homogeneous community. Am. J. Epidemiol. 1988; 128: 606–614. 6. E. Eaker, B. Packard, E. K. Wenger, Coronary heart disease in women: proceedings of an NIH workshop. New York: Haymarket Doyma, 1987.

7. M. A. Espeland, T. L. Bush, I. Mebane-Sims, M. L. Stefanick, S. Johnson, R. Sherwin, M. Waclawiw, Rationale, design, and conduct of the PEPI Trial. Postmenopausal Estrogen/Progestin Interventions. Control. Clin. Trials 1995; 16: 3S–19S. 8. V. T. Miller, R. L. Byington, M. A. Espeland, R. D. Langer, R. Marcus, S. Shumaker, M. P. Stern, Baseline characteristics of the PEPI participants. Control. Clin. Trials 1995; 16: 54S–65S. 9. P. C. MacDonald, Estrogen plus progestin in postmenopausal women – Act II. N. Engl. J. Med. 1986; 315: 959–961. 10. H. K. Ziel and W. D. Finkle, Increased risk of endometrial carcinoma among users of conjugated estrogens. N. Engl. J. Med. 1975; 293: 1167–1170. 11. D. C. Smith, R. Prentice, D. J. Thompson, and W. L. Herrmann, Association of exogenous estrogen and endometrial carcinoma. N. Engl. J. Med. 1975; 293: 1164–1167. 12. T. M. Mack, M. C. Pike, B. E. Henderson, R. I. Pfeffer, V. R. Gerkins, M. Arthur, and S. E. Brown, Estrogens and endometrial cancer in a retirement community. N. Engl. J. Med. 1976; 294: 1262–1267. 13. NS Weiss, DR Szekely, and D. F. Austin, Increasing incidence of endometrial cancer in the United States. N. Engl. J. Med. 1976; 294: 1259–1262. 14. M. E. Paterson, T. Wade-Evans, D. W. Sturdee, M. H. Thom, and J. W. Studd, Endometrial disease after treatment with oestrogens and progestogens in the climacteric. Br. Med. J. 1980; 280: 822–824. 15. M. H. Thom and J. W. Studd, Oestrogens and endometrial hyperplasia. Br. J. Hosp. Med. 1980; 23:506, 508–509, 511–513. 16. M. H. Thom, P. J. White, R. M. Williams, D. W. Sturdee, M. E. Paterson, T. Wade-Evans, and J. W. Studd, Prevention and treatment of endometrial disease in climacteric women receiving oestrogen therapy. Lancet 1979; 2: 455–457. 17. M. I. Whitehead and D. Fraser, The effects of estrogens and progestogens on the endometrium. Modern approach to treatment. Obstet. Gynecol. Clin. North Am. 1987; 14: 299–320. 18. L. Weinstein, Efficacy of a continuous estrogen-progestin regimen in the menopausal patient. Obstet. Gynecol. 1987; 69: 929–932. 19. L. Weinstein, C. Bewtra, and J. C. Gallagher, Evaluation of a continuous combined low-dose regimen of estrogen-progestin for treatment

POSTMENOPAUSAL ESTROGEN/PROGESTIN INTERVENTIONS TRIAL (PEPI) of the menopausal patient. Am. J. Obstet. Gynecol. 1990; 162: 1534–1589. 20. The Writing Group for the PEPI trial, Effects of estrogen or estrogen/progestin regimens on heart disease risk factors in postmenopausal women: the Postmenopausal Estrogen/Progestin Interventions (PEPI) Trial. JAMA 1995; 273: 199–208. 21. R. Lindsay, D. M. Hart, C. Forrest, and C. Baird, Prevention of spinal osteoporosis in oophorectomized women. Lancet 1980; 2: 1151–1154. 22. B. Ettinger, H. K. Genant, and C. E. Cann, Long-term estrogen replacement therapy prevents bone loss and fractures. Ann. Intern. Med. 1985; 102: 319—324. 23. G. A. Greendale, B. A. Reboussin, P. Hogan, V. M. Barnabei, S. Shumaker, S. Johnson, et al., Symptom relief and side effects of postmenopausal hormones: results from the Postmenopausal Estrogen/Progestin Interventions Trial. Obstet. Gynecol. 1998; 92: 982–988. 24. The Writing Group for the PEPI trial. Effects of hormone replacement therapy on endometrial histology in postmenopausal women. The Postmenopausal Estrogen/Progestin Interventions (PEPI) Trial. JAMA 1996; 275: 370–375. 25. The Writing Group for the PEPI trial. Effects of hormone therapy on bone mineral density: results from the postmenopausal estrogen/progestin interventions (PEPI) trial. JAMA 1996; 276: 1389–1396. 26. B. A. Reboussin, G. A. Greendale, and M. A. Espeland, Effect of hormone replacement therapy on self-reported cognitive symptoms: results from the Postmenopausal Estrogen/Progestin Interventions (PEPI) trial. Climacteric 1998; 1: 172–179. 27. J. E. Rossouw, L. P. Finnegan, W. R. Harlan, V. W. Pinn, C. Clifford, and J. A. McGowan, The evolution of the Women’s Health Initiative: perspectives from the NIH. J. Am. Med. Womens Assoc. 1995; 50: 50–5. 28. The Women’s Health Initiative Investigators, Design of the Women’s Health Initiative clinical trial and observational study. Control. Clin. Trials 1998; 19: 61–109. 29. Writing Group for the Women’s Health Initiative, Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results From the Women’s Health Initiative randomized controlled trial. JAMA 2002; 288: 321–333.

9

30. J. Hsia, R. D. Langer, J. E. Manson, L. Kuller, K. C. Johnson, S. L. Hendrix, M. Pettinger, S. R. Heckbert, N. Greep, S. Crawford, et al., Conjugated equine estrogens and coronary disease. Arch. Int. Med. 2006; 166: 357–365.

FURTHER READING PEPI published manuscripts—Baseline E. Barrett-Connor, H. G. Schrott, G. Greendale, D. Kritz-Silverstein, M. A. Espeland, M. P. Stern, T. Bush, J. A. Perlman, Factors associated with glucose and insulin levels in postmenopausal women. Diabetes Care. 1996. T. L. Bush, M. A. Espeland, and I. Mebane-Sims, The Postmenopausal Estrogen/Progestin Interventions (PEPI) Trial: overview. Control. Clin. Trials 1995; 16: 1S–2S. G. A. Greendale, P. Hogan, and S. Shumaker, Sexual functioning in postmenopausal women: the Postmenopausal Estrogen/Progestins Intervention Trial (PEPI). J. Women’s Health 1996; 5: 445–458. G. A. Greendale, P. Hogan, D. Kritz-Silverstein, R. Langer, S. R. Johnson, and T. Bush, Age at menopause in women participating in the Postmenopausal Estrogen/Progestins Interventions (PEPI) Trial: an example of bias introduced by selection criteria. Menopause 1995; 2: 27–34. G. A. Greendale, M. K. James, M. A. Espeland, E. Barrett-Connor, Can we measure prior postmenopausal estrogen/progestin use? The Postmenopausal Estrogen/Progestin Interventions Trial. Am. J. Epidemiol. 1997; 9: 763–770. G. A. Greendale, L. Bodin-Dunn, S. Ingles, R. Haile, and E. Barrett-Connor Leisure, home and occupational physical activity and cardiovascular risk factors in postmenopausal women: the Postmenopausal Estrogens/Progestins Intervention (PEPI) study. Arch. Intern. Med. 1996; 156: 418–424. G. A. Greendale, H. B. Wells, E. Barrett-Connor, R. Marcus, and T. L. Bush, Lifestyle factors and bone mineral density: the Postmenopausal Estrogen/Progestin Interventions study. J. Women’s Health 1995; 4: 231–246. S. Johnson, I. Mebane-Sims, P. E. Hogan, and D. B. Stoy, The Postmenopausal Estrogen/Progestin Interventions (PEPI) Trial: recruitment of postmenopausal women in the PEPI trial. Control. Clin. Trials 1995; 16: 20S–35S. S. L. Hall and G. A. Greendale, The relation of dietary vitamin C intake to bone mineral density: results from the PEPI study. Calcif. Tissue Int. 1998; 63: 183–189.

10

POSTMENOPAUSAL ESTROGEN/PROGESTIN INTERVENTIONS TRIAL (PEPI)

R. D. Marcus, G. Greendale, B. A. Blunt, T. L. Bush, S. Sherman, R. Sherwin, H. Wahner, and H. B. Wells, Correlates of bone mineral density in the Post-menopausal Estrogen/Progestin Interventions Trial (PEPI). J. Bone Miner. Res. 1994; 3: 1467–1476. V. T. Miller, R. L. Byington, M. A. Espeland, R. Langer, R. Marcus, S. Shumaker, M. P. Stern, The Postmenopausal Estrogen/Progestin Interventions (PEPI) Trial: Baseline characteristics of the PEPI participants. Control. Clin. Trials 1995; 16: 54S–65S. H. G. Schrott, C. Legault, M. L. Stefanick, V. T. Miller, J. LaRosa, P. D. Wood, and K. Lippel, Lipids in postmenopausal women: baseline findings of the Postmenopausal Estrogen/Progestin Interventions Trial. J. Women’s Health 1994; 3: 155–164. M. L. Stefanick, C. Legault, R. P. Tracy, G. Howard, G. Kessler, D. L. Lucas, and T. L. Bush, The distribution and correlates of plasma fibrinogen in middle-aged women: initial findings of the Postmenopausal Estrogen/Progestins Interventions (PEPI) study. Arterioscler. Thromb. Vasc. Biol. 1995; 15: 2085–2093. P. D. Wood, G. Kessler, Lippel K., M. L. Stefanick, C. H. Wasilauskas, and H. B. Wells, The Postmenopausal Estrogen/Progestin Interventions (PEPI) Trial: Physical and laboratory measurements in the PEPI trial. Controlled Clin Trials, 1995, 16: 36S–53S. PEPI published manuscripts—Outcomes E. Barrett-Connor, S. Slone, G. Greendale, D. Kritz-Silverstein, M. Espeland, S. R. Johnson, M. Waclawiw, and E. Fineberg, The Postmenopausal Estrogen/Progestin Interventions Study: primary outcomes in adherent women. Maturitas 1997; 27: 261–274. M. A. Espeland, M. L. Stefanick, D. KritzSilverstein, S. E. Fineberg, M. A. Waclawiw, M. K. James, and G. A. Greendale, Effect of postmenopausal hormone therapy on body weight and waist and hip girths. J. Clin. Endocrinol. Metabol. 1997; 82: 1549–1556. M. A. Espeland, S. M. Marcovina, V. Miller, P. D. Wood, C. Wasilauskas, R. Sherwin, H. Schrott, T. L. Bush, Effect of postmenopausal hormone therapy on lipoprotein(a) concentrations. Circulation 1998; 97: 979–986.

M. A. Espeland, P. E. Hogan, S. E. Fineberg, G. Howard, H. Schrott, M. A. Waclawiw, and T. L. Bush, Effect of postmenopausal hormone therapy on glucose and insulin concentrations. Diabetes Care 1998; 21: 1589–1595. G. A. Greendale, B. Melton, P. Hogan, V. Barnabei, S. Shumaker, S. Johnson, and E. Barrett-Connor, Effects of estrogen or estrogen/ progestin regimens on physical, cognitive, and affective symptoms: results from The Postmenopausal Estrogen/Progestin Interventions Trial. Obstet. Gynecol. 1998; 92: 982–988. G. A. Greendale, B. A. Melton, A. Sie, H. R. Singh, L. K. Olson, O. Gatewood, L. W. Bassett, C. Wasilauskas, T. Bush, and E. Barrett-Connor, The effects of estrogen and estrogen/progestin treatments on mammographic parenchymal density: Postmenopausal Estrogen/Progestin Interventions (PEPI) Investigators. Annals Ann. Intern. Med. 1999; 130: 262–269. R. D. Langer, J. J. Pierce, K. A. O’Hanlan, S. R. Johnson, M. A. Espeland, J. F. Trabal, V. M. Barnabei, M. J. Merino, R. E. Scully, Transvaginal ultrasonography compared with endometrial biopsy for the detection of endometrial disease. N. Engl. J. Med. 1997; 337: 1792–1798. C. Legault, M. A. Espeland, C. H. Wasilauskas, T. L. Bush, J. Trabal, H. L. Judd, S. R. Johnson, G. A. Greendale, Agreement in assessing endometrial pathology: The Postmenopausal Estrogen/Progestin Interventions Trial. J. Women’s Health 1998; 7: 435–441. H. G. Schrott, C. Legault, M. L. Stefanick, V. T. Miller, Effect of hormone replacement therapy on the validity of the Friedewald equation in postmenopausal women: The Postmenopausal Estrogen/Progestin Interventions Trial (PEPI). J. Clin. Epidemiol. 1999; 52: 1187–1195.

CROSS-REFERENCES Women’s Health Initiative

Power Transformations The assumptions in regression analysis include normally distributed errors of constant variance (see Scedasticity). Often these assumptions are more nearly satisfied not by the original response variable y, but by some transformation of y, z(y). For nonnegative responses, one frequently used transformation is log y. The original and transformed analyses can then be compared in a number of ways. Residuals can be plotted against fitted values, or assessed for normality by probability plots for various transformations (see Normal Scores). Another comparison is through analysis of the linear model using t or F tests – a correct transformation often yields a simple linear regression model, with no, or just a few, interaction or quadratic terms. A formal way of comparing transformations is to embed them in a parametric family and then to make inferences about the transformation parameter λ (see Model, Choice of). Transformations of three parts of the model are of differing complexity and importance. The most important is described in the following section. Transformation of the Response: Box and Cox [5]. The logarithmic transformation is one special case of the normalized power transformation [5]   yλ − 1 λ
= 0 λ−1 z(λ) = (1)  λy˙ y˙ log y λ = 0, where the geometric mean of the observations is written as y˙ = exp(Σ log yi /n). The regression model to be fitted is then z(λ) = X β + ε.

(2)

For fixed λ, the value of β is estimated by least squares giving a residual sum of squares for the z(λ) of R(λ). The maximum likelihood estimate λˆ minimizing R(λ) is found by numerical search (see Optimization and Nonlinear Equations), often over a grid of λ values. Exact minimization of R(λ) is not required, since simple rational values of λ are customary in the analysis of data: λ = 1, no transformation: λ = 1/2, the square root; λ = 0, the logarithmic and λ = −1, the reciprocal being widely used. An approximate minimum of R(λ) is, however, required to establish confidence intervals for and to

test hypotheses about λ. It is important that these comparisons of R(λ) do use the full form in (1) including the geometric mean. Omission of this term leads to meaningless comparisons – for most data, log y is very much smaller than y and so are the corresponding sums of squares, regardless of how well the regression models fit. Transformation of Explanatory Variables: Box and Tidwell [6]. For a regression model with p terms, it sometimes makes sense to consider models in which one (or perhaps more) of the explanatory variables is transformed, when the model is y=

p 

βj xj + βk xkλ + ε.

(3)

j
=k

Again the maximizing value of λ has to be found numerically, but the calculations are more straightforward than those for the Box–Cox transformation, since the scale of the observations is unaffected by the transformation. The residual sums of squares of y can be compared directly as λ varies. Transformation of Both Sides of the Model: Carroll and Ruppert [7]. The Box and Cox transformation often yields both approximately normal errors and a simple linear model. But sometimes the two transformations do not happen together. An example is the data on mandible length from Royston and Altman [10] plotted in Goodness of Fit. The analyses in Diagnostics, Forward Search, and Residuals showed that the log transformation yielded normal errors, but increased the evidence for the inclusion of a quadratic term in the linear model (see Polynomial Regression). If there is a simple model for y, the simplicity of the linear model can be maintained by subjecting both sides of the model to the same transformation. The purpose of the transformation is then to obtain normal errors of constant variance. Let E(Y ) = η = x T β be the simple linear model. The transformation model is then ηλ − 1 yλ − 1 = +ε λ−1 λy˙ λy˙ λ−1

λ
= 0,

y˙ log y = y˙ log η + ε

λ = 0.

(4)

The optimizing value of λ again minimizes the residual sum of squares of the transformed response. For given λ, estimating the parameters in general requires nonlinear estimation, although this is unaffected by

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

2

Power Transformations

division by λy˙ λ−1 . An example for the data on the volume of trees analyzed in Residuals is given by Atkinson [2]. The procedures using constructed variables described in Residuals provide tests for the value of λ, which avoid the need for calculation of λˆ whilst also giving information on the effect of individual observations on tests and parameter estimates. Numerous examples for the Box–Cox and Box–Tidwell transformations are given by Atkinson [1]. Cook and Weisberg [8, Chapter 13], describe interactive graphical methods for selecting a transformation. Diagnostic material on the transformation of both sides of the model is given by Hinkley [9], Shih [11], and by Atkinson [2]. Atkinson and Shephard [4] extend the Box–Cox transformation to time series analysis and describe the related diagnostic procedures. Diagnostic procedures for the effect of groups of observations via the Forward Search use the Fan Plot, described in greater detail in [3, Chapter 4].

References [1]

Atkinson, A.C. (1985). Plots, Transformations, and Regression. Oxford University Press, Oxford.

[2]

Atkinson, A.C. (1994). Transforming both sides of a tree, American Statistician 48, 307–313. [3] Atkinson, A.C. & Riani, M. (2000). Robust Diagnostic Regression Analysis. Springer-Verlag, New York. [4] Atkinson, A.C. & Shephard, N. (1996). Deletion diagnostics for transformation of time series, Journal of Forecasting 5, 1–17. [5] Box, G.E.P. & Cox, D.R. (1964). An analysis of transformations (with discussion), Journal of the Royal Statistical Society, Series B 26, 211–246. [6] Box, G.E.P. & Tidwell, P.W. (1962). Transformations of the independent variables, Technometrics 4, 531–550. [7] Carroll, R.J. & Ruppert, D. (1988). Transformation and Weighting in Regression. Chapman & Hall, London. [8] Cook, R.D. & Weisberg, S. (1999). Applied Regression Including Computing and Graphics. Wiley, New York. [9] Hinkley, D.V. (1985). Transformation diagnostics for linear models, Biometrika 72, 487–496. [10] Royston, P.J. & Altman, D.G. (1994). Regression using fractional polynomials of continuous covariates: parsimonious parametric modelling (with discussion), Applied Statistics 43, 429–467. [11] Shih, J.-Q. (1993). Regression transformation diagnostics in transform-both-sides model, Statistics and Probability Letters 16, 411–420.

A.C. ATKINSON

PREDICTING RANDOM EFFECTS IN COMMUNITY INTERVENTION

treatments while accounting for the random assignment of groups in the design. Historically, because the main focus of a trial is comparison of treatments over all groups, and groups are assigned at random to a treatment, the effect of an assigned group was not considered to be of interest. This perspective has resulted in many authors (4–8) limiting discussion to estimation of fixed effects. Support for this position stems from the fact that a group cannot be guaranteed to be included in the study. This fact, plus the result that the average of the group effects is zero, has been sufficient for many to limit discussion of group effects to estimating the group variance, not particular group effects. Nevertheless, when conducting a group randomized trial, it is natural to want to estimate response for a particular group. As a result of the limitations of random effects in a mixed model, some authors (9, 10) suggest that such estimates can be made, but they should be based on a different model. The model is conditional on the group assignment, and represents groups as fixed effects. With such a representation, response for individual groups can be directly estimated. However, the fixed effect model would not be suitable for estimating treatment effects, because the evaluation would not be based on the random assignment of groups to treatments. The apparent necessity of using different models to answer two questions in a group randomized trial has prompted much study (11). A basic question is whether a mixed model can be used to predict the mean of a group that has been randomly assigned to a treatment. If such prediction is possible, the mixed model would provide a unified framework for addressing a variety of questions in a group randomized trial, including prediction of combinations of fixed and random effects. A number of workers (12–15) argue that such prediction is possible. The predictor is called best linear unbiased predictor (BLUP) (16). Moreover, as the BLUP minimizes the expected mean squared error (EMSE), it is optimal.

EDWARD J. STANEK III University of Massachusetts Amherst, Massachusetts,

1

INTRODUCTION

The main objective in a group randomized trial is a comparison of the expected response between treatments, where the expected response for a given treatment is defined as the average expected response over all groups in the population. Often, interest may exist in the expected response for a particular group included in the trial. As a random sample of groups is included in the trial, it is usually represented as a random effect. In this context, predicting the expected response for an individual group requires predicting a random effect. For example, in a study of the impact of teaching paradigms on substance use of high school students in New Haven, Connecticut, high schools were randomly assigned to an intervention or control condition. The main evaluation of the intervention was a comparison of student response between intervention and control averaged over all high schools. Interest also existed in student response at particular high schools. As high schools were randomly assigned to conditions, the difference in response for a particular high school from the population average response is a random effect. 2

BACKGROUND

Since the early work on analysis of group randomized trials (1–3), models for response have included both fixed effects and random effects. Such models are called mixed models. Fixed effects appear directly as parameters in the model, whereas group effects are included as random variables. The random effects have mean zero and a nonzero variance. An advantage of the mixed model is the simultaneous inclusion of population parameters for

Wiley Encyclopedia of Clinical Trials, Copyright © 2007 John Wiley & Sons, Inc.

1

2

PREDICTING RANDOM EFFECTS IN COMMUNITY INTERVENTION

In light of these results, it may appear that methods for prediction of random effects are on firm ground. Some problems exist, however. The BLUP of a realized group is closer to the overall sample treatment mean than the sample group mean, a feature called shrinkage. This shrinkage results in a smaller average mean squared error. The reduction in EMSE is often attributed to ‘‘borrowing strength’’ from the other sample observations. However, the best linear unbiased predictor of a realized group is biased, whereas the sample group mean is unbiased. The apparent contradiction in terms is a result of two different definitions of bias. The BLUP are unbiased in the sense that zero average bias exists over all possible samples (i.e., unconditionally). The sample group mean is unbiased conditional on the realized group. A similar paradox may occur when considering the EMSE, where the mean squared error (MSE) for the best linear unbiased predictor may be larger than the MSE of the group mean for a realized group. This result can occur because, for the BLUP, the EMSE is best in the sense that the average MSE is minimum over all possible samples (i.e., unconditionally), whereas for the sample group mean, the MSE is evaluated for a realized group (17). These differences lead to settings, such as when the distribution of group means is bimodal, where the BLUP seems inappropriate for certain groups (18). For such reasons, predictors of random effects are somewhat controversial. Such controversies are not resolved here. Instead, an attempt is made to clearly define the problem, and then to describe some competing models and solutions. Statistical research in this area is active, and new insights and controversies may still emerge. 3

METHODS

A simple framework is provided that may be helpful for understanding these issues in the context of a group randomized trial. A study population is first defined, along with population/group parameters, which provides a finite population context for the group randomized trial. Next, random variables that

develop from random assignment of groups, and sampling subjects within a group, are explicitly represented. This context provides a framework for discussion of various mixed models and assumptions. Then, an outline of predictors of random effects is developed. This development is limited to the simplest setting. Finally, a broader discussion of other issues concludes the article. 3.1 Modeling Response for a Subject Suppose a group randomized trial is to be conducted to evaluate the impact of a substance abuse prevention program in high schools. It is assumed that, for each student, a measure of the perception of peer substance abuse response can be obtained from a set of items of a questionnaire administered to the student as a continuous response. The kth measure of response (possibly repeated) of student t in high school s is denoted by Ystk = yst + Wstk

(1)

indexing students by t = 1, . . ., M in each of N high schools, indexed by s = 1, . . ., N. Measurement error (corresponding to test, retest variability) is represented by the random variable W stk , and distinguishes yst (a fixed constant representing the expected response student t) from Y stk . The subscript k indicates an administration of the questionnaire, where potentially k > 1. The average expected response over students in school s M 1
yst , whereas the averis defined as µs = M t=1

age over all schools is defined as µ =

1 N

N


µs ;

s=1

µs will be referred to as the latent value of school s. Discussion is limited to the simplest setting where each school has the same number of students, an equal number of schools are assigned to each intervention, an equal number of students are selected from each assigned school, and a single measure of response is made on each selected student. These definitions provide the context for defining the impact of a substance abuse prevention program. To do so, imagine that each student could be measured with and without the intervention. If no substance abuse

PREDICTING RANDOM EFFECTS IN COMMUNITY INTERVENTION

program is implemented, let the expected response of a student be yst ; if an intervention program is in place, let the expected response be y∗st . The difference, y∗st − yst = δ st represents the effect of the intervention on student t in high school s. The average of these effects over students in school s is M 1
δst , whereas the average defined as δs = M t=1

over all schools is defined as δ =

1 N

N


δs . The

s=1

parameter δ is the main parameter of interest in a group randomized trial. To emphasize the effect of the school and of the student in the school, β s = (µs − µ) is defined as the deviation of the latent value of school s from the population mean and εst = (yst − µs ) as the deviation of the expected response of student t (in school s) from the school’s latent value. Using these definitions, the expected response of student t in school s is represented as yst = µ + βs + εst

(2)

This model is called a derived model (19). The effect of the intervention on a student can be expressed in a similar manner as δ st ∗ , where δ ∗ = δ − δ and δ ∗ = = δ + δs∗ + δst s s st δ st − δ s . Combining these terms, the expected response of student t in school s receiving the ∗ , where β ∗ intervention is y∗st = µ + δ + βs∗ + εst s ∗ = β s + δ s and εst = εst + δ st . When the effect of the intervention is equal for all students (i.e., δ st = δ), the expected response for student t (in school s) is represented by yst = µ + xs δ + βs + εst where xs is an indicator of the intervention, taking a value of 1 if the student receives the intervention, and 0 otherwise. It is assumed the effect of the intervention is equal for all students to simplify subsequent discussions. The latent value for school s under a given condition is given by µ + xs δ + β s .

exist (intervention and control), with n groups (i.e., schools) assigned to each treatment. A simple way to conduct the assignment is to randomly permute the list of schools, assigning the first n schools to the control and the next n schools to intervention. The school’s position in the permutation is indexed by i = 1, . . ., N, referring to control schools by i = 1, . . ., n and to the intervention schools by i = n + 1, . . ., 2n. A sample of students in a school can be represented in a similar manner by randomly permuting the students in a school, and then including the first j = 1, . . ., m students in the sample. The expected response (over measurement error) of a student in a school is represented as Y ij , using the indices for the positions in the permutations of students and schools. Note that Y ij is a random variable because the actual school and student corresponding to the positions will differ for different permutations. Once a permutation (say of schools) is selected, the school that occupies each position in the permutation is known. Given a selected permutation, the school is a ‘‘fixed’’ effect; models that condition on the schools in the sample portion of a permutation of schools will represent schools as fixed effects. Schools are represented as random effects in a model that does not condition on the permutation of schools. The resulting model is an unconditional model. The expected response for a school assigned to a position is represented explicitly as a random variable, and it accounts for the uncertainty in the assignment by a set of indicator random variables, U is , s = 1, . . ., N where U is takes the value of one when school s is assigned to position i, and the value 0 otherwise. Using these ranN
Uis µs represents a random dom variables, s=1

variable corresponding to the latent value of the school assigned to position i. Using β s = (µs − µ) and noting that for any permuN N

Uis = 1, Uis µs = µ + Bi , where tation, Bi =

3.2 Random Assignment of Treatment and Sampling The first step in a group randomized trial is random assignment of groups to treatments. It is assumed that two treatments

3

N


s=1

s=1

Uis βs represents the random effect

s=1

of the school assigned to position i in the permutation of schools. The random variables U is are used to represent permutations of schools. A similar set of indicator random variables U jt (s) that take

4

PREDICTING RANDOM EFFECTS IN COMMUNITY INTERVENTION

on a value of 1 when the jth position in a permutation of students in school s is student t, and 0 otherwise, relate yst to Y ij . For ease of exposition, refer to the school that will occupy position i in the permutation of schools as primary sampling unit (PSU) i, and to the student that will occupy position j in the permutation of students in a school as secondary sampling unit (SSU) j. PSUs and SSUs are indexed by positions (i and j), whereas schools and students are indexed by labels (s and t) in the finite population. As a consequence, the random variable corresponding to PSU i and SSU j is given by Yij =

N  M 

Uis Ujt(s) yst

effect (i.e., Bi , where i = 1). Once the school corresponding to the first PSU has been randomly assigned, the random variables U is for i = 1 and s = 1, . . ., N will be realized. If school s* is assigned to the first position, then the realized values (i.e., U 1s = u1s , s = 1, . . ., N) are u1s = 0 when s = s* and u1s = 1 when s = s*; the parameter for the realized random effect will correspond to βs∗ , the deviation of the latent value of school s* from the population mean. Methods are discussed for predicting the latent value of a realized PSU assigned to the control condition (i.e., i ≤ n) in the simplest setting when no measurement error exists. The model for SSU j in PSU i is the simple random effects model,

s=1 t=1

Using the representation response for a SSU in a PSU,

of

Yij = µ + xi δ + Bi + Eij noting

that

M
t=1

Ujt(s) = 1, δ

N


Uis xs = xi δ

s=1

because the treatment assigned to a position depends only on the position, and N
M
Uis Ujt(s) εst . Adding measurement Eij = s=1 t=1

error, the model is given by ∗ Yijk = µ + xi δ + Bi + Eij + Wijk

∗ where Wijk =

M N

s=1 t=1

Yij = µ + Bi + Eij

expected

Uis Ujt(s) Wstk . This model

includes fixed effects (i.e., µ and δ) and random effects (i.e., Bi and Eij ) in addition to measurement error, and hence is called a mixed model. 3.3 Intuitive Predictors of the Latent Value of a Realized Group Suppose that interest exists in predicting the latent value of a realized group (i.e., school), say the first selected group, µ + Bi = N
Uis µs , where i = 1. Before randomly s=1

assigning schools, as which school will be first is not known, the expected value of the first PSU is a random variable represented by the sum of a fixed (i.e., µ) and a random

(3)

The latent value of PSU i is represented by the random variable µ + Bi . The parameter for the latent value of school s is M 1
yst . The goal is to predict the latent µs = M t=1

value for the school corresponding to PSU i, which is referred to as the latent value of the realized PSU. It is valuable to develop some intuitive ideas about the properties of predictors. For school s, the latent value can be represented as the  sum of two random  m M

1 Ysj + Ysj ), variables (i.e.,µs = M j=1

where Ysj =

M
t=1

Ujt(s) yst represents response for

SSU j in school s. Let Y sI = Y sII =

1 M−m

j=m+1

M


1 m

m


Ysj and

j=1

Ysj represent random vari-

j=m+1

ables corresponding to the average responses of SSUs in the sample and remainder, respectively. Then, µs = f Y sI + (1 − f )Y sII , where the fraction of students selected in a school m . If school s is a control is given by f = M school (i.e., one of the first n PSUs), then the average response for students selected from the school, Y sI , will be realized after sampling and the only unknown quantity in the expression for µs will be Y sII , the average response of students not included in the sample. Framed

PREDICTING RANDOM EFFECTS IN COMMUNITY INTERVENTION

in this manner, the essential problem in predicting the latent value of a realized PSU is predicting the average response of the SSUs not included in the sample. The predictor of the latent value of a realized PSU will be close to the sample average for the PSU when the second-stage sampling fraction is large. For example, representing a predictor of Y sII by Yˆ sII , and assuming that f = 0.95, the predictor of the latent value of school s is µˆ s = 0.95Y sI + 0.05Yˆ sII . Even poor predictors of Y sII will only modestly affect the predictor of the latent value. This observation provides some guidance for assessing different predictors of the latent value of a realized random effect. As the second-stage sampling fraction is allowed to increase, a predictor should be closer and closer to the average response of the sample SSUs for the realized PSU.

4

MODELS AND APPROACHES

A brief discussion is provided of four approaches that have been used to predict the latent value of a realized group, limited to the simplest setting given by Equation (3). The four approaches correspond to Henderson’s approach (12, 20), a Bayesian approach (6), a super-population model approach (21, 22), and a random permutation model approach (22). An influential paper by Robinson (13) and its discussion brought prediction of realized random effects into the statistical limelight. This paper identified different applications (such as selecting sires for dairy cow breeding, Kalman filtering in time series analysis, Kriging in geographic statistics) in which the same basic problem of prediction of realized random effects occurred, and directed attention to the best linear unbiased predictor (BLUP) as a common solution. Other authors have discussed issues in predicting random effects under a Bayesian setting (23); random effects in logistic models (24–26); and in applications to blood pressure (27), cholesterol, diet, and physical activity (17). Textbook presentations of predictors are given by References 6 and (28–30). A general review in the context of longitudinal data is given by Singer and Andrade (31).

5

4.1 Henderson’s Mixed Model Equations Henderson developed a set of mixed model equations to predict fixed and random effects (12). The sample response for the model in Equation (3) is organized by SSUs in PSUs, resulting in the model Y = Xα + ZB + E

(4)

where Y is an nm × 1 response vector, X = 1nm is the design matrix for fixed effects, and Z = In ⊗ 1m is the design matrix for random effects, where 1nm denotes an nm × 1 column vector of ones, In is a n × n identity matrix, and A1 ⊗ A2 represents the Kronecker product formed by multiplying each element in the matrix A1 by A2 (32). The parameter α = E(Y ij ) is a fixed effect corresponding to the expected response, whereas the random effects are contained in B = (B1 B2 · · · Bn ) . It is assumed that E(Bi ) = 0, var(B) = G, E(Eij ) = 0, and var(E) = R so that
= var(ZB + E) = ZGZ + R. Henderson proposed estimating α (or a linear function of the fixed effects) by a linear combination of the data a Y. Requiring the estimator to be unbiased and have minimum variance lead to the generalized least squares estimator αˆ = (X
−1 X )−1 X
−1 Y. Difficulties in inverting
(in more complex settings) led Henderson to express the estimating equations as the solution to two simultaneous equations known as Henderson’s mixed model equations (20) ˆ = X R−1 Y X R−1 Xαˆ + X R−1 ZB ˆ = Z R−1 Y Z R−1 Xαˆ + (Z R−1 Z + G−1 )B These equations are easier to solve than the generalized least squares equations because the inverse of R and G are often easier to compute than the inverse of
. The mixed model equations were motivated by computational needs and developed from a matrix ˆ was a byproduct of identity (33). The vector B this identity and had no clear interpretation. ˆ could be interpreted as the predictor If B of a realized random effect, then the mixed model equations would be more than a computational device. Henderson (34) provided the motivation for such an interpretation by

6

PREDICTING RANDOM EFFECTS IN COMMUNITY INTERVENTION

developing a linear unbiased predictor of α + Bi that had minimum prediction squared error in the context of the joint distribution of Y and B. Using         Y Xα Y
ZG E = and var = B 0n B GZ G (5) Henderson showed that the best linear unbiˆ i , where ased predictor of α + Bi is αˆ + B ˆ = GZ
−1 (Y − Xα). ˆ i is the ith element of B ˆ B Using a variation of Henderson’s matrix identity, one may show that it is identical to the predictor obtained by solving the second of Henderson’s mixed model equations ˆ With the additional assumptions that for B. G = σ 2 In and R = σ e 2 Inm , where var(Bi ) = σ 2 , and that var(Eij ) = σ e 2 , the expresˆ i = k(Y i − Y), with ˆ i simplifies to B sion for B m n

2 1 k = 2 σ 2 , Yi = m Yij , and Y = n1 Y i. σ +σe /m

j=1

i=1

The coefficient k is always less than one, and ‘‘shrinks’’ the size of the deviation of sample mean for the ith PSU from the sample PSU average. The predictor of α + Bi is given by ˆ i = Y + k(Y i − Y) αˆ + B Henderson’s mixed model equations develop from specifying the joint distribution of Y and B. Only first and second moment assumptions are needed to develop the predictors. As discussed by Searle et al. (6), Henderson’s starting point was not the sample likelihood. However, if normality assumptions are added to Equation (5), then the conditional mean E(α + Bi |Y i = yi ) = α + k(yi − α). Replacing α by the sample average, y, the predictor of α + Bi conditional on Y is y + k(yi − y). 4.2 Bayesian Estimation The same predictor can be obtained using a hierarchical model with Bayesian estimation (6). Beginning with Equation (4), the terms in the model are classified as observable (including Y, X, and Z), and unobservable (including α, B, and E). The unobservable terms are considered as random variables, and hence use different notation to distinguish α from the

corresponding random variable A. The model is given by Y = XA + ZB + E and a hierarchical interpretation develops from considering the model in stages (28, 29). At the first stage, assume that the clusters corresponding to the selected PSUs are known, so that A = α 0 and B = β 0 . At this stage, the random variables E develop from selection of the SSUs. At the second stage, assume A and B are random variables, and have some joint distribution. The simplest case (which is considered here) assumes that the unobservable terms are independent and that A ∼ N(α, τ 2 ), B ∼ N(0n , G), and E ∼ N(0nm , R), where G = σ 2 In and R = σ e 2 Inm . Finally, the prior distributions are specified for α, τ 2 , σ 2 , and σ e 2 . Y is conditioned upon in the joint distribution of the random variables to obtain the posterior distribution. The expected value of the posterior distribution is commonly used to estimate parameters. The problem is simplified considerably by assuming that σ 2 , and σ e 2 are constant. The lack of prior knowledge of the distribution of A is represented by setting τ 2 = ∞. With these assumptions, the Bayesian estimate of B is the expected value of the posterior ˆ = GZ
−1 (Y − Xα)). ˆ This distribution (i.e., B predictor is identical to the predictor defined in Henderson’s mixed model equation. 4.3 Super-Population Model Predictors Predictors of the latent value of a realized group have also been developed in a finite population survey sampling context (21). To begin, suppose a finite population of M students in each of N schools is considered as the realization of a potentially very large group of students, called a super-population. Students or schools are not identified explicitly in the super-population, and are instead referred to as PSUs and SSUs. Nevertheless, a correspondence exists between a random variable, Y ij , and the corresponding realized value for PSU i and SSU j, yij , which would identify a student in a school. Predictors are developed in the context of a probability model specified for the super-population. This general strategy is referred to in survey sampling as model-based inference.

PREDICTING RANDOM EFFECTS IN COMMUNITY INTERVENTION

The random variable corresponding to the latent value for PSU i (where i ≤ n) given by M 1
Yi = M Yij , can be divided into two parts, j=1

Y i,I and Y i,II , such that Y i = f Y i,I + (1 − f )Y i,II , m 1
where Y i,I = m Yij corresponds to the averj=1

age response of SSUs that will potentially be M
1 Yij observed in the sample, Y i,II = M−m j=m+1

is the average response of the remaining ranm . Predicting the dom variables, and f = M latent value for a realized school simplifies to predicting an average of random variables not realized in the sample. For schools that are selected in the sample, only response for the students not sampled need be predicted. For schools not included in the sample, a predictor is needed for all students’ responses. Scott and Smith assume a nested probability model for the super-population, representing the variance between PSUs as σ 2 , and the variance between SSUs as σ e 2 . They derive a predictor for the average response of a PSU that is a linear function of the sample, unbiased, and has minimum mean squared error. For selected PSUs, the predictor simplifies to a weighted sum of the sample average response, and the average predicted response m for  not included in the sample, M yi +  SSUs ˆ ˆ M−m Y , where Y = y + k(y − y) and M

k=

i,II

σ2 . σ 2 +σe2 /m

i,II

i

This same result was derived

by Scott and Smith under a Bayesian framework. For a sample school, the predictor of the average response for students not included in the sample is identical to Henderson’s predictor and the predictor resulting from Bayesian estimation. For PSUs not included in the sample, Scott and Smith’s predictor reduces to the simple sample mean y. A substantial conceptual difference exists between Scott and Smith’s predictors and the mixed model predictors. The difference is a result of the direct weighting of the predictors by the proportion of SSUs that need to be predicted for a realized PSU. If this proportion is small, the resulting predictor will be close to the PSU sample mean. On the other hand, if only a small portion of the SSUs are observed in a PSU, Scott and Smith’s predictor will be very close to the mixed model predictors.

7

5 RANDOM PERMUTATION MODEL PREDICTORS An approach closely related to Scott and Smith’s super-population model approach can be developed from a probability model that develops from sampling a finite population. Such an approach is based on the two-stage sampling design and is design-based (35). As selection of a two-stage sample can be represented by randomly selecting a twostage permutation of a population, models under such an approach are referred to as random permutation models. This approach has the advantage of defining random variables directly from sampling a finite population. Predictors are developed that have minimum expected mean square error under repeated sampling in a similar manner as those developed by Scott and Smith. In a situation comparable with that previously described for Scott and Smith, predictors of the realized latent value are nearly identical to those derived by Scott and Smith. The only difference is the use of a slightly different shrinkage is  ∗ The predictor  constant. ∗ ˆ m M−m ˆ Y , where Y = y + given by y + M i

k∗ (y

6

i

−

y), k∗

M

=

σ ∗2 , σ ∗2 +σe2 /m

i,II

and

i,II

σ ∗2

= σ2 −

σe2 M.

PRACTICE AND EXTENSIONS

All of the predictors of the latent value of a realized group include shrinkage constants in the expressions for the predictor. An immediate practical problem in evaluating the predictors is estimating this constant. In a balanced setting, simple method of moment estimates of variance parameters can be substituted for variance parameters in the shrinkage constant. Maximum likelihood or restricted maximum likelihood estimates for the variance are also commonly substituted for variance parameters in the prediction equation. In the context of a Bayesian approach, the resulting estimates are called empirical Bayes estimates. Replacing the variance parameters by estimates of the variance will inflate the variance of the predictor using any of the approaches. Several methods have been developed that account for the larger variance (36–40).

8

PREDICTING RANDOM EFFECTS IN COMMUNITY INTERVENTION

In practice, groups in the population are rarely of the same size, or have identical intragroup variances. The first three approaches can be readily adapted to account for such unbalance when predicting random effects. The principal difference in the predictor is replacement of σ e 2 by σ ie 2 , the SSU variance within SSU i, when evaluating the shrinkage constant. The simplicity in which the methods can account for such complications is an appeal of the approaches. Predictors of realized random effects can also be developed using a random permutation model that represents the two-stage sampling. When SSU sampling fractions are equal, the predictors have a similar form with the shrinkage constant constructed from variance component estimates similar to components used in two-stage cluster sampling variances (41). Different strategies are required when second-stage sampling fractions are unequal (42). In many settings, a simple response error model will apply for a student. Such response error can be included in a mixed model such as Equation (4). When multiple measures are made on some students in the sample, the response error can be separated from the SSU variance component. Bayesian methods can be generalized to account for such additional variability by adding another level to the model hierarchy. Super-population models (43) and random permutation models (35) can also be extended to account for response error. Practical applications often involve much more complicated populations and research questions. Additional hierarchy may be present in the population (i.e., school districts). Variables may be available for control corresponding to districts, schools, and students. Measures may be made over time on the same sample of students or on different samples. Response variables of primary interest may be continuous, categorical, or ordinal. Some general prediction strategies have been proposed and implemented (29, 44, 45) using mixed models and generalized mixed models (30), often following a Bayesian paradigm. Naturally, the hypotheses and approaches in such settings are more complex. Active research is occurring in these

areas, which should lead to clearer guidance in the future. 7 DISCUSSION AND CONCLUSIONS The latent value of a group is a natural parameter of interest in a group randomized trial. Although such a parameter may be readily understood, development of an inferential framework for predicting such a parameter is not easy. Many workers struggled with ideas underlying interpretation of random effects in the mid-twentieth century. Predictors have emerged largely based on computing strategies and Bayesian models in the past 20 years. Such strategies have the appeal of providing answers to questions that have long puzzled statisticians. Computing software (based on mixed model and Bayesian approaches) is widely available and flexible, allowing multilevel models to be fit with covariates at different levels for mixed models. Although flexible software is not yet available for super-population model or random permutation model approaches, some evidence exists that when sampling fractions are small (<0.5), predictors and their MSE are very similar (35). Whether the different approaches predict the latent value of a realized group is a basic question that can still be asked. All of the predictors have the property of shrinking the realized group mean toward the overall sample mean. Although the predictors are unbiased and have minimum expected MSE, these properties hold over all possible samples, not conditionally on a realized sample. This use of the term ‘‘unbiased’’ differs from the popular understanding. For example, for a realized group, an unbiased estimate of the group mean is the sample group mean, whereas the BLUP is a biased estimate of the realized group mean. The rationale for preferring the biased estimate is that, in an average sense (over all possible random effects), the mean squared error is smaller. As this property refers to an average over all possible random effects, it does not imply smaller MSE for the realized group (18). In an effort to mitigate this effect, Raudenbush and Bryk (28) suggest including covariates that model realized group parameters to reduce the potential

PREDICTING RANDOM EFFECTS IN COMMUNITY INTERVENTION

biasing effect. Alternative strategies, such as conditional modeling frameworks, have been proposed (46–48), but increase in complexity with the complexity of the problem. Although increasing popularity of models that result in BLUP for realized latent groups exists, the basic questions that plagued researchers about interpretation of the predictors in the late twentieth century remain for the future. REFERENCES 1. H. Scheffe, A ‘mixed model’ for the analysis of variance. Ann. Math. Stat. 1956; 27: 23–36. 2. O. Kempthorne, Fixed and mixed models in the analysis of variance. Biometrics 1975; 31: 437–486. 3. M. B. Wilk and O. Kempthorne, Fixed, mixed, and random models. Amer. Stat. Assoc. J. 1955: 1144–1167. 4. H. Scheff´e, Analysis of Variance. New York: Wiley, 1959. 5. D. M. Murray, Design and Analysis of Grouprandomized Trials. New York: Oxford University Press, 1998. 6. S. R. Searle, G. Casella, and C. E. McCulloch, Prediction of random variables. In: Variance Components. New York: Wiley, 1992. 7. R. E. Kirk, Experimental Design. Procedures for the Behavioral Sciences. 3rd ed. New York: Brooks/Cole Publishing Company, 1995. 8. A. Donner and N. Klar, Design and Analysis of Cluster Randomized Trials in Health Research. London: Arnold, 2000. 9. C. Eisenhart, The assumptions underlying the analysis of variance. Biometrics 1947; 3: 1–21. 10. S. Searle, Linear Models. New York: Wiley, 1971. 11. M. L. Samuels, G. Casella, and G. P. McCabe, Interpreting blocks and random factors. J. Amer. Stat. Assoc. 1991; 86: 798–821. 12. C. R. Henderson, Applications of Linear Models in Animal Breeding. Guelph, Canada: University of Guelph, 1984. 13. G. K. Robinson, That BLUP is a good thing: the estimation of random effects. Stat. Sci. 1991; 6: 15–51. 14. D. A. Harville, Extension of the Gauss-Markov theorem to include the estimation of random effects. Ann. Stat. 1976; 4: 384–395. 15. R. C. Littell, G. A. Milliken, W. W. Stroup, and R. D. Wolfinger, SAS System for Mixed Models. Cary, NC: SAS Institute, 1996.

9

16. A. S. Goldberger, Best linear unbiased prediction in the generalized linear regression model. Amer. Stat. Assoc. J. 1962; 57: 369–375. 17. E. J. III Stanek, A. Well, and I. Ockene, Why not routinely use best linear unbiased predictors (BLUPs) as estimates of cholesterol, per cent fat from Kcal and physical activity? Stat. Med. 1999; 18: 2943–2959. 18. G. Verbeke and G. Molenberghs, Linear Mixed Models for Longitudinal Data. New York: Springer, 2000. 19. K. Hinkelmann and O. Kempthorne, Design and Analysis of Experiments. Volume 1. Introduction to Experimental Design. New York: Wiley, 1994. 20. C. R. Henderson, O. Kempthorne, S. R. Searle, and C. M. von Krosigk, The estimation of environmental and genetic trends from records subject to culling. Biometrics 1959: 192–218. 21. A. Scott and T. M. F. Smith, Estimation in multi-stage surveys. J. Amer. Stat. Assoc. 1969; 64: 830–840. 22. E. J. III Stanek and J. M. Singer, Estimating cluster means in finite population two stage clustered designs. International Biometric Society Eastern North American Region, 2003. 23. D. R. Cox, The five faces of Bayesian statistics. Calcutta Stat. Assoc. Bull. 2000; 50: 199–200. 24. T. R. Ten Have and A. R. Localio, Empirical Bayes estimation of random effects parameters in mixed effects logistic regression models. Biometrics 1999; 55: 1022–1029. 25. T. R. Ten Have, J. R. Landis, and S. L. Weaver, Association models for periodontal disease progression: a comparison of methods for clustered binary data. Stat. Med. 1995; 14: 413–429. 26. P. J. Heagerty and S. L. Zeger, Marginalized multilevel models and likelihood inference. Stat. Sci. 2000; 15: 1–26. 27. D. Rabinowitz and S. Shea, Random effects analysis of children’s blood pressure data. Stat. Sci. 1997; 12: 185–194. 28. S. R. Raudenbush and A. S. Bryk, Hierarchical Linear Models: Applications and Data Analysis Methods. 2nd ed. London: Sage Publications, 2002. 29. H. Goldstein, Multilevel Statistical Modeling. 3rd ed. Kendall’s Library of Statistics 3. London: Arnold, 2003. 30. C. E. McCulloch and S. R. Searle, Generalized, Linear, and Mixed Models. New York: Wiley, 2001.

10

PREDICTING RANDOM EFFECTS IN COMMUNITY INTERVENTION

31. J. M. Singer and D. F. Andrade, Analysis of longitudinal data. In: E. P. K. Sen and C. R. Rao (eds.), Handbook of Statistics. New York: Elsevier Science, 2000. 32. F. A. Graybill, Matrices with Applications in Statistics. Belmont, CA: Wadsworth International, 1983. 33. H. V. Henderson and S. R. Searle, On deriving the inverse of a sum of matrices. SIAM Rev. 1981; 23: 53–60. 34. C. R. Henderson, Selection index and expected genetic advance. In: Statistical Genetics and Plant Breeding. Washington, DC: National Academy of Sciences - National Research Council, 1963. 35. E. J. III Stanek and J. M. Singer, Predicting random effects from finite population clustered samples with response error. J. Amer. Stat. Assoc. 2004. 99: 1119–1130. 36. R. N. Kackar and D. A. Harville, Approximations for standard errors of estimators of fixed and random effects in mixed linear models. J. Amer. Stat. Assoc. 1984; 79: 853–862. 37. R. E. Kass and D. Steffey, Approximate Bayesian inference in conditionally independent hierarchical models (Parametric empirical Bayes models). J. Amer. Stat. Assoc. 1989; 84: 717–726. 38. D. A. Harville and D. R. Jeske, Mean squared error of estimation or prediction under general linear model. J. Amer. Stat. Assoc. 1992; 87: 724–731. 39. K. Das, J. Jiang, and J. N. K. Rao, Mean squared error of empirical predictor (Technical Report). Ottawa, Canada: Carleton University School of Mathematics and Statistics, 2001. 40. J. Wang and W. A. Fuller, The mean squared error of small area predictors constructed with estimated area variances. J. Amer. Stat. Assoc. 2003; 98: 716–723. 41. W. Cochran, Survey Sampling. New York: Wiley, 1977. 42. E. J. III Stanek and J. M. Singer, Predicting realized cluster parameters from two stage samples of an unequal size clustered population. Amherst, MA: Department of Biostat/Epid., University of Massachusetts, 2003. 43. H. Bolfarine and S. Zacks, Prediction Theory for Finite Populations. New York: SpringerVerlag, 1992. 44. X-H. Zhou, A. J. Perkins, and S. L. Hui, Comparisons of software packages for generalized linear multilevel models. Amer. Stat. 1999; 53: 282–290.

45. J. de Leeuw and I. Kreft, Software for multilevel analysis. In: A. Leyland and H. Goldstein (eds.), Multilevel Modelling of Health Statistics. Chichester: Wiley, 2001. 46. A. P. Dawid, Conditional independence in statistical theory. J. Royal Stat. Soc. 1979; 41: 1–31. 47. E. J. III Stanek and J. R. O’Hearn, Estimating realized random effects. Commun. Stat. Theory Meth. 1998; 27: 1021–1048. 48. G. Verbeke, B. Spiessens, and E. Lesaffre, Conditional linear mixed models. Amer. Stat. 2001; 55: 25–34.

PREFERENCE TRIALS

choice while randomizing those not expressing strong preferences in the usual manner (5). The effects of patient preferences and the advantages and disadvantages of the patient preference trial design will be discussed.

MARION K. CAMPBELL University of Aberdeen Aberdeen, United Kingdom

1

DAVID J. TORGERSON University of York York, United Kingdom

POTENTIAL EFFECTS OF PREFERENCE

In a standard randomized controlled trial, treatments are allocated to patients by means of a random process. By definition, patients cannot choose the treatment to which they get allocated. As such, if a patient has a strong preference to receive one particular treatment option, they have one of two choices:

When evaluating the effectiveness of interventions in health-care, it is now widely accepted that the randomized controlled trial is the gold standard design to adopt (1, 2). When randomization is adopted, this results in a number of benefits; for example, the groups generated should only differ by chance in baseline prognostic variables (i.e., bias is minimized), the potential for attribution is maximized, and the results can be analyzed using standard statistical testing. Acceptance of the standard randomized trial, however, requires that patients recruited to the trial be prepared to accept any of the interventions under study. Some participants may have explicit preferences for one of the treatments under evaluation, which may affect their willingness to participate in a standard randomized controlled trial, and they may refuse to be randomized (3). If the proportion of eligible participants who refuse randomization is significant, this jeopardizes the generalizability of the trial results (4). However, if the preferred treatment is only available within the context of a randomized trial (e.g., a novel treatment), many people may consent to take part hoping to get access to the new treatment. This consent hoping to be allocated to only one treatment can have significant consequences for the trial. If, for example, 50% of those being recruited to a trial of A versus B want intervention A, and this is only available within the trial, then they are likely to consent to participate in the trial; however, half of this 50% will be disappointed as they are going to be allocated to treatment B. A proposed solution to these problems has been the ‘‘patient preference trial,’’ which allocates patients with strong preferences to their treatment of

• Decline randomization to the trial. • Accept randomization to the trial in the

hope that they will be allocated to their preferred treatment. 1.1 Effects of Declining Randomization If a significant proportion of patients decline entry to the trial, this has a number of consequences for the trial conduct. First, it will lessen the generalizability of the results of the trial. For a trial to be truly generalizable, it must demonstrate that the individuals who consent to take part in the trial are representative of the wider population to whom the trial results are expected to apply. If, however, a significant proportion of patients decline on the basis of an explicit preference, this would indicate that those who consent to participate are markedly different from those who decline, compromising this assumption. A further consequence for the conduct of the trial if a significant number of patients decline to participate is that a longer recruitment window is likely to be required to achieve the target sample size. This has implications for the timeliness of the results because the publication of valid results requires that the target sample size is achieved. It also has cost consequences for the conduct of the trial as the trial team will have to be funded for a longer time period, leading to a more expensive trial for a potentially less generalizable result. The increased trial cost

Wiley Encyclopedia of Clinical Trials, Copyright  2007 John Wiley & Sons, Inc.

1

2

PREFERENCE TRIALS

also has knock-on effects for future research, as the more funders are required to channel money to this trial, the less money will be available for future research. 1.2 Effects of Accepting Randomization When Patients Have a Preference When patients accept randomization in the hope of being allocated to their preferred treatment, they can either be assigned to their treatment of choice or be assigned to the treatment they did not wish to receive. For patients who are not randomized to their treatment of choice, this can result in so called ‘‘resentful demoralization’’ (6). Resentful demoralization describes a pattern of behavior that can result if people are not allocated to their treatment of choice. This can involve difficulties with patient compliance with treatment, patients having a lower threshold for reporting adverse events of treatment, and an overly negative rating of their treatment effect. Patient compliance with treatment has been noted to be particularly problematic with treatments that require motivation, such as compliance with an exercise routine or compliance with dietary interventions (5). Conversely, patients allocated to their treatment of choice may rate their treatment overly positively. It is worthy of note, however, that although resentful demoralization is a known concept it remains more a theoretical concern because it is very difficult to accurately measure its effects and also to accurately attribute changes in patient compliance and outcome directly to initial preferences. The potential for patient preference to introduce systematic bias into the rating of treatment is particularly pertinent for subjective outcomes such as the patients assessments of the acceptability of the treatment or their self-assessed rating of their quality of life. For example, Clement et al. (7) highlighted the effect of initial preference on patients’ evaluations of different schedules of antenatal visits. In their trial, they compared traditional antenatal care (13 visits) with a new schedule of care (six to seven visits). The results showed that for women with no initial preference 7.4% of women allocated to traditional care were dissatisfied

with the frequency of visits compared with 31% allocated to new care. For those women who expressed an initial preference for traditional care, this difference in dissatisfaction rates was much more marked, with 3.5% randomized to traditional care being dissatisfied compared with 71.1% randomized to new style care. However, for women with an initial preference for new style care, the direction of effect was completely in the opposite direction: 48.9% randomized to traditional care were dissatisfied compared with 12.2% allocated to their preferred treatment of new care. These results highlight the significant impact preference can have on subjective trial outcomes. 2 THE PATIENT PREFERENCE DESIGN In an attempt to ameliorate the effects of preference in the standard randomized controlled trial, Brewin and Bradley (5) proposed the use of an alternative design: the partially randomized patient preference trial. Consider a trial of treatment A versus treatment B. In a patient preference design, patients who have an explicit preference for treatment A are allocated directly to treatment A. Similarly, patients who have an explicit preference for treatment B are allocated directly to treatment B. Only those patients who do not have a strong preference are randomized in the standard manner. This results in four parallel groups: those directly allocated to treatment A (group AllocA), those randomized to treatment A (RandA), those randomized to treatment B (RandB), and those directly allocated to treatment B (AllocB). A schematic outlining the resulting trial groups is presented in Figure 1. All four groups are then followed up, and their outcomes are compared. The effect of the intervention is still derived from the randomized comparison (RandA versus RandB); however, the effect of motivational factors can then also be estimated (from AllocA versus RandA and from AllocB versus RandB). An example of the use of the patient preference design is a trial by Bedi et al. (8) who compared the effectiveness of counseling compared with antidepressants for depression in primary care. These are very different

PREFERENCE TRIALS

3

Eligible for inclusion in trial Strong preference for treatment A expressed

No strong preference expressed

Strong preference for treatment B expressed

Randomize

Allocated to treatment A

Randomized to treatment A

Randomized to treatment B

Allocated to treatment B

Figure 1. The patient preference design.

treatments requiring different levels of active patient participation. The researchers were concerned that if a standard randomized controlled trial design was adopted the patients who did not get their preferred treatment (if they had a strong preference) might underestimate its effectiveness. To address these concerns, a patient preference design was adopted, where patients were first offered randomization but were allocated to their treatment of choice if they refused randomization on the basis of preference. In this trial, 103 patients agreed to randomization, and a further 220 were allocated directly to receive their treatment of choice (140 opted for counseling compared with 80 opting for antidepressants). The study found that there was no evidence of a difference in outcome between the randomized groups at 8 weeks. The investigators also noted that expressing a preference for either treatment did not appear to confer any additional benefit on outcome. 3 ADVANTAGES AND DISADVANTAGES OF THE PATIENT PREFERENCE DESIGN The patient preference design has a number of advantages and disadvantages. A key advantage is the comprehensive nature of the data collected and the additional inferences that can be drawn from explicit comparison of the results from the preference and randomized groups. Potential disadvantages have, however, been highlighted, especially the potential for selection bias across the trial groups, uncertainties about how the analysis of the data from the preference arms should

be integrated into the overall trial analysis, and the added cost of following up the nonrandomized patients. We will discuss these issues in turn. 3.1 Potential for Additional Insights Provided by the Data from the Preference Arms One of the purported advantages of the patient preference design is that the data gained from those in the preference arms provide valuable information on the impact of motivational factors, which can then be used to inform the most appropriate implementation of the trial intervention. This was apparent in a trial conducted by Henshaw et al. (9) who compared the effectiveness of surgical (vacuum aspiration) versus medical termination (mifepristone) of early pregnancies. In this trial, 363 women agreed to be recruited to the trial; 95 (26%) expressed an explicit preference for vacuum aspiration, 73 (20%) expressed an explicit preference for medical termination, and 195 (54%) agreed to be allocated to either method randomly. Reasons for preferences varied. Primary reasons for preferring medical treatment included fear of anesthesia and perceptions that medical treatment was less invasive and more ‘‘natural.’’ Conversely, primary reasons for preferring surgery included the relative speed of the technique, and fears about the physical and potential psychological sequelae of medical treatment. The results of the randomized comparison showed no evidence of a difference in acceptability between the two treatment approaches. There was also no evidence of difference in acceptability of

4

PREFERENCE TRIALS

the two techniques in those allocated to their treatment of choice. Further examination of the trial data showed, however, that women who had stated a preference for the surgical option tended to live much farther away from the hospital than those preferring the medical termination option. This primarily reflected a desire by women living farther from the hospital to take advantage of the fewer visits to hospital required in the surgical arm. This extra insight led the investigators to conclude that although there was no apparent superiority of effect with the surgical intervention, both treatment options should remain available. This additional insight would not have been possible to detect without the preference cohorts. 3.2 Potential for Selection Bias across the Trial Groups As patients with strong preferences can choose their treatment of choice in a patient preference design, it is possible that this could lead to selection bias across the different trial groups. It may be that preference could be determined by some underlying confounding factor. For example, in a trial of medical versus surgical management of a disease, those who have more severe disease may prefer surgery in the expectation that surgery is a more radical treatment and thus more suited to their needs. If the preference is a proxy for some other underlying measure of patient status, differential representation across preference and randomized groups may be problematic for future analysis. Such self-selection to trial groups was observed in a recent trial in the United Kingdom of surgical management versus continued optimized medical management for the treatment of gastroesophageal disease (10). In this patient preference trial, patients expressing a strong desire for surgery were directly allocated to surgical management (laparoscopic fundoplication) whereas those with a strong desire to remain on medical management were directly allocated to medical management (with proton pump inhibitors). Those who expressed no strong preference were randomized in the standard manner. On completion of recruitment,

however, it was apparent that there was a differential distribution of disease severity across the trial groups. Across key measures of self-reported quality of life at baseline, those choosing surgery reported a lower quality of life score compared with those recruited to the randomized comparisons. In contrast those choosing medical management reported better quality of life than those recruited to the randomized groups. As one would expect, there was no evidence of a difference in self-reported health across the randomized groups. This study showed that the introduction of preference arms can lead to a narrower spectrum of patients being recruited to the randomized element of a patient preference trial. 3.3 Analysis of the Data from the Preference Arms Although the comparison of data from those directly allocated to their chosen treatment with the randomized groups within a patient preference design is intuitively appealing, the true implications of such an analysis are unclear. The primary reason for this is that data from the non-randomized groups are not afforded the theoretical protection of randomization and, as such, are subject to the range of biases inherent in all observational data—especially the influence of confounding variables (whether known or unknown) (11). It has, therefore, been recommended that the primary analysis remains the comparison of the randomized groups. It is also unclear what the interpretation of a trial should be if those allocated to their preferred treatment appear to do worse that those randomized to the treatment. 3.4 Required Increase in Sample Size With the introduction of parallel preference arms, four groups of participants are routinely followed up in the patient preference design compared with two groups in a standard design. One of the key advantages of this additional data collection exercise has already been discussed: the extra data gained from those in the preference arms provide valuable information on the impact of motivational factors that could then be used to

PREFERENCE TRIALS

inform the most appropriate implementation of the trial intervention. However, in the standard randomized design, these patients with a strong preference would have refused randomization and hence would not have required time and resource to follow them up. This raises an interesting dilemma: are the extra data collected worth the extra cost? If resources for the conduct of research are unlimited, this is not an issue as all extra information is of intrinsic value; however, resources are limited, and when funders direct extra resources toward the follow-up of preference patients, this means that there is less money available for the conduct of future studies. As such this is an extra dimension which must be considered when adopting the preference design. An interesting illustration of this dilemma was presented in a randomized trial conducted by Cooper et al. (12). In this study of different treatments for the management of menorrhagia, women were randomized to be approached to take part in either (1) a standard randomized comparison or (2) a patient preference design. The results of this study showed that although the preference design included more women overall there was no apparent impact on recruitment to the randomized arms under the different designs. In this instance, the investigators concluded that the money that had been spent on following up the preference groups might have been better spent promoting overall recruitment to the standard randomized design. 4

ALTERNATIVE DESIGNS

The patient preference design is the most widely used design to accommodate for preferences, but a number of other designs have also been suggested. The two main alternatives are: • The two-step randomized design • The fully randomized preference design

4.1 Two-Step Randomized Design The two-step randomized design aims to measure the effect of preference by explicitly including allocation to receive preference into

5

the study design. Wennberg (13) proposed such a design in the early 1990s whereby potential trial participants are initially randomized either to receive their preferred treatment directly (Wennberg group A) or to be allocated to treatment using standard randomization (Wennberg group B). See Figure 2 for the design. ¨ Rucker (14) proposed a similar design to Wennberg, but included the possibility of a secondary randomization within the group initially randomized to receive their pref¨ erence. In the Rucker design, participants are initially randomized into the same two groups as the Wennberg design (i.e., to receive their preferred treatment directly or to be randomized to a treatment group). However, ¨ in the Rucker design, those initially allocated to receive their preference directly (similar to Wennberg group A) but who do not have a strong preference are subsequently randomized to treatment (Figure 2). The main advantage of this type of twostage design is that the effect of preference can be robustly measured (because it is planned for in the design); however, patients are still required to accept randomization, which may be problematic. Few studies have used this design in practice. 4.2 Fully Randomized Preference Design Recently, an alternative trial design has been proposed to measure the effect of patient preferences, the fully randomized preference design (15). This is appropriate for situations in which a new treatment is only available within the trial and where participants who have a preference would normally consent to participate in the hope of being allocated to the new treatment. In this design, preferences are recorded before randomization, allowing estimation of the effect of the intervention both among participants without a preference but also among those with a preference. For example, in a randomized trial of a physiotherapy intervention versus standard care for the management of back pain that used this approach (16), the following groups of participants resulted: 1. Indifferent participants allocated to physiotherapy

6

PREFERENCE TRIALS

Eligible for inclusion in trial

Random allocation

Randomized group

Randomized to A

Preference group

Receive preferred option A

Randomized to B

Receive preferred option B

(a) Wennberg design

Eligible for inclusion to trial

Random allocation

Randomized group

Randomized to A

Preference group

Randomized to B

No strong preference

Randomized to A

Randomized to B

Strong preference

Receive preferred Receive preferred option A option B

(b) Rücker design ¨ Figure 2. Two-step randomized designs. (a) Wennberg design. (b) Rucker design.

2. Indifferent participants allocated to standard care 3. Participants preferring physiotherapy allocated to receive physiotherapy 4. Participants preferring physiotherapy allocated to standard care By using this approach, the investigators were able to demonstrate that the treatment was effective across the different preference groups, and the results appeared not to have been affected by patient preference. Interestingly, the study showed the treatment did not work any better among those who really wanted the treatment and those who were allocated to receive it.

Another demonstration of the effect of eliciting participant preferences has occurred in a randomized trial of two treatments for neck pain (17). Participants were presented with two treatment options: the standard treatment, which consisted of between 5 and 10 physiotherapy treatments, or a brief intervention, which was conducted usually as a one-off treatment session with the aim of teaching the patient to ‘‘self-treat’’ using cognitive behavioral principles. As in the back pain trial, the patients were asked for their preferences before randomization. This resulted in six different groups of participants: 1. Indifferent participants randomized to standard care

PREFERENCE TRIALS

2. Indifferent participants randomized to brief intervention 3. Participants preferring standard care and randomized to standard care 4. Participants preferring standard care and randomized to brief intervention 5. Participants preferring brief intervention and randomized to brief intervention 6. Participants preferring brief intervention and randomized to standard care Figure 3 shows the main trial results. The figure shows that among those participants who were indifferent (groups 1 and 2) the standard care resulted in better outcomes relative to the brief intervention. (In this group, patient preferences have been eliminated so this is the ‘‘true’’ treatment effect unconfounded by preference.) The results among the subgroup of patients who had a preference for standard care (groups 3 and 4) were similar; however, those who were randomized to the brief intervention actually got slightly worse. The final group, those who preferred the brief intervention (groups 5 and 6) showed the most interesting result. For these patients, the direction of effect was actually reversed. Patients who wanted the brief intervention and were randomized to the brief intervention (group 5) did better than those who wanted the brief intervention but were randomized to standard care (group 6). This was despite our observing that standard care is the superior treatment in the absence of patient preference. This trial, therefore,

showed that the presence of preferences can influence treatment effects. In this instance, the proportion of participants who wanted the brief intervention was relatively small, so their results would not have strongly biased the trial’s overall findings. However, in clinical or health policy terms, the extra insights enhance the trial’s overall findings. Had participants’ preferences not been ascertained, we would have concluded that standard care was best and that all patients should be offered standard care. However, it is apparent that for those who express a strong desire for the brief intervention, this may indeed be the best treatment for them. Although the advantages of this design have been clearly demonstrated, the fully randomized preference design is not without its limitations. As with the standard randomized controlled trial design, it is highly likely that a proportion of patients who have strong preferences will refuse to be randomized under this design. As such, the results will only be able to inform the influence of weaker levels of preference. The interpretability of the results of this design would, however, be superior to those from a standard randomized controlled trial design as the influence of preference could at least be explored, even though all such analyses would be exploratory unless the trial had been specifically powered to address them. This design also relies on accurate measurement of preferences at baseline. To date, however, there has been little research available to guide how best to measure preference

Change in NPQ at 12 months

1

Figure 3. Preference Results of Neck Pain Trial.

0.5

Preference

0 −0.5

7

Indifferent

Brief Intervention

Usual Care

−1 −1.5 −2 −2.5 −3 −3.5 Brief Intervention

Usual Care

8

PREFERENCE TRIALS

in a valid, reliable manner. Further research in this field is required. 4.

5

DISCUSSION 5.

Participant preference can lead to bias in trials. Ignoring the role of preference on outcome could lead us to conclude something is effective when it is not, or is ineffective when it is not. It is, therefore, important when designing a trial to consider whether preference for particular treatments might be problematic for the trial under consideration. If patient preference could be an issue, it is important that it be accounted for; however, patients must be presented with full information on all treatments available within the trial to ensure that they make an informed choice whether to accept randomization or to choose their treatment. The patient-preference design advocated by Brewin and Bradley has been used successfully in a number of fields and has been shown to yield additional insights that can aid the interpretation of trial results. However, there have been a number of criticisms of the patient preference design, and investigators have been considering other ways of accommodating appropriately for preferences. A number of alternative designs exist, although they have not been used so widely in the literature. One such alternative is the fully randomized preference design, which has been used in a number of recent trials (16–20). As the results of these and other trials become available, together with future research on how best to measure preferences, recommendations for the most appropriate design to use for specific situations will become clearer. REFERENCES 1. S. J. Pocock, Clinical Trials: A Practical Approach. Chichester, UK: Wiley, 1983.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

2. B. Sibbald and M. Roland, Understanding controlled trials: why are randomised controlled trials important? BMJ. 1998; 316: 201. 3. K. Fairhurst and C. Dowrick, Problems with recruitment in a randomised controlled trial of counselling in general practice: causes and

16.

implications. J Health Serv Res Policy. 1996; 1: 77–80. C. Bradley, Designing medical and educational intervention studies. Diabetes Care. 1993; 16: 509–518. C. R. Brewin and C. Bradley, Patient preferences and randomised clinical trials. BMJ. 1989; 299: 313–315. T. D. Cook and D. T. Campbell, Quasiexperimentation: Design and Analysis Issues for Field Settings Chicago: Rand McNally, 1979. S. Clement, J. Sikorski, J. Wilson, and B. Candy, Merits of alternative strategies for incorporating patient preferences into clinical trials must be considered carefully [letter]. BMJ. 1998; 317: 78 N. Bedi, C. Chilvers, R. Churchill, M. Dewey, C. Duggan, et al., Assessing effectiveness of treatment of depression in primary care: partially randomised preference trial. Br J Psychiatry. 2000; 177: 312–318. R. C. Henshaw, S. A. Naji, I. T. Russell, and A. A. Templeton, Comparison of medical abortion with surgical vacuum aspiration: women’s preferences and acceptability of treatment. BMJ. 1993; 307: 714–717. A. Grant, S. Wileman, C. Ramsay, L. Bojke, D. Epstein et al. The place of minimal access surgery amongst people with gastro-oesophageal reflux disease—a UK collaborative study. The Reflux Trial (ISRCTN15517081). Health Technol Assess (in press). W. A. Silverman and D. G. Altman, Patient preferences and randomised trials. Lancet. 1996; 347: 171–174. K. G. Cooper, A. M. Grant, and A. M. Garratt, The impact of using a partially randomised patient preference design when evaluating managements for heavy menstrual bleeding. BJOG. 1997; 104: 1367–1373. J. E. Wennberg, M. J. Barry, F. J. Fowler, and A. Mulley, Outcomes research, PORTS, and healthcare reform. Ann NY Acad Sci. 1993: 703: 52–62. ¨ G. Rucker, A two-stage trial design for testing treatment, self selection and treatment preference effects Stat Med 1989; 8: 477–485. D. J. Torgerson, J. A. Klaber Moffett, and I. T. Russell, Patient preferences in randomised trials: threat or opportunity. J Health Serv Res Policy. 1996; 1: 194–197. J. Klaber Moffett, D. Torgerson, S. Bell-Syer, D. Jackson, H. Llewlyn-Phillips, et al., Randomised controlled trial of exercise for low

PREFERENCE TRIALS back pain: clinical outcomes, costs, and preferences. BMJ. 1999; 319: 279–283. 17. J. A. Klaber-Moffett, S. Richmond, D. Jackson, S. Coulton, S. Hahn, et al., Randomised trial of a brief physiotherapy intervention compared with usual physiotherapy for neck pain patients: outcomes, costs and patient preference. BMJ. 2005; 330: 75. 18. J. L. Carr, J. A. Klaber Moffett, E. Howarth, S. J. Richmond, D. J. Torgerson, et al., A randomised trial comparing a group exercise programme for back pain patients with individual physiotherapy in a severely deprived area Disabil Rehabil 2005; 27: 929–937. 19. K. J. Sherman, D. C. Cherkin, J. Erro, D. L. Miglioretti, R. A. Deyo, Comparing yoga, exercise, and a self-care book for chronic low back pain: a randomised, controlled trial. Ann Intern Med. 2005; 143: 849–856.

FURTHER READING

20. E. Thomas, P. R. Croft, S. M. Paterson, K. Dziedzic, E. M. Hay, What influences participants’ treatment preference and can it influence outcome? Results from a primary care-based randomised trial for shoulder pain. Br J Gen Pract. 2004; 54: 93–96.

Phase III trials

9

M. King, I. Nazareth, F. Lampe, P. Bower, M. Chandler, et al., Conceptual framework and systematic review of the effects of participants’ and professionals’ preferences in randomised controlled trials. Health Technol Assess. 2005; 9(35): 1–186. M. King, I. Nazareth, F. Lampe, P. Bower, M. Chandler, et al., Impact of participant and physician intervention preferences on randomised trials: a systematic review. JAMA. 2005; 293: 1089–1099.

CROSS-REFERENCES Large simple trials External validity

PREMARKET APPROVAL (PMA)

FDA regulations provide 180 days to review the PMA and to make a determination. In reality, the review time is normally longer. Before approving or denying a PMA, the appropriate FDA advisory committee may review the PMA at a public meeting and provide the FDA with the committee’s recommendation on whether the FDA should approve the submission. After the FDA notifies the applicant that the PMA has been approved or denied, a notice is published on the Internet (1) that announces the data on which the decision is based and (2) that provides interested persons an opportunity to petition FDA within 30 days for reconsideration of the decision. The regulation governing premarket approval is located in Title 21 Code of Federal Regulations (CFR) Part 814, Premarket Approval. A class III device that fails to meet PMA requirements is considered to be adulterated under section 501(f) of the FD&C Act and cannot be marketed. PMA requirements apply to Class III devices, which is the most stringent regulatory category for medical devices. Device product classifications can be found by searching the Product Classification Database. The database search provides the name of the device, classification, and a link to the Code of Federal Regulations, if any. The CFR provides the device type name, identification of the device, and classification information. A regulation number for Class III devices marketed prior to the 1976 Medical Device Amendments is provided in the CFR. The CFR for these Class III devices that require a PMA states that the device is Class III and will provide an effective date of the requirement for PMA. If the regulation in the CFR states that ‘‘No effective date has been established of the requirement for premarket approval,’’ a Class III 510(k) should be submitted. Please note that PMA devices often involve new concepts and many are not of a type marketed prior to the Medical Device Amendments. Therefore, they do not have a classification regulation in the CFR. In this case, the

Premarket approval (PMA) is the Food and Drug Administration (FDA) process of scientific and regulatory review to evaluate the safety and the effectiveness of Class III medical devices. Class III devices are those that support or sustain human life, are of substantial importance in preventing impairment of human health, or that present a potential, unreasonable risk of illness or injury. Because of the level of risk associated with Class III devices, the FDA has determined that general and special controls alone are insufficient to assure the safety and the effectiveness of class III devices. Therefore, these devices require a PMA application under section 515 of the Food, Drug and Cosmetic (FD&C) Act to obtain marketing clearance. Please note that some Class III preamendment devices may require a Class III 510(k). PMA is the most stringent type of device marketing application required by the FDA. The applicant must receive FDA approval of its PMA application prior to marketing the device. PMA approval is based on a determination by the FDA that the PMA contains sufficient valid scientific evidence to assure that the device is safe and effective for its intended use(s). An approved PMA is, in effect, a private license that grants the applicant (or owner) permission to market the device. The PMA owner, however, can authorize use of its data by another. Usually, the PMA applicant is the person who owns the rights, or otherwise has authorized access, to the data and other information to be submitted in support of FDA approval. This applicant may be an individual, partnership, corporation, association, scientific or academic establishment, government agency or organizational unit, or other legal entity. The applicant is often the inventor/developer and ultimately the manufacturer. This article was modified from the website of the United States Food and Drug Administration (http://www.fda.gov/cdrh/devadvice/pma/) by Ralph D’Agostino and Sarah Karl.

Wiley Encyclopedia of Clinical Trials, Copyright © 2008 John Wiley & Sons, Inc.

1

2

PREMARKET APPROVAL (PMA)

product classification database will only cite the device type name and the product code. If it is unclear whether the unclassified device requires a PMA, then use the three-letter product code to search the PMA database and the Premarket Notification 510(k) database. These databases can be found by clicking on the hypertext links at the top of the product classification database web page. Enter only the three-letter product code in the product code box. If there are 510(k)’s cleared by FDA and the new device is substantially equivalent to any of these cleared devices, then the applicant should submit a 510(k). Furthermore, a new type of device may not be found in the product classification database. If the device is a high-risk device (it supports or sustains human life, is of substantial importance in preventing impairment of human health, or presents a potential, unreasonable risk of illness or injury) and has been found to be not substantially equivalent (NSE) to a Class I, II, or III [Class III requiring 510(k)] device, then the device must have an approved PMA before marketing in the United States. Some devices that are found to be not substantially equivalent to a cleared Class I, II, or III (not requiring PMA) device, may be eligible for the de novo process as a Class I or Class II device. For additional information on the de novo process, see ‘‘New section 513(f)(2) - Evaluation of Automatic Class III Designation: Guidance for Industry and [Center for Device and Radiological Health] CDRH Staff’’. PMA requirements apply to Class III preamendment devices, transitional devices, and postamendment devices. 1

PREAMENDMENT DEVICES

A preamendment device is one that was in commercial distribution before May 28, 1976, which is the date the Medical Device Amendments were signed into law. After the Medical Device Amendments became law, the classification of devices was determined by FDA classification panels. Eventually, all Class III devices will require a PMA. However, preamendment Class III devices require a PMA only after the FDA publishes a regulation

calling for PMA submissions. The preamendment devices must have a PMA filed for the device by the effective date published in the regulation to continue marketing the device. The CFR will state the date that a PMA is required. Prior to the PMA effective date, the devices must have a cleared Premarket Notification 510(k) prior to marketing. Class III preamendment devices that require a 510(k) are identified in the CFR as Class III and include the statement ‘‘Date premarket approval application (PMA) or notice of completion of product development protocol (PDP) is required. No effective date has been established of the requirement for premarket approval.’’ Examples include intra-aortic balloon and control system (21 CFR 870. 3535), ventricular bypass (assist) device (21 CFR 870.3545), cardiovascular permanent pacemaker electrode (21 CFR 870.3680), and topical oxygen chamber for extremities (21 CFR 878.5650). 2 POSTAMENDMENT DEVICES A postamendment device is one that was first distributed commercially on or after May 28, 1976. Postamendment devices equivalent to preamendment Class III devices are subject to the same requirements as the preamendment devices. 3 TRANSITIONAL DEVICES Transitional devices are devices that were regulated by FDA as new drugs before May 28, 1976. Any Class III device that was approved by a New Drug Application (NDA) is now governed